I've got a API stage that's NOT using "Lambda Proxy integration" which has a Lambda function passing an error.
In the mapping template I have this:
$input.path("$.errorMessage")
Which results in the output of this:
{
"headers": {
"apiVersion": "20190218.1",
"isTesting": true
},
"body": {
"statusCode": 503,
"status": "Service Unavailable",
"title": "One or more of our data providers are currently offline for scheduled maintenance"
}
}
The header values are mapped to template headers and pull through correctly, however I need the body to transform to this:
{
"statusCode": 503,
"status": "Service Unavailable",
"title": "One or more of our data providers are currently offline for scheduled maintenance"
}
Whatever I have tried, body always returns as a blank string, an empty body, or an invalid JSON.
This is the closest I've got but it returns an invalid JSON:
$util.parseJson($input.path("$.errorMessage")).body
Result (comes back with no quotes):
{statusCode=503, status=Service Unavailable, title=One or more of our data providers are currently offline for scheduled maintenance}
Is it possible to do what I'm after? I can't find a reverse for $util.parseJson (i.e, stringify).
Thanks!
I think the original poster has probably moved on in the past 11 months, but in case anyone else stumbles across this question, $input.json('$.errorMessage.body') should work.
Related
Very short version
How do I include an ADF Variable inside a JSON POST request, in a Web Activity within ADF?
I feel like this should be a very simple string concatenation, but i can't get it to work
Detail
We have a requirement to run a query / SProc from within ADF, which will return a string containing an error message. That string is to then be passed via the Web Activity in ADF to a Logic App, in order to fire off an email, containing the error.
The setup of the logic app is copied from here:
https://www.mssqltips.com/sqlservertip/5718/azure-data-factory-pipeline-email-notification--part-1/
and then here (part 2)
https://www.mssqltips.com/sqlservertip/5962/send-notifications-from-an-azure-data-factory-pipeline--part-2/
In ADF, I used the Lookup activity, to run a query, which brings back the error (appears to work, the preview returns the correct string)
Then I use the Set Variable activity, to take the output of the lookup and store it in a variable.
Last Step is to fire off the POST using the Web Activity.
With this code (tweaked slightly to remove personal details) in my Web Activity, everything works fine and I receive an email
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "Pipeline finished!",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
But any attempt to put the contents of the Variable into the Subject part has failed.
This (for example) sends me an email with the subject literally being #variables('EmailSubject')
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#variables('EmailSubject')",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
But I've also attempted various other solutions that result in errors or the email subject just containing the literal thing that I put in there (e.g. + #variables('EmailSubject') +).
I also tried storing the entire JSON in the Variable, and then having the Web activity use only the variable, that returned no errors, but also did not send an email.
This attempt:
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#{variables('EmailSubject')}",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
Resulted in this input into the web activity - which actually includes the text of the error, which is a bonus ... (text = Job Duration Warning):
{
"url": "https://azureLogicAppsSiteHere",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": "{\n \"DataFactoryName\": \"DFNAMEHERE\",\n \"PipelineName\": \"pipeline1\",\n \"Subject\": \"{\"firstRow\":{\"\":\"Job Duration Warning\"},\"effectiveIntegrationRuntime\":\"DefaultIntegrationRuntime (West Europe)\",\"billingReference\":{\"activityType\":\"PipelineActivity\",\"billableDuration\":[{\"meterType\":\"AzureIR\",\"duration\":0.016666666666666666,\"unit\":\"DIUHours\"}]},\"durationInQueue\":{\"integrationRuntimeQueue\":0}}\",\n \"ErrorMessage\": \"Everything is okey-dokey!\",\n \"EmailTo\": \"me#myEmail.com\"\n}\t"
}
But then resulted in this error:
{
"errorCode": "2108",
"message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: f. Path 'Subject', line 4, position 17.'.\"}}",
"failureType": "UserError",
"target": "Web1",
"details": []
}
[Edit] The PREVIEW from the Lookup Activity is the text: Job Duration Warning BUT when I debug the pipeline, it lets me see the actual Output, which is this:
{
"count": 1,
"value": [
{
"": "Job Duration Warning"
}
],
"effectiveIntegrationRuntime": "DefaultIntegrationRuntime (West Europe)",
"billingReference": {
"activityType": "PipelineActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "DIUHours"
}
]
},
"durationInQueue": {
"integrationRuntimeQueue": 0
}
}
So it appears that the problem is that the Lookup Output isn't what I thought it was, so the variable can't be used in the Web Activity, as it contains unsupported characters or something along those lines.
I just tested this and it worked ok:
Create a String Parameter with the value Job Duration Warning
Set the Variable value to be #pipeline().parameters.ParamSubject
Include the variable in the web activity with an # in front of it
I then receive my expected email with the right subject. I just don't know how to get the string output of my query, into a variable / parameter, so that i can use it in the web activity.
I don't know how well this applies to other people's issues, but I found a solution that has worked for me.
In the SELECT query within the Lookup Activity - name the output (in my case, I called that column 'Subject'- i.e. SELECT xyz AS Subject
In the Lookup Activity, turn on the setting 'First Row Only'
In the Set Variable Activity, use the code: #activity('Lookup1').output.firstRow.subject
(where 'Lookup1' is the name of your Lookup Activity and Subject is the name of the column you are outputting)
In the Web Activity, reference the variable as follows:
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#{variables('EmailSubject')}",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
I'm just doing the preparation for an integration with EasyPost's Shipping API, which will be server side C#, but we always build a PostMan collection for new integrations, so that we can test data separately from the application if there's an issue.
While I do love the fact that EP provide C# libraries and examples, I'm struggling to find anything that just gives me a list of required headers and the raw JSON format for the body of any requests. It feels a bit like they're just being a little too helpful.
I'll be looking at the Orders endpoint probably.
I've got an account, I've checked all their documentation and searched the internet but haven't found anything so I'm hoping I'm not the first developer to want to use a client application for testing outside my code.
Update:
EasyPost now does have a public workspace https://www.postman.com/easypost-api
with at least 1 public collection
The curl examples are basically the same as json:
For the Address creation example:
-d "address[street1]=417 MONTGOMERY ST"
is the equivalent of
{ "address": { "street1": "417 MONTGOMERY ST" } }
(you might have to escape some characters to be valid json).
Check out How to stimulate cURL request to a request using postman for postman with HTTP Basic Auth.
EasyPost does not provide a public Postman collection; however, here is an example of a shipment Postman body (raw) that could be used. You can adjust the values, actions, objects, etc to your needs.
Using the following, you shouldn't need to pass any headers. You'll pass your API key under the username field with the Basic Auth authorization type.
{
"shipment": {
"to_address": {
"id": "adr_123..."
},
"from_address": {
"id": "adr_123..."
},
"parcel": {
"id": "prcl_123..."
},
"carrier_accounts": {
"id": "ca_123..."
},
"options": {
"address_validation_level": "0"
}
},
"format": "json",
"controller": "shipments",
"action": "create"
}
I am posting the results/logs of a CI/CD system to Microsoft Teams. While handling some failed builds with longer results, I stumbled upon the following error returned by the webhook URL https://outlook.office.com/webhook/bb6bfee7-1820-49fd-b9f9-f28f7cc679ff#<uuid1>/IncomingWebhook/<id>/<uuid2>:
Webhook message delivery failed with error: Microsoft Teams endpoint returned HTTP error 413 with ContextId tcid=3626521845697697778,server=DB3PEPF0000009A,cv=BmkbJ1NdTkv1EDoqr7n/rg.0..
As I observe, this is caused by too long payload posted to the Teams webhook URL.
The initial complex message (sections, titles, subtitles, formatted links, <pre> formatted text, etc.) was failing when the JSON payload was a above 18000 characters.
Testing a bit with the payload I observed that the more formatting I remove from the raw JSON payload, the longer can the Teams message be. The longest message I could post had (according cu cURL): Content-Length: 20711. The JSON payload for this message was:
{"themeColor":"ED4B35","text":"a....a"}
whitespaces in the JSON format seem not to count (i.e. adding spaces will not decrease the maximum message length that I can sent to the Teams webhook).
For reference, the initial message was looking similar to this:
{
"themeColor": "ED4B35",
"summary": "iris-shared-libs - shared-library-updates - failure",
"sections": [
{
"activityTitle": "Job: [iris-shared-libs](https://my.concourse.net/teams/hsm/pipelines/iris-shared-libs) - [shared-library-updates #89](https://my.concourse.sccloudinfra.net/teams/hsm/pipelines/iris-shared-libs/jobs/shared-library-updates/builds/89) (FAILURE)",
"activityImage": "https://via.placeholder.com/200.png/ED4B35/FFFFFF?text=F",
"facts": [
{
"name": "Failed step",
"value": "update-shared-libraries"
}
]
},
{
"text": "Trying a new strategy with gated versioned releases",
"facts": [
{
"name": "Repository",
"value": "[iris-concourse-resources](https://my.git.com/projects/IRIS/repos/iris-concourse-resources)"
},
{
"name": "Commit",
"value": "[2272145ddf9285c9933df398d63cbe680a62f2b7](https://my.git.com/projects/IRIS/repos/iris-concourse-resources/commits/2272145ddf9285c9933df398d63cbe680a62f2b7)"
},
{
"name": "Author",
"value": "me#company.com"
}
]
},
{
"activityTitle": "Job failed step logs part 1",
"text": "<pre>...very long log text goes here ...</pre>"
}
]
}
What is the actual maximum lengths of the Microsoft Teams connector webhook posted message?
The official page does not mention it. In the Feedback section at the bottom there is still an open question regarding "Messages size limits?" with the feedback: "We are currently investigating this."
From the tests I made so far, some limits observed (if this is independent of the server) are roughly, based on the JSON message payload (structure and formatting) between 18000 and 40000 (with length below 18000 never breaking and above 40000 always breaking).
Use case 18000: one long text for a section
Use case 40000: 600 facts with very short name and empty string as value
And removing a fragment of the JSON payload and adding an equal number of characters in another JSON value will not give you the same maximum.
I have observed a soft limit (message truncated, but no error) as well on the maximum number of sections: 10. The sections starting with the 11th are discarded.
I'm requesting a list of possible stock ticker symbols from the Yahoo Stock Lookup API, I used the CORS Anywhere service to avoid problems. Unfortunately the original json response from the Yahoo API gets formatted as 'text' in the CORS anywhere 'error' property.
I used this previously asked question as a reference on how to access nested properties.
this is what the json reponse looks like:
{
"headers": {
"normalizedNames": {},
"lazyUpdate": null
},
"status": 200,
"statusText": "OK",
"url": "https://cors-anywhere.herokuapp.com/http://d.yimg.com/autoc.finance.yahoo.com/autoc?query=Amd®ion=1&lang=en&callback=YAHOO.Finance.SymbolSuggest.ssCallback",
"ok": false,
"name": "HttpErrorResponse",
"message": "Http failure during parsing for https://cors-anywhere.herokuapp.com/http://d.yimg.com/autoc.finance.yahoo.com/autoc?query=Amd®ion=1&lang=en&callback=YAHOO.Finance.SymbolSuggest.ssCallback",
"error": {
"error": {
"line": 9848,
"column": 56,
"sourceURL": "capacitor://localhost/vendor-es2015.js"
},
"text": "YAHOO.Finance.SymbolSuggest.ssCallback({\"ResultSet\":{\"Query\":\"Amd\",\"Result\":[{\"symbol\":\"AMD\",\"name\":\"Advanced Micro Devices, Inc.\",\"exch\":\"NMS\",\"type\":\"S\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Equity\"},{\"symbol\":\"DOX\",\"name\":\"Amdocs Limited\",\"exch\":\"NMS\",\"type\":\"S\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Equity\"},{\"symbol\":\"AMDVX\",\"name\":\"American Century Mid Cap Value Fund R6 Class\",\"exch\":\"NAS\",\"type\":\"M\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Fund\"},{\"symbol\":\"AMDWX\",\"name\":\"Amana Mutual Funds Trust Developing World Fund Investor\",\"exch\":\"NAS\",\"type\":\"M\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Fund\"},{\"symbol\":\"AMDAX\",\"name\":\"AMIDEX35 Israel Fund Class A\",\"exch\":\"NAS\",\"type\":\"M\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Fund\"},{\"symbol\":\"AMDEX\",\"name\":\"AMIDEX35 Israel Mutual Fund\",\"exch\":\"NAS\",\"type\":\"M\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Fund\"},{\"symbol\":\"AMDCX\",\"name\":\"AMIDEX35 Israel Fund Class C\",\"exch\":\"NAS\",\"type\":\"M\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Fund\"},{\"symbol\":\"AMDRX\",\"name\":\"American Beacon Mid-Cap Value Fund R6 Class\",\"exch\":\"NAS\",\"type\":\"M\",\"exchDisp\":\"NASDAQ\",\"typeDisp\":\"Fund\"},{\"symbol\":\"AMDLY\",\"name\":\"Amada Holdings Co., Ltd.\",\"exch\":\"PNK\",\"type\":\"S\",\"exchDisp\":\"OTC Markets\",\"typeDisp\":\"Equity\"}]}});"
}
}
This is what I tried:
return this.yahooResult.subscribe(data => {return data['error']['text']['ResultSet']['Result']['symbol']});
I'm trying to access "symbol" : "AMD" from the Yahoo API response that is nested in the "text" property of the CORS repsonse.
I don't get much information from the error:
⚡️ [error] - ERROR
You don't have json there, you have a javascript snippet:
YAHOO.Finance.SymbolSuggest.ssCallback()
With some escaped text passed into it.
It's not a case of parsing this, its a case of fixing the error.
I think you are using the wrong API call. It has a callback parameter in it which is wrapping the reply in a function and breaking cors-anywhere's ability to parse the response.
The url you should be using is:
http://d.yimg.com/autoc.finance.yahoo.com/autoc?query=Amd®ion=1&lang=en
Not:
http://d.yimg.com/autoc.finance.yahoo.com/autoc?query=Amd®ion=1&lang=en&callback=YAHOO.Finance.SymbolSuggest.ssCallback
Then just plain JSON gets returned and cors-anywhere should be able to parse it.
I installed Orion and Fiware STH in order to use the minimalistic option.
The problem is that when i subscribe to an orion context then consult, the array is null.
"name": "temperature",
"values": []
In the STH log it says:
ERROR when collecting the raw data collection for retrieval (the collection may not exist).
I noticed that when i try to unsubscribe it returns a 200 OK message with the following body:
{
"subscriptionId": "xxx",
"statusCode": {
"code": "404",
"reasonPhrase": "No context element found",
"details": "subscriptionId: /xxx/"
}
}
EDIT: The request URL is:
192.168.60.117:8080/STH/v1/contextEntities/type/room/id/Room1/
(In my case the type is "room" and not "Room"). The headers are Accept,
Content-Type, Fiware-Service, Fiware-ServicePath
It looks like more a problem when you make the subscription to ORION than when the data is processed by STH, you can find how to make the subscription in Orion and how to notify to STH in this link: https://fiware-sth-comet.readthedocs.io/en/latest/data-storage/index.html, Also, if you provide more information of how you make the subscription and the logs I could have a better view of where is the problem and I can help you in a better way.