I am trying to create a test case result using a REST client, but get this error:
"Errors": ["Cannot parse input stream due to I/O error as JSON document: Parse error: expected '{' but saw '\uFFFF' [ chars read = >>>\uFFFF<<< ]"]
I get the same error when the name of the object, testcaseresult is not specified in the request body. Here are the steps to create a test case result using a browser REST client:
a) Generate the authorize key using "GET" method and the following URL:
https://rally1.rallydev.com/slm/webservice/v2.0/security/authorize
This is the response that I get back, with the security token: "123abc..."
{"OperationResult": {"_rallyAPIMajor": "2", "_rallyAPIMinor": "0", "Errors": [], "Warnings": [], "SecurityToken": "abc123..."}}
b) Use "POST" method, and the following URL:
https://rally1.rallydev.com/slm/webservice/v2.0/testcaseresult/create?key=abc123...
notice the security token in the end.
c) here is an example of a request body:
{
"testcaseresult":
{
"Build":"1",
"Tester":"/user/777",
"Date":"2010-09-04T19:56:05.000Z",
"TestCase":"/testcase/1111",
"Verdict":"Pass"
}
}
Only the required fields and the fields you want to set need to be referenced. Notice the outer key/value pair
{
"testcaseresult":{}
}
The fields that point to a full object, like "Tester" (points to User object) and "TestCase" (points to a TestCase object that owns the result) have to be referenced by their ObjectIDs:
"Tester":"/user/777",
"TestCase":"/testcase/1111",
Related
I have a Python Azure Function that produces custom logging messages when the Function executes. I need to pull out some of the JSON values nested in the logging message.
How can I use Kusto to access the nested JSON within the logging message string?
Example logging message:
Desired values marked with <----------
####### EventGrid trigger processing an event:
{
"id": "long-guid",
"data": {
"api": "FlushWithClose",
"requestId": "long-guid",
"eTag": "long-guid",
"contentType": "application/octet-stream",
"contentLength": 16264, <----------------------
"contentOffset": 0,
"blobType": "BlockBlob",
"blobUrl": "https://function.blob.core.windows.net/parentdir/childdir/file.name",
"url": "https://function.dfs.core.windows.net/parentdir/childdir/file.name", <---- JUST FILE.NAME here
"sequencer": "long-guid",
"identity": "long-guid",
"storageDiagnostics": {
"batchId": "long-guid"
}
},
"topic": "/subscriptions/long-guid/resourceGroups/resourceGroup/providers/Microsoft.Storage/storageAccounts/accountName",
"subject": "/blobServices/default/containers/containerName/blobs/childDir/file.name",
"event_type": "Microsoft.Storage.BlobCreated"
} #######
I imagine it has something to do with the Kusto extend function, but piping in...
| extend parsedMessage = todynamic(message)
| project timestamp, test = parsedMessage["id"]
...yields only an empty test column
message in your specific case isn't a valid JSON payload - as it has the ###... EventGrid trigger processing an event: prefix (and a somewhat similar suffix).
That is why todynamic() isn't able to process it and why you're not able to reference properties in the JSON payload that's included in it.
Ideally, you would change the payload you ingest to be a valid JSON payload, and re-type the target column to dynamic instead of string.
If you can't do that, you can use the substring() function or parse operator to get everything but the aforementioned prefix/suffix, and parse the output of that using todynamic()
though note that doing that each time you query the data bears runtime overhead that could be avoided by following the advice above.
I am trying to read output from my lambda function into a variable in my step function. The lambdas default output is
return {
'statusCode': 200,
'body': json.dumps('Hello from Lambda!')
}
What I would like to return is a json object like this
{
"version": version,
"bucket": bucket
}
where the version and bucket name gets passed from the lambda. in my step function, I am trying to capture this and insert it into an s3 url like this:
"S3Uri.$": "States.Format('s3://{}/path/to/script/{}/script.py',$.bucket, $.version)"
However, I am struggling with having the correct output from the lambda and how to grab the value in step functions. I've tried
return {
'statusCode': 200,
'body': json.dumps({
"version": version,
"bucket": bucket
}) }
And various ways of constructing the json object as a string to the body, eg
"{\"version\": \"" + version + "\",\"bucket\": \"" + bucket + "\"}"
But I can't find the right combo, and the job keeps failing. Here is an example error message
The JsonPath argument for the field '$.bucket' could not be found in
the input '{"statusCode": 200, "body": "{\"version\":
\"v0-1\", \"bucket\":
\"sagemaker-us-west-2-removed\"}"}'"
How should I construct the lambda output, and the corresponding step function variable to have the values pass through? Again, I want the lambda to tell step functions what bucket and version was used, and then have step functions insert these values into an s3 url string.
EDIT: here is the full error message for one of the attempts
{
"error": "States.Runtime",
"cause": "An error occurred while executing the state 'Postproc' (entered at the event id #38). The function 'States.Format('s3://{}/AAPM/AAPM_2207/prod/{}/meta/scripts/preproc/aapm-postproc.py',$.bucket, $.version)' had the following error: The JsonPath argument for the field '$.bucket' could not be found in the input '{\"statusCode\": 200, \"body\": \"{\\\"version\\\": \\\"v0-1\\\", \\\"bucket\\\": \\\"sagemaker-us-west-2-removed\\\"}\"}'"
}
As in this answer the trick was just to get rid of json dumps.
return {
'statusCode': 200,
'body': {
"version": version,
"bucket": bucket
}
}
and I could access them fine with $.bucket, $.version
Very short version
How do I include an ADF Variable inside a JSON POST request, in a Web Activity within ADF?
I feel like this should be a very simple string concatenation, but i can't get it to work
Detail
We have a requirement to run a query / SProc from within ADF, which will return a string containing an error message. That string is to then be passed via the Web Activity in ADF to a Logic App, in order to fire off an email, containing the error.
The setup of the logic app is copied from here:
https://www.mssqltips.com/sqlservertip/5718/azure-data-factory-pipeline-email-notification--part-1/
and then here (part 2)
https://www.mssqltips.com/sqlservertip/5962/send-notifications-from-an-azure-data-factory-pipeline--part-2/
In ADF, I used the Lookup activity, to run a query, which brings back the error (appears to work, the preview returns the correct string)
Then I use the Set Variable activity, to take the output of the lookup and store it in a variable.
Last Step is to fire off the POST using the Web Activity.
With this code (tweaked slightly to remove personal details) in my Web Activity, everything works fine and I receive an email
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "Pipeline finished!",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
But any attempt to put the contents of the Variable into the Subject part has failed.
This (for example) sends me an email with the subject literally being #variables('EmailSubject')
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#variables('EmailSubject')",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
But I've also attempted various other solutions that result in errors or the email subject just containing the literal thing that I put in there (e.g. + #variables('EmailSubject') +).
I also tried storing the entire JSON in the Variable, and then having the Web activity use only the variable, that returned no errors, but also did not send an email.
This attempt:
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#{variables('EmailSubject')}",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
Resulted in this input into the web activity - which actually includes the text of the error, which is a bonus ... (text = Job Duration Warning):
{
"url": "https://azureLogicAppsSiteHere",
"method": "POST",
"headers": {
"Content-Type": "application/json"
},
"body": "{\n \"DataFactoryName\": \"DFNAMEHERE\",\n \"PipelineName\": \"pipeline1\",\n \"Subject\": \"{\"firstRow\":{\"\":\"Job Duration Warning\"},\"effectiveIntegrationRuntime\":\"DefaultIntegrationRuntime (West Europe)\",\"billingReference\":{\"activityType\":\"PipelineActivity\",\"billableDuration\":[{\"meterType\":\"AzureIR\",\"duration\":0.016666666666666666,\"unit\":\"DIUHours\"}]},\"durationInQueue\":{\"integrationRuntimeQueue\":0}}\",\n \"ErrorMessage\": \"Everything is okey-dokey!\",\n \"EmailTo\": \"me#myEmail.com\"\n}\t"
}
But then resulted in this error:
{
"errorCode": "2108",
"message": "{\"error\":{\"code\":\"InvalidRequestContent\",\"message\":\"The request content is not valid and could not be deserialized: 'After parsing a value an unexpected character was encountered: f. Path 'Subject', line 4, position 17.'.\"}}",
"failureType": "UserError",
"target": "Web1",
"details": []
}
[Edit] The PREVIEW from the Lookup Activity is the text: Job Duration Warning BUT when I debug the pipeline, it lets me see the actual Output, which is this:
{
"count": 1,
"value": [
{
"": "Job Duration Warning"
}
],
"effectiveIntegrationRuntime": "DefaultIntegrationRuntime (West Europe)",
"billingReference": {
"activityType": "PipelineActivity",
"billableDuration": [
{
"meterType": "AzureIR",
"duration": 0.016666666666666666,
"unit": "DIUHours"
}
]
},
"durationInQueue": {
"integrationRuntimeQueue": 0
}
}
So it appears that the problem is that the Lookup Output isn't what I thought it was, so the variable can't be used in the Web Activity, as it contains unsupported characters or something along those lines.
I just tested this and it worked ok:
Create a String Parameter with the value Job Duration Warning
Set the Variable value to be #pipeline().parameters.ParamSubject
Include the variable in the web activity with an # in front of it
I then receive my expected email with the right subject. I just don't know how to get the string output of my query, into a variable / parameter, so that i can use it in the web activity.
I don't know how well this applies to other people's issues, but I found a solution that has worked for me.
In the SELECT query within the Lookup Activity - name the output (in my case, I called that column 'Subject'- i.e. SELECT xyz AS Subject
In the Lookup Activity, turn on the setting 'First Row Only'
In the Set Variable Activity, use the code: #activity('Lookup1').output.firstRow.subject
(where 'Lookup1' is the name of your Lookup Activity and Subject is the name of the column you are outputting)
In the Web Activity, reference the variable as follows:
{
"DataFactoryName": "#{pipeline().DataFactory}",
"PipelineName": "#{pipeline().Pipeline}",
"Subject": "#{variables('EmailSubject')}",
"ErrorMessage": "Everything is okey-dokey!",
"EmailTo": "me#myEmail.com"
}
I have been trying for a few days to get a parameter sent from the API Gateway in AWS to a Lambda function and I am having no success.
I decided to start from the beginning so I followed their walkthrough (http://docs.aws.amazon.com/apigateway/latest/developerguide/getting-started.html#getting-started-new-lambda)
I have checked this walkthrough twice and I have followed the steps to the letter.
Problem
When I test the API from Postman or in Swift I am getting the error:
{"message": "Could not parse request body into json: Unexpected character (\'-\' (code 45)) in numeric value: expected digit (0-9) to follow minus sign, for valid numeric value\n at [Source: [B#c036d15; line: 1, column: 3]"}
In postman, When I view the result as JSON I just get
Bad String
Lambda Function
The function is the basic example from the Walkthrough:
console.log('Loading event');
exports.handler = function(event, context) {
var name = (event.name === undefined ? 'No-Name' : event.name);
console.log('"Hello":"' + name + '"');
context.done(null, {"Hello":name}); // SUCCESS with message
};
When Tested from the Lambda Console and with the Test data I get the result:
{
"Hello": "TestUser123"
}
When Tested from the API Gateway Test, The result is also:
{
"Hello": "TestUser123"
}
Can anyone see why both test consoles are allowing this work but when tested with POSTMAN or used within a Swift Script it does not work ?
Edit 1
In postman, I have set the content-type to application/json
The script returns the default:
{
"Hello": "user"
}
However, When I add in the parameters name and TestUser123 in POSTMAN, this is when it returns the error.
Update 1
Ok, so I changed the mapping template to one that I found on another answer:
{ "name": "$input.params('name')" }
Now the result is:
{
"Hello": ""
}
Any Ideas why it is not getting the name?
I just got stuck with this today.
your mapping template is:
{ "name": "$input.params('name')" }
AWS uses AWS Velocity templates, which; even though looks like JSON, is different.
if you use
{ "name": $input.params('name') } // notice no quotes
for the mapping template right at the integration request step, then it should work as expected.
Read the error message very carefully, it actually tells you the problem. For example, I got
Could not parse request body into json: Unexpected character (\'\"\' (code 34)): was expecting comma to separate Object entries
So the problem is that I'm missing a comma. I check my Lambda's Integration Request - Body Mapping Template:
{
"age" : $input.json('$.persondata.age'),
"income" : $input.json('$.persondata.income')
"height" : $input.json('$.persondata.height')
}
Can you spot the problem? I am missing a comma after the income line.
Here is another example.
Could not parse request body into json: Unexpected character (\'}\' (code 125)): expected a value
When I look at the Integration Request - Body Mapping Template:
#set($inputRoot = $input.path('$'))
{
"age" : $inputRoot.age,
"height" : $inputRoot.height,
"income" : $inputRootincome
}
Can you spot the problem? I am missing a dot in $inputRootincome.
Error Message :
Could not parse request body into json: Could not parse payload into json: Unrecognized token \' \': was expecting (\'true\', \'false\' or \'null\')
Cause of the error : When string values inside the json are not assigned using double quotations in the aws mapping template the error occurs.
Solution : (Please Note : This example is for application/json type request template)
Actually the solution for the problem is, if you are using a value of type string in json then its value should be assigned inside a ("" - double quotation marks) in the mapping template.
The below shown example has the following attributes :
customerId - string (Please note : this value comes from a query parameter)
customerName - string
customerAge - integer
isPermanentEmployee - boolean
customerAddress - string (Please note this is an optional parameter)
And the mapping template should be defined like the example shown below
Refer the example below :
#set($inputRoot = $input.path('$'))
{
"CustomerId": "$input.params('customerId')",
"CustomerName": "$inputRoot.customerName",
"CustomerAge": $inputRoot.customerAge,
"IsPermanentEmployee": $inputRoot.isPermanentEmployee
"CustomerAddress ": #if($inputRoot.customerAddress == "") "" #elseif($inputRoot.customerAddress != "") "$inputRoot.customerAddress" #end
}
If you note the above mapping template, I would have given string values inside double quotation marks("") which will solve the error
Also this example contains how to handle optional parameters in aws mapping templates using #if#else statements.
It is likely that you had copy-pasted multiple lines in your "Integration Request" in the API gateway.
When copying a line and pasting it below, you might have copied the hidden character '\n' at the end of that line. This is probably causing issues at the lambda function.
Example: Copying the line containing age and pasting it twice and modifying them to have height and income respectively.
#set($inputRoot = $input.path('$'))
{
"age" : $inputRoot.age,
"height": $inputRoot.height,
"income": $inputRoot.income
}
Instead of copy-pasting, just type the line out for height and income.
I'm trying to use the hbase rest API. I've read the docs here (http://wiki.apache.org/hadoop/Hbase/Stargate), but they seem to be incomplete (doesn't cover json) and out of date.
I've also run across this gist, which is mostly what I'm going off of.
Here is the (insert) request I'm trying to send:
URI: http://server:8070/table/row1/data
Headers:
Content-Type : application/json
Accept : application/json
HTTP verb - POST
and the json payload (where key, column, and $ are all base64 encoded values):
{
"Row": {
"key": "NjQ=",
"Cell": [{
"column": "NjQ=",
"$": "NjQ="
}]
}
}
Here is the error I get:
Error 500 Can not deserialize instance of java.util.List out of START_OBJECT token
at [Source: org.mortbay.jetty.HttpParser$Input#3da0b822; line: 2, column: 5]
(through reference chain: org.apache.hadoop.hbase.rest.model.CellSetModel["Row"])
The error seems to suggest that there is something that should be an array and isn't. I've tried putting square brackets just about everywhere, and I can't get it to do anything other than change the nature of the error message slightly.
I had a look through the source code and that seems to suggest that the placement for the square brackets that I have is correct. However, as far as I can tell the entire request looks to be correct. I'm not very fluent in Java though, so maybe I'm missing something.
What is the proper JSON syntax for inserting a record with the hbase rest gateway?
This is now the correct format:
{
"Row": [{
"key": "NjQ=",
"Cell": [{
"column": "NjQ=",
"$": "NjQ="
}]
}]
}
Where:
key - is the row key
column - is the column name
$ - is the data that you are storing in the given table, column, and row.