AWS Step function string/json concatenation - json

I have orchestrated a data pipe line using AWS Step function.
In last state I want to send a custom notification. I'm using an Intrinsic function States.Format to format my message and subject. It works fine for Context object element. Here, I have tested that in Message parameter.
But it doesn't work with input JSON. This is my input JSON
{
"job-param":{
"pipe-line-name":"My pipe line name", "other-keys":"other values"
}
}
"Success State": {
"Type": "Task",
"Resource": "arn:aws:states:::sns:publish",
"Parameters": {
"Message.$": "States.Format('Execution Id:{}, completed successfully!', $$.Execution.Id)",
"Subject.$": "States.Format('[INFO] {} completed successfully!', $.job-param.pipe-line-name)",
"TopicArn": "arn:aws:sns:us-east-1:************:sns-topic"
},
"End": true
}
While saving this state machine, it gives me following error message:
The value for the field 'Subject.$' must be a valid JSON Path
I checked Input and Result path. They have this value. I can directly use this value as parameter. This is working fine. But I can't format with other string.
"Subject.$": "$.job-param.pipe-line-name"
Alternate approach would be to call lambda to customize and trigger SNS. But I want to avoid that.
Can I request some suggestions to fix this error?
Thanks in advance!

If you want to use any name with - in your JSON then you can write your JSON Path like this:
"Subject.$": "States.Format('[INFO] {} completed successfully!', $['job-param']['pipe-line-name'])",
But it would be easier if you change your input JSON and replace - with _:
"Subject.$": "States.Format('[INFO] {} completed successfully!', $.job_param.pipe_line_name)",

Related

Azure Logic App - Parse JSON with dynamic key/name

just want to know if and how I can parse a HTTP response with a dynamic name in a JSON?
I used the Azure Management API to receive the managed identities (system- and user assigned managed identities) to receive all managed identities.
With a foreach I am iterating the results.
If a resource has a system assigned managed identity and user assigned managed identity, the response looks like this:
{
"principalId": "<principalId1>",
"tenantId": "<tenantId>",
"type": "SystemAssigned, UserAssigned",
"userAssignedIdentities": {
"/subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<userAssignedIdentitiesName>": {
"principalId": "<principalId2>",
"clientId": "<clientId>"
}
}
}
Now, I would like to get the <principalId2>.
Unfortunately, the Name of the object is dynamic related to the scope of the resource /subscriptions/<subscriptionId>/resourcegroups/<resourceGroupName>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<userAssignedIdentitiesName>.
How can I parse the JSON to receive the needed <principalId2>?
For all other responses I can easily use the Data operations Parse JSON with the payload I inserted from the HTTP response.
Is there a way to use a wildcard? Otherwise, could I somehow just select the first object of userAssignedIdentities to receive the needed value?
Ok, this should work for you. This is the flow I tested with ...
Initialise JSON
Your JSON as a string, how you do that in your solution may differ slightly.
Initialize XPath Result
Defined as an Array and the expression is ...
xpath(xml(json(concat('{ root: ', replace(variables('JSON'), 'PrincipalId', 'principalId'), '}'))), '(//principalId)[2]')
Initialize Result
A bit more work again but defined as a String and the expression is ...
array(xpath(xml(base64ToString(variables('XPath Result')[0]?['$content'])), '//text()'))[0]
The end result should be your value ...

How to parse nested JSON, within a string, using Kusto

I have a Python Azure Function that produces custom logging messages when the Function executes. I need to pull out some of the JSON values nested in the logging message.
How can I use Kusto to access the nested JSON within the logging message string?
Example logging message:
Desired values marked with <----------
####### EventGrid trigger processing an event:
{
"id": "long-guid",
"data": {
"api": "FlushWithClose",
"requestId": "long-guid",
"eTag": "long-guid",
"contentType": "application/octet-stream",
"contentLength": 16264, <----------------------
"contentOffset": 0,
"blobType": "BlockBlob",
"blobUrl": "https://function.blob.core.windows.net/parentdir/childdir/file.name",
"url": "https://function.dfs.core.windows.net/parentdir/childdir/file.name", <---- JUST FILE.NAME here
"sequencer": "long-guid",
"identity": "long-guid",
"storageDiagnostics": {
"batchId": "long-guid"
}
},
"topic": "/subscriptions/long-guid/resourceGroups/resourceGroup/providers/Microsoft.Storage/storageAccounts/accountName",
"subject": "/blobServices/default/containers/containerName/blobs/childDir/file.name",
"event_type": "Microsoft.Storage.BlobCreated"
} #######
I imagine it has something to do with the Kusto extend function, but piping in...
| extend parsedMessage = todynamic(message)
| project timestamp, test = parsedMessage["id"]
...yields only an empty test column
message in your specific case isn't a valid JSON payload - as it has the ###... EventGrid trigger processing an event: prefix (and a somewhat similar suffix).
That is why todynamic() isn't able to process it and why you're not able to reference properties in the JSON payload that's included in it.
Ideally, you would change the payload you ingest to be a valid JSON payload, and re-type the target column to dynamic instead of string.
If you can't do that, you can use the substring() function or parse operator to get everything but the aforementioned prefix/suffix, and parse the output of that using todynamic()
though note that doing that each time you query the data bears runtime overhead that could be avoided by following the advice above.

How to get a json value dynamically?

in Jmeter, for the message below i want to get the value of tag CC2650HumidityData (or tag name and tag value) but I don't know the name of tag; it is contained in a variable.
I've used this json path:
$.message.inputData[*].${tagName}
$.message.inputData[*].vars.get("tagName")
but without success.
This is an example of message:
{
"message": {
"inputData": [
{
"CC2650HumidityData": "51",
"dateTime": "2020-12-18T08:35:21.342Z"
},
{
"CC2650BarometricPressureData": "21",
"dateTime": "2020-12-18T08:35:15.136Z"
}
]
}
}
Could you help me?
Your first option, to wit this one: $.message.inputData[*].${tagName} should work just fine
Just make sure that your tagName variable exists and has the anticipated value, it can be done using Debug Sampler and View Results Tree listener combination
Also be aware that you won't be able to use JSONPath Tester mode of the View Results Tree listener for testing JSONPath queries as it doesn't evaluate variables, you will need to run your test to see the variables values (or use this Dummy Sampler like I did)

How to perform validation of json files with cuelang?

I am evaluating cuelang.org to define data constraints, such as type or value checks on untrusted json payloads submitted to azure devops pipelines, before further processing.
When testing my definitions against invalid data I get no errors returned.
I want to have both type and value checks in place:
rules.cue:
#LoggingLinuxVM: {
resource_id: #NonEmptyString
version: =~"2020-07-21"
type: =~"Linux"
properties: #LoggingPropertiesLinuxVM
}
#LoggingPropertiesLinuxVM: {
CustomLogFiles: [...#NonEmptyString]
}
#NonEmptyString: string & !=""
invalid payload (missing type)
{
"resource_id": "/subscription/.../...",
"version": "2020-07-21",
"properties": {
"custom_log_files": [
"/var/log/dmesg",
"/var/log/messages"
]
}
}
when attempting to validate this with: cue vet payload.json rules.cue I expect to see validation errors, however none are raised and the exit status is 0 . Has anyone used cuelang in a similar fashion and can see where I am going wrong?
Turns out you have to provide the --schema flag to operate in the intended way. For this example, if the json were in payload.json the full command would be:
cue vet --schema '#LoggingLinuxVM' payload.json rules.cue

How to parse dynamic json reponse and get specific value and pass it as an input to next request

I get .json file as a response from an API and from that file I should parse and fins specific parameter and pass it as an input to the next request, how do I do that using Katalon.
If I say
response = JSON.parse("response.json");
it says it is unable to identify JSON as valid. Can someone help me out with the solution?
Your JSON is invalid, maybe it is a copy-paste issue.
The valid JSON should be
{
"responseStatusCode": "OK",
"data": {
"screenName": "employeeTimeslip",
"screenType": "Redirect",
"searchResultCount": 0,
"rows": [],
"tabs": [],
"searchParams": {
"employeeID": "000092926",
"timeslipNumber": "201900019701"
}
}
}
So, you were missing a "," between "OK" and "data" and two closing curly braces at the end of the file.
You can check JSON files for validity yourself using online JSON validators, for example, this one.
i found a way to read specific parameter from the json response file like below:
val scn = scenario("ClaimSubmission")
.exec(http("request_2")
.post("URL")
.headers(headers_2)
.body(RawFileBody("json file path"))
.check(jsonPath("$..timeslipnumber").find.saveAs("timeslipnumber")))
Timeslip number would be retrieved using : .check(jsonPath("$..timeslipnumber").find.saveAs("timeslipnumber")))