Bonjour!
Is there a function for removing/replacing all of a specific character in a JSON in Azure Logic Apps?
I am working with a HTTP request in Azure Logic Apps. The results I would like to see is a unnested JSON as a CSV-file.
My steps looks like this:
1.HTTP GET ->
2.Parse JSON ->
3.CSV-file
The parsed JSON contains alot of nested values which I would like to transform to columns instead.
My idea is to delete the "{" and "}" characters from the JSON before handling it as a CSV-file which I hope will result in success.
A small example of the structure:
"data": [
"consumption_stats": {
"energy": {
"hour": {
"count": 27745,
So my question is: how can I, in Azure Logic Apps, remove/replace these characters before I make it to an CSV-file?
Thank you
There is a replace function built into Logic Apps. You could simply replace the characters with an empty string?
Click "Add Dynamic Content" then go to the expression tree and type "Replace".
Related
I tried to design a test script of API Load Testing in JMeter 5.0. Here is my thread group info:
Below is the Body Data, I already added some variables and functions to it. You can also view the sample JSON Body Data at here:
{
"app_name": "Telekom",
"source": "MOENGAGE",
"moe_request_id": "req_${__threadNum}",
"events": [
{
"event_name": "Email Opened",
"event_code": "MOE_EMAIL_OPEN",
"event_uuid": "${__UUID}",
"event_time": ${__jexl3(${__time(,)}/1000,)},
"event_type": "CAMPAIGN_EVENT",
"event_source": "MOENGAGE",
"uid": "${__counter(,)}-${__RandomString(7,ABCDEFG123456)}${uid}",
"email_id": "${email}",
"event_attributes": {
"campaign_id": "${__RandomString(24,abcdefghijklmn123456789)}",
"campaign_name": "July2021_16072021_LigaSuper_English",
"campaign_type": "GENERAL",
"campaign_channel": "EMAIL",
"moe_delivery_type": "One Time",
"moe_campaign_tags": [
"engagement"
],
"moe_campaign_channel": "Email",
"u_em": "${email}"
},
"user_attributes": {
"PRODUCT_TYPE": "Unifi",
"moengage_user_id": "60dc48d4d722040a2e78b788",
"SERVICE_NO": "${uid}",
"id": "${email}"
},
"device_attributes": {}
}
]
}
I also include JSON Extractor to do correlation on the response body:
Here is the results in View Results Tree. Added Debug Sampler to see the whether my parameterization and correlation working or not. Fortunately, only my parameterization is working, while correlation is not.
Updated: This is example of response body:
I want to correlate at "rqUuid": "urn:uuid:1a5d8617-258c-49f7-b1a7-5b8ee71fb9fd" in the response body. Question is, is this the correct way to correlate the JSON Body? Do I need to correlate the response body although I already define the function "event_uuid": "${__UUID}" in the Body Data?
Thanks. Appreciate your help.
You are using a regular expression to get your data from a JSON response using the JSON Extractor. You need to use JSON Path Expressions instead of using regular expressions.
If you want to extract event_uuid from the response, your json path expressions would be events[*].event_uuid
To get the first event_uuid of your events array objects, you can use
events[0].event_uuid
JSON Extractor allows you executing JsonPath queries and it looks like you're trying to use a regular expression there, it will result into a syntax error, you can see jmeter.log file for details:
You're showing us request data and asking about extracting values from the response, in order to be able to help we need to see at least partial (or better full) response data and what part of it do you need.
So far I can only give the following piece of advice: switch to Regular Expression Extractor and there is a chance it will start working without any changes
You can use JSON Extractor or JSON JMESPath Extractor for extracting values from JSON responses.
Your JSON path expression should be responseHeader.rqUuid
You can evaluate the JSON Path expressions or JMESPath expression through the View Result tree or with online tools.
View Result Tree
You can generate the expressions online with http://jsonselector.com/
JMSE Path Evaluator https://jmespath.org
Sample JMX is uploaded to GitHub for your reference https://github.com/pragmatictesters/Pragmatic-Learning-JMeter-Examples/blob/master/TestPlan-StackOverFlow-68706730-JSON.jmx
I'm using a Microsoft REST API to query a Azure application, oauth and request goes without problem.
The response from InvokeHTTP has this format
{"#odata.context":"https://****.dynamics.com/api/data/v9.1/$metadata#endpoint","value":[ here comes the actual JSON result in format {
"#odata_etag" : "W/\"555598\"", "field":"value...},...]
,"#odata.nextLink":"https://****.dynamics.com/api/data/v9.1/endpoint?$skiptoken.....}
I need to extract the nextLink for pagination and Value to continue the flow and store the result.
When I try to parse with inferAvroSchema so I can start working with it throws this error "Illegal initial character: #odata.etag"
My Idea was to inferAvroSchema, then EvaluateJsonPath to extract the odata tags and then extract the values.
I tried using EvaluateJsonPath on the result asking to create an attribute for $.#odata.context but it doesn't find the item either, I'm sure is something about the #.
I can also replace all the # of the incoming flow for another char, but don't know if that makes sense.
I'm feeling that i'm not using a correct approach, but NIFI + odata doesn't give me results on google or here.
I'm open to any suggestions!
thank you!
Schema fields cannot contain #. You could replace the #, however you must be sure not to replace it in actual content like email addresses. Another solution is to transform the API response using JoltTransformJSON processor, such that your flow can work with it:
GenerateFlowFile:
For the JoltTransformJSON processor provide following Jolt specification:
[
{
"operation": "shift",
"spec": {
"\\#odata.nextLink": "next"
}
}
]
Leave the default values for the other properties. You can play around with Jolt here: http://jolt-demo.appspot.com/
EvaluateJsonPath:
Result:
Notice that the url is now part of the flowfile attributes.
Your hunch is correct, you can only have valid characters for the field names on the schema type you are using, avro or JSON.
You could get NiFi to remove illegal characters with the replacetext proceasor, have a read here on what is valid: http://avro.apache.org/docs/current/spec.html#names
i am getting a json data after hitting an API .
when i try to load that json into python using json.loads(response.text), I am getting a delimiter error .
when checked few fields in json dose not have "," separating them.
{
"id":"142379",
"label":"1182_Mailer_PROD",
"location":"Bangalore, India",
"targetType":"HTTPS performance",
"frequency":"15",
"fails":"2764",
"totalUptime":"85.32"
"tests":[
{"date":"09-24-2019 09:31","status":"Could not resolve: mailer.accenture.com (DNS server returned answer with no data)","responseTime":"0.000","dnsTime":"0.000","connectTime":"0.000","redirectTime":"0.000","firstbyteTime":"0.000","lastbyteTime":"0.000","pingLoss":"0.00","pingMin":"0.000","pingAvg":"0.000","pingMax":"0.000","size":"0","md5hash":"(null)"}
]
}
,
{
"id":"158651",
"label":"11883-GR dd-WSP",
"location":"Chicago, IL",
"targetType":"Performance transaction",
"frequency":"15",
"fails":"5919",
"totalUptime":"35.14"
,"tests":[
{"date":"09-24-2019 09:26","status":"Keywords not found - Working","responseTime":"0.669","stepresults":[
{"stepid":"1","date":"09-24-2019 09:26","status":"OK","responseTime":"0.453","dnsTime":"0.000","connectTime":"0.025","redirectTime":"0.264","firstbyteTime":"0.141","lastbyteTime":"0.024","size":"22351","md5hash":"ca002cf662980511a9faa88286f2ee96"},
{"stepid":"2","date":"09-24-2019 09:26","status":"Keywords not found - Working","responseTime":"0.216","dnsTime":"0.000","connectTime":"0.023","redirectTime":"0.000","firstbyteTime":"0.171","lastbyteTime":"0.022","size":"22457","md5hash":"38327404e4f2392979aa7dfa27118f4e"}
]}]
}
This is a small chunk of data from the response , as you could see "totalUptime":"85.32" doesn't have a comma separating it.
could you please let me know how can we load the data into python object even though the json is deformed
Deformed JSON is not JSON, so obviously you can't load it with a standard procedure. There are only two possibilities to load it:
Create your own parser
Modify the input to conform to the JSON standard
Both possibilities need you to define what format do you want to import. If it is OK for your format not to have commas then you have to define what your delimiters are.
From the example you posted is difficult to make any definitive assessment about how the input format is defined. So you probably will have to write a rudimentary parser and approximate it by try and error to the input you are trying to parse.
In JMeter, I need to extract some fields (City, Classification, and Chain) from a JSON response:
{
"StoreCode": "111243",
"StoreName": "Spencer - Sec 14 Gurgaon",
"Address1": "Gurgaon-Sector-14",
"Address2": "NCR",
"Pin": "110000",
"City": "NCR",
"Classification": "Vol 4",
"Chain": "Spencers",
"Version": "20281",
"VisitType": "Weekly"
}
Can it be done using the regular expression extractor? Is there another option?
If this piece of JSON is the all the response - it makes sense to use Regular Expression Extractor.
If you receive larger or more complicated structure - it's better to use special JSON Path Extractor available through plugin. JSON Path expressions for your JSON response would be something like $.City, $.Chain, etc.
See "Parsing JSON" chapter of Using the XPath Extractor in JMeter guide for more details on JSON Path language and on how to install the plugin.
Very easy with the plugin mentioned. See this for example. Here is link to plugin.
My biggest thing to understand was the flow. In your jmeter test you need to have an httprequest that returns data (in this case json data). So running your test you'd see json in the Response Data tab if you have a View Results Tree listener going. If you have this right click on the HttpRequest you want data from. ADD => Post Processors => jp#gc - JSON Path Extractor. Inside that extractor, you can name it anything you want.
The variable name should be one you already have defined in a User Defined Variables config element. The JSON Path would start with a dollar sign, then a dot, then the name of the key you want the value for from your json. So for me: $.logId the value from ... "logId": 4, ... in my json. It will store the number 4 in my userdefined variable. The default value can be set to something you'd never see like -1 or null or false etc...
BTW you reference your variable in your tests with ${variablename}. If putting into json and its a string do "${variablename}". Hope it helps.
Lots of the way to find out with the help of regular expression. Sharing one of them.
"City": "(.*)",
"Classification": "(.*)",
"Chain": "(.*)",
This could be the most basic json question ever. I'm creating a WCF REST service and have a HelloWorld test function that just returns a string. I'm testing the service in fiddler and the reponse body I'm getting back is:
"HelloWorld"
I also created a function that would just return a number (double) and the response body is:
1.0
Are those valid json responses? Are simple return types just returned in plain text like that (no markup with the brackets)?
Valid JSON responses start with either a { for an object or [ for a list of objects.
Native types are not valid JSON if not encapsulated. Try JSONlint to check validity.
RFC 4672, says no. Which doesn't mean it can't work, but it isn't strictly standards compliant. (Of course, neither are all the JSON readers...)
To quote from Section 2, "JSON Grammar":
A JSON text is a sequence of tokens. The set of tokens includes six
structural characters, strings, numbers, and three literal names.
A JSON text is a serialized object or array.
JSON-text = object / array
Only objects / maps, and arrays, at the top level.
According to the official website, you need to use a syntax like this:
You need to declare what you want beetween {}, like this:
{
"test": "HelloWorld"
}
No. The following is, for example:
{
"Foo": "HelloWorld"
}
You can try JSONLint to see what validates and what does not.