how to remove json element using json parse in logic app? - json

Input json -
{
"data": {
"Testresults": [
{
"mydata": {
"id": "111",
"uri": "url",
"type": "demo"
},
"name": "",
"address": "",
in side Parse Json schema added as below -
{
"properties": {
"data": {
"properties": {
"Testresults": {
"items": {
"properties": {
"name": {
"type": "string"
},
"address": {
"type": "string"
}
I have removed mydata from Parse Json schema still in the output getting same input json.
I want to remove mydata element from input json and pass it further , what should be done for that in parseJson?
expected output after Parse Json
{
"data": {
"Testresults": [
{
"name": "",
"address": "",

For your requirement, I post my steps below for your reference.
In my logic app, I use the json below as the input data:
{
"data": {
"Testresults": [
{
"mydata": {
"id": "111",
"uri": "url",
"type": "demo1"
},
"name": "Michael",
"address": "abc"
},
{
"mydata": {
"id": "222",
"uri": "url",
"type": "demo2"
},
"name": "Daniel",
"address": "def"
}
]
}
}
First I create a variable to store it.
Then, parse json.
After that, we can use liquid. We need to create an integration account and link to the logic app, then upload the liquid map shown as below:
{
"data": {
"Testresults": [
{% for testresult in content.data.Testresults %}
{
"name": "{{testresult.name}}",
"address": "{{testresult.address}}"
},
{% endfor %}
]
}
}
Now, we can use "Transform JSON to JSON" action to do convert it.
Run the logic app, I get the result we expect.(without "mydata")
Hope it would be helpful to your requirement~

Related

Merge Json Array Nodes and roll up a child element

I have a requirement to roll a collection of nodes that uses the current node name (within the collection) and for the value take each child nodes value (single node) into a string array, then use the parents key as the key.
Given.
{
"client": {
"addresses": [
{
"id": "27ef465ef60d2705",
"type": "RegisteredOfficeAddress"
},
{
"id": "b7affb035be3f984",
"type": "PlaceOfBusiness"
},
{
"id": "a8a3bef166141206",
"type": "EmailAddress"
}
],
"links": [
{
"id": "29a9de859e70799e",
"type": "Director",
"name": "Bob the Builder"
},
{
"id": "22493ad4c4fd8ac5",
"type": "Secretary",
"name": "Jennifer"
}
],
"Names": [
{
"id": "53977967eadfffcd",
"type": "EntityName",
"name": "Banjo"
}
]
}
}
from this the output needs to be
{
"client": {
"addresses": [
"RegisteredOfficeAddress",
"PlaceOfBusiness",
"EmailAddress"
],
"links": [
"Director",
"Secretary"
],
"Names": [
"EntityName"
]
}
}
What is the best way to achieve this? Any pointers to what/how to do this would be greatly appreciated.
Ron.
You can iterate over entries of your client object first with the help of the $each function, then get types for each of them, and combine via $merge:
{
"client": client
~> $each(function($list, $key) {{ $key: $list.type }})
~> $merge
}
Live playground: https://stedi.link/OpuRdE9

Converting nested JSON into CSV by adding the nested objects to s single column in Apache Nifi

I have a nested object JSON structure as given below;
{
"Bikes":[
{
"Name":"KTM",
"Model":"2017",
"Colour":"Yellow"
}
{
"Name":"Yamaha",
"Model":"2020",
"Colour":"Black"
}
],
"Cars":[
{
"Name":"BMW",
"Model":"2017",
"Colour":"Yellow"
}
{
"Name":"Audi",
"Model":"2020",
"Colour":"Black"
}
]
My output CSV should look this;
Bikes (Column 1)
{
"Name":"KTM",
"Model":"2017",
"Colour":"Yellow"
}
{
"Name":"Yamaha",
"Model":"2020",
"Colour":"Black"
}
Cars (Columns 2)
{
"Name":"BMW",
"Model":"2017",
"Colour":"Yellow"
}
{
"Name":"Audi",
"Model":"2020",
"Colour":"Black"
}
'''
I need to store the entire object Bikes in a single columns and likewise Cars in a single column
I am currently using a convert record processor to convert from JSON to CSV, MY avro schema for both JSON and CSV looks like this
{
"name": "Sydney",
"type": "record",
"namespace": "sydney",
"fields": [
{
"name": "Bikes",
"type": {
"type": "array",
"items": {
"name": "Vehicle",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "Model",
"type": "string"
},
{
"name": "Colour",
"type": "string"
}
]
}
}
},
{
"name": "Cars",
"type": {
"type": "array",
"items": {
"name": "Vehicle",
"type": "record",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "Model",
"type": "string"
},
{
"name": "Colour",
"type": "string"
}
]
}
}
}
]
}
but in the convert record processor I am getting this error
ConvertRecord[id=4d909c18-0177-1000-c1cd-9456a1775358] Failed to process StandardFlowFileRecord[uuid=63d04fc1-7edd-405b-8a9d-000bcdaa3d6c,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1611930760172-32, container=default, section=32], offset=744686, length=4],offset=0,name=test.json,size=4]; will route to failure: IOException thrown from ConvertRecord[id=4d909c18-0177-1000-c1cd-9456a1775358]: org.codehaus.jackson.JsonParseException: Unexpected character ('T' (code 84)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')
at [Source: org.apache.nifi.stream.io.NonCloseableInputStream#5b8393b6; line: 1, column: 2]
Could anyone help me out on this?
You probably want to use a jolt transform for this, which has its own processor. Restructuring a simple nested JSON like this is one of their documented examples.

Using a template variable that contains a JSON object with Postman Collection Runner

I am attempting to use the Postman collection runner to post several objects to an API from a JSON file structured something like this:
[
{
"id": "170",
"body": {
"name": "prod1",
"category": "category1"
},
"anotherProperty": "xyz"
},
{
"id": "171",
"body": {
"name": "prod3",
"category": "category1"
},
"anotherProperty": "dfg"
},
{
"id": "172",
"body": {
"name": "prod3",
"category": "category1"
},
"anotherProperty": "abc"
}
]
My problem seems to be with the body since it is an object:
Here is what I have in the Body > raw application/json of the request that my collection is using:
{
"$id": "{{id}}",
"body": {{body}},
"anotherProperty": "{{anotherProperty}}"
}
When viewing what it is plugging in it looks like:
{
"id": "170",
"body": {[object Object]}, // instead of the actual object
"anotherProperty": "xyz"
}
I needed to add the following to the Pre-request script:
let properties = pm.iterationData.get('properties');
pm.variables.set('properties', JSON.stringify(properties));
let body = pm.iterationData.get('body');
pm.variables.set('body', JSON.stringify(body));
And the raw JSON is plugged in with no problem!

avro runtime exception not a map when return in Json format

i have a avro schema for UKRecord, which contain a list of CMRecord(also avro schemaed):
{
"namespace": "com.uhdyi.hi.avro",
"type": "record",
"name": "UKRecord",
"fields": [
{
"name": "coupon",
"type": [
"null",
"string"
],
"default": null
},
{
"name": "cm",
"type": [
"null",
{
"type": "array",
"items": {
"type": "record",
"name": "CmRecord",
"fields": [
{
"name": "id",
"type": "string",
"default": ""
},
{
"name": "name",
"type": "string",
"default": ""
}
]
}
}
],
"default": null
}
]
}
in my java code, i create a UKRecord which has all fields populated correctly, eventually i need to return this object using a json based api, however it complained:
org.apache.avro.AvroRuntimeException: Not a map: {"type":"record","name":"CmRecord","namespace":"com.uhdyi.hi.avro","fields":[{"name":"id","type":"string","default":""},{"name":"name","type":"string","default":""}]}
the java code that write the object to json is :
ObjectWriter writer = ObjectMapper.writer();
if (obj != null) {
response.setHeader(HttpHeaders.Names.CONTENT_TYPE, "application/json; charset=UTF-8");
byte[] bytes = writer.writeValueAsBytes(obj); <-- failed here
...
}
obj is:
{"coupon": "c12345", "cm": [{"id": "1", "name": "name1"}, {"id": "2", "name": "name2"}]}
why do i get this error? please help!
Because you are using unions, Avro is uncertain how to interpret the JSON you are providing. Here's how you can change the JSON so Avro knows it's not null
{
"coupon": { "string": "c12345" },
"cm": { "array": [
{ "id": "1", "name": "name1" },
{ "id": "2", "name": "name2" }
]
}
}
I know, it's really annoying how Avro chose to handle nulls.

Swagger POST with json body

I am trying to write static .json file of server response using swagger. I'm stuck with post body and do not know how to describe it. It looks pretty much similar to Grooveshark api's where you have one page and different post parameters.
So, given grooveshark example (http://developers.grooveshark.com/docs/public_api/v3/)
Page that takes queries:
http://api.grooveshark.com/ws3.php?sig=cd3ccc949251e0ece014d620bbf306e7
POST body:
{
'method': 'addUserFavoriteSong',
'parameters': {'songID': 0},
'header': {
'wsKey': 'key',
'sessionID': 'sessionID'
}
}
How can I describe this with swagger?
without knowing a ton about how this API operates (such as, is "songID" the only parameter type?, I'm guessing you'd want something like this in your models section:
"models": {
"FavoriteSong": {
"id": "FavoriteSong",
"properties": {
"parameters": {
"type": "Parameter"
},
"header": {
"type": "Header"
}
}
},
"Parameter": {
"id": "Parameter",
"properties": {
"songID": {
"type": "integer",
"format": "int32"
}
}
}
"Header": {
"id": "Header",
"properties": {
"wsKey": {
"type": "string"
},
"sessionID": {
"type": "string"
}
}
}
}
}
And the operation would take the type "FavoriteSong" as a body type:
"parameters": [
{
"name": "body",
"description": "object to add",
"required": true,
"type": "FavoriteSong",
"paramType": "body"
}
]