I want to split and transfer the json data in NiFi, Here is my json structure look like this;
I want to split json by id1,id2 array of json transfer to respective processor group say example processor_group a,b. I tried with evaluate json path $.id1,$.id2 but i didn't get exact solution. Can you please help me out from this issue;
{
"id1": [{
"u_name": "aa"
}, {
"addr": "bb"
}],
"id2": [{
"u_name": "aa"
}, {
"addr": "bb"
}]
}
The processor you're looking for is SplitJSON.
Configure it as follows:
Then, you'll receive two FlowFiles:
First one will contain the id1:
[{
"u_name": "aa"
}, {
"addr": "bb"
}]
second one will contain id2:
[{
"u_name": "aa"
}, {
"addr": "bb"
}]
Here is how to get to the values you want with EvaluateJsonPath:
#varun_rathinam Accessing json in an array object via EvaluateJsonPath can be quite confusing. I also notice the structure of your json is kind of confusing with same values in both. I have adjusted id2 for cc and dd for testing so that I can tell id1 and id2 values apart.
The solution you want is (see template for exact string values):
Notice we use the normal tree for each json object ( $.object ) then access the array ( 0, 1 ) then access the array's objects. Also notice it is possible to access the json object array with or without a . before the [.
Reference:
https://community.cloudera.com/t5/Support-Questions/how-to-extract-fields-in-flow-file-which-are-surrounded-by/m-p/208635
You can also find my template during testing of your issue on my GitHub:
https://github.com/steven-dfheinz/NiFi-Templates/blob/master/NiFI_EvaluateJsonPath_Demo.xml
Related
I've got json file with some nested objects inside subs in it:
{
"version": 1,
"data": [{
"married": true,
"subs":[
{
"name":{
"sub1":{}
**},**
},
]
},
]}
If I add another 'name' object (with comma as separator), jsonDecode returns nothing.
if there goes single object, without comma - it's ok.
My Json structure is correct, and it's not restricted to use nested objects at all. Please anyone help.
This line has problem **},** and if changed to }, the problem will be solved..
you can check your json by online tools (e.g https://jsonformatter.curiousconcept.com/#)
I'm trying to capture multiple values from a JSON Response I get but seem to be unable to get them from the same random node.
I've tried to place multiple variables in the same extractor using ";" and this works but it goes through the nodes randomly and doesn't extract the values I need from the same one
The source would be something like
[
{
"Disabled": false,
"Group": null,
"Selected": false,
"Text": "Text1",
"Value": "Value1"
},
{
"Disabled": false,
"Group": null,
"Selected": false,
"Text": "Text2",
"Value": "Value2"
}
]
and I would like to get from any of the 2 nodes (randomly matched) both the Text and Value in either a array that I can use or 2 variables.
So far it seems to take value from one node and text from another (in lengthier sources)
so my desired outcome would be either text1 and value1 or text2 and value2, not a mix of both..
Add JSR223 PostProcessor as a child of the request which returns the above JSON
Put the following code into "Script" area:
def json = new groovy.json.JsonSlurper().parse(prev.getResponseData())
def randomEntry = json.get(org.apache.commons.lang3.RandomUtils.nextInt(0, json.size()))
vars.put('strBrandID', randomEntry.Value)
vars.put('strBrandName', randomEntry.Text)
That's it, you should be able to refer the Text/Value pairs as ${strBrandID} and ${strBrandName} where required
More information:
Groovy: Parsing and producing JSON
Apache Groovy - Why and How You Should Use It
I am creating with the d3 tree layout a tree. My data is as already as a tree but not with the d3js format ( {name: "", "childrend": []} ) but with a simple JSON tree format like :
[{
"A": [{
"AA": []
}, {
"AB": []
}, {
"B": [{
"BA": []
}, {
"BB": []
}]
}]
}]
Of course, the data is not with "A" and "B", is just for making the JSON more clear and give just a part of my data. (My data not following a pattern as the exemple)
I saw i could use tree.children() to change the name, but how can i dynamically do it ?!
I need to use this tree format with d3 tree layout.
So since you can write an accessor function, you can make it smarter than just returning a single property.
The function can be made to check each object key and return it if the corresponding value contains children.
I have this JSON that is returned from a REST-service I'm using.
{
"id": "6804",
"signatories": [
{
"id": "12125",
"fields": [
{
"type": "standard",
"name": "fstname",
"value": "John"
},
{
"type": "standard",
"name": "sndname",
"value": "Doe"
},
{
"type": "standard",
"name": "email",
"value": "john.doe#somwhere.com"
},
{
"type": "standard",
"name": "sigco",
"value": "Company"
}
]
}
]
}
Currently I'm looking into a way to parse this with json4s, iterating over the "fields" array, to be able to change the property "value" of the different objects in there. So far I've tried a few json libs and ended up with json4s.
Json4s allows me to parse the json into a JObject, which I can try extract the "fields" array
from.
import org.json4s._
import org.json4s.native.JsonMethods._
// parse to JObject
val data = parse(json)
// extract the fields into a map
val fields = data \ "signatories" \ "fields"
// parse back to JSON
println(compact(render(fields)))
I've managed to extract a Map like this, and rendered it back to JSON again. What I can't figure out though is, how to loop through these fields and change the property "value" in them?
I've read the json4s documentation but I'm very new to both Scala and it's syntax so I'm having a difficult time.
The question becomes, how do I iterate over a parsed JSON result, to change the property "value"?
Here's the flow I want to achieve.
Parse JSON into iterable object
Loop through and look for certain "names" and change their value, for example fstname, from John to some other name.
Parse it back to JSON, so I can send the new JSON with the updated values back.
I don't know if this is the best way to do this at all, I'd really appreciate input, maybe there's an easier way to do this.
Thanks in advance,
Best regards,
Stefan Konno
You can convert the json into an array of case class which is the easiest thing to do. For example: you can have case class for Fields like
case class Field(`type`: String, name: String, value: String)
and you can convert your json into array of fields like read[Array[Field]](json) where json is
[
{
"type": "standard",
"name": "fstname",
"value": "John"
},
...
]
which will give you an array of fields. Similarly, you can model for your entire Json.
As now you have an array of case classes, its pretty simple to iterate the objects and change the value using case classes copy method.
After that, to convert the array of objects into Json, you can simply use write(objects) (read and write functions of Json4s are available in org.json4s.native.Serialization package.
Update
To do it without converting it into case class, you can use transformField function
parse(json).transformField{case JField(x, v) if x == "value" && v == JString("Company")=> JField("value1",JString("Company1"))}
Hello I have a JSON in the following format.I need to parse this in the map function to get the gender information of all the records.
[
{
"SeasonTicket" : false,
"name" : "Vinson Foreman",
"gender" : "male",
"age" : 50,
"email" : "vinsonforeman#cyclonica.com",
"annualSalary" : "$98,501.00",
"id" : 0
},
{
"SeasonTicket": true,
"name": "Genevieve Compton",
"gender": "female",
"age": 28,
"email": "genevievecompton#cyclonica.com",
"annualSalary": "$46,881.00",
"id": 1
},
{
"SeasonTicket": false,
"name": "Christian Crawford",
"gender": "male",
"age": 53,
"email": "christiancrawford#cyclonica.com",
"annualSalary": "$53,488.00",
"id": 2
}
]
I have tried using JSONparser but am not able to get through the JSON structure.I have been advised to use JAQL and pig but cannot do so.
Any help would be appreciated.
What I understand is that you have a huge file with an array of JSONs. Of this, you need to read the same to a mapper and emit say <id : gender>. The challenge is that JSON falls across to multiple lines.
In this is the case, I would suggest you to change the default delimiter to "}" instead of "\n".
In this case, you will be able to get parts of the JSON into the map method as value. You can discard the key ie. byte offset and do slight re-fractor on the value like removing off unwanted [ ] or , and adding chars like "}" and then parse the remaining string.
This solution works because there is no nesting within JSON and } is a valid JSON end delimiter as per the given example.
For changing the default delimiter, just set the property textinputformat.record.delimiter to "}"
Please check out this example.
Also check this jira.