NIFI:Json Content Parsing in FlowFile - json

I have text attribute in GenerateFlowfile processor like this:
[{
"status": {
"runStatus": "STOPPED"
},
"component": {
"state": "STOPPED",
"id": "ea5db028-015d-1000-5ad5-80fd006dda92"
},
"revision": {
"version": 46,
"clientId": "ef592126-015d-1000-bf4f-93c7cf7eedc0"
}
} ]
and related groovy script in my ExecuteScript processor :
import org.apache.commons.io.IOUtils
import java.nio.charset.*
def flowFile = session.get();
if (flowFile == null) {
return;
}
def slurper = new groovy.json.JsonSlurper()
def attrs = [:] as Map<String,String>
session.read(flowFile,
{ inputStream ->
def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
text=flowFile.getAttribute('text')
def obj = slurper.parseText(text)
obj.each {k,v ->
attrs[k] = v.toString()
}
} as InputStreamCallback)
flowFile = session.putAllAttributes(flowFile, attrs)
session.transfer(flowFile, REL_SUCCESS)
but my processor still shows me exception like this, what should i change?

the problem that the root element of your json is an array
and you try to iterate it like map .each{k,v-> ... }
probably you want to take the first map to iterate it like in code below
def obj=[ [a:1] ]
//this line will throw exception:
//obj.each{k,v-> println "$k->$v" }
//but this one not
obj[0].each{k,v-> println "$k->$v" }
fyi: there is a EvaluateJsonPath processor that could extract attributes from json flowfile content and put result into attribute

Related

Parse nested json objects in groovy

I have a json file containing contact info grouped by city. I want to parse the json and create a list of names and numbers but after fiddling for an hour or so, I can't get this to work in groovy.
def ​json = '''{
"date":"2018-01-04T22:01:02.2125",
"boston": [
{
"name":"bob",
"phone":"242 123123",
"ext":"12",
"email":"bob#boston.com"
},
{
"name":"alice",
"phone":"212-123-345",
"ext":"1",
"email":"alice#boston.com"
}
],
"chicago": [
{
"name":"charlie",
"phone":"313-232-545",
"ext":"14",
"email":"charlie#chicago.com"
},
{
"name":"denise",
"phone":"414-123-546",
"ext":"9",
"email":"denise#chicago.com"
}
]
}'''
I have tried a few variations on the following theme but they all failed so far.
parsedjson = slurper.parseText(json)
phonelist = []
parsedjson.each{phonelist.add([it['name'],it['phone']])}
It's tricky with the json you have, as you need to look for the values which are lists... You can do this with a findAll, so given the json:
def ​json = '''{
"date":"2018-01-04T22:01:02.2125",
"boston": [
{
"name":"bob",
"phone":"242 123123",
"ext":"12",
"email":"bob#boston.com"
},
{
"name":"alice",
"phone":"212-123-345",
"ext":"1",
"email":"alice#boston.com"
}
],
"chicago": [
{
"name":"charlie",
"phone":"313-232-545",
"ext":"14",
"email":"charlie#chicago.com"
},
{
"name":"denise",
"phone":"414-123-546",
"ext":"9",
"email":"denise#chicago.com"
}
]
}'''
You can import the JsonSlurper and parse the json as you currently do:
import groovy.json.JsonSlurper
def parsedjson = new JsonSlurper().parseText(json)
Then;
def result = ​parsedjson.findAll { it.value instanceof List } // Find all entries with a list value
.values() // Get all the lists
.flatten() // Merge them into a single list
.collect { [it.name, it.phone] } ​​​​​ // grab the name and phone for each

How to get the exact json node instance using groovy?

Input
Json file :
{
"menu": {
"id": "file",
"value": "File",
"popup": {
"menuitem": [
{
"value": "New",
"onclick": ["CreateNewDoc()","hai"],
"newnode":"added"
}
]
}
}
}
Groovy code :
def newjson = new JsonSlurper().parse(new File ('/tmp/test.json'))
def value=newjson.menu.popup.menuitem.value
def oneclick=newjson.menu.popup.menuitem.onclick
println value
println value.class
println oneclick
println oneclick.class
Output:
[New]
class java.util.ArrayList
[[CreateNewDoc(), hai]]
class java.util.ArrayList
Here,
The json nodes which carries String and List returns the same class name with the groovy code above shown.
How can i differentiate that nodes value and oneclick. Logically I expect value should be a instance of String. but both returns as ArrayList.
How to get the exact type of node in json using groovy.
Update 1:
I don't exactly know, can do this like shown below. My expectation to get the results this,
New
class java.util.String
[CreateNewDoc(), hai]
class java.util.ArrayList
Here you go:
In the below script using closure to show the details of each value and its type
Another closure is used to show the each map in the menuitem list.
def printDetails = { key, value -> println "Key - $key, its value is \"${value}\" and is of typpe ${value.class}" }
def showMap = { map -> map.collect { k, v -> printDetails (k,v) } }
def json = new groovy.json.JsonSlurper().parse(new File('/tmp/test.json'))
def mItem = json.menu.popup.menuitem
if (mItem instanceof List) {
mItem.collect { showMap it }
}
println 'done'
You can quickly try the same online demo
menuitem is list, so you need to get property on concrete list element:
assert newjson.menu.popup.menuitem instanceof List
assert newjson.menu.popup.menuitem[0].value instanceof String
assert newjson.menu.popup.menuitem[0].onclick instanceof List
in your json the menuitem contains array of one object:
"menuitem": [
{
"value": "New",
"onclick": ["CreateNewDoc()","hai"],
"newnode":"added"
}
]
and when you try to access menuitem.value groovy actually returns a list of value attributes for all objects in menuitem array.
that's why menuitem.value returns array ["New"]
in this case
"menuitem": [
{
"value": "New",
"onclick": ["CreateNewDoc()","hai"],
"newnode":"added"
},
{
"value": "Old",
"onclick": ["CreateOldDoc()","hai"],
"newnode":"added"
}
]
menuitem.value will return array ["New", "Old"]
but menuitem[0].value will return the string value "New"
so in your groovy code to get attributes of first menu item:
def value=newjson.menu.popup.menuitem[0].value
def oneclick=newjson.menu.popup.menuitem[0].onclick

Unable to split the values in jsonobject in groovy

I'm new to groovy. I'm trying to split the values in json object in groovy but i cant seem to find a solution. Please find the sample code below
def inputFile = new File("C:\\graph.json")
def InputJSON = new JsonSlurper().parseFile(inputFile,'UTF-8')
InputJSON.each{println it}
def names = InputJSON.graph;
def name
for (int kk=0;kk<4;kk++)
{
name=names.JArray1[kk]
run.put(name.runid, name.rundetails);
println "test::"+name.runid+"--------------"+name.rundetails
}
graph.json
{
"graph": {
"JArray1": [
{
"runid": 1,
"rundetails":{
"01_Home":0.231,
"02_Login":0.561}
}
]
}
}
name.rundetails contains the below values
[01_Home:0.231, 02_Login:0.561]
I would like to split and add it as key and value in Hashmap like below format
Key:01_Home Value:0.231
Key:02_Login Value:0.561
How would i do that any advise on this would be helpful. Thanks in advance.
import groovy.json.*
def inputFile = new StringReader('''
{
"graph": {
"JArray1": [{
"runid": 1,
"rundetails": {
"01_Home": 0.231,
"02_Login": 0.561
}
}
]
}
}
''')
def json = new JsonSlurper().parse(inputFile)
json.graph.JArray1.each{run->
println "runid = ${run.runid}"
// at this point `run.rundetails` is a map like you want
println "details = ${run.rundetails}"
}
As I understand you need collection like:
[[Key:01_Home, Value:0.231], [Key:02_Login, Value:0.561]]
Then you may do:
println InputJSON.graph
.JArray1
.rundetails
.collectEntries{it}
.collect{[Key: it.key, Value: it.value]}

Parse JSON using groovy script (using JsonSlurper)

I am just two days old to groovy, I need to parse a json file with below structure. My actual idea is I need to run a set of jobs in different environments based on different sequences, so I came up with this format of json as a input file to my groovy
{
"services": [{
"UI-Service": [{
"file-location": "/in/my/server/location",
"script-names": "daily-batch,weekly-batch,bi-weekly-batch",
"seq1": "daily-batch,weekly-batch",
"seq2": "daily-batch,weekly-batch,bi-weekly-batch",
"DEST-ENVT_seq1": ["DEV1", "DEV2", "QA1", "QA2"],
"DEST-ENVT_seq2": ["DEV3", "DEV4", "QA3", "QA4"]
}]
}, {
"Mobile-Service": [{
"file-location": "/in/my/server/location",
"script-names": "daily-batch,weekly-batch,bi-weekly-batch",
"seq1": "daily-batch,weekly-batch",
"seq2": "daily-batch,weekly-batch,bi-weekly-batch",
"DEST-ENVT_seq1": ["DEV1", "DEV2", "QA1", "QA2"],
"DEST-ENVT_seq2": ["DEV3", "DEV4", "QA3", "QA4"]
}]
}]
}
I tried below script for parsing the json
def jsonSlurper = new JsonSlurper()
//def reader = new BufferedReader(new InputStreamReader(new FileInputStream("in/my/location/config.json"),"UTF-8"))
//def data = jsonSlurper.parse(reader)
File file = new File("in/my/location/config.json")
def data = jsonSlurper.parse(file)
try{
Map jsonResult = (Map) data;
Map compService = (Map) jsonResult.get("services");
String name = (String) compService.get("UI-Service");
assert name.equals("file-location");
}catch (E){
println Exception
}
I need to first read all the services (UI-service, Mobile-Service, etc..) then their elements and their value
Or you could do something like:
new JsonSlurper().parseText(jsonTxt).services*.each { serviceName, elements ->
println serviceName
elements*.each { name, value ->
println " $name = $value"
}
}
But it depends what you want (and you don't really explain in the question)
Example for reading from JsonParser object:
def data = jsonSlurper.parse(file)
data.services.each{
def serviceName = it.keySet()
println "**** key:${serviceName} ******"
it.each{ k, v ->
println "element name: ${k}, element value: ${v}"
}
}
other options:
println data.services[0].get("UI-Service")["file-location"]
println data.services[1].get("Mobile-Service").seq1

JSON output with Groovy

I have been experimenting with the groovy Jsonbuilder as you can see below trying to look at different ways to build JSON objects and arrays. After things started to make sense, I tried expanding to what is shown below. The question I have is, why does "content" show up in the json pretty string output? I actually have another json object displaying this.class information in json string outputs.
Any ideas? I'm new to this, so it could definitely be an obvious one.
def tt = ["test", "test1"]
def jjj = "jason"
def js3 = new groovy.json.JsonBuilder()
def js2 = new groovy.json.JsonBuilder(tt);
js3 hello: "$jjj", "$jjj": tt
def js4 = new groovy.json.JsonBuilder()
def result = js4([sdn: js3, openflow: js2, type: 3])
println js4.toPrettyString();
outputs:
{
"sdn": {
"content": {
"hello": "jason",
"jason": [
"test",
"test1"
]
}
},
"openflow": {
"content": [
"test",
"test1"
]
},
"type": 3
}
The problem can be restated as...
why does this:
import groovy.json.*
def js3 = new JsonBuilder(["test", "test1"])
def js4 = new JsonBuilder(js3)
println js4.toString()
print:
{"content":["test","test1"]}
and this:
import groovy.json.*
def js3 = new JsonBuilder(["test", "test1"])
def js4 = new JsonBuilder(js3.content)
println js4.toString()
prints this (?) :
["test","test1"]
The short answer is that JsonBuilder has a member named content, which represents the payload. When one JsonBuilder absorbs another, we want to replace the payload, and not nest it. This line is the way to replace the payload:
def js4 = new JsonBuilder(js3.content)
Ultimately, this stems from the fact that JsonBuilder.toString() (code here) calls JsonOutput.toJson(object) (code here).
An exercise for the reader is to experiment with:
class MyBuilder {
def content
}
def myB = new MyBuilder(content: ["test", "test1"])
println JsonOutput.toJson(myB)
println JsonOutput.toJson(myB.content)