So either I go back and tell someone that they should fix their JSON, or I need to find out what I am doing wrong. Here is the JSON, notice that parameter occurs three times:
String j= '''{
"jobname" : "test",
"parameters" : {
"parameter": {"name":"maxErrors", "value":"0"},
"parameter": {"name":"case", "value":"lower"},
"parameter": {"name":"mapTable", "value":"1"}
}
} '''
And I am trying to get each name & value. My code
def doc = new JsonSlurper().parseText(j)
def doc1 = doc.entrySet() as List
def doc2 = doc.parameters.entrySet() as List
println "doc1.size===>"+doc1.size()
println "doc1===>"+doc1
println "doc2.size===>"+doc2.size()
println "doc2===>"+doc2
And my results:
doc1.size===>2
doc1===>[jobname=test, parameters={parameter={name=mapTable, value=1}}]
doc2.size===>1
doc2===>[parameter={name=mapTable, value=1}]
How come I only get one parameter? Where are the other two? It looks like JSON only keeps one parameter and discards the other ones.
The JSON is not in the correct format. There should not be duplicate key in the same hierarchy or they will override each other.
It should have been an array of paramters.
Like this,
String j= '''{
"jobname" : "test",
"parameters" : [
{"name":"maxErrors", "value":"0"},
{"name":"case", "value":"lower"},
{"name":"mapTable", "value":"1"}
]
}
Related
I am looking for a way to update the given Json's values and keys in a dynamic way. The way the Json is delivered is Always the same(in Terms of structure). The only Thing that differs is the amount of Data that is provided. So for example there could sometimes be 30, sometimes only 10 nestings etc.
…
"ampdata": [
{
"nr": "303",
"code": "JGJGh4958GH",
"status": "AVAILABLE",
"ability": [ "" ],
"type": "wheeled",
"conns": [
{
"nr": "447",
"status": "",
"version": "3",
"format": "sckt",
"amp": "32",
"vol": "400",
"vpower": 22
}
]
}
As Json uses other keys/values than I in my DB, I Need to convert them. Additionally I Need to Change some values if they match explicit strings.
So for example: "Code" has to be renamed to"adrID" and "sckt" should map to the values "bike".
I tried a simple Groovy-Script to remove the key and or Change the value. There is no Problem in changing values, but in changing the key itself. So I tried removing the key and adding a new key. Unfortunately I could not figure out how to add a new key:value to the given json. So how can I add a new pair of key:value or rename the key, if that´s possible. Have a look at my code-example
def flowFile = session.get()
if (!flowFile) return
try {
flowFile = session.write(flowFile,
{ inputStream, outputStream ->
def text = IOUtils.toString(inputStream, StandardCharsets.UTF_8)
def obj = new JsonSlurper().parseText(text)
def objBuilder = new JsonBuilder(obj)
// Update ingestionDate field with today's date
for(i in 0..obj.data.size()-1){
obj.data[0].remove("postal_code")
objBuilder.data[0].postal_code=5
}
// Output updated JSON
def json = JsonOutput.toJson(obj)
outputStream.write(JsonOutput.prettyPrint(json).getBytes(StandardCharsets.UTF_8))
} as StreamCallback)
flowFile = session.putAttribute(flowFile, "filename", flowFile.getAttribute('filename').tokenize('.')[0]+'_translated.json')
session.transfer(flowFile, REL_SUCCESS)
} catch(Exception e) {
log.error('Error during JSON operations', e)
session.transfer(flowFile, REL_FAILURE)
}
...
def obj = new JsonSlurper().parse(inputStream, "UTF-8")
obj.data.each{e->
def value = e.remove("postal_code")
//set old value with a new key into object
e["postalCode"] = value
}
//write to output
def builder = new JsonBuilder(obj)
outputStream.withWriter("UTF-8"){ it << builder.toPrettyString() }
i searched and tried out a lot of things to solve my problem. But it seems that nothing will work. Maybe I just understand it wrong. I have a test.json File looking like this.
{
"TEST-A": [{ "app_id":"aaa" }],
"TEST-B": [{ "app_id":"bbb" }],
"TEST-C": [{ "app_id":"ccc" }]
}
and now i want to edit TEST-B with "xxx". What am i trying for example ist:
import groovy.json.JsonSlurper
def slurper = new groovy.json.JsonSlurper()
def inputFile = new File("../config/test.json")
def inputJSON = new JsonSlurper().parseText(inputFile.text)
def builder = new JsonBuilder(inputJSON)
builder.content.TEST-B[0] = 'xxx'
I thought that i already have a map to edit or do i need to use "assert"?
Greetings Frost.
The following code:
import groovy.json.*
def data = '''\
{
"TEST-A": [{ "app_id":"aaa" }],
"TEST-B": [{ "app_id":"bbb" }],
"TEST-C": [{ "app_id":"ccc" }]
}'''
def parsed = new JsonSlurper().parseText(data)
println "parsed is a java.util.Map: ${parsed instanceof Map}"
parsed.'TEST-B'[0].app_id = 'xxx'
println "map after change: $parsed"
def result = JsonOutput.toJson(parsed)
println "result is a String: ${result instanceof String}"
println "result: $result"
println "pretty result:\n${JsonOutput.prettyPrint(result)}"
when run, prints out:
~> groovy test.groovy
parsed is a java.util.Map: true
map after change: [TEST-A:[[app_id:aaa]], TEST-B:[[app_id:xxx]], TEST-C:[[app_id:ccc]]]
result is a String: true
result: {"TEST-A":[{"app_id":"aaa"}],"TEST-B":[{"app_id":"xxx"}],"TEST-C":[{"app_id":"ccc"}]}
pretty result:
{
"TEST-A": [
{
"app_id": "aaa"
}
],
"TEST-B": [
{
"app_id": "xxx"
}
],
"TEST-C": [
{
"app_id": "ccc"
}
]
}
and I believe accomplishes the essence of what you are trying to do. The thing to understand about JsonSlurper is after parsing the in-data, what you get back is a normal java.util.Map (or possibly a java.util.List depending on the in-data json).
In other words, the parsed variable above is just a Map where the keys are strings and the values are Lists of Maps.
The second thing to keep in mind is that keys like TEST-B are not valid indentifiers in groovy so you can not just write parsed.TEST-B because that would be interpreted as parsed.TEST - B, so you have to quote keys with special characters like this.
After you change the map and assuming you want to get back to a json representation, you have to use something like JsonOutput as in the code above.
To read the json file I have created below method in groovy. I am calling it from another class.
class WUPage{
#Shared
String fileContents ;
def jsonSlurper ;
def jsonObject ;
public String getValueFromJsonFile(String fileName, String jsonKey){
fileContents = new File(fileName).getText('UTF-8')
jsonSlurper = new JsonSlurper()
jsonObject = jsonSlurper.parseText(fileContents)
println " Json Key from method is : " + jsonObject.jsonKey
println " Json Key hardecoded is : " + jsonObject.pipe.id
return jsonObject.jsonKey
}
}
When I am calling this method from another class I am passing file name and the key to it like below
getValueFromJsonFile(jsonFIleName, "pipe.id")
I am getting below output
Json Key from method is : null
Json Key hardecoded is : [India]
From above output second line is correct when key is hardcoded. It seems it's not recognizing key coming from method parameter.
Can you please help me on this.
Json File is :
{
"source": [
{
"id": "manish",
"type": "csv",
"path": "/home/surya/f1.txt",
"delimiter": ",",
"tableName": "table1",
"schema": "f1,f2,f3,f4,f5"
}
],
"pipe": [
{
"id": "India",
"sql": "select f1,f2,f5 from table1"
}
],
"sinks": [
{
"id": "output1",
"type": "aaa",
"path": "/home/surya/out",
"format": "json"
}
]
}
It is because the dot in the key name is not doing what you expected. You didn't show your JSON data in the original post, but I can deduce that there is a top-level object "pipe" that contains an id key. Groovy can't magically know this, so it applies the entire parameter as a top-level key, not as compound deep keys.
You are trying to always get the key jsonKey from that map.
def json = [a: [b: 42]] // just some map, but that's what your JsonSlurper result is
def key = 'a' // start with a simple key
println json.key // wont work, because groovy will access the key `key`
// => null
println json."$key" // works, now
// => [b:42]
key = 'a.b'
println json."$key" // wont work, because you can not access recursive keys like that
// => null
So at that point you have to decide, what to do and how comples your path there can become and what's the source of your path (e.g. is is always a string from somewhere else or is this for developer convenience).
You can Eval a string
You can "split" the path at . and then reduce this list over the map
Instead of a string pass in a Closure, that accesses the data
I'm trying to parse a random JSON file in Grails.
First I need to get the name of each field
For example, given below JSON file,
{
"abbreviation": "EX",
"guid": "1209812-1l2kj1j-fwefoj9283jf-ae",
"metadata": {
"dataOrigin": "Example"
},
"rooms":
[
],
"site": {
"guid": "1209812-1l2kj1j-fwefoj9283jf-ae"
},
"title": "Example!!"
}
I want to find out the structure of the JSON file(lists of keys maybe), for example I want to save the list of keys such as 'abbreviation', 'guid', 'metadata', 'rooms', 'site', 'title' from this JSON file.
How would I do this?
(We need the name of the keys in order to get the value of that key, so with a arbitrarily structured JSON file I need to find out the keys first)
You can try below code
def filePath = "JSONFILE.json"
def text = new File(filePath).getText()
def json = JSON.parse(text)
def jsonKeys = json.collect{it.key}
println(jsonKeys)
This will print all json keys
From what dmahaptro commented, I figured out how to get all the keys within a JSON object.
Here is a simple sample code I wrote to test it
String jsonFile = new JsonSlurper().parseText(new URL(path to the json file).text)
JSONArray jsonParse = new JSONArray(jsonFile)
int len = jsonParse.length()
def names = []
def keys = []
(0..len-1).each {
JSONObject val = jsonParse.getJSONObject(it)
int numKeys = val.length()
names = val.names()
keys = val.keySet()
(0..numKeys-1).each {
def field = names[it]
println field +" : " + val."${field}"
}
}
This will print the key:value pair given a JSON file.
I am writing a Grails/Groovy app and I have a JSON object with a "string" name (grommet and widget) inside the params member that can change. That is, next time it might be acme and zoom. Here is the JSON:
def jx = """{
"job": "42",
"params": {
"grommet": {"name": "x", "data": "y"},
"widget": { "name": "a", "data": "b"}
}
}"""
I am trying to figure out how to get the string grommet . Code so far:
def dalist = new JsonSlurper().parseText(jx)
println dalist.job // Gives: 42
println dalist.params // Gives: [grommet:[name:x, data:y], widget:[name:a, data:b]]
println dalist.params[0] // Gives: null
Any idea how to get the string grommet? Iama going to keep hitting my head against a wall.
The params key on the JSON object is associated with a JSON object, not an array, so you cannot access it by index. JsonSlurper maps JSON objects to Groovy Maps, so you can access params by its keys, which are strings, e.g. dalist.params.grommet, which will give you the map [name: 'x', data: 'y'].
To access the keys on the params you can do dalist.params.keySet(), which will give you the list ['grommet', 'widget']. If you are interested in just knowing params keys, that should do the trick. If you need to get the 'grommet' string for some reason, you can do it by accessing the first element on that list, i.e. dalist.params.keySet()[0], but i don't know why you would want to know that. And i'm not sure if it is guaranteed that the first key of that map will always be 'grommet', as JSON objects are unordered by the spec (from json.org: An object is an unordered set of name/value pairs), but, in turn, Groovy maps are ordered (the default implementation is LinkedHashMap)... so i would assume that the order is preserved when parsing JSON to the Groovy world, but i'd try not to rely on that particular behavior hehe.
It's Map instance, try:
def params = dalist.params.entrySet() as List // entrySet() returns Set, but it's easier to use it as a List
println params
println params.size()
println params[0]
println params[0].key
println params[0].value
This might help you.
import groovy.json.JsonSlurper;
def jx='{"job":"42","params":{"grommet":{"name":"x","data":"y"},"widget":{"name":"a","data":"b"}}}'
def dalist = new JsonSlurper().parseText( jx )
assert dalist.params.getClass().name == "java.util.HashMap";
assert dalist.params.size() == 2;
def keys = dalist.params.collect{ a, b -> a}; // returns "[grommet, widget]"
assert !!dalist.params.get( "grommet" ) == true