Using jq, how can I take a json object from a file (input_02.json), and append it to output.json, while retaining everything already in output.json (e.g. an object originating from file input_01.json).
The object to be appended in both cases is literally the entire contents of the file, with the file's "id" field as the object's key.
I'm taking a large list of input files (all with the same syntax) and essentially combining them like that.
The command i'm using to create the object to be appended is as follows:
jq '{(.id):(.)} ' input_01.json
which gives me:
{
"input1_id": {
}
}
input_1.json:
{
"id": "input1_id",
"val": "testVal1"
}
input2.json:
{
"id": "input2_id",
"val": "testVal2"
}
desired output:
{
"input1_id": {
"id": "input1_id",
"val": "testVal1"
},
"input2_id": {
"id": "input2_id",
"val": "testVal2"
}
}
You’re on the right track with {(.id):(.)}. The following should handle the case you mentioned, and might give you some ideas about similar cases:
program.jq: map({(.id):(.)}) | add
Invocation:
jq -s -f program.jq input_01.json input_02.json
You could use "jf" for this from https://pypi.python.org/pypi/jf
$ pip install jf
$ jf 'chain(), {y["id"]: y for y in x}' input1.json input2.json
{
"input2_id": {
"id": "input2_id",
"val": "testVal2"
},
"input1_id": {
"id": "input1_id",
"val": "testVal1"
}
}
Related
I have two JSON files with the following content:
foo.json:
{
"name": "foo",
"block": {
"one": 1,
"two": "2"
},
"otherData": {
"two": 1,
"one": "2"
}
}
bar.json:
{
"name": "bar"
}
I want to copy the block from foo.json to bar.json in one line so bar.json looks like this:
{
"name": "bar",
"block": {
"one": 1,
"two": "2"
}
}
I tried:
jq --argjson block '{"block": "$(jq '.block' ./foo.json)"}' '. += [$block]' ./bar.json | sponge ./bar.json
The + operator can be used to combine multiple objects together. Having the object enclosed with {} selects the whole object for inclusion.
jq ' . + ( input | {block} )' bar.json foo.json | sponge bar.json
Note: sponge is a utility from the moreutils package, which needs to be installed separately on your system. See setup instructions on the moreutils page
Exercise caution while using the tool, because it overwrites whatever content that is coming in from the standard input to the target file specified. If not completely sure, verify the result by writing to standard output before running sponge.
In my JSON are some objects with a Description property. How do I copy this value over to the GMNotes property of the same object using JQ?
In other words, how does one go from
{
"ObjectStates": [
{
"Description": "",
"GMNotes": ""
},
{
"Description": "foo",
"GMNotes": ""
}
]
}
to
{
"ObjectStates": [
{
"Description": "",
"GMNotes": ""
},
{
"Description": "foo",
"GMNotes": "foo"
}
]
}
.ObjectStates[] | .GMNotes = .Description only returns the modified objects, as shown in the sandbox.
(I could easily do this in Perl. The point is using jq.)
You can use map() in combination with the update assignment operator |=:
jq '(.ObjectStates)|=map(.GMNotes=.Description)' file.json
https://jqplay.org/s/vFV_H4brlH
PS: instead of using map you could also just use the following command, the key is using |=.
jq '.ObjectStates[]|=(.GMNotes=.Description)' file.json
thanks chepner!
https://jqplay.org/s/NCGezXPjLE
I have a huge json file(15 GB) which looks like as follows:
{
"userActivities": {
"-L3ATRosRd-bDgSmX75Z": {
"deviceId": "60ee32c2fae8dcf0",
"dow": "Friday"
}
},
"users": {
"0GTDyAepIjcKMB1XulHCYLXylFS2": {
"ageRangeMin": 21,
"age_range": {
"min": 21
},
"gender": "male"
},
"0GTDyAepIjcKMB1S2": {
"ageRangeMin": 22,
"age_range": {
"min": 20
},
"gender": "male"
}
}
}
I want to extract the objects as if by .users[], but using the streaming parser (jq --stream). That is, I want my output to be as follows:
{"ageRangeMin":21,"age_range":{"min":21},"gender":"male"}
{"ageRangeMin":22,"age_range":{"min":20},"gender":"male"}
Any guidance/help is greatly appreciated. I'm unable to understand how jq --stream works.
If the goal is to just get objects at a certain depth of the json object tree, you can just truncate the stream.
$ jq --stream -nc 'fromstream(2|truncate_stream(inputs | select(.[0][:1] == ["users"])))'
Just make sure you're running the latest available jq. There's a bug in 1.5 for truncate_stream/1 that breaks for any other input greater than 1.
With your input in input.json, the following invocation:
$ jq -nc --stream '
fromstream(inputs|select(.[0][0] == "users"))|.[][]' input.json
yields:
{"ageRangeMin":21,"age_range":{"min":21},"gender":"male"}
{"ageRangeMin":22,"age_range":{"min":20},"gender":"male"}
The idea is to extract the "users" key-value pair first as a single-key object.
Note that the -n option must be used here.
I have files with json structure like this:
[
{
"uid": 11111,
"something": {
(...)
}
},
{
"uid": 22222,
"something": {
(...)
}
}
]
I'll read all files at one time (cat *) and i'd like to know which part is from which file, so i need to group it in some way.
So, my idea is to move content of each file to higher (parent) object with own members.
[
{
"var1": "val1"
"var2": "val2"
{
"uid": 11111,
"something": {
(...)
}
},
{
"uid": 22222,
"something": {
(...)
}
}
}
How to do that with jq?
#!/bin/bash
# For simplicity, assume each file in FILELIST contains a single JSON entity.
# Then instead of using cat FILELIST, use mycat FILELIST, e.g. mycat *.json
function mycat {
for file
do
jq --arg file "$file" '{"file": $file, "contents": .}' "$file"
done
}
If you have a sufficiently recent version of jq (e.g. jq 1.5) then one alternative would be:
jq '{file: input_filename, contents: .}' FILELIST
Could somebody help me to deal with jq command line utility to update JSON object's inner value?
I want to alter object interpreterSettings.2B263G4Z1.properties by adding several key-values, like "spark.executor.instances": "16".
So far I only managed to fully replace this object, not add new properties with command:
cat test.json | jq ".interpreterSettings.\"2B188AQ5T\".properties |= { \"spark.executor.instances\": \"16\" }"
This is input JSON:
{
"interpreterSettings": {
"2B263G4Z1": {
"id": "2B263G4Z1",
"name": "sh",
"group": "sh",
"properties": {}
},
"2B188AQ5T": {
"id": "2B188AQ5T",
"name": "spark",
"group": "spark",
"properties": {
"spark.cores.max": "",
"spark.yarn.jar": "",
"master": "yarn-client",
"zeppelin.spark.maxResult": "1000",
"zeppelin.dep.localrepo": "local-repo",
"spark.app.name": "Zeppelin",
"spark.executor.memory": "2560M",
"zeppelin.spark.useHiveContext": "true",
"spark.home": "/usr/lib/spark",
"zeppelin.spark.concurrentSQL": "false",
"args": "",
"zeppelin.pyspark.python": "python"
}
}
},
"interpreterBindings": {
"2AXUMXYK4": [
"2B188AQ5T",
"2AY8SDMRU"
]
}
}
I also tried the following but this only prints contents of interpreterSettings.2B263G4Z1.properties, not full object.
cat test.json | jq ".interpreterSettings.\"2B188AQ5T\".properties + { \"spark.executor.instances\": \"16\" }"
The following works using jq 1.4 or jq 1.5 with a Mac/Linux shell:
jq '.interpreterSettings."2B188AQ5T".properties."spark.executor.instances" = "16" ' test.json
If you have trouble adapting the above for Windows, I'd suggest putting the jq program in a file, say my.jq, and invoking it like so:
jq -f my.jq test.json
Notice that there is no need to use "cat" in this case.
p.s. You were on the right track - try replacing |= with +=