In my JSON are some objects with a Description property. How do I copy this value over to the GMNotes property of the same object using JQ?
In other words, how does one go from
{
"ObjectStates": [
{
"Description": "",
"GMNotes": ""
},
{
"Description": "foo",
"GMNotes": ""
}
]
}
to
{
"ObjectStates": [
{
"Description": "",
"GMNotes": ""
},
{
"Description": "foo",
"GMNotes": "foo"
}
]
}
.ObjectStates[] | .GMNotes = .Description only returns the modified objects, as shown in the sandbox.
(I could easily do this in Perl. The point is using jq.)
You can use map() in combination with the update assignment operator |=:
jq '(.ObjectStates)|=map(.GMNotes=.Description)' file.json
https://jqplay.org/s/vFV_H4brlH
PS: instead of using map you could also just use the following command, the key is using |=.
jq '.ObjectStates[]|=(.GMNotes=.Description)' file.json
thanks chepner!
https://jqplay.org/s/NCGezXPjLE
Related
Given the following input JSON:
{
"version": 2,
"models": [
{
"name": "first_table",
"tests": [
{
"dbt_utils.equal_rowcount": {
"compare_model": null
}
}
]
},
{
"name": "second_table",
"tests": [
{
"dbt_utils.equal_rowcount": {
"compare_model": null
}
}
]
}
]
}
How would I, using jq, replace the null (i.e., the value of "compare_model") with the value from the "name" key? Note that the key-value pairs in question here are not at the same level in the hierarchy: the former is nested in an object in an array, and it is this array that is at the same level as the latter.
For example, the output file should read:
{
"version": 2,
"models": [
{
"name": "first_table",
"tests": [
{
"dbt_utils.equal_rowcount": {
"compare_model": "first_table"
}
}
]
},
{
"name": "second_table",
"tests": [
{
"dbt_utils.equal_rowcount": {
"compare_model": "second_table"
}
}
]
}
]
}
FWIW, this is an intermediate step in some YAML (via yq, the Python wrapper variety of jq as opposed to the go variant) wrangling I'm doing on DBT config files.
(Bonus points if you can wrap the replacement text with parentheses and/or prefix it without breaking out of jq. :D If not, no worries -- this step I can do with another program.)
Needless to say, but your help is very much appreciated!
The key to a simple solution is to use |=, e.g.
.models |=
map(.name as $name
| (.tests[]."dbt_utils.equal_rowcount".compare_model =
$name))
To wrap the replacement value in parentheses, just add them:
.models |=
map("(\(.name))" as $name
| (.tests[]."dbt_utils.equal_rowcount".compare_model =
$name))
If you want the replacement to be conditional on the existing value being null, you could perhaps (depending on the exact requirements) use //=.
Using //= and walk
Here's another take on the problem:
.models
|= map("(\(.name))" as $name
| walk(if type=="object" and has("compare_model")
then .compare_model //= $name
else . end))
That the fields are not at the same level doesn't really matter here.
.models[] |= (.tests[]."dbt_utils.equal_rowcount".compare_model = "(\(.name))")
Online demo
Using jq, how can I take a json object from a file (input_02.json), and append it to output.json, while retaining everything already in output.json (e.g. an object originating from file input_01.json).
The object to be appended in both cases is literally the entire contents of the file, with the file's "id" field as the object's key.
I'm taking a large list of input files (all with the same syntax) and essentially combining them like that.
The command i'm using to create the object to be appended is as follows:
jq '{(.id):(.)} ' input_01.json
which gives me:
{
"input1_id": {
}
}
input_1.json:
{
"id": "input1_id",
"val": "testVal1"
}
input2.json:
{
"id": "input2_id",
"val": "testVal2"
}
desired output:
{
"input1_id": {
"id": "input1_id",
"val": "testVal1"
},
"input2_id": {
"id": "input2_id",
"val": "testVal2"
}
}
You’re on the right track with {(.id):(.)}. The following should handle the case you mentioned, and might give you some ideas about similar cases:
program.jq: map({(.id):(.)}) | add
Invocation:
jq -s -f program.jq input_01.json input_02.json
You could use "jf" for this from https://pypi.python.org/pypi/jf
$ pip install jf
$ jf 'chain(), {y["id"]: y for y in x}' input1.json input2.json
{
"input2_id": {
"id": "input2_id",
"val": "testVal2"
},
"input1_id": {
"id": "input1_id",
"val": "testVal1"
}
}
I have a list of objects and want to replace all occurrences of . with : when the key is Name using jq
input:
{
"Parameters": [
{
"Name": "TEST.AB.SOMETHING",
"Value": "hvfuycsgvfiwbiwbibibewfiwbcfwifcbwibcibc"
},
{
"Name": "TEST_GF_USER",
"Value": "ssssecret"
}
]
}
expected output:
{
"Parameters": [
{
"Name": "TEST:AB:SOMETHING",
"Value": "hvfuycsgvfiwbiwbibibewfiwbcfwifcbwibcibc"
},
{
"Name": "TEST_GF_USER",
"Value": "ssssecret"
}
]
}
You may split by . and join by :
jq '(.Parameters[].Name)|=(split(".")|join(":"))' file.json
The assignment is done using the update operator.
The trick is to use .Name |= gsub("\\.";":"). In your case (a flat list), it's simple. If you want to modify the keys of all objects in an arbitrary JSON text, the simplest would be to use walk/1:
walk( if type == "object" and has("Name") then .Name |= gsub("\\.";":")) else . end )
(If your jq does not have walk/1, then its jq definition can readily be found by googling.)
Could somebody help me to deal with jq command line utility to update JSON object's inner value?
I want to alter object interpreterSettings.2B263G4Z1.properties by adding several key-values, like "spark.executor.instances": "16".
So far I only managed to fully replace this object, not add new properties with command:
cat test.json | jq ".interpreterSettings.\"2B188AQ5T\".properties |= { \"spark.executor.instances\": \"16\" }"
This is input JSON:
{
"interpreterSettings": {
"2B263G4Z1": {
"id": "2B263G4Z1",
"name": "sh",
"group": "sh",
"properties": {}
},
"2B188AQ5T": {
"id": "2B188AQ5T",
"name": "spark",
"group": "spark",
"properties": {
"spark.cores.max": "",
"spark.yarn.jar": "",
"master": "yarn-client",
"zeppelin.spark.maxResult": "1000",
"zeppelin.dep.localrepo": "local-repo",
"spark.app.name": "Zeppelin",
"spark.executor.memory": "2560M",
"zeppelin.spark.useHiveContext": "true",
"spark.home": "/usr/lib/spark",
"zeppelin.spark.concurrentSQL": "false",
"args": "",
"zeppelin.pyspark.python": "python"
}
}
},
"interpreterBindings": {
"2AXUMXYK4": [
"2B188AQ5T",
"2AY8SDMRU"
]
}
}
I also tried the following but this only prints contents of interpreterSettings.2B263G4Z1.properties, not full object.
cat test.json | jq ".interpreterSettings.\"2B188AQ5T\".properties + { \"spark.executor.instances\": \"16\" }"
The following works using jq 1.4 or jq 1.5 with a Mac/Linux shell:
jq '.interpreterSettings."2B188AQ5T".properties."spark.executor.instances" = "16" ' test.json
If you have trouble adapting the above for Windows, I'd suggest putting the jq program in a file, say my.jq, and invoking it like so:
jq -f my.jq test.json
Notice that there is no need to use "cat" in this case.
p.s. You were on the right track - try replacing |= with +=
I have a hierarchically deep JSON object created by a scientific instrument, so the file is somewhat large (1.3MB) and not readily readable by people. I would like to get a list of keys, up to a certain depth, for the JSON object. For example, given an input object like this
{
"acquisition_parameters": {
"laser": {
"wavelength": {
"value": 632,
"units": "nm"
}
},
"date": "02/03/2525",
"camera": {}
},
"software": {
"repo": "github.com/username/repo",
"commit": "a7642f",
"branch": "develop"
},
"data": [{},{},{}]
}
I would like an output like such.
{
"acquisition_parameters": [
"laser",
"date",
"camera"
],
"software": [
"repo",
"commit",
"branch"
]
}
This is mainly for the purpose of being able to enumerate what is in a JSON object. After processing the JSON objects from the instrument begin to diverge: for example, some may have a field like .frame.cross_section.stats.fwhm, while others may have .sample.species, so it would be convenient to be able to interrogate the JSON object on the command line.
The following should do exactly what you want
jq '[(keys - ["data"])[] as $key | { ($key): .[$key] | keys }] | add'
This will give the following output, using the input you described above:
{
"acquisition_parameters": [
"camera",
"date",
"laser"
],
"software": [
"branch",
"commit",
"repo"
]
}
Given your purpose you might have an easier time using the paths builtin to list all the paths in the input and then truncate at the desired depth:
$ echo '{"a":{"b":{"c":{"d":true}}}}' | jq -c '[paths|.[0:2]]|unique'
[["a"],["a","b"]]
Here is another variation uing reduce and setpath which assumes you have a specific set of top-level keys you want to examine:
. as $v
| reduce ("acquisition_parameters", "software") as $k (
{}; setpath([$k]; $v[$k] | keys)
)