Replace subkey without exact path in jq - json

Example JSON file:
{
"u": "stuff",
"x": [1,2,3],
"y": {
"field": "value"
},
"z": {
"zz": {
"name": "change me",
"more": "stuff"
},
"randomKey": {
"name": "change me",
"random": "more stuff"
}
}
}
How can I update all the name fields to "something", maintaining the rest of the JSON file the same?
{
"u": "stuff",
"x": [1,2,3],
"y": {
"field": "value"
},
"z": {
"zz": {
"name": "something",
"more": "stuff"
},
"randomKey": {
"name": "something",
"random": "more stuff"
}
}
}
With a direct path, this would be easy, but the parent keys (z and randomKey in these case) varies.
I tried something like:
jq '.z | .. | .name? |= "something"' file.json
And it's updating the names, but putting also all the recursive stuff..

If it is acceptable to change the "name" field wherever it occurs, you could use walk/1:
walk(if type == "object" and has("name") then .name = "something" else . end)
Please note that walk/1 was only included with jq after jq 1.5 was released. If your jq does not have it, then you can find its definition on the jq FAQ, for example.
If you only want to modify the "name" field in the "z" context, then consider:
.z |= with_entries(if .value.name?
then .value.name = "something"
else . end)

Assuming every value within z has a name property, you could do this:
$ jq --arg newname 'something' '.z[].name = $newname' input.json
Using [] on an object will yield all the values contained in that object. And for each of those values, we were simply setting the name to the new name.
If you needed to be more selective with what gets updated, you'll have to add more conditions to what objects to update. In general, I'd use peak's approach, but here's another way it could be achieved using a structure similar to the first approach, assuming we only want to update objects that already have a name property:
$ jq --arg newname 'something' '(.z[] | select(has("name")).name) = $newname' input.json
It's important to wrap the LHS of the assignment in parentheses, we don't want to change the context prior to the assignment, otherwise we won't see the rest of the results.

Related

jq, replace null values on any level, not touching non-null or not existing

please assist to a newbie in jq. :)
I have to update a field with specific name that might occur on any level of JSON structure - and might not. Like with all *.description fields in JSON below:
{
"a": {
"b": [{
"name": "b0",
"description": "b0 has description"
},
{
"name": "b1",
"description": null
},
{
"name": "b2"
}
],
"description": null
},
"s": "Some string value"
}
I need to update "description" value with some dummy value if only it has null value, but do not touch existing values and do not create new fields where they do not exist. So desired result in this case is:
{
"a": {
"b": [{
"name": "b0",
"description": "b0 has description"
},
{
"name": "b1",
"description": "DUMMY DESCRIPTION"
},
{
"name": "b2"
}
],
"description": "DUMMY DESCRIPTION"
},
"s": "Some string value"
}
Here, .a.b[0].description left untouched because it existed and was not null; .a.b[1].description and .a.description are forced to "DUMMY DESCRIPTION" because these field existed and were null; and .a.b[2] as well as root level left untouched because there was no description field at all.
If for example I try to use command on known paths like below
jq '.known.level.description //= "DUMMY DESCRIPTION"' ........
it fails to skip non-existing fields like .a.b[2].description; and, sure, it works on known positions in JSON only. And if I try to do recursive search like:
jq '.. | .description? //= "DUMMY DESCRIPTION"' ........
it does not seem to work correctly on arrays.
What's the correct approach to walk through entire JSON in this case? Thanks!
What's the correct approach to walk through entire JSON in this case?
The answer is walk!
If your jq does not already have walk/1, you can google for it easily enough (jq "def walk"), and then include its def before using it, e.g. as follows:
walk(if type == "object" and has("description") and .description == null
then .description = "DUMMY DESCRIPTION"
else . end)
One option you could consider is using streams. You'll get paths and values to every item in the input. With that you could look for name/value pairs with the name "description" and update the value.
$ jq --arg replacement "DUMMY DESCRIPTION" '
fromstream(tostream | if length == 2 and .[0][-1] == "description"
then .[1] |= (. // $replacement)
else .
end)
' input.json

jq: How to match one of array and get sibling value

I have some JSON like this:
{
"x": [
{
"name": "Hello",
"id": "211"
},
{
"name": "Goodbye",
"id": "221"
},
{
"name": "Christmas",
"id": "171"
}
],
"y": "value"
}
Using jq, given a name value (e.g. Christmas) how can I get it's associated id (i.e. 171).
I've got as far as being able to check for presence of the name in one of the array's objects, but I can't work out how to filter it down
jq -r 'select(.x[].name == "Christmas")'
jq approach:
jq -r '.x[] | select(.name == "Christmas").id' file
171
The function select(boolean_expression) produces its input unchanged if boolean_expression returns true for that input, and produces no output otherwise.
It can also been done like:
jq '.x[] | select(.name == "Christmas").id'
Also you can try this at link online jq play

Getting only desired properties from nested array values with jq

The structure I ultimately want would be:
{
"catalog": [
{
"name": "X",
"catalog": [
{ "name": "Y", "uniqueId": "Z" },
{ "name": "Q", "uniqueId": "B" }
]
}
]
}
This is what the existing structure looks like except there are many other properties at each level (https://gist.github.com/ajcrites/e0e0ca4ca3a08ff2dc401ec872e6094c). I just want to filter those out and get a JSON format that looks specifically like this.
I have started out with: jq '.catalog', but this returns only the array. I still want the catalog property name there. I can do this with jq '{catalog: .catalog[]}, but this prints out each catalog object individually which makes the whole output invalid JSON. I still want the properties to be in the array. Is there a way to filter specific property key-values within arrays using jq?
The following transforms the given input to the desired output and may well be what you want:
{catalog}
| .catalog |= map( {name, catalog} )
| .catalog[].catalog |= map( {name, uniqueId} )
| .catalog |= .[0:1]
However, it's not clear to me that this is really what you want, as you don't discuss the duplication in the given JSON input. So maybe you don't really want the last line in the above, or maybe you want duplicates to be handled in some other way, or ....
Anyway, the trick to keeping things simple here is to use |=.
An alternative approach would be to use del to delete the unwanted properties (rather than selecting the ones you want), but in the present case, that would be (at best) tedious.
You could start by using tostream to convert your sample.json
into a stream of [path, value] arrays as you can see by running
jq -c tostream sample.json
This will generate
[["catalog",0,"catalog",0,"name"],"Y"]
[["catalog",0,"catalog",0,"prop11"],""]
[["catalog",0,"catalog",0,"uniqueId"],"Z"]
[["catalog",0,"catalog",0,"uniqueId"]]
[["catalog",0,"catalog",1,"name"],"Y"]
[["catalog",0,"catalog",1,"prop11"],""]
...
reduce and setpath can be used to convert back into the
original form with a filter such as:
reduce (tostream|select(length==2)) as [$p,$v] (
{};
setpath($p;$v)
)
Adding conditionals makes it easy to omit properties at any level.
For example the following removes leaf attributes starting with "prop":
reduce (tostream|select(length==2)) as [$p,$v] (
{};
if $p[-1]|startswith("prop")
then .
else setpath($p;$v)
end
)
With your sample.json this produces
{
"catalog": [
{
"catalog": [
{
"name": "Y",
"uniqueId": "Z"
},
{
"name": "Y",
"uniqueId": "Z"
}
],
"name": "X"
},
{
"catalog": [
{
"name": "Y",
"uniqueId": "Z"
},
{
"name": "Y",
"uniqueId": "Z"
}
],
"name": "X"
}
]
}
If the goal is to remove certain properties, then one could do so using walk/1. For example, to remove properties whose names start with "prop":
walk(if type == "object"
then with_entries(select(.key|startswith("prop") | not))
else . end)
The same approach would also be applicable if the focus is on retaining certain properties, e.g.:
walk(if type == "object"
then with_entries(select(.key == "name" or .key == "uniqueId" or .key == "catalog"))
else . end)
You could build up a file that contains paths into the json (expressed as arrays) that you want to keep. Then filter out values that do not fit in those paths.
paths.json:
["catalog","name"]
["catalog","catalog","name"]
["catalog","catalog","uniqueId"]
Then filter values based on their paths. Using streams is a great way to go for this since it gives you access to these paths directly:
$ jq --slurpfile paths paths.json '
def keep_path($path): any($paths[]; . == [$path[] | select(strings)]);
fromstream(tostream | select(length == 1 or keep_path(.[0])))
' input.json

substitute certain characters in strings found in an object

I have a list of objects and want to replace all occurrences of . with : when the key is Name using jq
input:
{
"Parameters": [
{
"Name": "TEST.AB.SOMETHING",
"Value": "hvfuycsgvfiwbiwbibibewfiwbcfwifcbwibcibc"
},
{
"Name": "TEST_GF_USER",
"Value": "ssssecret"
}
]
}
expected output:
{
"Parameters": [
{
"Name": "TEST:AB:SOMETHING",
"Value": "hvfuycsgvfiwbiwbibibewfiwbcfwifcbwibcibc"
},
{
"Name": "TEST_GF_USER",
"Value": "ssssecret"
}
]
}
You may split by . and join by :
jq '(.Parameters[].Name)|=(split(".")|join(":"))' file.json
The assignment is done using the update operator.
The trick is to use .Name |= gsub("\\.";":"). In your case (a flat list), it's simple. If you want to modify the keys of all objects in an arbitrary JSON text, the simplest would be to use walk/1:
walk( if type == "object" and has("Name") then .Name |= gsub("\\.";":")) else . end )
(If your jq does not have walk/1, then its jq definition can readily be found by googling.)

Update one value in array of dicts, using jq

I want to update a value in a dict, which I can only identify by another value in the dict. That is, given this input:
[
{
"format": "geojson",
"id": "foo"
},
{
"format": "geojson",
"id": "bar"
},
{
"format": "zip",
"id": "baz"
}
]
I want to change baz's accompanying format to 'csv':
[
{
"format": "geojson",
"id": "foo"
},
{
"format": "geojson",
"id": "bar"
},
{
"format": "csv",
"id": "baz"
}
]
I have found that this works:
jq 'map(if .id=="baz" then .format="csv" else . end)' my.json
But this seems rather verbose, so I wonder if there is a more elegant way to express this. jq seems to be missing some kind of expression selector, the equivalent of might be [#id='baz'] in xpath.
(When I started this question, I had [.[] |...], then I discovered map, so it's not quite as bad as I thought.)
A complex assignment is what you're looking for:
jq '(.[] | select(.id == "baz") | .format) |= "csv"' my.json
Perhaps not shorter but it is more elegant, as requested. See the last section of the docs at: http://stedolan.github.io/jq/manual/#Assignment
Edit: using map:
jq 'map((select(.id == "baz") | .format) |= "csv")' my.json