I have a json file input.json thus:
{
"foo":{
"prefix":"abc",
"body":[1,2,3]
},
"bar":{
"prefix":"def",
"body":[4,5,6]
}
}
I would like to sort it by the outer object names, with "bar" coming before "foo" in alphabetical order like so:
{
"bar":{
"prefix":"def",
"body":[4,5,6]
},
"foo":{
"prefix":"abc",
"body":[1,2,3]
}
}
to produce file output.json.
Versions of this question have been asked of Java/Javascript (here and here)
Is there a way to accomplish this using a command line tool like sed/awk or boost.json?
Using jq, you could use the keys built-in to get the key names in sorted order and form the corresponding value object
jq 'keys[] as $k | { ($k) : .[$k] }' json
Note that jq does have a field --sort-keys option, which cannot be used here, as it internally sorts the inner level objects as well.
Here's a variable-free jq solution:
to_entries | sort_by(.key) | from_entries
It is also worth noting that gojq, the Go implementation of jq, currently always sorts the keys within all JSON objects.
Related
Original Data
I have the following JSON:
{
"foo":[
"asd",
"fgh"
],
"bar":[
"abc",
"xyz",
"ert"
],
"baz":[
"something"
]
}
Now I want to transform it to a "flat" CSV, such that for every key in my object the list in the value is expanded to n rows with n being the number of entries in the respective list.
Expected Output
foo;asd
foo;fgh
bar;abc
bar;xyz
bar;ert
baz;something
Approaches
I guess I need to use to_entries and then for each .value repeat the same .key for the first column. The jq docs state that:
Thus as functions as something of a foreach loop.
So I tried combining
to_entriesto give the keys and values from my dictionary an accessible name
then build kind of a foreach loop around the .values
and pass the result to #csv
to_entries|map(.value) as $v|what goes here?|#csv
I prepared something that at least compiles here
Don't need to use _entries function, a simple key/value lookup and string interpolation should suffice
keys_unsorted[] as $k | "\($k);\( .[$k][])"
The construct .[$k][] is an expression that first expands the value associated with each key, i.e. .foo and then with object construction, produces multiple results for each key identified and stored in $k variable.
I have input like this:
{ "prop": ["apple", "banana"] }
{ "prop": ["kiwi", "banana"] }
{ "prop": ["cherry", "orange"] }
How do I print objects where prop contains at least one of kiwi and orange?
(The list of interesting values is longer than just 2, so I'd like to leverage the any function somehow.)
I've tried the following
jq 'select(any(.prop[] | contains(["kiwi", "orange"])))' < input.json
and various variants of the above, but can't figure out the right expressions.
The stream-oriented version of the built-in function any can be most easily used if one bears in mind its signature:
def any(generator; condition):
So we are led to:
select( any( .prop[]; . == "kiwi" or . == "orange" ))
or more succinctly:
select( any(.prop[]; IN("kiwi", "orange")))
whitelist
If the values of interest are provided as a JSON array, say $whitelist, you could tweak the above by substituting $whitelist[] for the explicit stream of values:
select( any(.prop[]; IN($whitelist[]) ))
I think you're looking for IN/2. It's implemented using any, but is far easier to grasp.
select(IN(.prop[]; "kiwi", "orange"))
Online demo
I have input JSON data in a bunch of files, with an IP address as one of the keys. I need to iterate over a the files, and I need to get "stuff" out of them. The IP address is different for each file, but I'd like to use a single jq command to get the data. I have tried a bunch of things, the closest I've come is this:
jq '.facts | keys | keys as $ip | .[0]' a_file
On my input in a_file of:
{
"facts": {
"1.1.1.1": {
"stuff":"value"
}
}
}
it returns the IP address, i.e. 1.1.1.1, but then how do I to go back do something like this (which is obviously wrong, but I hope you get the idea):
jq '.facts.$ip[0].stuff' a_file
In my mind I'm trying to populate a variable, and then use the value of that variable to rewind the input and scan it again.
=== EDIT ===
Note that my input was actually more like this:
{
"facts": {
"1.1.1.1": {
"stuff": {
"thing1":"value1"
}
},
"outer_thing": "outer_value"
}
}
So I got an error:
jq: error (at <stdin>:9): Cannot index string with string "stuff"
This fixed it- the question mark after .stuff:
.facts | keys_unsorted[] as $k | .[$k].stuff?
You almost got it right, but need the object value iterator construct, .[] to get the value corresponding to the key
.facts | keys_unsorted[] as $k | .[$k].stuff
This assumes that, inside facts you have one object containing the IP address as the key and you want to extract .stuff from it.
Optionally, to guard against objects that don't contain stuff inside, you could add ? as .[$k].stuff?. And also you could optionally validate keys against a valid IP regex condition and filter values only for those keys.
On the jq manual page there are a few examples of output formatting, particularly some shortcuts for when you want to just echo exactly what was in the input JSON.
What if I want to echo exactly what was in the input, but only for keys that match a certain pattern?
For example, given input like so ...
[
{"Name":"Widgets","Size":10,"SymUS":"Widg","SymCN":"Zyin","SymJP":"Kono"},
{"Name":"Blodgets","Size":400,"SymUS":"Blodg","SymAU":"Blod","SymJP":"Kado"},
{"Name":"Fonzes","Size":11,"SymRU":"Fyet","SymBR":"Foao"}
]
Say I want to select all objects where the Name ends in "ets" and then display the Name and all attributes of the form Sym*. All I know about those attributes is that there will be one or more per JSON object, and the names have the format Sym followed by a two-letter ISO country code.
I would like to just do this:
jq '.[] | select(.Name | endswith("ets")) | {Name, Sym*}'
but that's not a thing.
Is this just not something jq is designed to handle in a single operation? Should I do a first pass through the file to collect all the possible keys and then list them all explicitly via a slurpfile?
The key to a simple solution to the problem is to_entries, as described in the online manual. With your example data, the following filter produces the output shown below, in accordance with what I understand to be the expectations:
.[]
| select(.Name | test("ets$"))
| {Name} + (to_entries | map(select(.key|test("^Sym"))) | from_entries)
You might want to refine the regex tests, and/or make other minor adjustments.
Output:
{
"Name": "Widgets",
"SymUS": "Widg",
"SymCN": "Zyin",
"SymJP": "Kono"
}
{
"Name": "Blodgets",
"SymUS": "Blodg",
"SymAU": "Blod",
"SymJP": "Kado"
}
Say I have the following JSON, stored in my variable jsonVariable.
{
"id": 1,
"details": {
"username": "jamesbrown",
"name": "James Brown"
}
}
I parse this JSON with jq using the following:
echo $jsonVariable | jq '.details.name | select(.name == "James Brown")'
This would give me the output
James Brown
But what if I want to get the id of this person as well? Now, I'm aware this is a rough and simple example - the program I'm working with at the moment is 5 or 6 levels deep with many different JQ functions other than select. I need a way to select a parent's field when I am already 5 or 6 layers deep after carrying out various methods of filtering.
Can anyone help? Is there any way of 'going in reverse', back up to the parent? (Not sure if I'm making sense!)
For a more generic approach, save the value of the "parent" element at the detail level you want, then pipe it at the end of your filter:
jq '. as $parent | .details.name | select(. == "James Brown") | $parent'
Of course, for the trivial case you expose, you could omit this entirely:
jq 'select(.details.name == "James Brown")'
Also, consider that if your selecting filters return many matches for a single parent object, you will receive a copy of the parent object for each match. You may wish to make sure your select filters only return one element at the parent level by wrapping all matches below parent level into an array, or to deduplicate the final result with unique.
Give this a shot:
echo $jsonVariable | jq '{Name: .details.name, Id: .Id} | select(.name == "James Brown")'
Rather than querying up to the value you're testing for, query up to the root object that contains the value you're querying on and the values you wish to select.
You need the object that contains both the id and the name.
$ jq --arg name 'James Brown' 'select(.details.name == $name).id' input.json