I have a json file people.json:
{
"Joe" : {"Job" : "Clown", "Age" : 22},
"Sally" : {"Job" : "Programmer", "Age" : 32},
"Anne" : {"Job" : "Clown", "Age" : 29}
}
I would like to select everyone who is a Clown. My output should look like this:
{
"Joe" : {"Job" : "Clown", "Age" : 22},
"Anne" : {"Job" : "Clown", "Age" : 29}
}
I have tried the .. operator as in
cat people.json | jq '. | map(select(.Job == "Clown"))'
But it seems to match Joe and Anne at multiple levels and produces more output then I want. Any ideas? Thanks.
use with_entries to convert to/from an intermediate format that represents that data as an array of objects with key and value elements:
cat people.json | jq 'with_entries(select(.value.Job == "Clown"))'
as per the docs here: http://stedolan.github.io/jq/manual/
Here is a solution using reduce
. as $v
| reduce keys[] as $k (
{};
if $v[$k].Job == "Clown" then .[$k] = $v[$k] else . end
)
Related
I would like to create a JSON with array of nested objects with a grouping for different fields.
This is the CSV and Iwould like to group it by sid, year and quarter (first three fields):
S4446B3,2020,202001,2,345.45
S4446B3,2020,202001,4,24.44
S4446B3,2021,202102,5,314.55
S6506LK,2020,202002,3,376.55
S6506LK,2020,202003,3,76.23
After splitting the CSV with the following I get an object for each record.
split("\n")
| map(split(","))
| .[0:]
| map({"sid" : .[0], "year" : .[1], "quarter" : .[2], "customer_type" : .[3], "obj" : .[4]})
But for each sid I would like to get an array of objects nested like this :
[
{
"sid" : "S4446B3",
"years" : [
{
"year" : 2020,
"quarters" : [
{
"quarter" : 202001,
"customer_type" : [
{
"type" : 2,
"obj" : "345.45"
},
{
"type" : 4,
"obj" : "24.44"
}
]
}
]
},
{
"year" : 2021,
"quarters" : [
{
"quarter" : 202102,
"customer_type" : [
{
"type" : 5,
"obj" : "314.55"
}
]
}
]
}
]
},
{
"sid" : "S6506LK",
"years" : [
{
"year" : 2020,
"quarters" : [
{
"quarter" : 202002,
"customer_type" : [
{
"type" : 3,
"obj" : "376.55"
}
]
},
{
"quarter" : 202003,
"customer_type" : [
{
"type" : 3,
"obj" : "76.23"
}
]
}
]
}
]
}
]
It'd be more intuitive if sid, year, quarter, etc. were to be key names. With -R/--raw-input and -n/--null-input options on the command line, this will do that:
reduce (inputs / ",")
as [$sid, $year, $quarter, $type, $obj]
(.; .[$sid][$year][$quarter] += [{$type, $obj}])
And, to get your expected output you can append these lines to the above program.
| .[][] |= (to_entries | map({quarter: .key, customer_type: .value}))
| .[] |= (to_entries | map({year: .key, quarters: .value}))
| . |= (to_entries | map({sid: .key, years: .value}))
I'm working with Poloniex API. While using returnTicker function, the data comes like this.
curl "https://poloniex.com/public?command=returnTicker"
{
"BTC_BTS": {
"id": 14,
"last": "0.00000111",
"lowestAsk": "0.00000112",
"highestBid": "0.00000110",
"percentChange": "0.09900990",
"baseVolume": "3.12079869",
"quoteVolume": "2318738.79293715",
"isFrozen": "0",
"high24hr": "0.00000152",
"low24hr": "0.00000098"
},
"BTC_DASH": {
"id": 24,
"last": "0.00466173",
"lowestAsk": "0.00466008",
"highestBid": "0.00464358",
"percentChange": "0.02318430",
"baseVolume": "1.98111396",
"quoteVolume": "425.22973220",
"isFrozen": "0",
"high24hr": "0.00482962",
"low24hr": "0.00450482"
....
},
"USDT_GRT": {
"id": 497,
"last": "0.72811272",
"lowestAsk": "0.75999916",
"highestBid": "0.72740000",
"percentChange": "0.48594450",
"baseVolume": "133995.43411815",
"quoteVolume": "194721.36672887",
"isFrozen": "0",
"high24hr": "0.79000000",
"low24hr": "0.45000020"
},
"TRX_SUN": {
"id": 498,
"last": "500.00000000",
"lowestAsk": "449.99999999",
"highestBid": "100.00000000",
"percentChange": "0.00000000",
"baseVolume": "0.00000000",
"quoteVolume": "0.00000000",
"isFrozen": "0",
"high24hr": "0.00000000",
"low24hr": "0.00000000"
}
}
I want the output like this
BTC_BTS : 14 : 0.00000111 : 0.00000112 : 0.00000110 : 0.09900990 : 3.12079869 : 2318738.79293715 : 0 : 0.00000152 : 0.00000098
...
USDT_GRT : 497 : 0.72428700 : 0.75999958 : 0.72630001 : 0.47813685 : 133968.74968533 : 194695.96886712 : 0 : 0.79000000 : 0.45000020
TRX_SUN : 498 : 500.00000000 : 449.99999999 : 100.00000000 : 0.00000000 : 0.00000000 : 0.00000000 : 0 : 0.00000000 : 0.00000000
I am using jq and my problem is accesing the currency pair name.
I could do this;
14 : 0.00000111 : 0.00000112 : 0.00000110 : 0.09900990 : 3.12079869 : 2318738.79293715 : 0 : 0.00000152 : 0.00000098
...
497 : 0.72428700 : 0.75999958 : 0.72630001 : 0.47813685 : 133968.74968533 : 194695.96886712 : 0 : 0.79000000 : 0.45000020
498 : 500.00000000 : 449.99999999 : 100.00000000 : 0.00000000 : 0.00000000 : 0.00000000 : 0 : 0.00000000 : 0.00000000
by using this command;
curl "https://poloniex.com/public?command=returnTicker" |jq -r | jq '.[] | (.id|tostring) + " : " + (.last|tostring) + " : " + (.lowestAsk|tostring) + " : " + (.highestBid|tostring) + " : " + (.percentChange|tostring) + " : " + (.baseVolume|tostring) + " : " + (.quoteVolume|tostring) + " : " + (.isFrozen|tostring) + " : " + (.high24hr|tostring) + " : " + (.low24hr|tostring)'|jq -r
not only this, in every jq pipeline, I cant access the first element of json
I am not meaning the |jq .BTC_BTS or |jq .USDT_GRT pipeline.
|jq . gives whole json, |jq .[] gives the sub elements after the first element.
How can i access the first path?
By the way, I may have written stupid and long pipeline with jq. If you have any idea to convert whole json to a row-column data, I am open to your ideas.
Thank you all for your answers.
To be safe, it might be better not to assume that the ordering of the keys is the same in all the inner objects. Ergo:
keys_unsorted as $outer
| (.[$outer[0]] | keys_unsorted) as $keys
| $outer[] as $k
| [ $k, .[$k][$keys[]] ]
| join(" : ")
I think this does what you want.
curl -s "https://poloniex.com/public?command=returnTicker" | \
jq -r 'to_entries
| .[]
| [ .key, (.value | to_entries | .[] | .value) ]
| join(" : ")'
In a nutshell, put everything in an array and use join to produce the desired output.
Update
As luciole75w notes, my solution has too many steps. This is better.
jq -r 'to_entries[] | [ .key, .value[] ] | join(" : ")'
That said, I would use peak's solution. Mine does not guarantee that the columns are the same for each line.
In a JSON file I need to replace encrypted values with their clear text values as an initialization process using the command line tool jq. An application will then re-encrypt the values with its own keys, overwriting the clear text values. Encrypted values are represented as "$crypto" objects, containing information about the encryption method and which keys were used, looking like this:
{
"$crypto" : {
"type" : "x-simple-encryption",
"value" : {
"cipher" : "AES/CBC/PKCS5Padding",
"stableId" : "someId",
"salt" : "4J5ckE6+JaS8TLqAN4073g==",
"data" : "vBeHAPJXLl+X/8Enp9vxMA==",
"keySize" : 16,
"purpose" : "someDescription",
"iv" : "N2xCe5RiJibHv9hLY+OduA==",
"mac" : "VoOo1BKptwfqIJeSOb/qGA=="
}
}
}
These "$crypto" objects can be anywhere in the JSON structure. A sample input document looks like this:
{
"unknownKey1" : {
"unknownKey2" : {
"name" : "JWT_SESSION",
"properties" : {
"maxTokenLifeMinutes" : 120,
"tokenIdleTimeMinutes" : 30
}
},
"unknownKey3" : [
{
"unknownKey4" : "STATIC_USER",
"unknownKey5" : {
"unknownKey6" : "internal/user",
"unknownKey7" : "anonymous",
"unknownKey8" : {
"$crypto" : {
"type" : "x-simple-encryption",
"value" : {
"cipher" : "AES/CBC/PKCS5Padding",
"stableId" : "someId",
"salt" : "4J5ckE6+JaS8TLqAN4073g==",
"data" : "vBeHAPJXLl+X/8Enp9vxMA==",
"keySize" : 16,
"purpose" : "someDescription",
"iv" : "N2xCe5RiJibHv9hLY+OduA==",
"mac" : "VoOo1BKptwfqIJeSOb/qGA=="
}
}
}
},
"enabled" : true
}
]
}
}
So the value of "unknownKey8" was encrypted. I need that document to look like this:
{
"unknownKey1" : {
"unknownKey2" : {
"name" : "JWT_SESSION",
"properties" : {
"maxTokenLifeMinutes" : 120,
"tokenIdleTimeMinutes" : 30
}
},
"unknownKey3" : [
{
"unknownKey4" : "STATIC_USER",
"unknownKey5" : {
"unknownKey6" : "internal/user",
"unknownKey7" : "anonymous",
"unknownKey8" : "clearTextValue"
},
"enabled" : true
}
]
}
}
I have been able to find crypto objects in the input file using the following command:
cat input.json | jq 'paths | select(.[-1] == "$crypto")'
[
"unknownKey1",
"unknownKey3",
0,
"unknownKey5",
"unknownKey8",
"$crypto"
]
But I have not been able to make meaningful progress on performing the replacement.
The following performs the replacement described in the text. An invocation along the lines of
jq --arg cleartext "clearTextValue" -f decrypt.jq sample.json
is assumed. If your jq does not have walk/1, then either upgrade to jq 1.6 or include its def before its invocation (google search terms: jq def walk builtin.jq).
# input is assumed to be an object
def decrypt($value):
with_entries(if .value|type == "object"
then with_entries(if .value | (type == "object" and has("$crypto"))
then .value = $value else . end)
else . end) ;
walk(if type == "object" then decrypt($cleartext) else . end)
As an addition to the accepted answer and if using jq v1.5, use this as decrypt.jq
# input is assumed to be an object
def decrypt($value):
with_entries(if .value|type == "object"
then with_entries(if .value | (type == "object" and has("$crypto"))
then .value = $value else . end)
else . end) ;
# walk was added after the release of jq#1.5
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
walk(if type == "object" then decrypt($cleartext) else . end)
I'm trying to use JQ to find the most recent artifact in a Nexus API query. Right now, my JSON output looks something like:
{
"items" : [ {
"downloadUrl" : "https://nexus.ama.org/repository/Snapshots/org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.144121-1.jar",
"path" : "org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.144121-1.jar",
"id" : "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOTY1N2JiOTEyMTM1ZGRjZWQ",
"repository" : "Snapshots",
"format" : "maven2",
"checksum" : {
"sha1" : "7ac324905fb1ff15ef6020f256fcb5c9f54113ca",
"md5" : "bb25c483a183001dfdc58c07a71a98ed"
}
}, {
"downloadUrl" : "https://nexus.ama.org/repository/Snapshots/org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.204941-2.jar",
"path" : "org/sso/browser-manager/1.0-SNAPSHOT/browser-manager-1.0-20180703.204941-2.jar",
"id" : "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOWM4YjQ0NmRjYzFkODkxM2U",
"repository" : "Snapshots",
"format" : "maven2",
"checksum" : {
"sha1" : "b4ba2049ea828391c720f49b6668a66a8b0bca9c",
"md5" : "6757c55c0e6d933dc90e398204cca966"
}
} ],
"continuationToken" : null
}
I've managed to use JQ to repackage the data as:
.items[] | { "id" : .id, "date" : (.path | scan("[0-9]{8}\\.[0-9-]*")) }
output:
{
"id": "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOTY1N2JiOTEyMTM1ZGRjZWQ",
"date": "20180703.144121-1"
}
{
"id": "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOWM4YjQ0NmRjYzFkODkxM2U",
"date": "20180703.204941-2"
}
Now I'm a little stuck trying to figure out which of the two JSON objects is the most recent. How can I sort by date and extract the id for that object?
Is there a better way to filter/sort this data? My example has only 2 items[] in the JSON response, but there may be a larger number of them.
The filter sort_by/1 will sort your timestamps in chronological order, but it requires an array as input, so you could write:
.items
| map({ "id" : .id, "date" : (.path | scan("[0-9]{8}\\.[0-9-]*")) })
| sort_by(.date)
| .[-1]
The trailing .[-1] selects the last item, so with your input the result would be:
{
"id": "V0FEQS1TbmFwc2hvdHM6MzhjZDQ3NTQwMTBkNGJhOWM4YjQ0NmRjYzFkODkxM2U",
"date": "20180703.204941-2"
}
I have a JSON file like this:
{
"images1" :
{
"size" : "29x29",
"idiom" : "iphone",
"filename" : "Icon-Small#2x.png",
"scale" : "2x"
},
"images2" :
{
"size" : "29x30",
"idiom" : "iphone2",
"filename" : "Icon-Small#2x.png2",
"scale" : "22x"
}
}
I will pass JSON object name as an input. So if I know "images1" is the object, then I need all the keys and values of that object to be stored in two separate arrays, so that I can make use of them in further processing.
Any help is greatly appreciated.
Thanks
You can use the following :
jq ".$1 | { keys: keys_unsorted, values: [.[]] }"
where $1 should provide the name of the item you want to address (note that this assume you're using this in a script. You will probably want to use fedorqui's alternative instead).
It will produce an object whose keys property will be an array of the keys of $1 and values an array of the associated values :
$ echo '{
"images1" :
{
"size" : "29x29",
"idiom" : "iphone",
"filename" : "Icon-Small#2x.png",
"scale" : "2x"
},
"images2" :
{
"size" : "29x30",
"idiom" : "iphone2",
"filename" : "Icon-Small#2x.png2",
"scale" : "22x"
}
}' | jq ".images1 | { keys: keys_unsorted, values: [.[]] }"
{
"keys": [
"size",
"idiom",
"filename",
"scale"
],
"values": [
"29x29",
"iphone",
"Icon-Small#2x.png",
"2x"
]
}