Need help in parsing Json data using jq , I used to parse the data using json path as [?(#.type=='router')].externalIP. I am not sure how to do the same using jq.
The result from the query should provide the .externalIp from the type=router.
198.22.66.99
Json data snippet as below
[
{
"externalHostName": "localhost",
"externalIP": "198.22.66.99",
"internalHostName": "localhost",
"isUp": true,
"pod": "gateway",
"reachable": true,
"region": "dc-1",
"type": [
"router"
],
"uUID": "b5f986fe-982e-47ae-8260-8a3662f25fc2"
},
]
##
cat your-data.json | jq '.[]|.externalIP|select(type=="string")'
"198.22.66.99"
"192.22.66.29"
"192.22.66.89"
"192.66.22.79"
explanation:
.[] | .externalIP | select(type=="string")
for every array entry | get field 'externalIP' | drop nulls
EDIT/ADDENDUM: filter on type (expects router to be on index 0 of type array)
cat x | jq '.[]|select(.type[0] == "router")|.externalIP'
"198.22.66.99"
"192.22.66.89"
The description:
i would like to extract externalIP for only the array "type": [ "router" ]
The corresponding jq query is:
.[] | select(.type==["router"]) | .externalIP
To base the query on whether "router" is amongst the specified types:
.[] | select(.type|index("router")) | .externalIP
Related
In the JSON below, I need to find the value of the id key of every object where the value of state starts with "failed-"
[
{
"id": "RA_kwDOGETrS84EmTf2",
"state": "uploaded"
},
{
"id": "RA_kwDOGETrS84EmTf6",
"state": "failed-4325423"
},
{
"id": "RA_kwDOGETrS84EmTf7",
"state": "uploaded"
}
]
I got as far as extracting just the matching values of state:
.[] | .state | select(startswith("failed-"))
How do I find the corresponding values of id ?
.[] | select(.state | startswith("failed-")).id
Will output:
"RA_kwDOGETrS84EmTf6"
Trick is to pass state in the select() to startswith(), and then get .id of the result
Demo
.[] | select(.state | test("^failed")).id
is another way
I am trying to convert JSON to CSV for selected keys using jq.
file.json
{
"_ref": "ipv4address/Li5pcHY0X2FkZHJlc3yMDIuMS8w:10.202.202.1",
"discovered_data": {
"bgp_as": 64638,
"device_model": "catalyst37xxStack",
"device_port_name": "Vl2002",
"device_port_type": "propVirtual",
"device_type": "Switch-Router",
"device_vendor": "Cisco",
"discovered_name": "Test_Device.network.local",
"discoverer": "Network Insight",
"first_discovered": 1580161888,
"last_discovered": 1630773758,
"mac_address": "aa:bb:cc:dd:ee:ff",
"mgmt_ip_address": "10.202.202.1",
"os": "15.2(4)E10",
"port_speed": "Unknown",
"port_vlan_name": "TEST-DATA",
"port_vlan_number": 2002
},
"ip_address": "10.202.202.1",
"is_conflict": false,
"mac_address": "",
"names": ["Test_Device"],
"network": "10.202.202.0/23",
"network_view": "TEST VIEW",
"objects": [],
"status": "USED",
"types": [
"UNMANAGED"
],
"usage": []
}
my desired output is:
names,ip_address,discovered_data.mac_address,discovered_data.discovered_name
Test_Device,10.202.202.1,aa:bb:cc:dd:ee:ff,Test_Device.network.local
So far, I have tried using following command but getting some syntax error:
jq -r 'map({names,ip_address,discovered_data.mac_address,discovered_data.discovered_name}) | (first | keys_unsorted) as $keys | map([to_entries[] | .value]) as $rows | $keys,$rows[] | #csv' < file.json
Assuming the JSON has been fixed, consider the output of:
(null
| {names,
ip_address,
"discovered_data.mac_address",
"discovered_data.discovered_name"} | keys_unsorted) as $keys
| $keys,
({names: .names[],
ip_address,
"discovered_data.mac_address": .discovered_data.mac_address,
"discovered_data.discovered_name": .discovered_data.discovered_name }
| [.[]])
| #csv
Assuming jq is invoked with the -r command-line option, this has the advantage of producing valid CSV. If you prefer to have all the key names and values unquoted, you might wish to consider using join(",") instead of #csv, or some more sophisticated variation if you want to have your cake and eat it.
This may just not be possible due to how conceptually streaming/filtering JSON works, but let's suppose I have something like the following JSON:
[
{
"name": "account_1",
"type": "account"
},
{
"name": "account_2",
"type": "account"
},
{
"name": "user_1",
"type": "user"
},
{
"name": "user_2",
"type": "user"
}
]
And now I want to print out only the user objects.
I know I can filter to just the streaming type entities with something like this:
cat file.json | jq --stream 'select(.[0][1] == "type" and .[1] == "user" | .)'
Which would produce:
[
[
2,
"type"
],
"user"
]
[
[
3,
"type"
],
"user"
]
Is there any way I can print out the parent objects of those types instead of the type entities? E.g. I'd like to get out:
[
{
"name": "user_1",
"type": "user"
},
{
"name": "user_2",
"type": "user"
}
]
Without streaming, this is a pretty straightforward exercise. E.g.:
cat file.json | jq '.[] | select(.type=="user")'
In reality the actual input file is around 5GB, so I need to use streaming input, but I can't seem to get the jq syntax right with --stream enabled. E.g.
cat file.json | jq --stream '.[] | select(.type=="user")'
Produces:
jq: error (at <stdin>:3): Cannot index array with string "type"
jq: error (at <stdin>:5): Cannot index array with string "type"
...
(edited to include desired output)
Just truncate the top-level array.
jq -n --stream 'fromstream(1 | truncate_stream(inputs)) | select(.type == "user")'
Online demo
jqplay does not support the --stream option, so the above demo has the output of --stream as the JSON input.
I have the following JSON data:
{
"status": "ok",
"ok": true,
"data": "MFR-L",
"stores": [{
"name": "KOLL",
"lat": 52.93128,
"lng": 6.962956,
"dist": 1,
"x10": 1.129,
"isOpen": true
},
{
"name": "Takst",
"lat": 52.9523773,
"lng": 6.981644,
"dist": 1.3,
"x10": 1.809,
"isOpen": false
}]
}
I'm trying to convert it to a flat file using JQ, but I keep running into all sorts of problems, especially because of the file types ("cannot index boolean with string", etc).
This post has helped me flatten the contents of the array so far, like this:
jq -r -s 'map(.stores | map({nm: .name, lt: .lat} | [.nm, .lt])) | add [] | #csv
How can I get the contents higher up in the hierarchy to map to the array contents?
You could always collect the values you want from the parent objects separately from the child objects and combine them later.
e.g.,
$ jq -r '[.data] + (.stores[] | [.name, .lat, .lng, .dist]) | #csv' input.json
yields
"MFR-L","KOLL",52.93128,6.962956,1
"MFR-L","Takst",52.9523773,6.981644,1.3
There are several ways in which the illustrative JSON might be "flattened" (e.g. to CSV), but the following two approaches may be of interest. (I've omitted the invocation of #csv for ease-of-reading.)
$ jq '[.data, .stores[][]]' in.json
[
"MFR-L",
"KOLL",
52.93128,
6.962956,
1,
1.129,
true,
"Takst",
52.9523773,
6.981644,
1.3,
1.809,
false
]
$ jq '.data as $data | .stores[] | [$data, .[]]' in.json
[
"MFR-L",
"KOLL",
52.93128,
6.962956,
1,
1.129,
true
]
[
"MFR-L",
"Takst",
52.9523773,
6.981644,
1.3,
1.809,
false
]
Here is another approach which uses jq variables and string interpolation:
.data as $data
| .stores[]
| "\($data),\(.name),\(.lat),\(.lng),\(.dist),\(.x10),\(.isOpen)"
output with sample data:
"MFR-L,KOLL,52.93128,6.962956,1,1.129,true"
"MFR-L,Takst,52.9523773,6.981644,1.3,1.809,false"
I have the following json file:
{
"FOO": {
"name": "Donald",
"location": "Stockholm"
},
"BAR": {
"name": "Walt",
"location": "Stockholm"
},
"BAZ": {
"name": "Jack",
"location": "Whereever"
}
}
I am using jq and want to get the "name" elements of the objects where 'location' is 'Stockholm'.
I know I can get all names by
cat json | jq .[] | jq ."name"
"Jack"
"Walt"
"Donald"
But I can't figure out how to print only certain objects, given the value of a sub key (here: "location" : "Stockholm").
Adapted from this post on Processing JSON with jq, you can use the select(bool) like this:
$ jq '.[] | select(.location=="Stockholm")' json
{
"location": "Stockholm",
"name": "Walt"
}
{
"location": "Stockholm",
"name": "Donald"
}
To obtain a stream of just the names:
$ jq '.[] | select(.location=="Stockholm") | .name' json
produces:
"Donald"
"Walt"
To obtain a stream of corresponding (key name, "name" attribute) pairs, consider:
$ jq -c 'to_entries[]
| select (.value.location == "Stockholm")
| [.key, .value.name]' json
Output:
["FOO","Donald"]
["BAR","Walt"]
I had a similar related question: What if you wanted the original object format back (with key names, e.g. FOO, BAR)?
Jq provides to_entries and from_entries to convert between objects and key-value pair arrays. That along with map around the select
These functions convert between an object and an array of key-value
pairs. If to_entries is passed an object, then for each k: v entry in
the input, the output array includes {"key": k, "value": v}.
from_entries does the opposite conversion, and with_entries(foo) is a
shorthand for to_entries | map(foo) | from_entries, useful for doing
some operation to all keys and values of an object. from_entries
accepts key, Key, name, Name, value and Value as keys.
jq15 < json 'to_entries | map(select(.value.location=="Stockholm")) | from_entries'
{
"FOO": {
"name": "Donald",
"location": "Stockholm"
},
"BAR": {
"name": "Walt",
"location": "Stockholm"
}
}
Using the with_entries shorthand, this becomes:
jq15 < json 'with_entries(select(.value.location=="Stockholm"))'
{
"FOO": {
"name": "Donald",
"location": "Stockholm"
},
"BAR": {
"name": "Walt",
"location": "Stockholm"
}
}
Just try this one as a full copy paste in the shell and you will grasp it.
# pass the multiline string to the jq, use the jq to
# select the attribute named "card_id"
# ONLY if its neighbour attribute
# named "card_id_type" has the "card_id_type-01" value.
# jq -r means give me ONLY the value of the jq query no quotes aka raw
cat << EOF | \
jq -r '.[]| select (.card_id_type == "card_id_type-01")|.card_id'
[
{ "card_id": "id-00", "card_id_type": "card_id_type-00"},
{ "card_id": "id-01", "card_id_type": "card_id_type-01"},
{ "card_id": "id-02", "card_id_type": "card_id_type-02"}
]
EOF
# this ^^^ MUST start first on the line - no whitespace there !!!
# outputs:
# id-01
or with an aws cli command
# list my vpcs or
# list the values of the tags which names are "Name"
aws ec2 describe-vpcs | jq -r '.| .Vpcs[].Tags[]
|select (.Key == "Name") | .Value'|sort -nr
Note that you could move up and down in the hierarchy both during the filtering phase and during the selecting phase :
kubectl get services --all-namespaces -o json | jq -r '
.items[] | select( .metadata.name
| contains("my-srch-string")) |
{ name: .metadata.name, ns: .metadata.namespace
, nodePort: .spec.ports[].nodePort
, port: .spec.ports[].port}
'