I am trying to transform the map of the object in json to List using JQ ...
Source:
{
"title": "title",
"keyword": "keyword",
"desc": {
"user1": {
"name": "kumar",
"Duration": null,
"Time": null,
" Dominance": "Upper Field First"
},
"user2": {
"id": "user2",
"name": "user2",
"Duration": null,
"Time": null,
" Dominance": "Upper Field First"
}
}
}
Target:
[
{
"id": "user1",
"name": "kumar",
"Duration": null,
"Time": null,
"Dominance": "Upper Field First"
},
{
"id": "user2",
"name": "user2",
"Duration": null,
"Time": null,
"Dominance": "Upper Field First"
}
]
I tried various options like , but not able to get the extact thing i need.
.desc | . as $in| keys[]
to_entries
JQ Workspace ::: https://jqplay.org/s/MiJ9w1Sz5L
The following filter will do what you want, assuming you've fixed the input:
.desc
| to_entries
| map( {id: .key} + .value)
To understand this, simply read up on to_entries in the online manual (see jq).
Related
Guy, long term looking at this board and learning a lot but now stuck with little issue. Im working with Linux shell script that reads json (no problem here). What Im trying to do is get value from entry that has specific Type.
By parsing a json with just jq -r '.', I get
{
"records": [
{
"id": 01,
"type": "SOA",
"name": "#",
"data": "1800",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
},
{
"id": 02,
"type": "A",
"name": "#",
"data": "test.com",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
}
],
"links": {},
"meta": {
"total": 2
}
}
Then, I use "jq -r '.records' " and get:
[
{
"id": 01,
"type": "SOA",
"name": "#",
"data": "1800",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
},
{
"id": 02,
"type": "A",
"name": "#",
"data": "test.com",
"priority": null,
"port": null,
"ttl": 1800,
"weight": null,
"flags": null,
"tag": null
}
]
What I need to do is get data value of the type A. Currently, we have type SOA and A, but I need to only get data from A.
I can use dummy way of "jq -r '.records[1].data'" and it gives me correct response of test.com, but I want a more dynamic way of searching for specific type (in this case "A") and then giving the data value.
Thanks guys!
Is the field called domain_records or just records?
Use select to match your criteria
jq -r '.records[] | select(.type == "A").data'
test.com
Demo
I have the following simplified json structure: Notice an array of values, which have children, whose children could have children.
{
"value": [
{
"id": "12",
"text": "Beverages",
"state": "closed",
"attributes": null,
"iconCls": null
},
{
"id": "10",
"text": "Foods",
"state": "closed",
"attributes": null,
"iconCls": null,
"children": [
{
"id": "33",
"text": "Mexican",
"state": "closed",
"attributes": null,
"iconCls": null,
"children": [
{
"id": "6100",
"text": "Taco",
"count": "3",
"attributes": null,
"iconCls": ""
}
]
}
]
}
]
}
How do I flatten a json structure using jq? I would like to print each element just once, but in a flat structure. An example output:
{
"id": "12",
"category": "Beverages"
},
{
"id": "10",
"category": "Foods"
},
{
"id": "33",
"category": "Mexican"
},
{
"id": "6100",
"category": "Tacos"
}
My attempt doesn't seem to work at all:
cat simple.json - | jq '.value[] | {id: .id, category: .text} + {id: .children[]?.id, category: .children[]?.text}'
.. is your friend:
.. | objects | select( .id and .text) | {id, category: .text}
If your actual input is that simple, recursively extracting id and text from each object under value should work.
[ .value | recurse | objects | {id, category: .text} ]
Online demo
I was totally going in the wrong direction
Not really. Going in that direction, you would have something like:
.value[]
| recurse(.children[]?)
| {id, category: .text}
I have looked at many examples, but I cannot get my code to work. The structure of this JSON seems to be different than any others I've seen. I need to iterate through these results and get "id" and "legalName".
[
"URL: https://myurl.com/rest/api",
{
"value": [
{
"id": 271,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "01",
"legalName": "Test1",
"naicsCode": null,
"status": "Active",
"links": " "
},
{
"id": 272,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "02",
"legalName": "Test2",
"naicsCode": null,
"status": "Terminated",
"links": " "
}
],
"Count": 6
}
]
I've tried lots of different code examples, but none of them have worked. This returns nothing. I can't seem to get down to the properties I need.
$var = Get-RestData("https://myurl.com/rest/api/") | ConvertTo-Json
$var["value"] | select id
You can use the ConvertFrom-Json cmdlet to convert from a JSON formatted string to a PSCustomObject with which you can iterate over.
$json = #"
[
"URL: https://myurl.com/rest/api",
{
"value": [
{
"id": 271,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "01",
"legalName": "Test1",
"naicsCode": null,
"status": "Active",
"links": " "
},
{
"id": 272,
"client": "#{id=245; clientCode=; clientName=}",
"dbaName": null,
"federalEIN": "",
"legalCode": "02",
"legalName": "Test2",
"naicsCode": null,
"status": "Terminated",
"links": " "
}
],
"Count": 6
}
]
"#
$json | ConvertFrom-Json | ForEach-Object { $_.value } | Select id, legalName
which returns:
id legalName
-- ---------
271 Test1
272 Test2
I am trying to flatten a JSON file by using jq command. But the output got duplicated.
Please see my jqplay here: https://jqplay.org/s/gwvMIH_fed
My input JSON:
{
"cost": 0.0,
"interval": "0:01:00",
"namespace": "Microsoft.ApiManagement/service",
"resourceregion": "australiaeast",
"timespan": "2019-05-22T00:00:00Z/2019-05-22T00:02:00Z",
"value": [
{
"id": "/my-api/providers/Microsoft.Insights/metrics/Capacity",
"name": {
"localizedValue": "Capacity",
"value": "Capacity"
},
"resourceGroup": "my-group",
"timeseries": [
{
"data": [
{
"average": 15,
"count": null,
"maximum": null,
"minimum": null,
"timeStamp": "2019-05-22T00:00:00+00:00",
"total": null
},
{
"average": 16,
"count": null,
"maximum": null,
"minimum": null,
"timeStamp": "2019-05-22T00:01:00+00:00",
"total": null
}
],
"metadatavalues": []
}
],
"type": "Microsoft.Insights/metrics",
"unit": "Percent"
}
]
}
My expected output:
{
"apiId": "/my-api/providers/Microsoft.Insights/metrics/Capacity",
"metrics": "Capacity",
"timestamp": "2019-05-22T00:00:00+00:00",
"value": 15
}
{
"apiId": "/my-api/providers/Microsoft.Insights/metrics/Capacity",
"metrics": "Capacity",
"timestamp": "2019-05-22T00:01:00+00:00",
"value": 16
}
Could anyone please have a look at the provided jqplay URL and advise.
Each .[] is like a "for" loop, so the multiplicative behavior you observe is essentially the result of having nested for loops. It would seem that what you want is closer to:
.value[] as $v
| $v.timeseries[].data[] as $d
| {"apiId": $v.id,
"metrics": $v.name.value,
"timestamp": $d.timeStamp,
"value": $d.average }
With your JSON as input, this produces two JSON objects, though the second of these differs very slightly from what you give as the expected output.
I have a deep json. Sometimes, I need to look for the json path for a key containing certain word.
{
"apiVersion": "v1",
"kind": "Pod",
"metadata": {
"creationTimestamp": "2019-03-28T21:09:42Z",
"labels": {
"bu": "finance",
"env": "prod"
},
"name": "auth",
"namespace": "default",
"resourceVersion": "2786",
"selfLink": "/api/v1/namespaces/default/pods/auth",
"uid": "ce73565a-519d-11e9-bcb7-0242ac110009"
},
"spec": {
"containers": [
{
"command": [
"sleep",
"4800"
],
"image": "busybox",
"imagePullPolicy": "Always",
"name": "busybox",
"resources": {},
"terminationMessagePath": "/dev/termination-log",
"terminationMessagePolicy": "File",
"volumeMounts": [
{
"mountPath": "/var/run/secrets/kubernetes.io/serviceaccount",
"name": "default-token-dbpcm",
"readOnly": true
}
]
}
],
"dnsPolicy": "ClusterFirst",
"nodeName": "node01",
"priority": 0,
"restartPolicy": "Always",
"schedulerName": "default-scheduler",
"securityContext": {},
"serviceAccount": "default",
"serviceAccountName": "default",
"terminationGracePeriodSeconds": 30,
"tolerations": [
{
"effect": "NoExecute",
"key": "node.kubernetes.io/not-ready",
"operator": "Exists",
"tolerationSeconds": 300
},
{
"effect": "NoExecute",
"key": "node.kubernetes.io/unreachable",
"operator": "Exists",
"tolerationSeconds": 300
}
],
"volumes": [
{
"name": "default-token-dbpcm",
"secret": {
"defaultMode": 420,
"secretName": "default-token-dbpcm"
}
}
]
},
"status": {
"conditions": [
{
"lastProbeTime": null,
"lastTransitionTime": "2019-03-28T21:09:42Z",
"status": "True",
"type": "Initialized"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2019-03-28T21:09:50Z",
"status": "True",
"type": "Ready"
},
{
"lastProbeTime": null,
"lastTransitionTime": null,
"status": "True",
"type": "ContainersReady"
},
{
"lastProbeTime": null,
"lastTransitionTime": "2019-03-28T21:09:42Z",
"status": "True",
"type": "PodScheduled"
}
],
"containerStatuses": [
{
"containerID": "docker://b5be8275555ad70939401d658bb4e504b52215b70618ad43c2d0d02c35e1ae27",
"image": "busybox:latest",
"imageID": "docker-pullable://busybox#sha256:061ca9704a714ee3e8b80523ec720c64f6209ad3f97c0ff7cb9ec7d19f15149f",
"lastState": {},
"name": "busybox",
"ready": true,
"restartCount": 0,
"state": {
"running": {
"startedAt": "2019-03-28T21:09:49Z"
}
}
}
],
"hostIP": "172.17.0.37",
"phase": "Running",
"podIP": "10.32.0.4",
"qosClass": "BestEffort",
"startTime": "2019-03-28T21:09:42Z"
}
}
Currently If i need the podIP, then I do that this way to find the object which has the search keyword and then I build the path
curl myson | jq "[paths]" | grep "IP" --context=10
Is there any nice shortcut to simplify this? What I really need is - all the paths which could have the matching key.
spec.podIP
spec.hostIP
select paths containing keyword in their last element, and use join(".") to generate your desired output.
paths
| select(.[-1] | type == "string" and contains("keyword"))
| join(".")
.[-1] returns the last element of an array,
type == "string" is required because an array index is a number and numbers and strings can't be checked for their containment.
You may want to specify -r option.
As #JeffMercado implicitly suggested you can set the query from command line without touching the script:
jq -r 'paths
| select(.[-1] | type == "string" and contains($q))
| join(".")' file.json --arg q 'keyword'
You can stream the input in, which provides paths and values. You could then inspect the paths and optionally output the values.
$ jq --stream --arg pattern 'IP' '
select(length == 2 and any(.[0][] | strings; test($pattern)))
| "\(.[0] | join(".")): \(.[1])"
' input.json
"status.hostIP: 172.17.0.37"
"status.podIP: 10.32.0.4"
shameless plug
https://github.com/TomConlin/json_to_paths
because sometime you do not even know the component you want to filter for before you see what is there.
json2jqpath.jq file.json
.
.apiVersion
.kind
.metadata
.metadata|.creationTimestamp
.metadata|.labels
.metadata|.labels|.bu
.metadata|.labels|.env
.metadata|.name
.metadata|.namespace
.metadata|.resourceVersion
.metadata|.selfLink
.metadata|.uid
.spec
.spec|.containers
.spec|.containers|.[]
.spec|.containers|.[]|.command
.spec|.containers|.[]|.command|.[]
.spec|.containers|.[]|.image
.spec|.containers|.[]|.imagePullPolicy
.spec|.containers|.[]|.name
.spec|.containers|.[]|.resources
.spec|.containers|.[]|.terminationMessagePath
.spec|.containers|.[]|.terminationMessagePolicy
.spec|.containers|.[]|.volumeMounts
.spec|.containers|.[]|.volumeMounts|.[]
.spec|.containers|.[]|.volumeMounts|.[]|.mountPath
.spec|.containers|.[]|.volumeMounts|.[]|.name
.spec|.containers|.[]|.volumeMounts|.[]|.readOnly
.spec|.dnsPolicy
.spec|.nodeName
.spec|.priority
.spec|.restartPolicy
.spec|.schedulerName
.spec|.securityContext
.spec|.serviceAccount
.spec|.serviceAccountName
.spec|.terminationGracePeriodSeconds
.spec|.tolerations
.spec|.tolerations|.[]
.spec|.tolerations|.[]|.effect
.spec|.tolerations|.[]|.key
.spec|.tolerations|.[]|.operator
.spec|.tolerations|.[]|.tolerationSeconds
.spec|.volumes
.spec|.volumes|.[]
.spec|.volumes|.[]|.name
.spec|.volumes|.[]|.secret
.spec|.volumes|.[]|.secret|.defaultMode
.spec|.volumes|.[]|.secret|.secretName
.status
.status|.conditions
.status|.conditions|.[]
.status|.conditions|.[]|.lastProbeTime
.status|.conditions|.[]|.lastTransitionTime
.status|.conditions|.[]|.status
.status|.conditions|.[]|.type
.status|.containerStatuses
.status|.containerStatuses|.[]
.status|.containerStatuses|.[]|.containerID
.status|.containerStatuses|.[]|.image
.status|.containerStatuses|.[]|.imageID
.status|.containerStatuses|.[]|.lastState
.status|.containerStatuses|.[]|.name
.status|.containerStatuses|.[]|.ready
.status|.containerStatuses|.[]|.restartCount
.status|.containerStatuses|.[]|.state
.status|.containerStatuses|.[]|.state|.running
.status|.containerStatuses|.[]|.state|.running|.startedAt
.status|.hostIP
.status|.phase
.status|.podIP
.status|.qosClass
.status|.startTime