I have this output from a pipe
{
"pipelineName": "pipelineName-AAAAQ6UFM",
"pipelineVersion": 2,
"stageStates": [
{
"stageName": "Approval",
"inboundTransitionState": {
"enabled": true
},
"actionStates": [
{
"actionName": "Approval",
"latestExecution": {
"status": "InProgress",
"token": "aaaa-aaaa-4316-a95f-2efc51d05761"
}
}
],
"latestExecution": {
"pipelineExecutionId": "fc73f4cb-c5a9-44a8-8fc1-d7e50259f485",
"status": "InProgress"
}
}
]
}
I am trying to write a json like this
{
"pipelineName": "pipelineName-AAAAQ6UFM",
"stageName": "Approval",
"actionName": "Approval",
"token": "aaaa-aaaa-4316-a95f-2efc51d05761",
"result": {
"status": "Approved",
"summary": ""
}
}
I could maybe set two variables from the pipeoutput with the read command but I don't know how to set both of them.
token
jq -r ' .stageStates[] | select(.stageName == "Approval") | .actionStates[0].latestExecution.token'
pipelineName
jq -r '.pipelineName'
Then I might be able to write the json with the jq command.
What would be the best way to do this ?
Based on your select(.stageName == "Approval"), it would appear that you are attempting to parameterize by the "stageName", so the following might be close to what you're looking for:
"Approval" as $stage
| { pipelineName, stageName: $stage, actionName: $stage }
+ (.stageStates[]
| select(.stageName == $stage).actionStates[]
| select(.actionName == $stage)
| {token: .latestExecution.token, result: {status: "Approved", summary: ""}})
You can use just jq to create the json:
jq ' .stageName = .stageStates[0].stageName
| .actionName = .stageStates[0].actionStates[0].actionName
| .token = .stageStates[0].actionStates[0].latestExecution.token
| .result = { "status": "Approved", "summary": "" }
| del(.stageStates, .pipelineVersion)
' file.json
Related
I am trying to get key#value pairs of JSON file below using jq
{
"STUFF_RELATED1": "STUFF_RELATED1",
"STUFF_RELATED2": "STUFF_RELATED2",
"THINGS": {
"THING_2": {
"details": {
"stuff_branch": "user/dev"
},
"repository": "path/to/repo",
"branch": "master",
"revision": "dsfkes4s34jlis4jsj4lis4sli3"
},
"THING_1": {
"details": {
"stuff_branch": "master"
},
"repository": "path/to/repo",
"branch": "master",
"revision": "dsfkes4s34jlis4jsj4lis4sli3"
}
},
"STUFF": {
"revision": "4u324i324iy32g",
"branch": "master"
}
}
The key#value pair should look like this:
THING_1#dsfkes4s34jlis4jsj4lis4sli3
Currently I have tried this on my own:
jq -r ' .THINGS | keys[] as $k | "($k)#(.[$k].revision)" ' things.json
But it does not give the resul that I really want.:( Thanks in advance!
You need to escape ( :
jq -r ' .THINGS | keys[] as $k | "\($k)#\(.[$k].revision)" ' things.json
(It was hard to come up with a title that summarizes the issue, so feel free to improve it).
I have a JSON file with the following content:
{
"Items": [
{
"ID": {
"S": "ID_Complete"
},
"oldProperties": {
"L": [
{
"S": "[property_A : value_A_old]"
},
{
"S": "[property_B : value_B_old]"
}
]
},
"newProperties": {
"L": [
{
"S": "[property_A : value_A_new]"
},
{
"S": "[property_B : value_B_new]"
}
]
}
},
{
"ID": {
"S": "ID_Incomplete"
},
"oldProperties": {
"L": [
{
"S": "[property_B : value_B_old]"
}
]
},
"newProperties": {
"L": [
{
"S": "[property_A : value_A_new]"
},
{
"S": "[property_B : value_B_new]"
}
]
}
}
]
}
I would like to manipulate the data using jq in such a way that for each item in Items[] that has a new value for property_A (under newProperties list) generate an output with the corresponding id, old and new (see desired output below) fields regardless of the value that property has in the oldProperties list. Moreover, if property_A does not exist in the oldProperties, I still need the old field to be populated with a null (or any fixed string for what it's worth).
Desired output:
{
"id": "id_Complete",
"old": "[property_A : value_A_old]",
"new": "[property_A : value_A_new]"
}
{
"id": "ID_Incomplete",
"old": null,
"new": "[property_A : value_A_new]"
}
Note: Even though property_A doesn't exist in the oldProperties list, other properties may (and will) exist.
The problem I am facing is that I am not able to get an output when the desired property does not exist in the oldProperties list. My current jq command looks like this:
jq -r '.Items[] |
{ id:.ID.S,
old:.oldProperties.L[].S | select(. | contains("property_A")),
new:.newProperties.L[].S | select(. | contains("property_A")) }'
Which renders only the ID_Complete case, while I need the other as well.
Is there any way to achieve this using this tool?
Thanks in advance.
Your list of properties appear to be values of some object. You could map them out into an object to then diff the objects, then report on the results.
You could do something like this:
def make_object_from_properties:
[.L[].S | capture("\\[(?<key>\\w+) : (?<value>\\w+)\\]")]
| from_entries
;
def diff_objects($old; $new):
def _prop($key): select(has($key))[$key];
([($old | keys[]), ($new | keys[])] | unique) as $keys
| [ $keys[] as $k
| ({ value: $old | _prop($k) } // { none: true }) as $o
| ({ value: $new | _prop($k) } // { none: true }) as $n
| (if $o.none then "add"
elif $n.none then "remove"
elif $o.value != $n.value then "change"
else "same"
end) as $s
| { key: $k, status: $s, old: $o.value, new: $n.value }
]
;
def diff_properties:
(.oldProperties | make_object_from_properties) as $old
| (.newProperties | make_object_from_properties) as $new
| diff_objects($old; $new) as $diff
| foreach $diff[] as $d ({ id: .ID.S };
select($d.status != "same")
| .old = ((select(any("remove", "change"; . == $d.status)) | "[\($d.key) : \($d.old)]") // null)
| .new = ((select(any("add", "change"; . == $d.status)) | "[\($d.key) : \($d.new)]") // null)
)
;
[.Items[] | diff_properties]
This yields the following output:
[
{
"id": "ID_Complete",
"old": "[property_A : value_A_old]",
"new": "[property_A : value_A_new]"
},
{
"id": "ID_Complete",
"old": "[property_B : value_B_old]",
"new": "[property_B : value_B_new]"
},
{
"id": "ID_Incomplete",
"old": null,
"new": "[property_A : value_A_new]"
},
{
"id": "ID_Incomplete",
"old": "[property_B : value_B_old]",
"new": "[property_B : value_B_new]"
}
]
It seems like your data is in some kind of encoded format too. For a more robust solution, you should consider defining some functions to decode them. Consider approaches found here on how you could do that.
This filter produces the desired output.
def parse: capture("(?<key>\\w+)\\s*:\\s*(?<value>\\w+)") ;
def print: "[\(.key) : \(.value)]";
def norm: [.[][][] | parse | select(.key=="property_A") | print][0];
.Items
| map({id:.ID.S, old:.oldProperties|norm, new:.newProperties|norm})[]
Sample Run (assumes filter in filter.jq and data in data.json)
$ jq -M -f filter.jq data.json
{
"id": "ID_Complete",
"old": "[property_A : value_A_old]",
"new": "[property_A : value_A_new]"
}
{
"id": "ID_Incomplete",
"old": null,
"new": "[property_A : value_A_new]"
}
Try it online!
I've been trying to use jq parser to help me extract information from json files.
Here is an example snippet
{
"main_attribute": {
"name": {
"display_name": "abc"
},
"address": {
"unit": "1",
"street": "Dundas",
"suburb": "Syd",
"state": "NSW"
},
"financial_debt": {
"bank_loan": true
}
},
"secondary_attr": {
"income": {
"pretax": 100000
},
"automobile": {
"make": "Citroen",
"model": 2015,
"new": true
},
"property": {
"property_owned": 1,
"owned_since": 2000,
"first_sale": true
},
"education": {
"degree": "MS",
"graduated": 1990,
"financial_debt": {
"bank_loan": false
}
}
}
}
I need to find the blocks where "financial_debt" is true. This field could be either in the main_attribute (as a global value) or in the secondary attribute.
Expected output:
financial_debt: bank_loan on "automobile" and "property"
Can you please advise how to go about doing this search using jq?
This is by no means the most efficient way, but it is functional. It returns a boolean value specifying whether or not there is a true boolean value under the financial_debt property.
jq '[recurse | .financial_debt? | select(. != null) | recurse | booleans] | any'
tostream can be used to find paths containing "financial_debt" as follows:
tostream
| select(length==2)
| select(.[0] | contains(["financial_debt"]))
with this filter in filter.jq and data in data.json
$ jq -M -c -f filter.jq data.json
produces
[["main_attribute","financial_debt","bank_loan"],true]
[["secondary_attr","education","financial_debt","bank_loan"],false]
This intermediate result can be used along with reduce, setpath, getpath and a filter such as
. as $d
| reduce ( tostream
| select(length==2)
| select(.[0] | contains(["financial_debt"]))) as [$p,$v] (
{}
; setpath($p[:-1]; $d | getpath($p[:-1]))
)
to produce
{
"main_attribute": {
"financial_debt": {
"bank_loan": true
}
},
"secondary_attr": {
"education": {
"financial_debt": {
"bank_loan": false
}
}
}
}
I have a json of this structure:
{
"nodes": {
"60e327ee58a0": {
"nodeinfo": {
"network": {
"mesh": {
"bat0": {
"interfaces": {
"wireless": [
"<mac-address-removed>"
],
"tunnel": [
"<mac-address-removed>"
]
}
}
},
"mac": "<mac removed>",
"addresses": [
"<ipv6 removed>",
"<ipv6 removed>"
]
},
"hardware": {
"model": "TP-Link TL-WR841N/ND v10",
"nproc": 1
},
"software": {
"batman-adv": {
"compat": 15,
"version": "2015.1"
},
"autoupdater": {
"branch": "stable",
"enabled": true
},
"firmware": {
"release": "v2016.1+1.0.1",
"base": "gluon-v2016.1"
},
"status-page": {
"api": 1
},
"fastd": {
"enabled": true,
"version": "v17"
}
},
"hostname": "Antoniusweg12",
"system": {
"site_code": "ffmsd03"
},
"node_id": "60e327ee58a0"
},
"lastseen": "2016-04-14T12:39:04",
"flags": {
"gateway": false,
"online": true
},
"firstseen": "2016-03-16T15:14:04",
"statistics": {
"clients": 1,
"gateway": "de:ad:be:ef:43:02",
"rootfs_usage": 0.6041666666666667,
"loadavg": 0.09,
"uptime": 1822037.41,
"memory_usage": 0.8124737210932025,
"traffic": {
"rx": {
"packets": 50393821,
"bytes": 5061895206
},
"forward": {
"packets": 173,
"bytes": 17417
},
"mgmt_rx": {
"packets": 47453745,
"bytes": 6623785282
},
"tx": {
"packets": 1205695,
"bytes": 173509528,
"dropped": 5683
},
"mgmt_tx": {
"packets": 37906725,
"bytes": 11475209742
}
}
}
},
"30b5c2b042f4": {
<next block...>
And I want to query it with jq for the hostname, the mac or the IPv6.
cat nodes.json |jq -c '.nodes[] | select(.nodes[]| contains("Antoniusweg12"))'
Most examples do not fit this kind of json structure as the objects have an index
Thanks for help in advance.
If you're going to filter, you need to drill down to the property that you want to check for and see if it matches your criteria. You can't expect to just give a name and you'll magically be presented with the results you want.
Searching by hostname, it is found on the .nodeinfo.hostname property of each node:
$ jq -c --arg hostname "Antoniusweg12" \
'.nodes[] | select(.nodeinfo.hostname == $hostname)' nodes.json
Similarly for the mac address, it's found on the .nodeinfo.network.mac property:
$ jq -c --arg mac "aa:bb:cc:dd:ee:ff" \
'.nodes[] | select(.nodeinfo.network.mac == $mac)' nodes.json
For the ip addresses, there's an array of them but it's not that much different in the query. They're found on the .nodeinfo.network.addresses property:
$ jq -c --arg ip "aaaa:bbbb:cccc:dddd::1" \
'.nodes[] | select(.nodeinfo.network.addresses[] == $ip)' nodes.json
Here's another take on the question. Suppose you want to find all occurrences of the key "hostname" for which the value is "Antoniusweg12",
no matter where the key/value combination occurs.
The following will reveal the path to the key/value combination of interest:
paths as $p
| select ( $p[-1] == "hostname" and getpath($p) == "Antoniusweg12" )
| $p
The result for the given input JSON:
[
"nodes",
"60e327ee58a0",
"nodeinfo",
"hostname"
]
If you wanted the path to the containing object, then replace the final $p with $p[0:-1]; and if you want the containing object itself: getpath($p[0:-1])
Here is a solution which searches for nodes where the specified $needle is present in any of the addresses, mac or hostname fields.
"<ipv6 removed>" as $needle # set to whatever you like
| foreach (.nodes|keys[]) as $k (
.
; .
; ( .nodes[$k].nodeinfo.network.addresses?
+ [ .nodes[$k].nodeinfo.network.mac?
, .nodes[$k].nodeinfo.hostname?
]
) as $haystack
| if $haystack | index($needle)
then {($k): .nodes[$k]}
else empty
end
)
EDIT: I now realize a filter of the form foreach E as $X (.; .; R) can almost always be rewritten as E as $X | R so the above is really just
"<ipv6 removed>" as $needle
| (.nodes|keys[]) as $k
| ( .nodes[$k].nodeinfo.network.addresses?
+ [ .nodes[$k].nodeinfo.network.mac?
, .nodes[$k].nodeinfo.hostname?
]
) as $haystack
| if $haystack | index($needle)
then {($k): .nodes[$k]}
else empty
end
I have this input:
{
"users": [
{
"name": "tester-01",
"user": {
"username": "tester01"
}
},
{
"name": "tester-02",
"user": {
"username": "tester02"
}
}
],
"current-user": "tester-02"
}
Using jq (1.5), I want to print the username that matches current-user.
Can anyone share how this would be done or tips to get me started?
This was my solution.
cat 99 | jq -r '."current-user" as $foo | .users[] | select(.name == $foo).user'
{
"username": "tester02"
}
The select solution is good. Here is an alternative that uses foreach.
foreach .users[] as $u (
."current-user"
; .
; if . == $u.name then $u.user.username else empty end
)