Create one JSON from another, with extra data va JQ - json

I have this file:
[
"smoke-tests",
"push-apps-manager"
]
I'd like to get this output using JQ:
{
"errands": [
{"name": "smoke-tests", "post_deploy": true},
{"name": "push-apps-manager", "post_deploy": true}
]
}
It seems so simple, yet, I have so much difficulty here...

It's a little tricky, since you need to embed the input into the list bound to the errands key. Start by creating the sequence of name/post_deploy objects:
% jq '.[] | {name: ., post_deploy: true}' names.json
{
"name": "smoke-tests",
"post_deploy": true
}
{
"name": "push-apps-manager",
"post_deploy": true
}
Then wrap that in the list in the outer object:
% jq '{errands: [.[] | {name: ., post_deploy: true}]}' names.json
{
"errands": [
{
"name": "smoke-tests",
"post_deploy": true
},
{
"name": "push-apps-manager",
"post_deploy": true
}
]
}
You can also use the map function (which I rarely remember how to use correctly, but it turns out is pretty simple here):
% jq '{errands: map({name:., post_deploy: true})}' names.json

Here is another approach. If you are new to jq it may be easiest to work towards the goal in small steps.
1) Start with the identity filter
.
which produces as expected
[
"smoke-tests",
"push-apps-manager"
]
2) next add the outer object with the "errands" key:
{ "errands": . }
which produces
{
"errands": [
"smoke-tests",
"push-apps-manager"
]
}
3) next move the data into an array
{ "errands": [ . ] }
which produces
{
"errands": [
[
"smoke-tests",
"push-apps-manager"
]
]
}
4) add the inner object with the "name" and "post_deploy" keys
{ "errands": [ { "name": ., "post_deploy": true } ] }
which produces
{
"errands": [
{
"name": [
"smoke-tests",
"push-apps-manager"
],
"post_deploy": true
}
]
}
5) Now we're really close. All we need to do is take advantage of jq's Object Construction behavior when an expression produces multiple results :
{ "errands": [ { "name": .[], "post_deploy": true } ] }
which gives us the desired result
{
"errands": [
{
"name": "smoke-tests",
"post_deploy": true
},
{
"name": "push-apps-manager",
"post_deploy": true
}
]
}

Related

JQ get data from an array of array

How can I get a value from an array of arrays. This is JSON from which I have to get the data:
{
"status": "success",
"data": {
"resultType": "matrix",
"result": [
{
"metric": {
"myMetric": "ABC"
},
"values": [
[
1633597734,
"64.54166666666667"
],
[
1633598034,
"65.51666666666667" <-- wanted value
]
]
}
]
}
}
I have to get value from the last array in values.
I tried the following:
jq .data.result.values input.json
jq .data.result.values[] input.json
jq .data.result.values[][] input.json
For every one the result is:
jq: error (at json:0): Cannot index array with string "values"
How can I get value from the last array in values?
You can use a negative index to enumerate from the back of an array: to get the last element of the last array in the values of the last element of result:
.data.result[-1].values[-1][-1]
demo: https://jqplay.org/s/YMpB8-A4-Q
filter:
.data.result[].values[1][1]?
input:
{
"status": "success",
"data": {
"resultType": "matrix",
"result": [
{
"metric": {
"myMetric": "ABC"
},
"values": [
[
1633597734,
"64.54166666666667"
],
[
1633598034,
"65.51666666666667"
]
]
}
]
}
}
output:
"65.51666666666667"

jq update json document to alter an array element

I know this has to be simple, but for some reason it's eluding me how to find an element given a condition and modify one of its fields. The doc should be fully output (sed style) with the edit made.
{
"state": "wait",
"steps": {
"step1": [
{ "name":"Foo", "state":"wait" },
{ "name":"Bar", "state":"wait" }
],
"step2": [
{ "name":"Foo", "state":"wait" },
{ "name":"Zoinks", "state":"ready" }
],
"step3": [
{ "name":"Foo", "state":"cancel" }
]
}
}
I'm expecting something like this should be workable.
jq '. | (select(.steps[][].name=="Foo" and .steps[][].state=="wait") |= . + {.state:"Ready"}'
or
jq '. | (select(.steps[][]) | if (.name=="Foo" and .state=="wait") then (.state="Ready") else . end)
The desired output, of course, would be
{
"state": "wait",
"steps": {
"step1": [
{ "name":"Foo", "state":"ready" },
{ "name":"Bar", "state":"wait" }
],
"step2": [
{ "name":"Foo", "state":"ready" },
{ "name":"Zoinks", "state":"ready" }
],
"step3": [
{ "name":"Foo", "state":"cancel" }
]
}
}
Instead, when I'm not getting cryptic errors, I'm either modifying a top-level field in the document or modifying the field for all the elements or repeated the entire doc multiple times.
Any insights greatly appreciated.
Thanks.
p.s. is there a better syntax than [] to wildcard the named-elements under steps? Or after the pipe to identify the indices discovered by the select?
Pipe the output of .steps[][] into a select call that chooses the objects with the desired name and state values, then set the state value on the result.
$ jq '(.steps[][] | select(.name == "Foo" and .state == "wait")).state = "ready"' tmp.json
{
"state": "wait",
"steps": {
"step1": [
{
"name": "Foo",
"state": "ready"
},
{
"name": "Bar",
"state": "wait"
}
],
"step2": [
{
"name": "Foo",
"state": "ready"
},
{
"name": "Zoinks",
"state": "ready"
}
],
"step3": [
{
"name": "Foo",
"state": "cancel"
}
]
}
}
You can help confirm this using diff (the first jq just normalizes the formatting so that only the changes made by the second one show up in the diff):
$ diff <(jq . tmp.json) <(jq '...' tmp.json)
7c7
< "state": "wait"
---
> "state": "ready"
17c17
< "state": "wait"
---
> "state": "ready"

How can i insert object which got arrays into arrays using jq?

I may not discribed it properly, I tried many times and looked up the manaual on jq Manaual , and i got no idea how to insert object which contains array into a json file by jq command, anyway, here is the origin test.json:
[
{
"gate": [
{
"pro1": "1"
}
],
"home": [
{
"mem1": "1"
}
],
"holder": "1"
}
]
And i wish to be like this after insert:
[
{
"gate": [
{
"pro1": "1"
}
],
"home": [
{
"mem1": "1"
}
],
"holder": "1"
},
{
"gate": [
{
"pro1": "2"
}
],
"home": [
{
"mem1": "2"
}
],
"holder": "1"
}
]
Could it possibly done by jq?
Since you haven't provided more detail I am going to assume that you simply want to append a hard-coded object to an existing array. (If that's not what you mean you'll need to be more precise in your question.)
You can add items to the end of a list with the + operator. So in your case:
jq '.+[{"gate":[{"pro1": "2"}], "home":[{"mem1": "2"}],"holder": "1"}]' input.json
[
{
"gate": [
{
"pro1": "1"
}
],
"home": [
{
"mem1": "1"
}
],
"holder": "1"
},
{
"gate": [
{
"pro1": "2"
}
],
"home": [
{
"mem1": "2"
}
],
"holder": "1"
}
]
The . takes the existing input, which is an array. The +[ {...} ] concatenates it with another array of one object. If you want to put the new item(s) at the start instead of the end, swap it round: [ {...} ]+.

How to merge object TargetGroupARNs and TargetGroupArn unique in bash shell?

I have two json files.
I wanna merge objects in targetgroup.json and autoscaling.json with TargetGroupARNs and TargetGroupArn unique value in bash shell.
I am using jq version 1.4 and unable upgrade to last version because some cause.
#cat autoscaling.json
{
"AutoScalingGroups": [
{
"AutoScalingGroupARN": "arn:aws:autoscaling:ap-northeast-1:050073205187:autoScalingGroup:29f22791-b9cf-49c2-9ba6-b9ac2759d767:autoScalingGroupName/asg-dev-srv-01-20190327_133322",
"ServiceLinkedRoleARN": "arn:aws:iam::050073205187:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling",
"TargetGroupARNs": [
"arn:aws:elasticloadbalancing:ap-northeast-1:050073205187:targetgroup/dev-trial-tg/ef79ad5c9df9014e"
]
},
{
"AutoScalingGroupARN": "arn:aws:autoscaling:ap-northeast-1:050073205187:autoScalingGroup:cf662dc3-4e27-4022-a165-ebba8ca9488b:autoScalingGroupName/asg_group",
"ServiceLinkedRoleARN": "arn:aws:iam::050073205187:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling",
"TargetGroupARNs": [
"arn:aws:elasticloadbalancing:ap-northeast-1:050073205187:targetgroup/dev-trial1-tg/580ab71538a59063"
]
}
]
}
#cat targetgroup.json
[
{
"TargetGroupArn": "arn:aws:elasticloadbalancing:ap-northeast-1:050073205187:targetgroup/dev-trial-tg/ef79ad5c9df9014e",
"TargetGroupName": "dev-trial-tg"
},
{
"TargetGroupArn": "arn:aws:elasticloadbalancing:ap-northeast-1:050073205187:targetgroup/dev-trial1-tg/580ab71538a59063",
"TargetGroupName": "dev-trial1-tg"
}
]
My desired is:
[
{
"AutoScalingGroups": [
{
"AutoScalingGroupARN": "arn:aws:autoscaling:ap-northeast-1:050073205187:autoScalingGroup:29f22791-b9cf-49c2-9ba6-b9ac2759d767:autoScalingGroupName/asg-dev-srv-01-20190327_133322",
"ServiceLinkedRoleARN": "arn:aws:iam::050073205187:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling",
"TargetGroupARNs": [
"TargetGroupArn": "arn:aws:elasticloadbalancing:ap-northeast-1:050073205187:targetgroup/dev-trial-tg/ef79ad5c9df9014e",
"TargetGroupName": "dev-trial-tg"
],
},
{
"AutoScalingGroupARN": "arn:aws:autoscaling:ap-northeast-1:050073205187:autoScalingGroup:cf662dc3-4e27-4022-a165-ebba8ca9488b:autoScalingGroupName/asg_group",
"ServiceLinkedRoleARN": "arn:aws:iam::050073205187:role/aws-service-role/autoscaling.amazonaws.com/AWSServiceRoleForAutoScaling",
"TargetGroupARNs": [
"TargetGroupArn": "arn:aws:elasticloadbalancing:ap-northeast-1:050073205187:targetgroup/dev-trial1-tg/580ab71538a59063",
"TargetGroupName": "dev-trial1-tg"
]
}
]
}
]
I tried, but it was too hard for me. I hope you can help me.
This is solution from #oguzismail
jq -n 'input | reduce (input|to_entries)[] as $p (.; .AutoScalingGroups[$p.key].TargetGroupARNs = $p.value)' autoscaling.json targetgroup.json

Non destructive assignation with jq

I got the following data:
{
"things": [
{
"name": "lkj",
"something": [
"hike"
],
"more_data": "important",
"other_stuff": "very important"
},
{
"name": "iou",
"different_more_data": "very important too",
"more_different_data": [
"even more"
]
}
]
}
Each of things has an id called "name", with jq I can edit it like:
jq '(.things[]) |= {name,something:["changed"]}'
{
"things": [
{
"name": "lkj",
"something": [
"changed"
]
},
...
Unfortunately I lose everything not declared in the right hand of the assignation operation.
Is there a way to make assignations without losing data? So that the result is like this:
{
"things": [
{
"name": "lkj",
"something": [
"changed"
],
"more_data": "important",
"other_stuff": "very important"
},
{
"name": "iou",
"something": [
"changed"
],
"different_more_data": "very important too",
"more_different_data": [
"even more"
]
}
]
}
You can simply modify your query so that it looks like:
.things[] |= (.something = ["changed"])
You can also use |= (or one of its siblings, such as +=) instead of = in the RHS expression, e.g.
.things[] |= (.something += ["changed"])
If you want to update some, but not all, items, you can still use the above forms. A straightforward approach is to use if ... then ... else ... end, for example:
.things[] |= (if .name == "lkj" then .something = ["changed"] else . end)
Using select on the LHS of |=
jq (or at least jq since version 1.4) does support the use of select on the LHS of |=, e.g.
(.things[] | select(.name=="lkj")) |= (.something += ["changed"])
With jq's map function:
jq '.things |= map(.something = ["changed"])' jsonfile
map(x) - apply specified filter x for each item of the input array
.something = ["changed"] - set key something to an object with array ["changed"] as a value
The output:
{
"things": [
{
"name": "lkj",
"something": [
"changed"
],
"more_data": "important",
"other_stuff": "very important"
},
{
"name": "iou",
"different_more_data": "very important too",
"more_different_data": [
"even more"
],
"something": [
"changed"
]
}
]
}