Extract child key and flatten into parent object output in JQ - json

I'm trying to simplify a nested JSON structure using JQ
Input
[
{
"defaultBranchRef": {
"name": "main"
},
"nameWithOwner": "KyleMit/stack-posts"
},
{
"defaultBranchRef": {
"name": "master"
},
"nameWithOwner": "KyleMit/HelloSERN"
}
]
Desired Output
[
{
"nameWithOwner": "KyleMit/stack-posts",
"defaultBranchName": "main"
},
{
"nameWithOwner": "KyleMit/HelloSERN",
"defaultBranchName": "master"
}
]
Attempted Solution
I can retrieve each of the individual fields at any depth
Get nameWithOwner
[.[]|.nameWithOwner] // ["KyleMit/stack-posts","KyleMit/HelloSERN"]
Get defaultBranchRef.name
[.[]|.defaultBranchRef.name] // ["main","master"]
But can't quite get how to combine fields / selectors to return mulitple values


Just update |= the defaultBranchRef field to the content of its name subfield:
jq '.[].defaultBranchRef |= .name'
[
{
"defaultBranchRef": "main",
"nameWithOwner": "KyleMit/stack-posts"
},
{
"defaultBranchRef": "master",
"nameWithOwner": "KyleMit/HelloSERN"
}
]
Demo


Open to other solutions, but one way to combine is to use Object Construction like
[.[]|{nameWithOwner, defaultBranchName: .defaultBranchRef.name}]
Which results in the following output:
{
"nameWithOwner": "KyleMit/stack-posts",
"defaultBranchName": "main"
},
{
"nameWithOwner": "KyleMit/HelloSERN",
"defaultBranchName": "master"
}
]

Related

How to check if json contains another json using jq?

I have just started using jq and I need to check if a given json is present in another json using jq?
Suppose this is my json_input:
{
"info": {
"values": [
{
"data": [
{
"name": "name",
"value": "val"
}
]
}
]
}
}
I need to check if the above json input is present inside the following available_json:
{
"info": {
"values": [
{
"data": [
{
"name": "name",
"value": "val"
},
{
"name": "name2",
"value": "val2"
},
{
"name": "name3",
"value": "val3"
}
],
"key1":"val1",
"key2":"val2"
}
],
"priority":1,
"objects":[
{
"name":"a"}
]
}
}
Both json are stored in variables and should report the presence for any json_input given as input based on any available_json (generic). How can this be done using jq?
Or Is there any other better way like converting both json to string and then comparing?
PS: The json object key info is fixed and the values can change.

This is so trivial that one might not even think of it: Using the jq filter contains:
jq 'contains({
"info": {
"values": [
{
"data": [
{
"name": "name",
"value": "val"
}
]
}
]
}
})' available.json
Output will be true or false. If you run jq -e (--exit-status), it will set the exit status accordingly, which allows you to use it together with if or &&/|| in your shell.
If you the input_json is also stored in a file:
jq --slurpfile input_json input_json 'contains($input_json[0])' available.json
If the JSON document is stored in a variable, then --argjson instead of --slurpfile:
jq --argjson input_json "$input_json" 'contains($input_json)' available.json
or simply relying on parameter expansion of your shell:
jq "contains($input_json)" available.json

Insert whole json to the field in Dynamo DB using AWS Step Functions

I am trying to PutItem into the DynamoDB using the AWS Step Functions.
I managed to save Item with simple string fields (S), but one of the fields should store the whole JSON payload. So it should be the Map (M).
But my payload includes nested Maps also.
Example JSON:
{
"firstMap": {
"field": "something",
},
"secondMap": {
"nestedMap": {
"field": "something",
},
"anotherNestedMap": [
{
"field": "something",
"oneMoreNestedMap": {
"andOneMore": {
"field": "something",
},
"arrayComesHere": [
{
"andAgainNestedMap": {
"field": "something",
},
"andAgain": [
{
"field": "something",
"alsoNestedArray": [
{
"field": "something"
}
]
}
]
}
]
},
"letItBeFinalOne": [
{
"field": "something"
}
]
}
]
...
What I want to do is to just say, hey Step Function, insert please this whole JSON into the item field like this
"Item": {
...
"whole_payload": {
"M.$": "$"
},
} ...
But it fails, cause it accepts only one Map to be handled.
So I need to directly iterate over all nested maps and mark them with 'M'.
Is there a way to make it resolve it by itself?
Like in Typescript I can use aws.DynamoDB.DocumentClient() and just put a whole JSON to the field and it resolves all the maps by itself

Came across a thread for similar request to AWS Step functions team. New feature enhanced allows something closer to what you are looking for.
Sample snippet:
...
"Parameters": {
"TableName" : "dynamodb-table",
"Item":{
"requestId" : {
"S.$": "$.requestId"
},
"payload": {
"S.$":"States.JsonToString($)"
}
}
...
AWS Reference

Creating a new element in an json object using jq

Is there a way to create a new element in an existing json object using jq? Example below:
Let's say I have this json object and would like to add a new element to foo:
json='{
"id": "<id>>",
"name": "<name>",
"properties": {
"State": "<state>",
"requests": [],
"foo": [
{
"id": "<id1>",
"bar1": [
{
"baz1": "*"
}
]
},
{
"id": "<id2>",
"bar2": [
{
"baz2": "*"
}
]
}
]
}
}'
This command works to do that:
json2=$($json1 | jq '.properties.foo += [ { "id": "<id3>", "bar3": [ { "baz3": "*"} ] } ]')
However, running that same command without a preexisting foo element fails (example array below):
json3='{
"id": "<id>>",
"name": "<name>",
"properties": {
"State": "<state>",
"requests": []
}
}'
Is there a way in jq to create that element in the json object if one already does not exist?
Thanks!

There is nothing wrong with your jq program, which can be seen by running:
jq '.properties.foo += [ { "id": "<id3>", "bar3": [ { "baz3": "*"} ] } ]' <<< "$json3"
It looks like the problem is with your invocation but since it's not clear what $json1 is, I'll just guess that the above is sufficient for you to resolve the issue.

How to delete array elements that end with 1?

I need to remove all array elements that have the name field ending with 1.
Input:
{
"foo": "bar",
"data": {
"code": "abc123",
"items": [
{
"name": "exp1"
},
{
"name": "exp2"
},
{
"name": "exp11"
}
]
}
}
Desired output:
{
"foo": "bar",
"data": {
"code": "abc123",
"items": [
{
"name": "exp2"
}
]
}
}
My attempt:
jq 'del(.data.items[] | select(.name | endswith("1")))' input
Which results in Invalid path expression.

You can use this jq filter:
jq '.data.items|=map(select(.name|endswith("1")|not))' file
This replace .data.items with the a new array having objects whose names don't end with 1.

Your attempt will work with recent versions of jq (that is, more recent than version 1.5).
Yet another variant (perhaps the most concise robust alternative):
.data.items|=map(select(.name|test("[^1]$")))

Flatten JSON array of objects that contain an array with Jolt

I'm using Jolt 0.1.0, and trying to transform the following JSON:
{
"records": [
{
"collectionId": "COLLECTION1",
"recordIds": [
"recA",
"recB"
]
},
{
"collectionId": "COLLECTION1",
"recordIds": [
"recC",
"recD",
"recE"
]
},
{
"collectionId": "COLLECTION2",
"recordIds": [
"recF",
"recG"
]
}
]
}
... to this:
{
"records": [
"COLLECTION1:recA",
"COLLECTION1:recB",
"COLLECTION1:recC",
"COLLECTION1:recD",
"COLLECTION1:recE",
"COLLECTION2:recF",
"COLLECTION2:recG"
]
}
I've made several attempts with the modify-default-beta operator and the concat function, but can't seem to make it work.

Very similar to https://github.com/bazaarvoice/jolt/issues/656 which required 4 steps.