Want to read "Items" from the AWS CLI response. and want to write "for loop" on "Items". But I am not able to do in shell script.
The output format of AWS cli command is in JSON format.
Sh Code:
[ ~]$ response_scan=`aws dynamodb scan --table-name recovery-plan --max-items 10 --attributes-to-get '["job_id", "job_type", "launch_category"]'`
[ ~]$ echo $response_scan
{
"Count": 166,
"Items": [
{ "launch_category": { "S": "TOT" }, "job_type": { "S": "TEST" }, "job_id": { "S": "39504214122e" } },
{ "job_type": { "S": "TEST" }, "job_id": { "S": "8c48-914d0aa2a186" } },
{ "job_type": { "S": "TEST" }, "job_id": { "S": "cbd07892491d" } },
{ "job_type": { "S": "TEST1" }, "job_id": { "S": "7afef48b0283" } },
{ "job_type": { "S": "TEST" }, "job_id": { "S": "7d678fab68e1" } }
],
"NextToken": "eyJFasasaseGNsdXasasaslX2Ftb3VudCasasI6IDEwfQ==",
"ScannedCount": 166,
"ConsumedCapacity": null
}
Can anyone help me to iterate over response_scan["Items"] ?
What I am doing exactly:
I want to add field - launch_category to items/row which not have this field.
Value of launch_category is TOT for TEST and TOT1 for TEST1
response_scan=$(aws dynamodb scan --table-name recovery-plan --max-items 10 --attributes-to-get '["job_id", "job_type", "launch_category"]' --query Items[].job_type.S --output text )
Or you can use jq to parse json - https://stedolan.github.io/jq/
The above answer is still working, updating here as per AWS Documentation using --projection-expression is recommended instead of --attribute-to-get.
--attributes-to-get (list)
This is a legacy parameter. Use ProjectionExpression instead. For more information, see AttributesToGet in the Amazon DynamoDB Developer Guide .
response_scan=$(aws dynamodb scan
--table-name recovery-plan
--max-items 10
--projection-expression "job_id, job_type,launch_category"
--query Items[].job_type.S --output text
)
AttributesToGet
Related
Is there a command to use jq to add onto this type of map?
append an array of maps using keys (ie, vm1, vm2, vm3)
Note: I have an existing vm_map {} in a json file and i want to add to the vm_map
this is my new_json.json file
{
"gcs_config": [
{
"bucket_name": "somebucket",
"bucket_readers": [],
"bucket_writers": []
}
],
"label_application": "someapp",
"label_environment": "dev",
"lits_vm_zone": "somezone",
"project_id": "someproject",
"region": "someregion",
"storage_bucket_required": true,
"vm_map" : {}
}
expected: using jq to add onto vm_maps map. I will have an empty vm_map and each time it runs, i will add a x amount of new entries.
{
"gcs_config": [
{
"bucket_name": "somebucket",
"bucket_readers": [],
"bucket_writers": []
}
],
"label_application": "someapp",
"label_environment": "dev",
"lits_vm_zone": "zone-a",
"project_id": "someproject",
"region": "someregion",
"storage_bucket_required": true,
"vm_map": {
"vm1": {
"host": "vm1",
"network": "10.1.1.1",
"name": "vm1"
},
"vm2": {
"host": "123",
"network": "10.1.12",
"name": "vm2"
}
}
}
The file you describe is not valid JSON. I'm assuming you mean
{
"vm_map": {
"vm1": {
"host": "vm1",
"network": "xxxxx",
"name": "xxxxxxx"
},
"vm2": {
"host": "vm2",
"network": "xxxxx",
"name": "xxxxxxx"
}
}
}
You can use this:
jq \
--arg VMHOST "$VMHOST" \
--arg NETWORK_IP "$NETWORK_IP" \
--arg VM_NAME "$VM_NAME" \
'
.vm_map[ $VMHOST ] = {
host: $VMHOST,
network: $NETWORK_IP,
name: $VM_NAME
}
'
Is there a way to create a new element in an existing json object using jq? Example below:
Let's say I have this json object and would like to add a new element to foo:
json='{
"id": "<id>>",
"name": "<name>",
"properties": {
"State": "<state>",
"requests": [],
"foo": [
{
"id": "<id1>",
"bar1": [
{
"baz1": "*"
}
]
},
{
"id": "<id2>",
"bar2": [
{
"baz2": "*"
}
]
}
]
}
}'
This command works to do that:
json2=$($json1 | jq '.properties.foo += [ { "id": "<id3>", "bar3": [ { "baz3": "*"} ] } ]')
However, running that same command without a preexisting foo element fails (example array below):
json3='{
"id": "<id>>",
"name": "<name>",
"properties": {
"State": "<state>",
"requests": []
}
}'
Is there a way in jq to create that element in the json object if one already does not exist?
Thanks!
There is nothing wrong with your jq program, which can be seen by running:
jq '.properties.foo += [ { "id": "<id3>", "bar3": [ { "baz3": "*"} ] } ]' <<< "$json3"
It looks like the problem is with your invocation but since it's not clear what $json1 is, I'll just guess that the above is sufficient for you to resolve the issue.
I have json file which is constructed in simmilar way:
[
{
"_id":"1234",
"org":"org1",
"int":
{"url":"http://url.com.uk:1234"}},
{
"_id":"4321",
"org":"org2",
"int":
{"url":"http://url.com.us:4321"}},
...
]
Now im "jumping" from one entry to another and checking if under URL application is working properly. After check i want to add/update field "status". But i can't update whole file, im just getting:
$ jq --arg mod "GOOD" '.[0].int + {stat: $mod}' tmp.json
{
"url": "http://url.com.uk:1234",
"stat": "GOOD"
}
How can i with jq command get new updated whole file, not just only part of it?
If you put your data in data.json and the changes you want to make to
each record into a separate arg.json argument file like
{
"1234": { "int": { "stat": "GOOD" } },
"4321": { "int": { "stat": "BAD", "xxx": "yyy" } }
}
and run jq as
$ jq -M --argfile arg arg.json 'map(. + $arg[._id])' data.json
then it will output the updated data, e.g.
[
{
"_id": "1234",
"org": "org1",
"int": {
"stat": "GOOD"
}
},
{
"_id": "4321",
"org": "org2",
"int": {
"stat": "BAD",
"xxx": "yyy"
}
}
]
Note that the + replaces keys. If you want to merge keys you can use * e.g.
$ jq -M --argfile arg arg.json 'map(. * $arg[._id])' data.json
which generates
[
{
"_id": "1234",
"org": "org1",
"int": {
"url": "http://url.com.uk:1234",
"stat": "GOOD"
}
},
{
"_id": "4321",
"org": "org2",
"int": {
"url": "http://url.com.us:4321",
"stat": "BAD",
"xxx": "yyy"
}
}
]
If you want to update the data in place you could use sponge
as described in the answer Manipulate JSON with jq
e.g.
$ jq -M --argfile arg arg.json 'map(. * $arg[._id])' data.json | sponge data.json
You can map to array and resign the int by operation, like:
jq --arg mod "GOOD" '.[] | .int=.int + {stat: $mod}' tmp.json
{
"_id": "1234",
"org": "org1",
"int": {
"url": "http://url.com.uk:1234",
"stat": "GOOD"
}
}
{
"_id": "4321",
"org": "org2",
"int": {
"url": "http://url.com.us:4321",
"stat": "GOOD"
}
}
While putting below JSON in dynamo DB using AWS CLI with below command:
aws dynamodb put-item --table-name ScreenList --item file://tableName.json
I am getting Parameter validation failed Exception.I have gone rigorously through AWS docs but failed to find example to insert a complicated json.Every small help is welcome.
The updated Json :
{
"itemName": {
"S": "SCREEN_LIST"
},
"productName": {
"S": "P2P_MOBITEL"
},
"screenList": {
"L": [
{
"menu": {
"L": [
{
"M": {
"menuId": {
"N": "1"
},
"menuText": {
"S": "ENG_HEADING"
},
"menuType": {
"S": "Dynamic"
}
}
}
]
},
"M": {
"screenFooter": {
"S": "F_LANGUAGE_CHANGE"
},
"screenHeader": {
"S": "H_LANGUAGE_CHANGE"
},
"screenId": {
"S": "LANGUAGE_CHANGE"
},
"screenType": {
"S": ""
}
}
}
]
}
}
It seems that you are defining complex types incorrectly. According to AWS documentation you should define a list like this:
"L": ["Cookies", "Coffee", 3.14159]
and a map should be defined like this:
"M": {"Name": {"S": "Joe"}, "Age": {"N": "35"}}
which means that a menu map should be defined like this:
"menu": {
"L": [
{
"M": {
"menuId": {"N" :"1"},
"menuText": {"S" :"PACKS_SCREEN"},
"menuType": {"S" :"Dynamic"}
}
}
]
}
Notice the "M" and "L" attributes.
You should change the rest of your JSON in a similar fashion.
You can find full JSON definition here in the Options section.
UPDATE
Now your list definition is incorrect. You have:
"screenList":{
"L":[
{
"menu":{ ... },
"M":{ ... }
}
]
}
While it should be:
"screenList":{
"L":[
{
"M":{ ... }
},
{
"M":{ ... }
},
]
}
I've been attempting to programmatically update the AWS Route 53 DNS records, so I've been using jq to update the following JSON file;
{
"Comment": "Update 'A' record for drivepoc.biz zone file",
"Changes": [
{
"Action": "UPSERT",
"ResourceRecordSet": {
"Name": "www.domain.biz.",
"Type": "A",
"TTL": 60,
"ResourceRecords": [
{
"Value": "123.123.123.123"
}
]
}
}
]
}
So, the existing entry "Value": "123.123.123.123" needs to remain, but needs to have additional entry of "Value": "456.456.456.456". The nearest I got to do this was:
cat a_record.json | jq '.Changes[0].ResourceRecordSet.ResourceRecords |= .+ ["Value: 456.456.456.456"]'
but this puts it outside the braces and the quotes are wrong;
"ResourceRecords": [
{
"Value": "52.18.219.57"
},
"Value": "456.456.456.456"
]
Instead of what is required;
"ResourceRecords": [
{
"Value": "52.18.219.57"
},
{
"Value": "456.456.456.456"
}
]
Can anyone give me any tips please?
You're adding an object to that array, not a string. Create an object to be inserted.
.Changes[].ResourceRecordSet.ResourceRecords += [{Value:"456.456.456.456"}]