I am using JQ to parse JSON returned by Salesforce CLI.
My bash script is:
orgCreateResult=$(sfdx force:org:display -u mockScreenOrg --json)
orgCreateStatus=$(echo $orgCreateResult | ./bash-scripts/jq-win64.exe .status -r)
echo "orgCreateResult: "
echo "$orgCreateResult"
echo "orgCreateStatus: $orgCreateStatus"
instanceUrl=$(echo $checkDevHubResult | ./bash-scripts/jq-win64.exe .result -r)
echo "instanceUrl: $instanceUrl"
I am using bash terminal in VS Code on windows. Output I am getting from this script is :
orgCreateResult:
{
"status": 0,
"result": {
"username": "test-******#example.com",
"devHubId": "****#**.com",
"id": "************",
"createdBy": "*****#**.com",
"createdDate": "2019-05-13T14:15:32.000+0000",
"expirationDate": "2019-06-12",
"status": "Active",
"edition": "Developer",
"orgName": "HPESC",
"accessToken": "************************************",
"instanceUrl": "https://nosoftware-ruby-2532-dev-ed.cs6.my.salesforce.com",
"clientId": "PlatformCLI",
"alias": "mockScreenOrg"
}
}
orgCreateStatus: 0
instanceUrl:
I am able to read status field from JSON but it kinda fails with result object. I want to read instanceUrl field from JSON to use later in my script. Not sure if I am doing something wrong. I am very new to bash and JQ
Try this:
instanceUrl=$(echo $orgCreateResult | ./bash-scripts/jq-win64.exe .result.instanceUrl -r)
Related
i have a details.json file with a lot of entries and a shops.txt file like below. I like to have a little script which compares two values and just return the matching json entries.
[
{
"userName": "Anne",
"email": "anne#stack.com",
"company": {
"name": "Stack GmbH",
},
"details": {
"key": "EFHJKI-KJEFT-DHMNEB",
"prod": "Car",
},
"store": {
"id": "05611a7f-a679-12ad-a3u2-0745e3650a03",
"storeName": "shop-a57ca0a3-120c-1a73-153b-fa4231cab768",
}
},
{
"userName": "Tom",
"email": "tom#stack.com",
"company": {
"name": "Stack GmbH",
},
"details": {
"key": "DFSGSE-FGEAR-GWRTGW",
"prod": "Bike",
},
"store": null
},
]
This is the other file "shops.txt" (can be a lot more of shops inside)
shop-a57ca0a3-120c-1a73-153b-fa4231cab768
The script is looping through the shops, for every shop it loops through the json and should compare the currentShop with the store.shop from json and then echo the user and the shop.
But I can not access the specific parameters inside the json. How can I do this?
#!/bin/bash
shops="shops.txt"
while IFS= read -r line
do
currentShop="$line"
jq -c '.[.userName, .store.storeName]' details.json | while read i; do
if [[ $i.store.storeName == *$currentShop* ]]; then
echo $i.userName
echo $currentShop
fi
done
done < "$shops"
First of all, you might want to 'clean' your json, remove any trailing ,'s etc.
After looping through each line in the file we just need one select() to get the matching object.
The script could look something like:
#!/bin/bash
while read shop; do
echo "Check: $shop"
jq -r --arg storeName "$shop" '.[] | select(.store.storeName == "\($storeName)") | "\(.userName) - \(.store.storeName)"' details.json
done < "shops.txt"
Which will produce
Check: shop-a57ca0a3-120c-1a73-153b-fa4231cab768
Anne - shop-a57ca0a3-120c-1a73-153b-fa4231cab768
I guess this could be combined into a single jq call, but it seems like you want to loop over each entry found
You can test this jq selector on this online JqPlay Demo.
I was able to access the values with the following command:
echo $i | jq -r '.userName'
#!/bin/bash
DESCRIBE_VPC=$(aws ec2 describe-vpcs --region us-west-2)
The Json value retrieved from aws ec2 describe-vpcs --region us-west-2 stores in DESCRIBE_VPC which comes to the output format below.
> echo $DESCRIBE_VPC
{
"Vpcs": [
{
"VpcId": "vpc-12345678910",
"InstanceTenancy": "default",
"Tags": [
{
"Value": "arn:aws:cloudformation:us-west-2:12345678910:stack/vpc/0123456-vpcid",
"Key": "stack-id"
},
{
"Value": "vpc-type",
"Key": "Name"
},
],
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-123456",
"CidrBlock": "123.456.89.10",
"CidrBlockState": {
"State": "associated"
}
}
],
"State": "available",
"DhcpOptionsId": "dpt-01234567",
"OwnerId": "12345678910",
"CidrBlock": "123.456.789.10",
"IsDefault": false
}
]
}
[root#ip bin]# jq '.Vpcs' $DESCRIBE_VPC
jq: error: Could not open file {: No such file or directory
jq: error: Could not open file "Vpcs":: No such file or directory
parse error: Invalid numeric literal at line 1, column 57
Any suggestions here how to parse the json values stored in variable?
Actually you have to use the (json) content of a shell variable in place of a command's (jq) input. This input should be a file or a stream, but you have it in a shell variable. There are many ways to do this, this is a simple one:
echo "$DESCRIBE_VPC" | jq '.Vpcs'
or
printf "%s" "$DESCRIBE_VPC" | jq '.Vpcs'
or this (for bash shell):
jq '.Vpcs' <<< "$DESCRIBE_VPC"
Also jq can accept variables and json variables. For example you could do this:
jq -n --argjson x "$DESCRIBE_VPC" '$x.Vpcs'
But the first one is usually better.
I have a windows batch script to perform POST request using curl and reads the data from JSON file, it works fine with only a single object in the file and it looks like this.
curl -u username#password -H "Content-Type: application/json" -d #file.json http://apiurl.com
and the json file is this:
{
"name": "Empty name",
"properties": {
"active": "True",
"subcity_zone": "East Hararge",
"woreda": "Meta"
}
}
But now I want to send the request each object in the array by iterating each item. So, How do I iterate each JSON object from the file?
Here is what the new JSON array file looks like:
[{
"name": "test facility I",
"properties": {
"active": "True",
"city": "",
"subcity_zone": "East Hararge",
"woreda": "Meta"
}
},
{
"name": "test facility II",
"properties": {
"active": "True",
"subcity_zone": "East Hararge",
"woreda": "Girawa"
}
}]
Using jq:
jq -c '.[]' file | while read js; do
curl -u username#password -H "Content-Type: application/json" -d #<(echo "$js") http://apiurl.com
done
The jq command extracts the each object in one line that is read by read command into the $js variable.
The <(echo "$js") creates a temporary file that is passed to curl.
Haven't used jq before but I'm wanting to build a shell script that will get a JSON response and extract just the values. To learn I thought I would try on my blog's WP API but for some reason I'm getting an error of:
jq: error (at :322): Cannot index array with string "slug"
When researching for and testing previous questions:
jq: Cannot index array with string
jq is sed for JSON
JSON array to bash variables using jq
How to use jq in a shell pipeline?
How to extract data from a JSON file
The above reading I've tried to code:
URL="http://foobar.com"
RESPONSE=$(curl -so /dev/null -w "%{http_code}" $URL)
WPAPI="/wp-json/wp/v2"
IDENTIFIER="categories"
if (("$RESPONSE" == 200)); then
curl -s {$URL$WPAPI"/"$IDENTIFIER"/"} | jq '.' >> $IDENTIFIER.json
result=$(jq .slug $IDENTIFIER.json)
echo $result
else
echo "Not returned status 200";
fi
An additional attempt changing the jq after the curl:
curl -s {$URL$WPAPI"/"$IDENTIFIER"/"} | jq '.' | $IDENTIFIER.json
result=(jq -r '.slug' $IDENTIFIER.json)
echo $result
I can modify the uncompress with the python JSON tool:
result=(curl -s {$URL$WPAPI"/"$IDENTIFIER"/"} | python -m json.tool > $IDENTIFIER.json)
I can save the JSON to a file but when I use jq I cannot get just the slug and here are my other trys:
catCalled=$(curl -s {$URL$WPAPI"/"$IDENTIFIER"/"} | python -m json.tool | ./jq -r '.slug')
echo $catCalled
My end goal is to try to use jq in a shell script and build a slug array with jq. What am I doing wrong in my jq and can I use jq on a string without creating a file?
Return from curl after uncompress per comment request:
[
{
"id": 4,
"count": 18,
"description": "",
"link": "http://foobar.com/category/foo/",
"name": "Foo",
"slug": "foo",
"taxonomy": "category",
},
{
"id": 8,
"count": 9,
"description": "",
"link": "http://foobar.com/category/bar/",
"name": "Bar",
"slug": "bar",
"taxonomy": "category",
},
{
"id": 5,
"count": 1,
"description": "",
"link": "http://foobar.com/category/mon/",
"name": "Mon",
"slug": "mon",
"taxonomy": "category",
},
{
"id": 11,
"count": 8,
"description": "",
"link": "http://foobar.com/category/fort/",
"name": "Fort",
"slug": "fort",
"taxonomy": "category",
}
]
eventually my goal is trying to get the name of the slug's into an array like:
catArray=('foo','bar','mon', 'fort')
There are 2 issues here:
slug is not a root level element in your example json. The root level element is an array. If you want to access the slug property of each element of the array, you can do so like this:
jq '.[].slug' $IDENTIFIER.json
Your example json has trailing commas after the last property of each array element. Remove the commas after "taxonomy": "category".
If I take your sample json, remove the errant commas, save it to a plain text file called test.json and run the following command:
jq '.[].slug' test.json
I get the following output:
"foo"
"bar"
"mon"
"fort"
Preprocessing
Unfortunately, the JSON-like data shown as having been produced by curl is not strictly JSON. jq does not have a "relaxed JSON" mode, so in order to use jq, you will have to preprocess the JSON-like data, e.g. using hjson (see http://hjson.org/):
$ hjson -j input.qjson > input.json
jq
With the JSON in input.json:
$ jq -c 'map(.slug)' input.json
["foo","bar","mon","fort"]
your string is not json, notice how the last member of your objects ends with a comma,
{foo:"bar",baz:9,}
this is legal in javascript, but it's illegal in json. if you are supposed to be receiving json from that endpoint, then contact the people behind it and tell them to fix the bug (it's breaking the json specs by ending objects's last member with a comma, which is illegal in json.) - until it's fixed, i guess you can patch it with a little regex, but it's a dirty quickfix, and probably not very reliable, but running it through
perl -p -0777 -e 's/\"\,\s*}/\"}/g;' makes it legal json..
My json looks like this :
{
"20160522201409-jobsv1-1": {
"vmStateDisplayName": "Ready",
"servers": {
"20160522201409 jobs_v1 1": {
"serverStateDisplayName": "Ready",
"creationDate": "2016-05-22T20:14:22.000+0000",
"state": "READY",
"provisionStatus": "PENDING",
"serverRole": "ROLE",
"serverType": "SERVER",
"serverName": "20160522201409 jobs_v1 1",
"serverId": 2902
}
},
"isAdminNode": true,
"creationDate": "2016-05-22T20:14:23.000+0000",
"totalStorage": 15360,
"shapeId": "ot1",
"state": "READY",
"vmId": 4353,
"hostName": "20160522201409-jobsv1-1",
"label": "20160522201409 jobs_v1 ADMIN_SERVER 1",
"ipAddress": "10.252.159.39",
"publicIpAddress": "10.252.159.39",
"usageType": "ADMIN_SERVER",
"role": "ADMIN_SERVER",
"componentType": "jobs_v1"
}
}
My key keeps changing from time to time. So for example 20160522201409-jobsv1-1 may be something else tomorrow. Also I may more than one such entry in the json payload.
I want to echo $KEYS and I am trying to do it using jq.
Things I have tried :
| jq .KEYS is the command i use frequently.
Is there a jq command to display all the primary keys in the json?
I only care about the hostname field. And I would like to extract that out. I know how to do it using grep but it is NOT a clean approach.
You can simply use: keys:
% jq 'keys' my.json
[
"20160522201409-jobsv1-1"
]
And to get the first:
% jq -r 'keys[0]' my.json
20160522201409-jobsv1-1
-r is for raw output:
--raw-output / -r: With this option, if the filter’s result is a string then it will be written directly to standard output rather than being formatted as a JSON string with quotes. This can be useful for making jq filters talk to non-JSON-based systems.
Source
If you want a known value below an unknown property, eg xxx.hostName:
% jq -r '.[].hostName' my.json
20160522201409-jobsv1-1