I use JQ 1.5 in a Windows10 enviroment (PowerShell).
I built a jq statement that works on the example data on jqplay but throws a error on my enviroment:
Sample: Code share
Code:
. | { last_update: .starbase_detailed_scan.last_update_time, user_name: .starbase_detailed_scan.owner_name, alliance_id: .starbase_detailed_scan.owner_alliance_id, drydocks: .starbase_detailed_scan.num_drydocks, tier: .starbase_detailed_scan.owner_level, defence_plattform: .starbase_detailed_scan.num_defence_platforms, shield_triggered: .starbase_detailed_scan.player_shield.triggered_on, shield_end: .starbase_detailed_scan.player_shield.expiry_time, parsteel: .resources."325683920".current_amount, tritanium: .resources."743985951".current_amount, dilithium: .resources."2614028847".current_amount, user_id: .starbase_detailed_scan.owner_user_id, defence_rating: .starbase_detailed_scan.defense_rating }
The problem are the JSON objects with a numeric identifier. On jqplay I got the correct values. On PowerShell jq I get an error. I expected that this is a PowerShell problem so I tried to move the filter into a filter file. The error is then gone but I get only NULL as value for the three objects.
Numbers in the json path need to be marked Oldschool like:
.starbase_detailed_scan.resources["2614028847"]
BR
Timo
Related
I am currently working on a bash script that combines the output of both the aws iam list-users and aws iam list-user-tags commands in a CSV file containing all users along with their respective information and assigned tags. To parse the JSON output of those commands I choose to use jq.
Retrieving parsing and converting (JSON to CSV) the list-user output works fine and produces the expected comma-separated list of values.
The output of list-user-tags does not quite behave that way. Its JSON output has the following schema:
{
Tags: [
{
Key: "Name",
Value: "NameOfUser"
},
{
Key: "Email",
Value: "EmailOfUser"
},
{
Key: "Company",
Value: "CompanyOfUser"
}
]
}
Unfortunately the order of the tags is not consistent across users (and possibly across queries) which currently makes it impossible for me to maintain the order defined in the CSV file. On top of that there is the possibility of one or multiple missing tags.
What I am looking for is a way to achieve the following (preferably using jq):
Select a tags "Value"-value by its "Key"-value
Check whether it is existent and if not add an empty entry
Put the value in the exact same place every time (maintain a certain order)
Repeat for every entry in the original output
Convert the resulting array of values into CSV
What I tried so far:
aws iam list-user-tags --user-name abcdef --no-cli-pager \
| jq -r '[.Tags[] | select(.Key=="Name"),select(.Key=="Email"),select(.Key=="Company") | .Value // ""] | #csv'
Any help is much appreciated!
Let's suppose your sample "schema" is in a file named schema.jq
Then
jq -n -f schema.jq | jq -r '
def get($key):
map(select(.Key == $key))
| if length == 0 then null else .[0].Value end;
.Tags | [get("Name"), get("Email"), get("Company")] | #csv
'
produces the following CSV:
"NameOfUser","EmailOfUser","CompanyOfUser"
It should be easy to adapt this illustration to your needs.
I have a JSON which goes like this:
{
"results":[
{
"uri":"www.xxx.com"
}
]
}
EDIT
When uri is not present, JSON looks like this:
{
"results":[
]
}
In some cases, uri is present and in some cases, it is not.
Now, I want to use jq to return boolean value if uri is present or not.
This is what I wrote so far but despite uri being present, it gives null.
${search_query_response} contains the JSON
file_status=$(jq -r '.uri' <<< ${search_query_response})
Can anyone guide me?
Since you use jq, it means you are working within a shell script context.
If the boolean result is to be handled by the shell script, you can make jq set its EXIT_CODE depending on the JSON request success or failure status, with jq -e
Example shell script using the EXIT_CODE from jq:
if uri=$(jq -je '.results[].uri') <<<"$search_query_response"
then
printf 'Search results contains an URI: %s.\n' "$uri"
else
echo 'No URI in search results.'
fi
See man jq:
-e / --exit-status:
Sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null, or 4 if no valid result was ever produced. Normally jq exits with 2 if there was any usage problem or system error, 3 if there was a jq program compile error, or 0 if the jq program ran.
Another way to set the exit status is with the halt_error builtin function.
The has function does the job:
jq '.results|map(has("uri"))|.[]'
map the has function on .results.
I have a jq filter that converts (influxdb) json input to csv for further parsing. However, this filter fails when influxdb returns an error. I'm trying to improve my jq filter to detect this, however I can't get this to work. I need something like https://stackoverflow.com/a/41829748 but can't seem to get this to work. Any ideas?
Example data
{"results":[{"statement_id":0,"series":[{"name":"energyv3","columns":["time","value"],"values":[["2015-07-30T23:59:00Z",56980800],["2015-07-31T23:59:00Z",95108400]]}]}]}
{"error":"error parsing query: found EOF, expected integer at line 1, char 34"}
Desired outcome
"\"time\",\"value\""
"\"2015-07-30T23:59:00Z\",56980800"
"\"2015-07-31T23:59:00Z\",95108400"
"error parsing query: found EOF, expected integer at line 1, char 34"
i.e.
For input with .results key: data formatted as csv (works OK)
For input with .error key: only error string (doesn't work)
Current filter used
select(.results) | (.results[0].series[0].columns), (.results[0].series[0].values[]) | #csv
Attempt to combine filters
((select(.error) | {error}) // null) + select(.results) | (.results[0].series[0].columns), (.results[0].series[0].values[]) | #csv
Based on your attempts, and the assumption that each object contain either results or error, this should do it:
( .results[0].series | .[0].columns, .[]?.values[] ) // [ .error ] | #csv
REPL demo
This question already has answers here:
Parse or view JSON data fields using JQ tool utility where field names have a "-" dash in the key name
(4 answers)
Closed 4 years ago.
I use jq 1.5 in a Windows Environment to modify an json object that i receiving from Amazon s3. I have there a funny Problem. I use jq to extract single keys of the object:
{
"s3_direct_url": "https://fanzo-photos.s3.amazonaws.com/photos/images/034/005/322/screen1.jpg",
"url": "https://fanzo-photos.s3.amazonaws.com",
"fields": {
"key": "photos/images/034/005/322/screen1.jpg",
"success_action_status": "200",
"Content-Type": "image/jpeg",
"acl": "public-read",
"policy": "eyJleHBpcmF0aW9uIjoiMjAxOC0xMS0wMlQxMzo0NzoxNVoiLCJjb25kaXRpb25zIjpbeyJidWNrZXQiOiJmYW56by1waG90b3MifSx7ImtleSI6InBob3Rvcy9pbWFnZXMvMDM0LzAwNS8zMjIvc2NyZWVuMS5qcGcifSx7InN1Y2Nlc3NfYWN0aW9uX3N0YXR1cyI6IjIwMCJ9LHsiQ29udGVudC1UeXBlIjoiaW1hZ2UvanBlZyJ9LHsiYWNsIjoicHVibGljLXJlYWQifSx7IngtYW16LWNyZWRlbnRpYWwiOiJBS0lBSlkzWVRCV1NMQzQ2SFdCQS8yMDE4MTEwMi91cy1lYXN0LTEvczMvYXdzNF9yZXF1ZXN0In0seyJ4LWFtei1hbGdvcml0aG0iOiJBV1M0LUhNQUMtU0hBMjU2In0seyJ4LWFtei1kYXRlIjoiMjAxODExMDJUMTI0NzE1WiJ9XX0=",
"x-amz-credential": "AKIAJY3YTBWSLC46HWBA/20181102/us-east-1/s3/aws4_request",
"x-amz-algorithm": "AWS4-HMAC-SHA256",
"x-amz-date": "20181102T124715Z",
"x-amz-signature": "52d8246536e8743fba8e7668cb65a08a1142221d54a58676b6ab14e3835482a3"
},
"id": 34005322,
"media_type": "InputMedia"
}
If i extract informations from the 'fields' object without a '-' in the key name that works fine. If i try to extract a key with '-' in the name i got following error:
jq: error: amz/0 is not defined at <top-level>, line 1:
.fields.x-amz-credential
jq: error: credential/0 is not defined at <top-level>, line 1:
.fields.x-amz-credential
jq: 2 compile errors
exit status 3
Update:
After the hint with the FAQ and the " i rebuild the jq command and tested it in jqplay: .fields."Content-Type" where it works as expected. Under powershell that variant didn't working since the powershell didn't excepting the quotating.
.\jq .fields."Content-Type" jq: error: Type/0 is not defined at <top-level>, line 1: .fields.Content-Type jq: 1 compile error
BR
Timo
Any tips?
Yes! If you cannot find the answer in the onine jq manual, check the jq FAQ:
𝑸: How can I access the value of a key with hyphens or $ or other
special characters in it? Why does .a.["$"] produce a syntax error?
A: The basic form for accessing the value of a key is .["KEYNAME"]
where "KEYNAME" is any valid JSON string, but recent versions of jq
also allow ."KEYNAME".
Using the basic form might require explicit use of the pipe symbol, as
in .["a-b"]|.["x-y"], but this can be abbreviated to .["a-b"]["x-y"].
I have a curl command which results in the following example json:
json={"id":"12345","key":"ABC-DEF","url":"https://google.com"}
Now, I want to parse this, and get the key out of it and store it in a variable. What I did was the following:
json={"id":"12345","key":"ABC-DEF","url":"https://google.com"}
ID=$(echo $json | jq '.key' )
But the above gives me a error as: parse error: Invalid numeric literal at line 1, column 4. Can someone help me with this? Thanks!
You will need to quote the input string so the shell doesn't do anything with your string
json='{"id":"12345","key":"ABC-DEF","url":"https://google.com"}'
ID=$(echo "$json" | jq '.key' )