I am trying to get values from a json file from a url using curl and then printing specific keys with jq command (e.g. company). Unfortunately when I use:
jq '.[] | .company' JB.json
I get the error:
parse error: Invalid numeric literal at line 1, column 9. I have checked the downloaded file with less and it looks exactly like in the url.
Some people suggested to use the -R option but it prints:
jq: error: Cannot iterate over string
The url of the file is: https://jobs.github.com/positions.json?description=python&location=new+york
If you use curl -sS to retrieve the file, the '//'-style comments are skipped:
curl -Ss 'https://jobs.github.com/positions.json?description=python&location=new+york' | jq '.[] | .company'
"BentoBox"
"Aon Cyber Solutions"
"Sesame"
"New York University"
So presumably your JB.json contains the "//"-style comments. The simplest workaround would probably be to filter out those first two lines (e.g. using sed (or jq!)) first.
Here's a jq-only solution:
< JB.json jq -Rr 'select( test("^//")|not)' |
jq '.[] | .company'
Related
I see questions about selecting multiple values from an array using JQ, but I have a string that originally I just need the value after the last /, which is easily selected:
Input:
https://www.googleapis.com/compute/v1/projects/test-project-1/zones/europe-west1-b/instanceGroups/test-instance-group-1
JQ:
jq -r '.[]|.zone|=split("/")[-1]|"\(.name) \(.zone)"'
Output:
test-instance-group-1 europe-west1-b
However for the actual instances, the zone isn't listed, so must be extracted from the same key instance.
Input:
https://www.googleapis.com/compute/v1/projects/test-project-1/zones/europe-west1-b/instances/test-instance-1
JQ:
jq -r '.[]|.instance|=split("/")[-1]|"\(.instance)'
Output:
test-instance-1
However, I also want to extract the zone infomation, from the input as well, which I presume is selected with =split("/")[-3] however no matter how I format the request to JQ, I get errors:
$ jq -r '.[]|.instance|=split("/")[-1][-3]|"\(.instance)"'
jq: error (at <stdin>:47): Cannot index string with number
How can I extract two strings, from the same value/key ?
You're looking for something like this:
jq -r '.[].instance | split("/") | "\(.[-1]) \(.[-3])"'
I'm trying to call an api and in the same line parse its output with jq.
If I do all in one line it gives me an error:
bash-4.2$ call_api -s "job=work_1*&status=Executing" | $PATH_API/jq '.statuses[].name // empty' --raw-output
parse error: Unfinished string at EOF at line 2301, column 61
But if I redirect the api's output to a file and then apply jq to that file it doesn't raise any error:
bash-4.2$ call_api -s "job=work_1*&status=Executing" > /tmp/kk
bash-4.2$ $PATH_API/jq '.statuses[].name // empty' --raw-output /tmp/kk
work_1_002103ALLGDC
work_1_000049EVEFIN
work_1_002038ALF003
Do you know why? Is it possible to fix it to can achieve it in one line?
This might be a problem with call_api itself - since it's not defined what that is. Some programs (e.g. curl, more) behave slightly differently depending on where their output goes. Typically the big distinction is between tty and not-a-tty but you may find differences between pipe and regular file. There should be no inherent difference in jq's behavior.
richard#sophia:~/cc/dpbuilder (bug/T-10571) $ jq -r '.[]' < fred.json
Hello
World
richard#sophia:~/cc/dpbuilder (bug/T-10571) $ jq -r '.[]' fred.json
Hello
World
richard#sophia:~/cc/dpbuilder (bug/T-10571) $ cat fred.json | jq -r '.[]'
Hello
World
richard#sophia:~/cc/dpbuilder (bug/T-10571) $
Also check how large the output is - you may be running into limits in terms of piped output. In particular, is it greater than 64k?
I'm parsing some json Redfish data using jq.
Trying to pull value for #odata.id from redfish.txt file below.
Invoking with recommended jq .["#odata.id"] doesn't seem to quite work to pull just value itself which is: /redfish/v1/Systems
Any suggestions welcomed. Output below... :)
Thanks,
Nick
root#ubuntu-xenial:/var/opt# cat redfish.txt
{"#odata.context":"/redfish/v1/$metadata#ServiceRoot.ServiceRoot","#odata.id":"/redfish/v1","#odata.type":"#ServiceRoot.v1_2_0.ServiceRoot","AccountService":{"#odata.id":"/redfish/v1/Managers/iDRAC.Embedded.1/AccountService"},"Chassis":{"#odata.id":"/redfish/v1/Chassis"},"Description":"Root Service","EventService":{"#odata.id":"/redfish/v1/EventService"},"Id":"RootService","JsonSchemas":{"#odata.id":"/redfish/v1/JSONSchemas"},"Links":{"Sessions":{"#odata.id":"/redfish/v1/Sessions"}},"Managers":{"#odata.id":"/redfish/v1/Managers"},"Name":"Root Service","Oem":{"Dell":{"#odata.type":"#DellServiceRoot.v1_0_0.ServiceRootSummary","IsBranded":0,"ManagerMACAddress":"d0:96:69:51:d4:70","ServiceTag":"XXXX"}},"RedfishVersion":"1.2.0","Registries":{"#odata.id":"/redfish/v1/Registries"},"SessionService":{"#odata.id":"/redfish/v1/SessionService"},"Systems":{"#odata.id":"/redfish/v1/Systems"},"Tasks":{"#odata.id":"/redfish/v1/TaskService"},"UpdateService":{"#odata.id":"/redfish/v1/UpdateService"}}
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems
{
"#odata.id": "/redfish/v1/Systems"
}
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems | jq .#odata.id
jq: error: syntax error, unexpected FIELD, expecting QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.#odata.id
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.#odata.id
jq: 2 compile errors
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems | jq .["#odata.id"]
{
"#odata.id": "/redfish/v1/Systems"
}
"/redfish/v1/Systems"
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems | jq .["odata.id"]
{
"#odata.id": "/redfish/v1/Systems"
}
"/redfish/v1/Systems"
root#ubuntu-xenial:/var/opt#
You could simply use the filter:
.Systems["#odata.id"]
That is, at a bash or bash-like prompt, you'd type something like:
jq '.Systems["#odata.id"]' redfish.txt
i stumbled across this, as this is the highest rated site for a duckduckgo query of "jq" and "#odata.id"
Unfortunately the answer is incorrect ... after some messing around I have got the correct answer:
jq '.Systems.[]"#odata.id"'
Since the things in the square brackets [] are an array that are a 0 indexed (start at zero) numeric array, you'll get an error trying to reference an element called "#odata.id" (i.e. .Systems["#odata.id"]), while .Systems[1] would work (if there are two elements in the array)
If you want all the elements ... use the above, if you want a specific element, you can use:
jq '.Systems[#]."odata.id"'
There should be an earlier component (in the output) that will tell you how many elements (hopefully), something like "Members#odata.count"
BUT ... after i typed this and checked your data i realized i was solving my case, that looked like yours, but wasn't. In actually looking at your data, what you need is ...
jq '.Systems."#odata.id"'
Because you DON'T have an array, you have have normal key that just needs proper quoting ...
Hopefully this helps
Utilizing the code at How do I use a shell-script as Chrome Native Messaging host application as a template and given the file file.json which contains
{"text":"abc"}
following the code at Iterate over json with jq and the jq documentation
$ cat file.json | jq --raw-output '.text'
outputs
abc
Am not certain how to incorporate the pattern at this Answer
while read -r id name date; do
echo "Do whatever with ${id} ${name} ${date}"
done< <(api-producing-json | jq --raw-output '.newList[] | "\(.id) \(.name) \(.create.date)"')
into the template at the former Answer for the purpose of capturing the single property "text" (abc) from the JSON within the loop using jq for the ability to pass that text to another system call then printf the message to client.
What we are trying to achieve is
json=$(<bash program> <captured JSON property>)
message='{"message": "'$json'"}'
where the {"text":"abc"} is sent to the Native Messaging host from client (Chromium app).
How to use jq within the code at the former Answer to get the JSON property as a variable?
Assuming that file.json contains the JSON as indicated, I believe all you will need is:
json=$(jq '{message: .}' file.json)
If you then echo "$json", the result will be:
{
"message": {
"text": "abc"
}
}
I'm using jq to parse some of my logs, but some of the log lines can't be parsed for various reasons. Is there a way to have jq ignore those lines? I can't seem to find a solution. I tried to use the --seq argument that was recommended by some people, but --seq ignores all the lines in my file.
Assuming that each log entry is exactly one line, you can use the -R or --raw-input option to tell jq to leave the lines unparsed, after which you can prepend fromjson? | to your filter to make jq try to parse each line as JSON and throw away the ones that error.
I have log stream where some messages are in json format.
I want to pipe the json messages through jq, and just echo the rest.
The json messages are on a single line.
Solution: use grep and tee to split the lines in two streams, those starting with "^{" pipe through jq and the rest just echo to terminal.
kubectl logs -f web-svjkn | tee >(grep -v "^{") | grep "^{" | jq .
or
cat logs | tee >(grep -v "^{") | grep "^{" | jq .
Explanation:
tee generates 2nd stream, and grep -v prints non json info, 2nd grep only pipes what looks like json opening bracket to jq.
This is an old thread, but here's another solution fully in jq. This allows you to both process proper json lines and also print out non-json lines.
jq -R . as $line | try (fromjson | <further processing for proper json lines>) catch $line'
There are several Q&As on the FAQ page dealing with the topic of "invalid JSON", but see in particular the Q:
Is there a way to have jq keep going after it hits an error in the input file?
In particular, this shows how to use --seq.
However, from the the sparse details you've given (SO recommends a minimal example be given), it would seem it might be better simply to use inputs. The idea is to process one JSON entity at a time, using "try/catch", e.g.
def handle: inputs | [., "length is \(length)"] ;
def process: try handle catch ("Failed", process) ;
process
Don't forget to use the -n option when invoking jq.
See also Processing not-quite-valid JSON.
If JSON in curly braces {}:
grep -Pzo '\{(?>[^\{\}]|(?R))*\}' | jq 'objects'
If JSON in square brackets []:
grep -Pzo '\[(?>[^\[\]]|(?R))*\]' | jq 'arrays'
This works if there are no []{} in non-JSON lines.