jq not parsing json with spaces [duplicate] - json

This question already has answers here:
Difference between single and double quotes in Bash
(7 answers)
When to wrap quotes around a shell variable?
(5 answers)
Closed 5 years ago.
I’m trying to run the following command that reads JSON from a file and formats it with jq :
jq -n -r --arg m $(<$1) '$m | fromjson | {records:[{value:.}]}'
It produces the desired output when the input JSON does not contain spaces, such as {"test":"helloworld"} :
{
"records": [
{
"value": {
"test": "helloworld"
}
}
]
}
However, for an input like {"test":"hello world"} it would give the following error:
jq: error: syntax error, unexpected QQSTRING_START, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
world"}
jq: 1 compile error
Can’t figure out what’s causing this problem.
Thanks for any help :)

It's not a jq problem but a quoting issue (as highlighted in the error).
Change the --arg option to have the value within double quote:
arg='{"test":"hello world"}'
jq -n -r --arg m "$arg" '$m | fromjson | {records:[{value:.}]}'

You've encountered a shell issue: you are missing quotation marks around $(<$1). The hint is that the space makes a difference.
By the way, when there are several moving parts (as there are here), it would be wise to try to isolate the problem before resorting to stackoverflow. That way, you will often solve the problem yourself; if not, it will at least make it easier for others to focus on the real (and hopefully interesting) issue.

Related

adding a value with jq fails if the value has spaces [duplicate]

This question already has answers here:
Passing variables (containing spaces) to curl --data field
(3 answers)
Closed 17 days ago.
When I try to add a bash string, that contains spaces with jq an error message will be generated. The error doesn't occur, if there are no spaces.
the code that generates the error
value="value with spaces"
echo {} | jq ". +={"key":"'$value'"}"
error message
jq: error: syntax error, unexpected $end, expecting QQSTRING_TEXT or QQSTRING_INTERP_START or QQSTRING_END (Unix shell quoting issues?) at <top-level>, line 1: . +={"key":"value jq: 1 compile error
expected output
{"key":"value with spaces"}
note
without spaces in the value bash variable, the command will run just fine
You need to properly format the string you are using in jq.
You can create the variable in the same way:
VAL="value with spaces"
And using it like this:
jq -n --arg myvar "${VAL}" '{key: $myvar}'
{
"key": "value with spaces"
}

Parse JSON key starting in % with jq [duplicate]

This question already has answers here:
How to use jq when the variable has reserved characters?
(3 answers)
Closed last month.
Assume the following output from a REST API (stored as test.json here):
{
"system": {
"power": 10.5,
"%mem": 0.5,
"%cpu": 12.4
}
}
I can easily query the first item:
$ jq '.system.power' test.json
10.5
However:
$ jq '.system.%cpu' test.json
jq: error: syntax error, unexpected '%', expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.system.%cpu
jq: 1 compile error
According to JSON.org, a string is a sequence of zero or more Unicode characters, wrapped in double quotes, using backslash escapes. Any valid string can be used as a JSON key. So %cpu seems indeed valid and, indeed, jq can handle this elsewhere just fine:
$ jq '.system | has("%cpu")' test.json
true
Question: How can I extract the lower two properties from the system object?
JSON keys are always strings, but jq sometimes allows to use them without string (if they would be a valid "identifier"). You must quote it if it contains special characters such as %:
jq '.system."%cpu"' test.json
or jq '.system["%cpu"]' test.json

Retrieve value of #odata.id with jq in Redfish output

I'm parsing some json Redfish data using jq.
Trying to pull value for #odata.id from redfish.txt file below.
Invoking with recommended jq .["#odata.id"] doesn't seem to quite work to pull just value itself which is: /redfish/v1/Systems
Any suggestions welcomed. Output below... :)
Thanks,
Nick
root#ubuntu-xenial:/var/opt# cat redfish.txt
{"#odata.context":"/redfish/v1/$metadata#ServiceRoot.ServiceRoot","#odata.id":"/redfish/v1","#odata.type":"#ServiceRoot.v1_2_0.ServiceRoot","AccountService":{"#odata.id":"/redfish/v1/Managers/iDRAC.Embedded.1/AccountService"},"Chassis":{"#odata.id":"/redfish/v1/Chassis"},"Description":"Root Service","EventService":{"#odata.id":"/redfish/v1/EventService"},"Id":"RootService","JsonSchemas":{"#odata.id":"/redfish/v1/JSONSchemas"},"Links":{"Sessions":{"#odata.id":"/redfish/v1/Sessions"}},"Managers":{"#odata.id":"/redfish/v1/Managers"},"Name":"Root Service","Oem":{"Dell":{"#odata.type":"#DellServiceRoot.v1_0_0.ServiceRootSummary","IsBranded":0,"ManagerMACAddress":"d0:96:69:51:d4:70","ServiceTag":"XXXX"}},"RedfishVersion":"1.2.0","Registries":{"#odata.id":"/redfish/v1/Registries"},"SessionService":{"#odata.id":"/redfish/v1/SessionService"},"Systems":{"#odata.id":"/redfish/v1/Systems"},"Tasks":{"#odata.id":"/redfish/v1/TaskService"},"UpdateService":{"#odata.id":"/redfish/v1/UpdateService"}}
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems
{
"#odata.id": "/redfish/v1/Systems"
}
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems | jq .#odata.id
jq: error: syntax error, unexpected FIELD, expecting QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.#odata.id
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.#odata.id
jq: 2 compile errors
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems | jq .["#odata.id"]
{
"#odata.id": "/redfish/v1/Systems"
}
"/redfish/v1/Systems"
root#ubuntu-xenial:/var/opt# cat redfish.txt | jq .Systems | jq .["odata.id"]
{
"#odata.id": "/redfish/v1/Systems"
}
"/redfish/v1/Systems"
root#ubuntu-xenial:/var/opt#
You could simply use the filter:
.Systems["#odata.id"]
That is, at a bash or bash-like prompt, you'd type something like:
jq '.Systems["#odata.id"]' redfish.txt
i stumbled across this, as this is the highest rated site for a duckduckgo query of "jq" and "#odata.id"
Unfortunately the answer is incorrect ... after some messing around I have got the correct answer:
jq '.Systems.[]"#odata.id"'
Since the things in the square brackets [] are an array that are a 0 indexed (start at zero) numeric array, you'll get an error trying to reference an element called "#odata.id" (i.e. .Systems["#odata.id"]), while .Systems[1] would work (if there are two elements in the array)
If you want all the elements ... use the above, if you want a specific element, you can use:
jq '.Systems[#]."odata.id"'
There should be an earlier component (in the output) that will tell you how many elements (hopefully), something like "Members#odata.count"
BUT ... after i typed this and checked your data i realized i was solving my case, that looked like yours, but wasn't. In actually looking at your data, what you need is ...
jq '.Systems."#odata.id"'
Because you DON'T have an array, you have have normal key that just needs proper quoting ...
Hopefully this helps

How can jq reference a property with a non-identifier character in the name (like "#")? [duplicate]

This question already has answers here:
Escape field name in jq that contains '#' and '-'? [duplicate]
(2 answers)
Closed 3 years ago.
It seems unlikely this hasn't been asked, but I can not find it anywhere.
I am trying to filter response from elasticsearch and need to reference the #timestamp field. However, jq uses # to specify formatters and I find nothing in documentation about how to escape the # character.
$ jq '.hits.hits[] | ._source.#timestamp' < debug.response.json
jq: error: syntax error, unexpected $end, expecting QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.hits.hits[] | ._source.#timestamp
jq: 1 compile error
You can use the generic object index syntax, like:
.hits.hits[]._source["#timestamp"]

To be assigned value as variable in jq [duplicate]

This question already has answers here:
Passing bash variable to jq
(10 answers)
Closed 5 years ago.
json processor jq for a package.json
{
"someparent" : {
"somechild" : "oldvalue"
}
}
If I run the following command (to change oldvalue to somevalue):
jq '.someparent["somechild"] = "somevalue" "$aDirection/package.json"'
It is successfully changed. However, if I give a variable instead of someValue:
aVar="anotherValue"
jq '.someparent["somechild"] = "$aVar" "$aDirection/package.json"'
It is not working.
What I already tried:
["aVar"] #interpreted as string
["$aVar"] #interpreted as string
"$aVar" # aVar is not defined
$avar #aVar is not defined
The single quotes around the jq filter are in the wrong place. In your attempt made it is also enclosing the actual JSON file input. The right syntax when applying filter over files is
jq '<filter>' json-file
In your case the right filter is just
.someparent["somechild"] = $aVar
and to use a shell variable in to the jq, you need to use the --arg field. Putting together both of the options. The first variable after --arg is used in the context of jq and the one following it is the actual shell variable.
aVar="anotherValue"
jq --arg aVar "$aVar" '.someparent["somechild"] = $aVar' "$aDirection/package.json"
# ^^^^ Remember the single quote
# terminating here and not extended
Just to be clear: using the OP's approach can be made to work, e.g.
jq '.someparent["somechild"] = "'"$aVar"'"' "$aDirection/package.json"
but using --arg (for strings) and/or --argjson is far better. An additional option for environment variables is to use the jq function env.