This must be incredibly simple but the man page makes no sense to me.
curl example.com/json gives me
{
"stats": {
"storage_server.disk_total": XXXXXXXXXX
},
"counters": {}
}
and I want to extract the value XXXXXXXXXX of the disk_total. What is the syntax to do this?
To get deeply nested values by their key:
$ jq '.. |."storage_server.disk_total"? | select(. != null)'
.. is a shortcut for the zero-argument recurse -- an analog of the XPath // operator.
For learning how to construct jq queries, it is more useful to look at the tutorial and manual than the "man" page. There's also a FAQ.
The inner key name has a period in it, and therefore the .keyname shorthand cannot be used for it. So you could write:
.stats["storage_server.disk_total"]
or if your jq allows it:
.stats."storage_server.disk_total"
These are both abbreviations for:
.stats | .["storage_server.disk_total"]
Tho dot in `storage_server.disk_total" needs to be escaped to prevent it from being interpreted as an object key separator. so you can use:
jq '.stats."storage_server.disk_total"'
assuming that XXXXXXXXXX is a valid JSON number in your real JSON.
Related
I have the following JSON structure:
{
"host1": "$PROJECT1",
"host2": "$PROJECT2",
"host3" : "xyz",
"host4" : "$PROJECT4"
}
And the following environment variables in the shell:
PROJECT1="randomtext1"
PROJECT2="randomtext2"
PROJECT4="randomtext3"
I want to check the values for each key, if they have a "$" character in them, replace them with their respective environment variable(which is already present in the shell) so that my JSON template is rendered with the correct environment variables.
I can use the --args option of jq but there are quite a lot of variables in my actual JSON template that I want to render.
I have been trying the following:
jq 'with_entries(.values as v | env.$v)
Basically making each value as a variable, then updating its value with the variable from the env object but seems like I am missing out on some understanding. Is there a straightforward way of doing this?
EDIT
Thanks to the answers on this question, I was able to achieve my larger goal for a part of which this question was asked
iterating over each value in an object,
checking its value,
if it's a string and starts with the character "$"
use the value to update it with an environment variable of the same name .
if it's an array
use the value to retrieve an environment variable of the same name
split the string with "," as delimiter, which returns an array of strings
Update the value with the array of strings
jq 'with_entries(.value |= (if (type=="array") then (env[.[0][1:]] | split(",")) elif (type=="string" and startswith("$")) then (env[.[1:]]) else . end))'
You need to export the Bash variables to be seen by jq:
export PROJECT1="randomtext1"
export PROJECT2="randomtext2"
export PROJECT4="randomtext3"
Then you can go with:
jq -n 'with_entries((.value | select(startswith("$"))) |= env[.[1:]])'
and get:
{
"host1": "randomtext1",
"host2": "randomtext2",
"host3": "xyz",
"host4": "randomtext3"
}
Exporting a large number of shell variables might not be such a good idea and does not address the problem of array-valued variables. It might therefore be a good idea to think along the lines of printing the variable=value details to a file, and then combining that file with the template. It’s easy to do and examples on the internet abound and probably here on SO as well. You could, for example, use printf like so:
printf "%s\t" ${BASH_VERSINFO[#]}
3 2 57 1
You might also find declare -p helpful.
See also https://github.com/stedolan/jq/wiki/Cookbook#arbitrary-strings-as-template-variables
I have this json:
{"temperature":"21", "humidity":"12.3", "message":"Today ID 342 is running"}
I want to use jq to obtain this json:
{"temp":"21", "hum":"12.3", "id":"342"}
As you can see, what i want to do is extract the ID number 342 and put it in the new json with a different key name. I think i should use a regex but i don't know how to insert it in jq syntax.
I can create another json using the basic command:
cat old.json | jq '{temp:.temperature,hum:.humidity, id:.message}' > new.json
I know i can select substring using square brackets, but i don't want to use them because they don't take into account strings with different lengths and structure. I want to use a regex because i know that the ID number comes lways after the "ID" part.
You're right that a regex is the way to go here. Fortunately, the jq manual has a large section on using them.
jq '
{
temp: .temperature,
hum: .humidity,
id: (.message | capture("ID (?<id>[[:digit:]]+)").id)
}' <old.json >new.json
You can see this running with your sample data at https://jqplay.org/s/k-ZylbOC6W
Consider a JSON object like
{
"foo": 42,
"baz": -12,
"bar{label1=\"value1\"}": 12.34
}
constructed by jq using some data source. The actual key names and their amount may vary, but the result will always be an object with numbers (int or float) as values. The keys may contain quotation marks, but no whitespaces.
Can I use jq to format the object into a Prometheus-compatible format so I can just use the output to push the data to a Prometheus Pushgateway?
The required result would look like
foo 42
bar{label1="value1"} 12.34
baz -12
i.e. space-separated with newlines (no \r) and without quotes except for the label value.
I can't use bash for post-processing and would therefore prefer a pure jq solution if possible.
Use keys_unsorted to get object keys (keys does the same as well but the former is faster), generate desired output by means of string interpolation.
$ jq -r 'keys_unsorted[] as $k | "\($k) \(.[$k])"' file
foo 42
baz -12
bar{label1="value1"} 12.34
And, by adding -j option and printing line feed manually as #peak suggested you can make this portable.
On a Windows platform, jq will normally use CR-LF for newlines; to prevent this, use the -j command-line option and manually insert the desired 'newline' characters like so:
jq -rj 'to_entries[] | "\(.key) \(.value)\n"' file
I have a JSON document that looks like:
simple: 42
normal:
description: "NORMAL"
combo:
one:
description: "ONE"
two:
description: "TWO"
arbitrary:
foo: 42
I want to use a jq expression to generate the following:
["normal", "one", "two"]
The condition to select the key is that its corresponding value is an object type that has a key description. In this case, keys simple and arbitrary don't qualify.
I'm having a hard time to craft the filter. Looked into with_entries and recurse/2 but can't solve it myself.
TIA.
It's not clear to me whether the YAML that you gave is just a "view" of your JSON or whether you actually want to start with YAML. If your document really is YAML, then one approach would be to use a tool
(such as yaml2json or yq) to convert the yaml to JSON, and then run jq
as shown below; another would be to use jq as a text-processor,
but in that case you could just as well use awk.
yaml2json input.yaml |
jq -c '[.. | objects | to_entries[]
| select(.value | has("description")?) | .key]'
Output
["normal","one","two"]
Streaming parser
This type of problem is also well-suited to jq's streaming parser, which is especially handy when dealing with very large JSON texts. Using jq --stream, a suitable jq filter would be:
[select(length==2) | .[0] | select(.[-1] == "description") | .[-2]]
The ordering of the results will depend on the ordering of the keys produced by the YAML-to-JSON conversion tool.
I'm trying to use jq to get a value from the JSON that cURL returns.
This is the JSON cURL passes to jq (and, FTR, I want jq to return "VALUE-I-WANT" without the quotation marks):
[
{
"success":{
"username":"VALUE-I-WANT"
}
}
]
I initially tried this:
jq ' . | .success | .username'
and got
jq: error (at <stdin>:0): Cannot index array with string "success"
I then tried a bunch of variations, with no luck.
With a bunch of searching the web, I found this SE entry, and thought it might have been my saviour (spoiler, it wasn't). But it led me to try these:
jq -r '.[].success.username'
jq -r '.[].success'
They didn't return an error, they returned "null". Which may or may not be an improvement.
Can anybody tell me what I'm doing wrong here? And why it's wrong?
You need to pipe the output of .[] into the next filter.
jq -r '.[] | .success.username' tmp.json
tl;dr
# Extract .success.username from ALL array elements.
# .[] enumerates all array elements
# -r produces raw (unquoted) output
jq -r '.[].success.username' file.json
# Extract .success.username only from the 1st array element.
jq -r '.[0].success.username' file.json
Your input is an array, so in order to access its elements you need .[], the array/object-value iterator (as the name suggests, it can also enumerate the properties of an object):
Just . | sends the input (.) array as a whole through the pipeline, and an array only has numerical indices, so the attempt to index (access) it with .success.username fails.
Thus, simply replacing . | with .[] | in your original attempt, combined with -r to get raw (unquoted output), should solve your problem, as shown in chepner's helpful answer.
However, peak points out that since at least jq 1.3 (current as of this writing is jq 1.5) you don't strictly need a pipeline, as demonstrated in the commands at the top.
So the 2nd command in your question should work with your sample input, unless you're using an older version.