I know that jq can take serialized JSON strings to format/filter/modify the content. However, as far as I know the output is always human readable form of JSON. Is there a way in Bash or jq itself, to stringify the output?
For example, given
echo '{"foo" : "bar"}' | jq -r .
I'm looking for an output that is "{\"foo\" : \"bar\"}"
You want -R, aka --raw-input, to make jq treat your input as a string instead of as an object.
echo '{"foo" : "bar"}' | jq -R .
By "stringify", you seem to mean "encode a JSON value as a JSON string".
$ echo '{"foo": "bar"}' | jq '#json'
"{\"foo\":\"bar\"}"
Related
For example:
echo '{"p":2}{"q":3}' | jq '.'
How do I select the first object? I want the object below:
{"p":2}
You would use the -n command-line option, e.g.:
jq -n input
or
jq -n 'first(inputs)'
Hello I am trying to write unix script/command where I have to list out all filenames from given directory with filename format string-{number}.txt(eg: filename-1.txt,filename-2.txt) from which I have to form a json object. any pointers would be helpful.
[{
"filenumber": "1",
"name": "filename-1.txt"
},
{
"filenumber": "2",
"name": "filename-2.txt"
}
]
In the above json file-number should be read from {number} format of the each filename
A single call to jq should suffice :
shopt -s extglob
printf "%s\0" *-+([0-9]).txt | \
jq -sR 'split("\u0000") |
map({filenumber:capture(".*-(?<n>.*)\\.txt").n,
name:.})'
Very easy for the command-line tool xidel and its integrated EXPath File Module:
$ xidel -se '
array{
for $x in file:list(.,false(),"*.txt")
return {
"filenumber":extract($x,"(\d+)\.txt",1),
"name":$x
}
}
'
Intuitively, I'd say you can do this with jq. However, in practice I've rarely been able to achieve what I wanted with jq :-)
With some lunch break puzzling, I've come up with this beauty:
ls | jq -R '{filenumber:input_line_number, name:.}' | jq -s .
Instead of ls you could use any other command that produces a newline separated list of strings.
I have tried with multiple examples to achieve exact use case of mine and finally found this working fine exactly how I wanted Thanks
for file in $(ls *.txt); do file_version=$(echo $file | sed 's/\(^.*-\)\(.*\)\(.txt.*$\)/\2/'); jq -n --arg name "$file_version" --arg path "$file" '{name: $name, name: $path}'; done | jq -n '.urls |= [inputs]'
I have a first json file like this:
{
"env_vars": {
"TERRAFORM_CFG_TLS_CERT": "-----BEGIN CERTIFICATE----\\nMIIIqzCCB5O"
}
}
If I use the command:
echo <file> | jq -r '.env_vars'
The result is as expected (the backslash are still there):
{
"TERRAFORM_CFG_TLS_CERT": "-----BEGIN CERTIFICATE----\\nMIIIqzCCB5O"
}
But if i execute this command:
cat <file> | jq -r '.env_vars' | jq -r 'keys[] as $k | "\($k)=\"\(.[$k])\""'
The result is:
TERRAFORM_CFG_TLS_CERT: "-----BEGIN CERTIFICATE----\nMIIIqzCCB5O"
=> One backslash has been removed... why ?
How to avoid this ?
Thanks.
Using the -r option tells jq to "translate" the JSON string into a "raw" string by interpreting the characters that are special to JSON (see e.g. http://json.org). Thus, following the [mcve] guidelines a bit more closely, we could start with:
$ jq . <<< '"X\\nY"'
"X\\nY"
$ jq -r . <<< '"X\\nY"'
X\nY
If you check the json.org specification of strings, you'll see this is exactly correct.
So if for some reason you want each occurrence of \\ in the JSON string to be replaced by two backslash characters (i.e. JSON: "\\\\"), you could use sub or gsub. That's a bit tricky, because the first argument of these functions is a regex. Behold:
$ jq -r 'gsub("\\\\"; "\\\\")' <<< '"X\\nY"'
X\\nY
You should output the string as json to preserve the escapes. By taking a string and outputting it raw, you're getting exactly what that string was, a literal backslash followed by an n.
$ ... | jq -r '.env_vars | to_entries[] | "\(.key): \(.value | tojson)"'
If any of the values are non-strings, add a tostring to the filter.
I'm using jq to parse a JSON file as shown here. However, the results for string values contain the "double-quotes" as expected, as shown below:
$ cat json.txt | jq '.name'
"Google"
How can I pipe this into another command to remove the ""? so I get
$ cat json.txt | jq '.name' | some_other_command
Google
What some_other_command can I use?
Use the -r (or --raw-output) option to emit raw strings as output:
jq -r '.name' <json.txt
So for a file containing just {"name": "Google"} then yes
sample='{"name":"Google"}'
echo $sample| jq '.name'
"Google"
using --raw-input helps
echo $sample| jq --raw-output '.name'
Google
But I stumbled upon this question because I was using --raw-output on a json array like this
sample='[{"name":"Yahoo"},{"name":"Google"}]'
echo $sample | jq --raw-output 'map(.name)'
[
"Yahoo",
"Google"
]
And I didn't understand why the quotes remained. I came across this post, and now I know adding | .[] does the trick!
echo $sample | jq --raw-output 'map(.name)| .[]'
Yahoo
Google
I have a JSON and I need to extract a base64-encoded value by particular key and decode it.
JSON has the following structure:
[
{
"LockIndex": 0,
"Key": "Arul/key1",
"Flags": 0,
"Value": "MzAKCg==",
"CreateIndex": 369,
"ModifyIndex": 554
}
]
In the above JSON, I need to extract only "Value":"MzAKCg==" and decode the base64-encoded "MzAKCg==" value. I would like to perform this using shell scripting.
Please assist.
jq has recently added support for base64 encoding and decoding
https://stedolan.github.io/jq/manual/#Formatstringsandescaping
#base64:
The input is converted to base64 as specified by RFC 4648.
#base64d:
The inverse of #base64, input is decoded as specified by RFC 4648.
Note: If the decoded string is not UTF-8, the results are undefined.
For your data, the command would be
jq -r 'map(.Value | #base64d)' < file.json
https://github.com/stedolan/jq/issues/47
It's not released yet, but you can install the latest development version to use it.
brew reinstall --HEAD jq
Once the next version of jq is released then you can switch back to the latest stable version.
Using jq and base64:
jq -r '.[].Value' < file.json | base64 --decode
if this JSON will have always the same structure, you can use cut -d and later decode the value, for example:
$echo "Value": "MzAKCg==" | cut -d ":" -f 2 | base64 -D
30
This worked for me: curl --silent localhost:8500/v1/kv/key1 | jq -r '.[0].Value | #base64d'
use built-in in jq base64 decoder:
echo '{"val" : "MzAKCg=="}' | jq -r '.val |=#base64d'
yields:
{ "val": "30\n\n" }
side note: jq is simply a file symbolic for jq-win64.exe
if you want to keep the origin struct, you could use like jq -r '.|=map(.key|=#base64d)
With xidel:
xidel -s input.json -e '$json()/binary-to-string(base64Binary(Value))'
or
xidel -s input.json -e 'binary-to-string(base64Binary($json//Value))'