I'm tying to use jq to make to exportable variables in bash. I ran in to an issue with strings that contain spaces. I tried modifying my jq but all I get are invalid character errors. I have tried both single quotes or double quotes but neither have worked for me. What am I missing here?
input:
{
"PD_TOKEN":"Token y_NbAkKc66ryYTWUXYEu",
"API":"http://cool.api/",
"HELP_URL":"https://help.com"
}
jq:
jq -r 'to_entries|map("\(.key)=\(.value|tostring)")' /json
current result:
PD_TOKEN=Token y_NbAkKc66ryYTWUXYEu
wanted result (note the quotation marks):
PD_TOKEN="Token y_NbAkKc66ryYTWUXYEu"
You can use #sh instead of tostring to escape shell-safely:
$ jq -r 'to_entries | map("\(.key)=\(.value|#sh)")[]' infile.json
PD_TOKEN='Token y_NbAkKc66ryYTWUXYEu'
API='http://cool.api/'
HELP_URL='https://help.com'
Additionally, without map:
Using the array iterator in the first step (h/tip oguz ismail):
jq -r 'to_entries[] | "\(.key)=\(.value|#sh)"' infile.json
Iterate over keys (h/tip ikegami):
jq -r 'keys[] as $key | "\($key)=\(.[$key]|#sh)"' infile.json
Related
I am trying to format a json string using jq with expected output like this:
[
{
"command": [
"printf 'this is a text'"
]
}
]
However, I cannot get it to work for the single quotes ('), e.g. $ jq -n '[{"command": ["printf 'this is a text'"]}]' gives me a compile error.
I also thought about escaping all double quotes e.g. jq -n "[{\"command\": [\"printf 'this is a text'\"]}]", this is fine however the json string is passed in from a function, I can replace all double quotes with \" first and then run the jq command but it's not very elegant.
Is there a better way to handle the single quotes inside a json string?
Here are four alternatives that should work with a bash or bash-like shell. They can be adapted for other shells as well.
jq -n $'[{"command": ["printf \'this is a text\'"]}]'
cat << EOF | jq .
[{"command": ["printf 'this is a text'"]}]
EOF
jq --arg cmd "printf 'this is a text'" -n '[{command: [ $cmd ]}]'
VAR="[{\"command\": [\"printf 'this is a text'\"]}]"
jq -n --argjson var "$VAR" '$var'
See also How to escape single quotes within single quoted strings
I have a first json file like this:
{
"env_vars": {
"TERRAFORM_CFG_TLS_CERT": "-----BEGIN CERTIFICATE----\\nMIIIqzCCB5O"
}
}
If I use the command:
echo <file> | jq -r '.env_vars'
The result is as expected (the backslash are still there):
{
"TERRAFORM_CFG_TLS_CERT": "-----BEGIN CERTIFICATE----\\nMIIIqzCCB5O"
}
But if i execute this command:
cat <file> | jq -r '.env_vars' | jq -r 'keys[] as $k | "\($k)=\"\(.[$k])\""'
The result is:
TERRAFORM_CFG_TLS_CERT: "-----BEGIN CERTIFICATE----\nMIIIqzCCB5O"
=> One backslash has been removed... why ?
How to avoid this ?
Thanks.
Using the -r option tells jq to "translate" the JSON string into a "raw" string by interpreting the characters that are special to JSON (see e.g. http://json.org). Thus, following the [mcve] guidelines a bit more closely, we could start with:
$ jq . <<< '"X\\nY"'
"X\\nY"
$ jq -r . <<< '"X\\nY"'
X\nY
If you check the json.org specification of strings, you'll see this is exactly correct.
So if for some reason you want each occurrence of \\ in the JSON string to be replaced by two backslash characters (i.e. JSON: "\\\\"), you could use sub or gsub. That's a bit tricky, because the first argument of these functions is a regex. Behold:
$ jq -r 'gsub("\\\\"; "\\\\")' <<< '"X\\nY"'
X\\nY
You should output the string as json to preserve the escapes. By taking a string and outputting it raw, you're getting exactly what that string was, a literal backslash followed by an n.
$ ... | jq -r '.env_vars | to_entries[] | "\(.key): \(.value | tojson)"'
If any of the values are non-strings, add a tostring to the filter.
I have the date in the file which looks like
test,test
test1,test1
I want to convert it into like:
{"test":"test","test1":"test1"}
I have tried jq for this purpose jq -R -s -c 'split("\n")'
But its oupting in the format ["test,test","test1,test1",""]
jq 1.5 has inputs, which allows a simple and efficient solution:
jq -R -n -c '[inputs|split(",")|{(.[0]):.[1]}] | add' input.txt
Important: don't forget the -n (--null-input) option, otherwise you'll lose the first line.
Alternative
If your jq does not have inputs, then it's time to upgrade if at all possible. Otherwise:
jq -R -s '
split("\n")
| map(if index(",") then split(",")|{(.[0]):.[1]}
else empty end)
| add' input.txt
As #peak indicates, use the inputs with the split function. But to merge the key/values into one single object, use the reduce method:
jq -Rn '[inputs|split(",")| {(.[0]): .[1]}] | reduce .[] as $obj ({}; . + $obj) ' input.csv
The reduce method reduces each item in the array into a single item. In this case, we indicate that each item should be assigned to the $obj variable, and that we start out with the empty {} object. The second argument to the reduce method indicates how to "reduce" things down to a single item. In this case, we are adding/merging the $obj we assigned with the {} object we started out with and then returning the resulting object to be used in the next iteration. After all the iterations have completed, the final item (in this case, the combined object) is returned.
What you ask is possible to achieve with just standar unix shell utilities (assuming your input in file.txt):
bash $ echo { \"$(<file.txt sed 's/,/":"/g' | paste -s -d, - | sed 's/,/","/g')\" }
{ "test":"test","test1":"test1" }
bash $
resulting output is a valid json
I have a sample json response as shown below which i am trying to parse using jq in shell script.[{"id":1,"notes":"Demo1\nDemo2"}]
This is the command through which I am trying to access notes in the shell script.
value=($(curl $URL | jq -r '.[].notes'))
When I echo "$value" I only get Demo1. How to get the exact value: Demo1\nDemo2 ?
To clarify, there is no backslash or n in the notes field. \n is JSON's way of encoding a literal linefeed, so the value you should be expecting is:
Demo1
Demo2
The issue you're seeing is because you have split the value on whitespace and created an array. Each value can be accessed by index:
$ cat myscript
data='[{"id":1,"notes":"Demo1\nDemo2"}]'
value=($(printf '%s' "$data" | jq -r '.[].notes'))
echo "The first value was ${value[0]} and the second ${value[1]}"
$ bash myscript
The first value was Demo1 and the second Demo2
To instead get it as a simple string, remove the parens from value=(..):
$ cat myscript2
data='[{"id":1,"notes":"Demo1\nDemo2"}]'
value=$(printf '%s' "$data" | jq -r '.[].notes')
echo "$value"
$ bash myscript2
Demo1
Demo2
I'm using jq to parse a JSON file as shown here. However, the results for string values contain the "double-quotes" as expected, as shown below:
$ cat json.txt | jq '.name'
"Google"
How can I pipe this into another command to remove the ""? so I get
$ cat json.txt | jq '.name' | some_other_command
Google
What some_other_command can I use?
Use the -r (or --raw-output) option to emit raw strings as output:
jq -r '.name' <json.txt
So for a file containing just {"name": "Google"} then yes
sample='{"name":"Google"}'
echo $sample| jq '.name'
"Google"
using --raw-input helps
echo $sample| jq --raw-output '.name'
Google
But I stumbled upon this question because I was using --raw-output on a json array like this
sample='[{"name":"Yahoo"},{"name":"Google"}]'
echo $sample | jq --raw-output 'map(.name)'
[
"Yahoo",
"Google"
]
And I didn't understand why the quotes remained. I came across this post, and now I know adding | .[] does the trick!
echo $sample | jq --raw-output 'map(.name)| .[]'
Yahoo
Google