I have a JSON where I am storing it in a variable in my Bash script like this:
RAW_JSON="{"secretKey": "ADFGHJKGBNJK"}"
I wanted to store the value of secretKey in another variable called secretValue. How can I do that?
To preserve double quotation (") marks around the key and value, the syntax is:
$ RAW_JSON='{"secretKey": "ADFGHJKGBNJK"}'
$ echo "$RAW_JSON"
{"secretKey": "ADFGHJKGBNJK"}
Then jq can be used to return the value:
$ secretValue=$(echo "$RAW_JSON" | jq -r .secretKey)
$ echo "$secretValue"
ADFGHJKGBNJK
The -r parameter removes the quotes from the result.
Related
I have a simple file with this test line:
mmm#gmail.com 31460 147557432
My goal is to send as json data.
In my while loop I can echo the variables in the second line of my code example.
However, when I attempt to assign them to jsonstring and echo, the values are not populated.
What do I need to do to pass these values to my json string?
while read emailvar idvar expirevar; do
echo "$emailvar:$expirevar:$idvar"
jsonstring=$idvar $emailvar $expirevar
echo "$jsonstring"
#jsonstring='{"user_id":"$idvar","email":"$emailvar","custom_attributes":{"Program_Expires_at":"$expirevar"}}'
done < "tempdata.txt"
#!/bin/bash
while read line;
do
line_array=($line)
emailvar=${line_array[0]}
expirevar=${line_array[1]}
idvar=${line_array[2]}
jsonstring='{"user_id": "'$idvar'", "email": "'$emailvar'", "custom_attributes":{"Program_Expires_at": "'$expirevar'"}'
echo $jsonstring
done < 'tempdata.txt'
Output:
You have to escape the whitespace to make it part of the string, rather than creating a simple command with some pre-command assignments.
jsonstring=$idvar\ $emailvar\ $expirevar
more commonly written as
jsonstring="$idvar $emailvar $expirevar"
In your commented assignment, you used single quotes, which prevent parameter expansion. You need to use double quotes, which requires manually escaping the interior double quotes. More robust, though, is to use a tool like jq to generate the JSON for you: it will take care of escaping any characters in your variables to generate valid JSON.
jsonstring=$(jq -n \
--arg id "$idvar" \
--arg email "$emailvar" \
--arg expire "$expirevar" \
'{user_id: $id,
email: $email,
custom_attributes: {Program_Expires_at: $expire}}'
)
It seems this is essentially a problem with how bash handles variables and parameter expansion. I believe the solution here basically adding bunch of double quotes.
Double quotes can be used to enable parameter expansion for multiple variables. For JSON output in this bash script, we'll need to use nested double-quotes.
To fix this, we can:
put double quotes (") surrounding the value for jsonstring
escape double quotes surrounding strings used within the value for jsonstring with \
If you'd like $idvar and $expirevar to be interpreted as numbers instead of strings, you don't need escaped double-quotes around these values.
For example:
#!/bin/bash
while read emailvar idvar expirevar; do
jsonstring="{\"user_id\":$idvar,\"email\":\"$emailvar\",\"custom_attributes\":{\"Program_Expires_at\":$expirevar}}"
echo "$jsonstring"
done < "tempdata.txt"
Example output:
user#pc: bash ./script.sh
{"user_id":31460,"email":"mmm#gmail.com","custom_attributes":{"Program_Expires_at":147557432}}
user#pc: bash ./script.sh | jq .
{
"user_id": 31460,
"email": "mmm#gmail.com",
"custom_attributes": {
"Program_Expires_at": 147557432
}
}
I'm tying to use jq to make to exportable variables in bash. I ran in to an issue with strings that contain spaces. I tried modifying my jq but all I get are invalid character errors. I have tried both single quotes or double quotes but neither have worked for me. What am I missing here?
input:
{
"PD_TOKEN":"Token y_NbAkKc66ryYTWUXYEu",
"API":"http://cool.api/",
"HELP_URL":"https://help.com"
}
jq:
jq -r 'to_entries|map("\(.key)=\(.value|tostring)")' /json
current result:
PD_TOKEN=Token y_NbAkKc66ryYTWUXYEu
wanted result (note the quotation marks):
PD_TOKEN="Token y_NbAkKc66ryYTWUXYEu"
You can use #sh instead of tostring to escape shell-safely:
$ jq -r 'to_entries | map("\(.key)=\(.value|#sh)")[]' infile.json
PD_TOKEN='Token y_NbAkKc66ryYTWUXYEu'
API='http://cool.api/'
HELP_URL='https://help.com'
Additionally, without map:
Using the array iterator in the first step (h/tip oguz ismail):
jq -r 'to_entries[] | "\(.key)=\(.value|#sh)"' infile.json
Iterate over keys (h/tip ikegami):
jq -r 'keys[] as $key | "\($key)=\(.[$key]|#sh)"' infile.json
I am trying to format a json string using jq with expected output like this:
[
{
"command": [
"printf 'this is a text'"
]
}
]
However, I cannot get it to work for the single quotes ('), e.g. $ jq -n '[{"command": ["printf 'this is a text'"]}]' gives me a compile error.
I also thought about escaping all double quotes e.g. jq -n "[{\"command\": [\"printf 'this is a text'\"]}]", this is fine however the json string is passed in from a function, I can replace all double quotes with \" first and then run the jq command but it's not very elegant.
Is there a better way to handle the single quotes inside a json string?
Here are four alternatives that should work with a bash or bash-like shell. They can be adapted for other shells as well.
jq -n $'[{"command": ["printf \'this is a text\'"]}]'
cat << EOF | jq .
[{"command": ["printf 'this is a text'"]}]
EOF
jq --arg cmd "printf 'this is a text'" -n '[{command: [ $cmd ]}]'
VAR="[{\"command\": [\"printf 'this is a text'\"]}]"
jq -n --argjson var "$VAR" '$var'
See also How to escape single quotes within single quoted strings
I have a sample json response as shown below which i am trying to parse using jq in shell script.[{"id":1,"notes":"Demo1\nDemo2"}]
This is the command through which I am trying to access notes in the shell script.
value=($(curl $URL | jq -r '.[].notes'))
When I echo "$value" I only get Demo1. How to get the exact value: Demo1\nDemo2 ?
To clarify, there is no backslash or n in the notes field. \n is JSON's way of encoding a literal linefeed, so the value you should be expecting is:
Demo1
Demo2
The issue you're seeing is because you have split the value on whitespace and created an array. Each value can be accessed by index:
$ cat myscript
data='[{"id":1,"notes":"Demo1\nDemo2"}]'
value=($(printf '%s' "$data" | jq -r '.[].notes'))
echo "The first value was ${value[0]} and the second ${value[1]}"
$ bash myscript
The first value was Demo1 and the second Demo2
To instead get it as a simple string, remove the parens from value=(..):
$ cat myscript2
data='[{"id":1,"notes":"Demo1\nDemo2"}]'
value=$(printf '%s' "$data" | jq -r '.[].notes')
echo "$value"
$ bash myscript2
Demo1
Demo2
I am writing a bash script to use with badips.com
This command:
wget https://www.badips.com/get/key -qO -
Will return something like this:
{"err":"","suc":"new key 5f72253b673eb49fc64dd34439531b5cca05327f has been set.","key":"5f72253b673eb49fc64dd34439531b5cca05327f"}
Or like this:
{"err":"","suc":"Your Key was already present! To overwrite, see http:\/\/www.badips.com\/apidoc.","key":"5f72253b673eb49fc64dd34439531b5cca05327f"}
I need to parse the key value out (5f72253b673eb49fc64dd34439531b5cca05327f) into a variable in the script. I would prefer to use grep to do it but can't get it right.
Instead of parsing with some grep, you have the perfect tool for this: jq.
See:
jq '.key' file
or
.... your_commands .... | jq '.key'
will return
"5f72253b673eb49fc64dd34439531b5cca05327f"
See another example, for example to get the suc attribute:
$ cat a
{"err":"","suc":"new key 5f72253b673eb49fc64dd34439531b5cca05327f has been set.","key":"5f72253b673eb49fc64dd34439531b5cca05327f"}
{"err":"","suc":"Your Key was already present! To overwrite, see http:\/\/www.badips.com\/apidoc.","key":"5f72253b673eb49fc64dd34439531b5cca05327f"}
$ jq '.suc' a
"new key 5f72253b673eb49fc64dd34439531b5cca05327f has been set."
"Your Key was already present! To overwrite, see http://www.badips.com/apidoc."
You could try the below grep command,
grep -oP '"key":"\K[^"]*(?=")' file
Using perl :
wget https://www.badips.com/get/key -qO - |
perl -MJSON -MFile::Slurp=slurp -le '
my $s = slurp "/dev/stdin";
my $d = JSON->new->decode($s);
print $d->{key}
'
Not as strong as precedent one, but that don't require to install new modules, a stock perl can do it :
wget https://www.badips.com/get/key -qO - |
perl -lne 'print $& if /"key":"\K[[:xdigit:]]+/'
awk keeps it simple
wget ... - | awk -F: '{split($NF,k,"\"");print k[2]}'
the field separator is :;
the key is always in the last field, in awk this field is accessed using $NF (Number of Fields);
the split function splits $NF and puts the pieces in array k, according to separator "\"" that is just a single double quote character;
the second field of the k array is what you want.