I have a specific problem. I make email variable in bash script like:
tDestinationEmail=email#email.com
and if I enter an email after the sign = my json should look like this:
{"ToAddresses": ["email#email.com"]}
I try to do it with
tDestinationEmailJSON='{"ToAddresses": ["$tDestinationEmail"]}'
but on the output i getting
{"ToAddresses": ["$tDestinationEmail"]}
Maybe whom know solution how i can do it ? Please suggest!
jq is the right tool for JSON data manipulation:
tDestinationEmail="email#email.com"
jq -nc --arg email "$tDestinationEmail" '{ToAddresses: [$email]}'
The output:
{"ToAddresses":["email#email.com"]}
From man page of bash:
"Enclosing characters in single quotes preserves the literal value of each character within the quotes."
So when you call $tDestinationEmail will be treated as string and will not call the value of the variable.
Using double quotes:
tDestinationEmailJSON='{"ToAddresses": ['"\"$tDestinationEmail\""']}'
Related
I am writing a JSON template for AWS SSM document.
Once of the commands I am trying to run looks like this:
"ssh-keygen -q -t rsa -N '' <<< ""$'\n'"y" 2>&1 >/dev/null",
However, due to the "" after the <<< JSON thinks that this is a new line and is expecting a comma.
Is there a way I can tell JSON that that is a single command and need to be treated as a single line?
You need to escape the double quotes. So " inside your json string becomes \".
I have this which works to declare a JSON string in a bash script:
local my_var="foobar"
local json=`cat <<EOF
{"quicklock":"${my_var}"}
EOF`
The above heredoc works, but I can't seem to format it any other way, it literally has to look exactly like that lol.
Is there any way to get the command to be on one line, something like this:
local json=`cat <<EOF{"quicklock":"${my_var}"}EOF`
that would be so much nicer, but doesn't seem to take, obviously simply because that's not how EOF works I guess lol.
I am looking for a shorthand way to declare JSON in a file that:
Does not require a ton of escape chars.
That allows for dynamic interpolation of variables.
Note: The actual JSON I want to use has multiple dynamic variables with many key/value pairs. Please extrapolate.
I'm not a JSON guy, don't really understand the "well-formed" arguments in the discussion above, but, you can use a 'here-string' rather than a 'here-document', like this:
my_var="foobar"
json=`cat <<<{\"quicklock\":\"${my_var}\"}`
why not use jq? It's pretty good at managing string interpolation and it lints your structure.
$ echo '{}' >> foo.json
$ declare myvar="assigned-var"
$ jq --arg ql "$myvar" '.quicklock=$ql' foo.json
the text that comes out on the other end of that call to jq can then be cat into a file or whatever you wanna do. text would look something like this:
{"quicklock": "assigned-var"}
You can do this with printf:
local json="$(printf '{"quicklock":"%s"}' "$my_var")"
(Never mind that SO's syntax highlighting looks odd here. Posix shell command substitution allows nesting one level of quotes.)
A note (thanks to Charles Duffy's comment on the question): I'm assuming $my_var is not controlled by user input. If it is, you'll need to be careful to ensure it is legal for a JSON string. I highly recommend barring non-ASCII characters, double quotes, and backslashes. If you have jq available, you can use it as Charles noted in the comments to ensure you have well-formed output.
You can define your own helper function to address the situation with missing bash syntax:
function begin() { eval echo $(sed "${BASH_LINENO[0]}"'!d;s/.*begin \(.*\) end.*/\1/;s/"/\\\"/g' "${BASH_SOURCE[0]}"); }
Then you can use it as follows.
my_var="foobar"
json=$(begin { "quicklock" : "${my_var}" } end)
echo "$json"
This fragment displays the desired output:
{ "quicklock" : "foobar" }
This is just a proof of concept. You can define your syntax in any way you want (such as end of the input by the custom EOF string, correctly escape invalid characters). For example, since Bash allows function identifiers using characters other than alphanumeric characters, it is possible to define such a syntax:
json=$(/ { "quicklock" : "${my_var}" } /)
Moreover, if you relax the first criterion (escape characters), ordinary assignment will nicely solve this problem:
json="{ \"quicklock\" : \"${my_var}\" }"
How about just using the shell's natural concatenation of strings? If you concatenate ${mybar} rather than interpolate it, you can avoid escapes and get everything on one line:
my_var1="foobar"
my_var2="quux"
json='{"quicklock":"'${my_var1}'","slowlock":"'$my_var2'"}'
That said, this is a pretty crude scheme, and as others have pointed out you'll have problems if the variables, say, contain quote characters.
Since no escape chars is strong requirement here is a here-doc based solution:
#!/bin/bash
my_var='foobar'
read -r -d '' json << EOF
{
"quicklock": "$my_var"
}
EOF
echo "$json"
It will give you the same output as the first solution I mentioned.
Just be careful, if you would put first EOF inside double quotes:
read -r -d '' json << "EOF"
$my_var would not be considered as a variable but as a plain text, so you would get this output:
{
"quicklock": "$my_var"
}
My question is similar to this, where I am running into issues with putting JSON into a file. The issue is, no matter how I've formatted my strings inside the userData section of the CloudFormation template, I can't seem to capture an env $variable while maintaining a valid JSON object (with double quotes around the keys and values)
Below are two different ways I've tried to get the object into a file (via echo and cat << EOF < env-config.json) with virtually every combination of string escaping (single quotes wrapped around double quotes escaped around object keys...etc..)
echo '{\"development\": {\"EnvironmentConfig\": {\"api\": \" 'http://$ip:8000/api' \"}}}' >> env-config.json\n"
cat << EOF > env-config.json
{\"development\": {\"EnvironmentConfig\": {\"api\": \" 'http://$ip:8000/api' \"}}}
EOF
How can I place my perfectly formatted JSON object into a file while capturing an env $variable in it from the userData section of CloudFormation?
Thank you!
edit
Tools involved: gulp-ng-config, bash, cloudformation, json
Using gulp-ng-config to create a module with constants with the env-config.json file
I found out the answer, I needed single quotes around the url portion (as well as the double quotes) of my JSON like the below. This is what the whole line would look like in Cloudformation, I hope this helps someone:
"echo '{\"development\": {\"EnvironmentConfig\": {\"api\": \"'http://$ip:8000/api'\"}}}' >> env-config.json\n",
Wanted to parse json:
{"FileStatus":"accessTime":1472892839430,"blockSize":134217728,"childrenNum":0,"fileId":17226,"group":"admin","length":115714,"modificationTime":1469649837471,"owner":"admin","pathSuffix":"","permission":"755","replication":2,"storagePolicy":0,"type":"FILE"}}
I tried something like this but not able to get it.
$ {"FileStatus":{"accessTime":1472892839430}} | jq '.FileStatus.accessTime'
Error:
-bash: {FileStatus:{accessTime:1472892839430}}: command not found`
Can someone help me to parse this whole json.
To make a command read a string on stdin in bash, use a "here string" like this:
$ jq '.FileStatus.accessTime' <<<'{"FileStatus":{"accessTime":1472892839430}}'
1472892839430
Also, you need to properly quote the text, so that bash doesn't try to interpret in some way you don't intend. When you want to preserve it literally, use single quotes (').
I'm trying to pull variables from an API in json format and then put them back together with one variable changed and fire them back as a put.
Only issue is that every value has quote marks in it and must go back to the API separated by commas only.
example of what it should see with redacted information, variables inside the **'s:
curl -skv -u redacted:redacted -H Content-Type: application/json -X PUT -d'{properties:{basic:{request_rules:[**"/(req) testrule","/test-body","/(req) test - Admin","test-Caching"**]}}}' https://x.x.x.x:9070/api/tm/1.0/config/active/vservers/xxx-xx
Obviously if I fire them as a plain array I get spaces instead of commas. However I tried outputting it as a plain string
longstr=$(echo ${valuez[#]})
output=$(echo $longstr |sed -e 's/" /",/g')
And due to the way bash is interpreted it seems to either interpret the quotes wrong or something else. I guess it might well be the single ticks encapsulating after the PUT -d as well but I'm not sure how I can throw a variable into something that has single ticks.
If I put the raw data in manually it works so it's either the way the variable is being sent or the single ticks. I don't get an error and when I echo the line out it looks perfect.
Any ideas?
valuez=( "/(req) testrule" "/test-body" "/(req) test - Admin" "test-Caching" )
# Temporarily set IFS to some character which is known not to appear in the array.
oifs=$IFS
IFS=$'\014'
# Flatten the array with the * expansion giving a string containing the array's elements separated by the first character of $IFS.
d_arg="${valuez[*]}"
IFS=$oifs
# If necessary, quote or escape embedded quotation marks. (Implementation-specific, using doubled double quotes as an example.)
d_arg="${d_arg//\"/\"\"}"
# Substitute the known-to-be-absent character for the desired quote+separator+quote.
d_arg="${d_arg//$'\014'/\",\"}"
# Prepend and append quotes.
d_arg="\"$d_arg\""
# insert the prepared arg into the final string.
d_arg="{properties:{basic:{request_rules:[${d_arg}]}}}"
curl ... -d"$d_arg" ...
if you have gnu awk with version 4 and above, which support FPAT
output=$(echo $longstr |awk '$1=$1' FPAT="(\"[^\"]+\")" OFS=",")
Explanation
FPAT #
This is a regular expression (as a string) that tells gawk to create the fields based on text that matches the regular expression. Assigning a value to FPAT overrides the use of FS and FIELDWIDTHS for field splitting. See Splitting By Content, for more information.
If gawk is in compatibility mode (see Options), then FPAT has no special meaning, and field-splitting operations occur based exclusively on the value of FS.
valuez=( "/(req) testrule" "/test-body" "/(req) test - Admin" "test-Caching" )
csv="" sep=""
for v in "${valuez[#]}"; do csv+="$sep\"$v\""; sep=,; done
echo "$csv"
"/(req) testrule","/test-body","/(req) test - Admin","test-Caching"
If it's something you need to do repeatedly, but it into a function:
toCSV () {
local csv sep val
for val; do
csv+="$sep\"$val\""
sep=,
done
echo "$csv"
}
csv=$(toCSV "${valuez[#]}")