How to echo variable inside single quotes using Bash? - json

users.I want to run a curl command with a bash script.
(below command works perfectly in terminal)
curl -i -H "Content-Type: application/json" -X POST -d '{"mountpoint":"/gua-la-autentica-1426251559"}' http://127.0.0.1:5000/connect
But unable to run this command in bash. When mountpoint value is given in a variable($final).
final="/gua-la-autentica-1426251559"
curl -i -H "Content-Type: application/json" -X POST -d '{"mountpoint":'$final'}' http://127.0.0.1:5000/connect
Could someone please help me, how to echo variable inside single quotes?

JSON string values should be quoted and so should parameter expansions. You can achieve this by using double quotes around the entire JSON string and escaping the inner double quotes, like this:
curl -i -H "Content-Type: application/json" -X POST -d "{\"mountpoint\":\"$final\"}" http://127.0.0.1:5000/connect
As mentioned in the comments, a more robust approach would be to use a tool such as jq to generate the JSON:
json=$(jq -n --arg final "$final" '{ mountpoint: $final }')
curl -i -H "Content-Type: application/json" -X POST -d "$json" http://127.0.0.1:5000/connect

I'd probably do it with a printf format. It makes it easier to see formatting errors, and gives you better control over your output:
final="/gua-la-autentica-1426251559"
fmt='{"mountpoint":"%s"}'
curl -i -H "Content-Type: application/json" -X POST \
-d "$(printf "$fmt" "$final")" \
http://127.0.0.1:5000/connect
I don't know where you're getting $final in your actual use case, but you might also want to consider checking it for content that would break your JSON. For example:
Portable version:
if expr "$final" : '[A-Za-z0-9./]*$'; then
curl ...
else
echo "ERROR"
fi
Bash-only version (perhaps better performance but less portable):
if [[ "$final" ]~ ^[A-Za-z0-9./]*$ ]]; then
curl ...
...
Checking your input is important if there's even the remote possibility that your $final variable will be something other than what you're expecting. You don't have valid JSON anymore if it somehow includes a double quote.
Salt to taste.

Related

How to execute the same curl request on a sequence of server IPs (URLs)

Command
curl -v -H "Accept: application/json" -H "Content-type: application/json" \
-X POST -d '{"test": "some data"}' http://XX.XX.X.001:8080/services/test
I want to execute the above same curl command on different servers (IP addresses). Is there any way or CURL command so I can execute this service on all servers, instead of changing the IP address every time manually?
I have servers IP in some sequence, e.g.
http://XX.XX.X.002:8080/services/test
http://XX.XX.X.003:8080/services/test
...
You can either use shell's brace expansion in bash, e.g. {2..15} for 2,3,4,...,15:
curl ... http://x.x.x.{2..15}:8080/services/test
or, curl's alphanumeric series [] operator in the URL part (notice the double-quotes):
curl ... "http://x.x.x.[2-15]:8080/services/test"
Alternatively, you can use arithmetic for loop:
for ((i=2; i<=15; i++)); do
curl ... "http://x.x.x.$i:8080/services/test"
done
You could use a loop:
for HOST in 0.0.0.1:8080 0.0.0.2:8080; do curl -v -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{"my": "data"}' http://$HOST/services/test ; done
If your hosts are sequential then randomir's answer is a lot cleaner.

How to pass bash variable to JSON

I'm trying to write a sample script where I'm generating names like 'student-101...student-160'. I need to post JSON data and when I do, I get a JSON parse error.
Here's my script:
name="student-10"
for i in {1..1}
do
r_name=$name$i
echo $r_name
curl -i -H 'Authorization: token <token>' -d '{"name": $r_name, "private": true}' "<URL>" >> create_repos_1.txt
echo created $r_name
done
I always get a "Problems parsing JSON" error. I've tried various combination of quotes, etc but nothing seems to work!
What am I doing wrong?
First, your name property is a string, so you need to add double quotes to it in your json.
Second, using single quotes, bash won't do variable expansion: it won't replace $r_name with the variable content (see Expansion of variable inside single quotes in a command in bash shell script for more information).
In summary, use:
-d '{"name": "'"$r_name"'", "private": true}'
Another option is to use printf to create the data string:
printf -v data '{"name": "%s", "private": true}' "$r_name"
curl -i -H 'Authorization: token <token>' -d "$data" "$url" >> create_repos_1.txt
Don't; use jq (or something similar) to build correctly quoted JSON using variable inputs.
name="student-10"
for i in {1..1}
do
r_name=$name$i
jq -n --arg r_name "$r_name" '{name: $r_name, private: true}' |
curl -i -H 'Authorization: token <token>' -d #- "<URL>" >> create_repos_1.txt
echo created $r_name
done
The #- argument tells curl to read data from standard input (via the pipe from jq) to use for -d.
Something like "{\"name\": \"$r_name\", \"private\": true}" may work, but it is ugly and will also fail if r_name contains any character which needs to be quoted in the resulting JSON, such as double quotes or ASCII control characters.

cURL throwing globbing error when JSON data in variable contains a space

Using bash v4.3.48(1) on Ubuntu 16.04.2 LTS and bash v3.2.57 on macOS Sierra 10.12.5 I have this code which POSTs JSON data stored in a variable:
#1
JSON_DATA="Test"; \
curl -X POST "https://www.techotaku.com/json.php" \
-H "Content-Type: application/json" \
-d '{"data":"'$JSON_DATA'"}'
This returns:
Array
(
[data] => Test
)
However, if $JSON_DATA contains a space...
#2
JSON_DATA="A Test"; \
curl -X POST "https://www.techotaku.com/json.php" \
-H "Content-Type: application/json" \
-d '{"data":"'$JSON_DATA'"}'
...it throws an error:
curl: (3) [globbing] unmatched close brace/bracket in column 6
I thought this might be due to incorrect use of quotes, but if I echo the JSON data...
echo '{"data":"'$JSON_DATA'"}'
...it appears well-formed:
{"data":"A Test"}
Also, if I include the JSON data as a string and not a variable, it works.
#3
curl -X POST "https://www.techotaku.com/json.php" \
-H "Content-Type: application/json" \
-d '{"data":"A Test"}'
The only way I can get it to work if the variable contains a space is to enclose the JSON data in double-quotes and escape the literal double-quotes in the string:
#4
JSON_DATA="A Test"; \
curl -X POST "https://www.techotaku.com/json.php" \
-H "Content-Type: application/json" \
-d "{\"data\":\"$JSON_DATA\"}"
I would have thought that all 4 examples would have worked. So, can anyone throw any light on why only examples 1,3 and 4 work and example 2 doesn't?
Many thanks, Steve.
You should quote your variables to prevent word splitting by shell. Your second example:
'{"data":"'$JSON_DATA'"}'
doesn't work because it's being expanded by bash to two words:
{"data":"A Test"}
so -d option receives 2 arguments ({"data":"A and Test"}). If JSON_DATA contained globbing metacharacters, those would cause filename expansion.
To fix it, quote it:
'{"data":"'"$JSON_DATA"'"}'
Note the use of alternating single and double quotes. This will be expanded to a single word:
{"data":"A Test"}
Here's a more detailed answer I wrote some time ago which also mentions another convenient option, the use of a here-document.

Cannot expand and substitute the contents of a variable in Bash

I am using Gitbash, GNU bash, version 4.3.46(2)-release (x86_64-pc-msys).
I am running a bash script that looks like this
CODE CHUNK 1
curl -i -X POST \
-H "Content-Type:application/json" \
-H "x-customheader:customstuff" \
-d \
'{
Gigantic json payload contents in here
}' \
'http://localhost:5000/api/123'
This works fine. Basically it posts a giant payload to an endpoint and all is well. The problem is when I attempt to substitute the url for a value from a variable, I get a curl error,
curl: (1) Protocol "'http" not supported or disabled in libcurl
CODE CHUNK 2
stuff=\'http://localhost:5000/api/123\'
curl -i -X POST \
-H "Content-Type:application/json" \
-H "x-customheader:customstuff" \
-d \
'{
Gigantic json payload contents in here
}' \
$stuff
If I echo $stuff immediately after the stuff=\'http://localhost:5000/api/123\', I get 'http://localhost:5000/api/123'. This is the same value as I had hard-coded in code chunk 1, single ticks and all. There is something hiding behind the scenes in how that url is being evaluated after the variable has been expanded. I need to get the same behavior as a hard coded url.
Look closely at this error message:
curl: (1) Protocol "'http" not supported or disabled in libcurl
Notice the single-quote in front of http.
The curl command surely knows the http protocol, but not the 'http protocol!
The way you wrote it, the single-quotes are part of the value of stuff:
stuff=\'http://localhost:5000/api/123\'
Remove those, write like this:
stuff='http://localhost:5000/api/123'
If you have variables inside your real string,
and you want them expanded, then use double-quotes instead of single-quotes:
stuff="http://localhost:5000/api/123"
Equally important,
when you use $stuff as a parameter of curl,
you must double-quote it.
If you just write curl $stuff,
then the shell may interpret some characters in $stuff before passing to curl.
To protect from that,
you must write curl "$stuff".
The complete command:
curl -i -X POST \
-H "Content-Type:application/json" \
-H "x-customheader:customstuff" \
-d \
'{
Gigantic json payload contents in here
}' \
"$stuff"
Finally, make sure that after each \ at the end of lines,
there's nothing after the \ on each line,
the \ must be at the very end.
Why do you define stuff with the ' ' ?
Try it like that:
stuff="http://localhost:5000/api/123"
curl -i -X POST \
-H "Content-Type:application/json" \
-H "x-customheader:customstuff" \
-d \
'{
Gigantic json payload contents in here
}' \
"$stuff"
Also, don't put variables in single quotes, because bash is not available to understand them.
stuff="http://localhost:5000/api/123"
echo "$stuff"
>> http://localhost:5000/api/123
echo '$stuff'
>> $stuff

How to PUT a json object with an array using curl

I have a series of data to enter into database. The user interface to enter the data isn't good for bulk entry, so I'm trying to formulate a command line equivalent. When I examine the network request of the UI in chrome, I see a PUT request of a json object. When I try to replicate the request
curl -H 'Accept: application/json' -X PUT '{"tags":["tag1","tag2"],"question":"Which band?","answers":[{"id":"a0","answer":"Answer1"},{"id":"a1","answer":"answer2"}]}' http://example.com/service`
I get a error
curl: (3) [globbing] nested braces not supported at pos X
Where X is the character position of first "[".
How can I PUT a json object that includes an array?
Your command line should have a -d/--data inserted before the string you want to send in the PUT, and you want to set the Content-Type and not Accept.
curl -H 'Content-Type: application/json' -X PUT -d '[JSON]' \
http://example.com/service
Using the exact JSON data from the question, the full command line would become:
curl -H 'Content-Type: application/json' -X PUT \
-d '{"tags":["tag1","tag2"],
"question":"Which band?",
"answers":[{"id":"a0","answer":"Answer1"},
{"id":"a1","answer":"answer2"}]}' \
http://example.com/service
Note: JSON data wrapped only for readability, not valid for curl request.
Although the original post had other issues (i.e. the missing "-d"), the error message is more generic.
curl: (3) [globbing] nested braces not supported at pos X
This is because curly braces {} and square brackets [] are special globbing characters in curl.
To turn this globbing off, use the "-g" option.
As an example, the following Solr facet query will fail without the "-g" to turn off curl globbing:
curl -g 'http://localhost:8983/solr/query?json.facet={x:{terms:"myfield"}}'
It should be mentioned that the Accept header tells the server something about what we are accepting back, whereas the relevant header in this context is Content-Type
It's often advisable to specify Content-Type as application/json when sending JSON. For curl the syntax is:
-H 'Content-Type: application/json'
So the complete curl command will be:
curl -H 'Content-Type: application/json' -H 'Accept: application/json' -X PUT -d '{"tags":["tag1","tag2"],"question":"Which band?","answers":[{"id":"a0","answer":"Answer1"},{"id":"a1","answer":"answer2"}]}' http://example.com/service`
The only thing that helped is to use a file of JSON instead of json body text. Based on How to send file contents as body entity using cURL
Try using a single quote instead of double quotes along with -g
Following scenario worked for me
curl -g -d '{"collection":[{"NumberOfParcels":1,"Weight":1,"Length":1,"Width":1,"Height":1}]}" -H "Accept: application/json" -H "Content-Type: application/json" --user test#testmail.com:123456 -X POST https://yoururl.com
WITH
curl -g -d "{'collection':[{'NumberOfParcels':1,'Weight':1,'Length':1,'Width':1,'Height':1}]}" -H "Accept: application/json" -H "Content-Type: application/json" --user test#testmail.com:123456 -X POST https://yoururl.com
This especially resolved my error curl command error : bad url colon is first character