Passing variables (containing spaces) to curl --data field - json

Arguments containing spaces will not properly pass to the curl command.
Quotes are not passed correctly in the --data field.
If I just echo the variable 'curlData' that I use in the curl command I get everything as it should be; ex :
$echo $curlData
'{"name":"jason","description","service"}'
I don't understand why curl dont expend this 'curlData' variable as expected:
curl --data '{"name":"jason","description","service"}'
Here's a sample of my code:
read -p "Name : " repoName
read -p "Description []: " repoDescription
curlData="'"{'"'name'"':'"'$repoName'"','"'descripton'"':'"'$repoDescription'"'}"'"
curl --data $curlData $apiURL
And the error:
curl: (3) [globbing] unmatched close brace/bracket in column 26
Thank your for your help, I feel i'm in Quote-ception right now.


Quote all variable expansions,
To make sure that curlData is a valid JSON value with properly escaped special-characters etc., use jq for producing it.
curlData="$(jq --arg name "$repoName" --arg desc "$repoDescription" -nc '{name:$name,description:$desc}')"
curl --data "$curlData" "$apiURL"


If you have access to any form of package management, I highly recommend jo.
curlData=$(jo name="$repoName" description="$repoDescription")
curl -d "$curlData" "$apiURL"


I had similar issue, which was very difficult to even understand. I used the below construct in a number of curl commands present in my shell script. It always worked like a charm. Until one fine day I had to pass a variable which was string containing spaces (eg. modelName="Abc def").
curl -X 'PUT' \
'http://localhost:43124/api/v1/devices/'$Id'' \
-H 'accept:*/*' \
-H 'Authorization:Bearer '$token'' \
-H 'Content-Type:application/json' \
-d '{
"modelName":"'$modelName'",
"serialNumber":"'$childSN'"
}'
Worked for me after the below change
curl -X 'PUT' \
'http://localhost:43124/api/v1/devices/'$Id'' \
-H 'accept:*/*' \
-H 'Authorization:Bearer '$token'' \
-H 'Content-Type:application/json' \
-d '{
"modelName":'\""$modelName"\"',
"serialNumber":"'$childSN'"
}'
I took help from the accepted answer by #oguz. Pasting this response , just in case anyone is in similar situation

Related

CURL ERROR: parse error: Invalid numeric literal at line 1, column 8

I'm doing a bash script, when working out such an error ... I can't figure out what the problem is. I'm not good at scripting yet.
#!/bin/bash
data=LOGIN
password=123PASSWD
note_link=$(curl -s 'https://cryptgeon.nicco.io' \
-H 'X-Requested-With: XMLHttpRequest' \
--data-urlencode "data=$data" \
--data "has_manual_pass=false&duration_hours=0&dont_ask=false&data_type=T&notify_email=&notify_ref=" \
| jq -r --arg arg $password '.note_link + "#" + $arg')
echo "note URL is $note_link"

curl's -s option is silencing the errors as well, but you want to see the errors in this case to be able to understand what is going wrong, so use -sS instead.
Also, jq can only parse json. If the input is not json, it will fail with the error you get. You should first try to parse the output with jq, and if it fails, display it.
#!/bin/bash
data=LOGIN
password=123PASSWD
curl_output=$(curl -sS 'https://cryptgeon.nicco.io' \
-H 'X-Requested-With: XMLHttpRequest' \
--data-urlencode "data=$data" \
--data "has_manual_pass=false&duration_hours=0&dont_ask=false&data_type=T&notify_email=&notify_ref=")
if note_link=$(jq -r --arg pass "$password" '.note_link + "#" + $pass' <<<"$curl_output" 2>/dev/null); then
echo "note URL is $note_link"
else
printf >&2 %s\\n "Could not parse the curl output:" "$curl_output"
fi

Shell script not working for curl request

I want to make a curl request with Databox to push somemetrix and want to do it in shell script.
Here is the databox POST request example (which works like a charm)
curl https://push.test \
-u token
: \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/vnd.databox.v2+json' \
-d '{
"data":[
{
"$testcount": 50,
"test_name": "test"
}
]
}'
When I form the json body as a separate json string and try to pass as parameter, it doesn't work and gives a json parsing error. I am not sure what am I am doing wrong here. can someone help? I am new to shell scripts
#!/bin/bash
JSON_STRING= '{"data" : [{"$testcount":50,"testname":"test"}]}'
echo "$JSON_STRING"
curl https://testpush \
-u token
: \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/vnd.databox.v2+json' \
-d '$JSON_STRING'
error :
{"type":"invalid_json","message":"Invalid request body - JSON parse error"}
I have added my token for the request, so the authorisation should work.

You have excess whitespace around the =.
Also, $JSON_STRING in the last line of the second script should be in double quotes instead of the single quotes, to get it expanded into what you just set it to.
Btw., if data gets out of hand or is sensitive, you might want to look into the possibility to start the data with the letter # and then have the rest be a file name that contains the data.

Interpolate command output into GitHub REST request

I am trying to create a pull request comment automatically whenever CI is run. The output of a given command is written to a file (could also just be stored inside an environment variable though). The problem is, I usually get the following response:
curl -XPOST -d "{'body':'$RESULT'}" https://api.github.com/repo/name/issues/number/comment
{
"message": "Problems parsing JSON",
"documentation_url": "https://developer.github.com/v3/issues/comments/#create-a-comment"
}
This is usually due to unescpaed characters, like \n, \t, " etc.
Is there any easy way to achieve this on the command line or in bash, sh, with jq or Python? Using the Octokit.rb library is works straight away, but I don't want to install Ruby in the build environment.

You can use jq to create your JSON object. Supposing you have your comment content in RESULT variable, the full request would be :
DATA=$(echo '{}' | jq --arg val "$RESULT" '.| {"body": $val}')
curl -s -H 'Content-Type: application/json' \
-H 'Authorization: token YOUR_TOKEN' \
-d "$DATA" \
"https://api.github.com/repos/:owner/:repo/issues/:number/comments"

The post "Using curl POST with variables defined in bash script functions" proposes multiple techniques for passing a varialbe like $RESULT in a curl POST parameter.
generate_post_data()
{
cat <<EOF
{
"body": "$RESULT"
}
EOF
}
Then, following "A curl tutorial using GitHub's API ":
curl -X POST \
-H "authToken: <yourToken" \
-H "Content-Type: application/json" \
--data "$(generate_post_data)" https://api.github.com/repo/name/issues/number/comment

Passing a shell variable to a JSON request to curl?

Let's take the following example:
curl -i -X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "Player.Open", "params":{"item":false}}' \
http://example.com/jsonrpc
Now I want to have the boolean value of "item" be set in a shell script variable such as:
PRIVATE=false
read -p "Is this a private? (y/[n]) " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
PRIVATE=true
fi
And I want to pass in the value of PRIVATE to item. I have tried every which way but no luck. Can anyone shed some light?

You can do it this way:
curl -i -X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "Player.Open", "params":{"item":'"$PRIVATE"'}}' \
http://example.com/jsonrpc

Instead of your existing -d ... line above, you could try the following:
-d "{\"jsonrpc\": \"2.0\", \"method\": \"Player.Open\", \"params\":{\"item\":$PRIVATE}}" \
That is: when using double quote speechmarks ("), bash substitutes values for variables referenced $LIKE_THIS (not the case for single quotes you were using). The downside is that you then need to escape any double-quotes in the string itself (using the backslash, as above).

This abomination works too.
$ npm run script -- madrid
# script
json='{"city":"'"$1"'"}'
curl -X POST -d $json http://localhost:5678/api/v1/weather

curl error when using url string from string

I'm now testing some restful APIs using bash shell script.
I want to read url from file then make a json data string with the url in file.
For the test, below codes work fine. It's not reading from file.
#!/bin/bash
URL=http://test.com/test.jpg
curl -X POST \
-H "Content-Type:application/json" \
-H "accept:application/json" \
--data '{"url":"'"$URL"'"}' \
http://api.test.com/test
But, it returns some error when I'm using the codes like below.
#!/bin/bash
FILE=./url.txt
cat $FILE | while read line; do
echo $line # or whaterver you want to do with the $line variable
curl -X POST \
-H "Content-Type:application/json" \
-H "accept:application/json" \
--data '{"url":"'"$line"'"}' \
http://api.test.com/test
done
But, it returns error when I use the string from reading file.
This is error message.
Illegal unquoted character ((CTRL-CHAR, code 13)): has to be escaped using backslash to be included in string value
at [Source: org.apache.catalina.connector.CoyoteInputStream#27eb679c; line: 1, column: 237]
How to solve this issue?
Why it returns error when I use the string from file reading?

It appears that your file is in dos format with \n\r line terminators. Try running dos2unix on it to strip the \rs. Additionally, no need to cat the file, use redirection, like so
while read -r line; do
echo $line # or whaterver you want to do with the $line variable
curl -X POST \
-H "Content-Type:application/json" \
-H "accept:application/json" \
--data '{"url":"'"$line"'"}' \
http://api.test.com/test
done < "$FILE"
Also, pass -r to read to prevent backslash escaping