Edit: This question has been edited for context. 1.sh and 2.sh scripts in the original question are replaced by their real-world implementations: monitor.sh and api-check.sh respectively.
Function:
function checkAlive() {
script="$1"
args="${#:2}"
startCheck=$(date +%s)
echo "$#"
tempResult="$( "$#" )"
T_RC="$?"
if [ "$T_RC" -ne 0 ]; then
if [ "$result" = "" ]; then
result="ERRORS: $tempResult"
else
result="$result ,, $tempResult"
fi
fi
RC="$((RC > T_RC ? RC : T_RC))"
timePassed
}
api-check.sh:
method="$1"
url="https://$2"
jwt="$3"
body="$4"
echo "$body"
if ! response="$(curl -s --request "$method" "$url" -w "%{http_code}" --header \
'Authorization: Bearer '"$jwt" --header 'Content-Type: application/json' --header 'Accept: application/json' --data-raw "$body")"; then
echo "$url curl error getting code: $response"
exit 2
fi
monitor.sh:
api_url="foo"
oauth_token"bar"
body='{"address":"Test address"}'
checkAlive "api-check.sh" '"POST" "'$api_url'" "'$oauth_token'" '"'$body'"''
Execution Flow:
monitor.sh passes the string json body as an argument to checkAlive function (defined above).
Within checkAlive, the echo "$#" returns:
api-check.sh "POST" "foo" "bar" '{"address":"Test address"}'
The api-check.sh script never executes properly due to some escaping issue again
Question:
What json body should I pass as an argument to checkAlive to escape correctly OR Which line of code and in which script do I need to modify to evaluate the json body correctly?
do func() not function func()
just "$var". not anything else. Just make sure any $var is inside " ". Nothing more required.
Do:
api_url="foo"
oauth_token"bar"
body='{"address":"Test address"}'
checkAlive "api-check.sh" "POST" "$api_url" "$oauth_token" "$body"
Related
I am trying to submit a POST request using Bash that includes JSON data with a variable equal to that of a random string. I get the string dynamically by parsing a file which can vary in content... and may contain regex characters and also new lines.
I will provide an example string here with a curl request that works successfully with the API I am posting this request to. I can only get the request to go through with the string hardcoded into the JSON data and not while assigning a variable to the string like for instance stringVar and using the variable in the JSON data. I could sure use some help where I am messing this up
my working shell script looks something like this
#!/bin/bash
curl -i -H "Authorization: Bearer <MY_API_TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" -X POST 'https://api.example.com/v1/endpoint' -d '{
"name": "my-project",
"files": [
{
"data": "helloWorldFunction() {\n echo \"hello world\" \n}"
},
],
}'
This works, however I need to change data's value from the string
helloWorldFunction() {\n echo "hello world" \n}
to a variable
I have tried settings the variable in different ways in the JSON content from reading suggestions on other questions. For instance I have tried tried changing my shell script to
#!/bin/bash
stringVar="helloWorldFunction() {\n echo \"hello world\" \n}"
curl -i -H "Authorization: Bearer <MY_API_TOKEN>" -H "Content-Type: application/json" -H "Accept: application/json" -X POST 'https://api.example.com/v1/endpoint' -d '{
"name": "my-project",
"files": [
{
"data": "${stringVar}"
},
],
}'
i have tried setting the variable like this
"${stringVar}" | turns into ${stringVar}
"$stringVar" | turns into "$stringVar"
and
"'${stringVar}'"
"'$stringVar'"
both seem to return this error
{"error":{"code":"bad_request","message":"Invalid JSON"}}curl: (3) unmatched brace in URL position 1: {\n
and
stringVar
stringVar
$stringVar
"'"$stringVar"'"
""$stringVar""
${stringVar}
all seem to return this error
{"error":{"code":"bad_request","message":"Invalid JSON"}}
Ahh any help on what I am doing wrong would be great.
Thanks in advance y'all
In order to interpolate the value of a variable, you need to enclose the string in double-quotes("), but the JSON also requires literal double-quotes.
The easiest solution is probably to use a here-document to feed the data into curl's standard input, as in #Gilles Quénot's answer. But you can still pass it in via the command line; you just have to be careful with the quotes.
This is one way:
curl ... -d '{
"name": "my-project",
"files": [
{
"data": "'"$stringVar"'"
}
]
}'
The JSON here is mostly contained within single quotation marks ('...'). But right after opening the pair of literal "s that will enclose the value of data, we close the single quotes and switch our shell quotation mode to double quotes in order to include the value of $stringVar. After closing the double quotes around that expansion, we go back into single quotes for the rest of the JSON, starting with closing the literal double-quotes around the value of data.
In a language that used + for string concatenation, it would look like '... "data": "' + "$stringVar" + '"... ', but in the shell you just put the strings next to each other with no operator to concatenate them.
As an alternative, you could put the whole thing in double-quotes, but then you need backslashes to include the literal double quotes that are part of the JSON:
curl ... -d "{
\"name\": \"my-project\",
\"files\": [
{
\"data\": \"$stringVar\"
}
]
}"
So that requires a lot more changes if you're starting from plain JSON; it also looks messier, IMO.
You can also use a tool that knows how to build JSON and let it worry about quoting etc. Here's one way to build it with jq:
jq -n --arg data "$stringVar" '{
"name": "my-project",
"files": [
{
"data": $data
}
]
}'
Using --arg creates a variable inside jq – I named it data – which can then be included in an expression with the syntax $varname ($data in this case). Despite the similarity of syntax, that's not a shell interpolation; we're passing the literal text $data to jq, and jq itself is what replaces it with the value of the variable (which was passed as the second argument to --arg).
There's another tool called jo, which doesn't manipulate JSON but rather produces it, from input that is easier to generate in the shell. Here's one way to construct the desired object with it:
jo name=my-project files="$(jo -a "$(jo data="$stringVar")")"
Either way you can include the constructed JSON in your curl command line like this:
curl ... -d "$(jo or jq command goes here)"
Do not generate such JSON by hand. Use a tool like jq to do it for you.
#!/bin/bash
stringVar="helloWorldFunction() {\n echo \"hello world\" \n}"
jq -n --arg s "$stringVar" '{name: "my-project", files: [{data: $s}]}' |
curl -i -H "Authorization: Bearer <MY_API_TOKEN>" \
-H "Content-Type: application/json" \
-H "Accept: application/json" \
-X POST \
-d #- \
'https://api.example.com/v1/endpoint'
Like this:
#!/bin/bash
stringVar="..."
curl -i -H "Authorization: Bearer <MY_API_TOKEN>" \
-H "Content-Type: application/json" \
-H "Accept: application/json" -X POST 'https://api.example.com/v1/endpoint' \
-d#/dev/stdin <<EOF
{
"name": "my-project",
"files": [
{
"data": $stringVar
},
],
}
EOF
You then should take care about what you fed in the variable, this have to be valid JSON
As an alternative to jq and curl you could use xidel to generate the JSON and submit the POST-request.
With command-line options:
#!/bin/bash
stringVar='helloWorldFunction() {\n echo "hello world" \n}'
xidel -s --variable var="$stringVar" \
-H "Authorization: Bearer <MY_API_TOKEN>" \
-H "Content-Type: application/json" \
-H "Accept: application/json"
-d '{serialize(
{"name":"my-project","files":array{{"data":$var}}},
{"method":"json"}
)}' \
"https://api.example.com/v1/endpoint" \
-e '$raw'
Or with the x:request() function in-query:
#!/bin/bash
stringVar='helloWorldFunction() {\n echo "hello world" \n}'
xidel -s --variable var="$stringVar" -e '
x:request({
"headers":(
"Authorization: Bearer <MY_API_TOKEN>",
"Content-Type: application/json",
"Accept: application/json"
),
"post":serialize(
{"name":"my-project","files":array{{"data":$var}}},
{"method":"json"}
),
"url":"https://api.example.com/v1/endpoint"
})/raw
'
$raw / /raw returns the raw output, like curl. If the API-endpoint returns JSON, then you can use $json / /json to parse it.
I'm doing a bash script, when working out such an error ... I can't figure out what the problem is. I'm not good at scripting yet.
#!/bin/bash
data=LOGIN
password=123PASSWD
note_link=$(curl -s 'https://cryptgeon.nicco.io' \
-H 'X-Requested-With: XMLHttpRequest' \
--data-urlencode "data=$data" \
--data "has_manual_pass=false&duration_hours=0&dont_ask=false&data_type=T¬ify_email=¬ify_ref=" \
| jq -r --arg arg $password '.note_link + "#" + $arg')
echo "note URL is $note_link"
curl's -s option is silencing the errors as well, but you want to see the errors in this case to be able to understand what is going wrong, so use -sS instead.
Also, jq can only parse json. If the input is not json, it will fail with the error you get. You should first try to parse the output with jq, and if it fails, display it.
#!/bin/bash
data=LOGIN
password=123PASSWD
curl_output=$(curl -sS 'https://cryptgeon.nicco.io' \
-H 'X-Requested-With: XMLHttpRequest' \
--data-urlencode "data=$data" \
--data "has_manual_pass=false&duration_hours=0&dont_ask=false&data_type=T¬ify_email=¬ify_ref=")
if note_link=$(jq -r --arg pass "$password" '.note_link + "#" + $pass' <<<"$curl_output" 2>/dev/null); then
echo "note URL is $note_link"
else
printf >&2 %s\\n "Could not parse the curl output:" "$curl_output"
fi
Just curious to know the difference between below three executions, If I use a variable to hold the json data and pass the same to curl command, json msg become corrupt and server returns incorrect result, What is the best way to pass json from a variable while invoking curl command ? Trying to make the last line in the snippet work.
my $tsFormatted = '2021-06-10';
my $metaLoad = '{"validDate": "'.$tsFormatted.'"}';
my $curl_out_1 = `curl -s -X POST -H 'Content-Type: application/json' $mw_meta_load_api --key $mw_access_key --cert $mw_access_cert --data '{"validDate": "2021-06-10"}'`;
my $curl_out_2 = `curl -s -X POST -H 'Content-Type: application/json' $mw_meta_load_api --key $mw_access_key --cert $mw_access_cert --data '{"validDate": "'.$tsFormatted.'"}'`;
my $curl_out_3 = `curl -s -X POST -H 'Content-Type: application/json' $mw_meta_load_api --key $mw_access_key --cert $mw_access_cert --data $metaLoad`;
Backticks interpolate as double-quoted strings, so we can easily view the difference by using double-quoted string literals instead of backticks.
my $tsFormatted = '2021-06-10';
my $metaLoad = '{"validDate": "'.$tsFormatted.'"}';
# qq`...`, qq"..." and "..." are all the same thing.
say qq`... --data '{"validDate": "2021-06-10"}'`;
say qq`... --data '{"validDate": "'.$tsFormatted.'"}'`;
say qq`... --data $metaLoad`;
Output:
... --data '{"validDate": "2021-06-10"}'
... --data '{"validDate": "'.2021-06-10.'"}'
... --data {"validDate": "2021-06-10"}
The contents of string literals (including backticks) aren't treated as Perl code.
Of those three, only the first is correct.
The first passes the following two arguments to curl:
--data
{"validDate": "2021-06-10"}
The second passes the following two arguments to curl:
--data
{"validDate": ".2021-06-10."}
The third passes the following three arguments to curl:
--data
{validDate:
2021-06-10}
What you want:
use Cpanel::JSON::XS qw( encode_json );
use String::ShellQuote qw( shell_quote );
my $ts_formatted = '2021-06-10';
my $data = { validDate => $ts_formatted };
my $data_formatted = encode_json($data);
my $cmd = shell_quote(
curl => (
$mw_meta_load_api,
'-s',
-X => 'POST',
-H => 'Content-Type: application/json',
'--key' => $mw_access_key,
'--cert' => $mw_access_cert,
'--data' => $data_formatted,
)
);
my $out = `$cmd`;
die("Can't execute shell: $!\n") if $? == -1;
die("curl killed by signal ".( $? & 0x7F )."\n") if $? & 0x7F;
die("curl exited with error ".( $? >> 8 )."\n") if $? >> 8;
Also, have you considered using Net::Curl::Easy instead of shelling out?
I want to make a curl request with Databox to push somemetrix and want to do it in shell script.
Here is the databox POST request example (which works like a charm)
curl https://push.test \
-u token
: \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/vnd.databox.v2+json' \
-d '{
"data":[
{
"$testcount": 50,
"test_name": "test"
}
]
}'
When I form the json body as a separate json string and try to pass as parameter, it doesn't work and gives a json parsing error. I am not sure what am I am doing wrong here. can someone help? I am new to shell scripts
#!/bin/bash
JSON_STRING= '{"data" : [{"$testcount":50,"testname":"test"}]}'
echo "$JSON_STRING"
curl https://testpush \
-u token
: \
-X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/vnd.databox.v2+json' \
-d '$JSON_STRING'
error :
{"type":"invalid_json","message":"Invalid request body - JSON parse error"}
I have added my token for the request, so the authorisation should work.
You have excess whitespace around the =.
Also, $JSON_STRING in the last line of the second script should be in double quotes instead of the single quotes, to get it expanded into what you just set it to.
Btw., if data gets out of hand or is sensitive, you might want to look into the possibility to start the data with the letter # and then have the rest be a file name that contains the data.
Let's take the following example:
curl -i -X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "Player.Open", "params":{"item":false}}' \
http://example.com/jsonrpc
Now I want to have the boolean value of "item" be set in a shell script variable such as:
PRIVATE=false
read -p "Is this a private? (y/[n]) " -n 1 -r
if [[ $REPLY =~ ^[Yy]$ ]]; then
PRIVATE=true
fi
And I want to pass in the value of PRIVATE to item. I have tried every which way but no luck. Can anyone shed some light?
You can do it this way:
curl -i -X POST \
-H "Content-Type: application/json" \
-d '{"jsonrpc": "2.0", "method": "Player.Open", "params":{"item":'"$PRIVATE"'}}' \
http://example.com/jsonrpc
Instead of your existing -d ... line above, you could try the following:
-d "{\"jsonrpc\": \"2.0\", \"method\": \"Player.Open\", \"params\":{\"item\":$PRIVATE}}" \
That is: when using double quote speechmarks ("), bash substitutes values for variables referenced $LIKE_THIS (not the case for single quotes you were using). The downside is that you then need to escape any double-quotes in the string itself (using the backslash, as above).
This abomination works too.
$ npm run script -- madrid
# script
json='{"city":"'"$1"'"}'
curl -X POST -d $json http://localhost:5678/api/v1/weather