Total noob in BASH. Trying to learn.
I have the following bash script to make an API request:
#!/bin/bash
if [ $1 = "new_event" ]; then
a='https://www.googleapis.com/calendar/v3/calendars/'
b=$2
c='/events?access_token='
d=$3
path=$a$b$c$d
echo $4
OUTPUT="$(curl -s -H "Content-Type: application/json" $path -d $4 )"
echo "${OUTPUT}"
fi
Positional arguments are 'new_event', calendarId, access token and json string.
If I run the script I get:
first echo is the json string I pass as parameter in the call:
' {"guestsCanSeeOtherGuests": false, "location": "", "description": "TEST", "reminders": {"useDefault": false}, "start": {"dateTime": "2017-07-06T14:00:00", "timeZone": "America/Sao_Paulo"}, "end": {"dateTime": "2017-07-06T15:00:00", "timeZone": "America/Sao_Paulo"}, "guestsCanInviteOthers": false, "summary": "TEST", "status": "tentative", "attendees": []} '
second echo gives me parsing error.
BUT, if I copy the echoed json string and replace $4 for it, everything works.
OUTPUT="$(curl -s -H "Content-Type: application/json" $path -d ' {"guestsCanSeeOtherGuests": false, "location": "", "description": "TEST", "reminders": {"useDefault": false}, "start": {"dateTime": "2017-07-06T14:00:00", "timeZone": "America/Sao_Paulo"}, "end": {"dateTime": "2017-07-06T15:00:00", "timeZone": "America/Sao_Paulo"}, "guestsCanInviteOthers": false, "summary": "TEST", "status": "tentative", "attendees": []} ' )"
Any hint why is it not working with the positional argument while it works if I paste it's content?
Thanks!
When you pass the JSON string as a parameter, it assigns the content contained inside the single quote as a whole string but when you pass it to curl it was subjected to word splitting.
To show what it looks like here's a sample script to demonstrate it.
This script will receive the string and pass it to the second script.
#!/bin/bash
./params $1
This second script simulates what curl sees. It will print the number of parameters it receives.
#!/bin/bash
echo $#
Guess what the output is:
27
To fix your issue and make it simpler, drop the outermost quote and quote everything inside the $().
OUTPUT=$(curl -s -H "Content-Type: application/json" "$path" -d "$4" )
Related
So I'm writing a script that needs to create a json object, and post it with curl.
This is working:
curl --header "Content-Type: application/json" --request POST --data '{ "_type": "_test", "_device": "123.123.123.123", "_system": "web-services", "result": "success", "_time": "123", "error": "" }' $data_pipeline
$data_pipeline contains the URL for the post request
if $json_string contains the string including single quotes:
'{ "_type": "_test", "_device": "123.123.123.123", "_system": "web-services", "result": "success", "_time": "123", "error": "" }'
This should work but it doesn't:
curl --header "Content-Type: application/json" --request POST --data $json_string $data_pipeline
First I was creating the $json_object without the single quotes, and tried to add them on the command line for CURL. If I don't escape the single quotes, $json_string is sent as a literal, instead of expanding the variable. I escaped the single quotes, and it did not help, I even tried a double escape in case that was needed, and still it is not working. It only works if I put the entire json string by hand, but not if I put it in a variable. How can I fix this? The json is dynamically created by the script, using jq, and the json is valid as I can successfully run the post by hand, I just need it to work with a variable. It won't work without the single quotes, but the single quotes don't work when I use a variable to hold the json, it doesn't matter if I put the single quotes in the variable, or try to do it outside of the variable... How do I fix this?
The json object is built with this code:
json_string=\'$( jq -n \
--arg _type "$_type" \
--arg _device "$_device" \
--arg _system "$_system" \
--arg result "$result" \
--arg _time "$_time" \
--arg error "$error" \
'{_type: $_type, _device: $_device, _system: $_system, result: $result, _time: $_time, error: $error}' )\'
Originally I was creating the json_string without the ' but I added that in attempt to get the single quotes wrapped around the json.
Thanks!
#!/bin/bash
_type="hello"
# let's have some fun
_device="single'quote"
_system='double"quote'
error='multi
line'
encode()
{
printf "%s" "$1" | sed 's/"/%22/' | tr '\n' ' '
}
# you don't need encode() if your strings are not json-offensive
json_string="{
_type: \"$_type\"
_device: \"$_device\"
_system: \"$(encode "$_system")\"
error: \"$(encode "$error")\"
}"
set -x
curl \
-X POST \
-d "$json_string" \
-H 'content-type: application/json' \
http://httpbin.org/anything
Output:
+ curl -X POST -d '{
_type: "hello"
_device: "single'\''quote"
_system: "double%22quote"
error: "multi line"
}' -H 'content-type: application/json' http://httpbin.org/anything
{
"args": {},
"data": "{\n\t_type: \"hello\"\n\t_device: \"single'quote\"\n\t_system: \"double%22quote\"\n\terror: \"multi line\"\n}",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Content-Length": "92",
"Content-Type": "application/json",
"Host": "httpbin.org",
"User-Agent": "curl/7.58.0",
},
"json": null,
"method": "POST",
"url": "http://httpbin.org/anything"
}
I have a Nexus Repository server where my artifacts are stored. I want to write a shell script to download artifacts from here. When using the curl request curl --user username:password -X GET "http://your_ip:your_port/service/rest/v1/search?repository=your_repository" -H "accept: application/json" I get a list of the items in my repository which looks like this:
{
"items": [
{
"id": "dGVzdC1hcHA6ZDM1MTBiN2FkMThkODJjZGU1NjNhMWVlMWFmOWIwMGQ",
"repository": "test-app",
"format": "maven2",
"group": "no.ahj",
"name": "test-app",
"version": "1.0-20190715.130341-2",
"assets": [
{
"downloadUrl": "http://192.168.56.2:8081/repository/test-app/no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom",
"path": "no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom",
"id": "dGVzdC1hcHA6Yzc3MDE2OWMwYjJlM2VkODU0MGMyOGEwOWQ0Njk4ZTQ",
"repository": "test-app",
"format": "maven2",
"checksum": {
"sha1": "5fd032774dd3ae6fbbd6484b3dc6ef2582d9b397",
"md5": "3a6aa8e295a734fdb8a8df782c0a14d5"
}
},
I would like my shell script to run this curl request, extract the value from the downloadURL field, store it in some variable and then use wget with this variable to download the file. So my question is this: How can I take the URL from downloadURL and store/use it in my shell script?
A way to parse it with likely no external dependencies (as python is installed by default on most Linux distributions) is just to use python:
user#host ~ % JSON=$(curl --user username:password -X GET "http://your_ip:your_port/service/rest/v1/search?repository=your_repository" -H "accept: application/json")
user#host ~ % echo $JSON | python -c 'import sys, json; print(json.load(sys.stdin)["items"][0]["assets"][0]["downloadUrl"])'
http://192.168.56.2:8081/repository/test-app/no/ahj/test-app/1.0-SNAPSHOT/test-app-1.0-20190715.130341-2.pom
If you are going to do a lot of JSON parsing in this script, it may be worth considering writing the entire script in Python, too, instead of shell script.
I need to concatenate 2 variables, but when i execute the bash code i get this error filters.api.malformed_request_body.
I need to get the current IPs, to be able to add a new IP to the filter, that's because i need to use two variables, $a is the current IP that is in my firewall rule, and $b is the new IP that i will add.
From Cloudflare
To preserve existing values, issue a GET request and based on the response, determine which fields (and respective values) to include in your PUT request and that way, avoid any undesired overwrites.
Code:
a=122.16.89.10
b=137.77.77.77
curl -X PUT \
-H "X-Auth-Email: EMAIL" \
-H "X-Auth-Key: KEY" \
-H "Content-Type: application/json" \
-d '[
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {'$a'" "'$b'})",
"description": "Block IP"
}
]' "https://api.cloudflare.com/client/v4/zones/ZONE/filters"
I also tried: "(ip.src in {'$a $b'})" and:
new_filter="$a $b"
...
...
"(ip.src in {'$new_filter'})"
If i echo $new_filter it shows the correct result:
new_filter="$a $b"
echo $new_filter
#122.16.89.10 137.77.77.77
When i use the variable $new_filter it also show this error curl: (3) [globbing] unmatched close brace/bracket in column 13 line 13 is this one -H "Content-Type: application/json" \.
None worked, why? I get this error:
{
"result": null,
"success": false,
"errors": [
{
"code": 10014,
"message": "filters.api.malformed_request_body"
}
],
"messages": []
}
This works: "(ip.src in {'$a'})".
Well, let's take your first example and modify it to print the JSON body:
export a=122.16.89.10
export b=137.77.77.77
echo '[
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {'$a'" "'$b'})",
"description": "Block IP"
}
]'
The output is this:
{
"id": "ID",
"paused": false,
"expression": "(ip.src in {122.16.89.10" "137.77.77.77})",
"description": "Block IP"
}
]
You can see that JSON is not valid. The double quotes are unbalanced in expression.
Try "expression": "(ip.src in {'$a' '$b'})", instead -- that will produce valid JSON.
I have a windows batch script to perform POST request using curl and reads the data from JSON file, it works fine with only a single object in the file and it looks like this.
curl -u username#password -H "Content-Type: application/json" -d #file.json http://apiurl.com
and the json file is this:
{
"name": "Empty name",
"properties": {
"active": "True",
"subcity_zone": "East Hararge",
"woreda": "Meta"
}
}
But now I want to send the request each object in the array by iterating each item. So, How do I iterate each JSON object from the file?
Here is what the new JSON array file looks like:
[{
"name": "test facility I",
"properties": {
"active": "True",
"city": "",
"subcity_zone": "East Hararge",
"woreda": "Meta"
}
},
{
"name": "test facility II",
"properties": {
"active": "True",
"subcity_zone": "East Hararge",
"woreda": "Girawa"
}
}]
Using jq:
jq -c '.[]' file | while read js; do
curl -u username#password -H "Content-Type: application/json" -d #<(echo "$js") http://apiurl.com
done
The jq command extracts the each object in one line that is read by read command into the $js variable.
The <(echo "$js") creates a temporary file that is passed to curl.
This is a small command line script that is used to post a json body in an http server. I am finding difficulties to pass the first command line argument $1 to the json body.
#!/bin/bash
curl -X POST -d '{ "game": 16, "id": $(($1)) }' http://localhost:10000/
The command does not fail, however the http body contains exactly
{ "game": 16, "id": $(($1)) }
I want to run the script ./script 123 and send the json body
{ "game": 16, "id": 3 }
How can I do this using bash?
You can also use single quotes so you don't have to escape double quotes like this:
#!/bin/bash
curl -X POST -d '{ "game": 16, "id": '$1' }' http://localhost:10000/
Using single quotes will print literal characters. You need to use double quotes for string interpolation. Try:
curl -X POST -d "{ \"game\": 16, \"id\": $1 }" http://localhost:10000/