I'm using curl to send some json data. Part of the data is a request counter that needs to be incremented after each call.
I would like to reduce the code below by incrementing it right after evaluating it. I'm not sure how to format the variable within the json string though.
Thank you in advance!
#!/bin/bash
reqcnt=0
curl http://myurl.com --data-binary '{"requestCounter":'${reqcnt}'}'
((reqcnt++))
Expected:
#!/bin/bash
reqcnt=0
curl http://myurl.com --data-binary '{"requestCounter":'${((reqcnt++)}'}'
Edit
Taking into account the great answer by Inian, I noticed there are cases where I need to save the output of curl. For some reason the arithmetic operation is not performed on the variable in that case:
res=$(curl http://myurl.com --data-binary {"requestCounter":'"$((reqcnt++))"'}')
Related
I have a bash script which sends curl post requests. I want to pass data as bash script parameters. However one of the parameter has spaces in the string and it fails with the error below.
Error parsing JSON data.\n\tString not terminated on line
In shell script, I'm sending an argument like this format {"name":"'$2'"}
Could you please help me to solve that issue?
Thanks
jq is good not only for manipulating existing JSON data, but creating new data, as it does things like correctly handling characters that can't appear unescaped in JSON strings, and proper quoting. Something like
curl ... -d"$(jq -n --arg val "$2" '{name: $val}')"
It would be better if you add enough data while asking question.
I assume your json will be like
{
"name": "argument passed"
}
curl -XPOST "your/url/here" -H 'Content-Type:application/json' -d'{"name":"'$1'"}'
Save the above command as post_request.sh(Feel free to change the name).
Run using below comand.
sh post_request.sh "argument passed"
"argument passed" will be your name with space.
Using GET I need to pass a json value to a URL via the command line within a bash script.
This works:
curl -i "http://MYURL:8080/admin/rest_api/api?api=trigger_dag&dag_id=spark_submit&conf=\{\"filename\":\"myfile.csv\"\}"
If I want to expand on the json value, I would prefer to pass a variable via the URL parameter for readability. Somethig like ... but this doesn't appear to work correctly.
generate_post_data =
{
"filename": "myfile.csv"
}
curl -i "http://MYURL:8080/admin/rest_api/api?api=trigger_dag&dag_id=spark_submit&conf=${generate_post_data}"
You need to properly set the variable and you should url encode it using the --data-urlencode option.
#!/bin/bash
generate_post_data="filename=myfile.csv"
curl -G "http://MYURL:8080/admin/rest_api/api?api=trigger_dag&dag_id=spark_submit" --data-urlencode $generate_post_data
From the manpage:
--data-urlencode <data>
(HTTP) This posts data, similar to the other -d, --data options with
the exception that this performs URL-encoding.
To be CGI-compliant, the part should begin with a name followed
by a separator and a content specification. The part can be
passed to curl using one of the following syntaxes:
For more info you can use man curl and then /data-urlencode to jump to the section on it.
I am getting response from the below command in JSON format in .sh file (using shell script):
**curl https://api.flipkart.net/sellers/skus/$col1/listings -H "Content-Type: application/json" -H "Authorization: Bearer $value"**
Here is the response I am getting:
{"listingId":"LSMSW","skuId":"M7489","fsn":"ACCCQD","attributeValues":{"actual_stock_count":"1","mrp":"199","seller_listing_state":"current","procurement_sla":"2","zonal_shipping_charge":"0","stock_count":"10","local_shipping_charge":"0","listing_status":"ACTIVE","max_order_quantity_allowed":"3","fulfilled_by":"seller","fk_release_date":"2016-08-29 10:00:08","selling_price":"529","inventory_count":"10","national_shipping_charge":"0","sku_id":"M7489"},"listingValidations":null}
Now I want to extract all the value in a separate variable for reuse it
like:
LISTINGID='listing id value'
SKUID='skuId value'
and many more.
If anyone know the answer please comment with explanation.
Here is a solution using jq. If data.json contains the sample data then the following bash script
#!/bin/bash
jq -M -r '
.listingId
, .skuId
, .attributeValues.mrp
' data.json | \
while read -r listingId
read -r skuId
read -r mrp; do
echo "$listingId $skuId $mrp"
done
produces
LSMSW M7489 199
The jq command in the script writes out the specified attributes on separate lines which are read into bash variables by the successive read -r commands. It should be clear how to extend or modify this to meet your requirements – the primary constraint is that for each line written by jq there should be a corresponding read -r.
I am not aware of any bash json parsers. I would go with Python or Php and parse the Json string in one of these languages and export values from there. In Php there is the putenv() method.
Good luck!
Edit:
Check here
Parsing JSON with Unix tools
I'm trying to POST data from a bash variable using curl, however, I'm unable to get this to work. This is the command that I'm using:
escape() { printf "%q" "$1"; }
curl -d "$(escape "$client")" -X POST -v https://$server/clients
The client variable look like this:
{"roles":["test"],"softwareName":"Some Soft","passwordSalt":"aaa","clientID":"full-client-2","contactPerson":"Test","contactPersonEmail":"a#b.org","description":"test","name":"Full Client-2","organization":"Some Org","passwordAlgorithm":"sha512","passwordHash":"bbb"}
And on the server I'm receiving the following:
{ '{"roles":': { '"test"': { '\"test\"\': '' } } }
I think its a problem with the escaping but I can't figure this out.
I've had a look at a number of other questions about this on here, but it seems most people need to insert variable into a literal that they are then trying to post. My problem is around using an entire variable as the json body. I've tried to use their answers to help me out but I haven't had any luck so far.
Don't try to quote it; use a here document:
curl -d#- -X POST -v https://"$server"/clients <<JSON
{"roles":["test"],"softwareName":"Some Soft","passwordSalt":"aaa","clientID":"full-client-2","contactPerson":"Test","contactPersonEmail":"a#b.org","description":"test","name":"Full Client-2","organization":"Some Org","passwordAlgorithm":"sha512","passwordHash":"bob"}
JSON
#- tells the -d option to look in standard input for the data, rather than using a hard-coded string.
If the text is in a variable, nothing more needs to be done; just quote the variable to prevent the shell from processing it:
curl -d "$client" -X POST -v https://"$server"/clients
I know I can use
curl --data "param1=value1¶m2=value2" http://hostname/resource
or
curl --request POST https://$url?param1=value1¶m2=value2
But what do I need to do if param1 is value and param2 is a JSON?
It just does not work(tm) if I just toss the JSON in there, even using a variable
$json='{"data":"value"}'
curl --request POST https://$url?param1=value1¶m2=$json
What is the trick here?
Note that I HAVE TO make only one call.
Thank you!
Ok, if we escape everything (using python) here's what it looks like
>>> x
'{"data": "value"}'
>>> urllib.urlencode({'param1':'value1', 'param2':x})
'param2=%7B%22data%22%3A+%22value%22%7D¶m1=value1'
Or, using the curl option
curl localhost:8080 --data-urlencode 'param1={"data":"value"}'
Will send to the server
param1=%7B%22data%22%3A%22value%22%7D
You may notice that the first version has a +, which probably comes from the space in the json encoded, not sure it works or if it can be removed