Mixing JSON with normal data in cURL request in bash - json

I know I can use
curl --data "param1=value1&param2=value2" http://hostname/resource
or
curl --request POST https://$url?param1=value1&param2=value2
But what do I need to do if param1 is value and param2 is a JSON?
It just does not work(tm) if I just toss the JSON in there, even using a variable
$json='{"data":"value"}'
curl --request POST https://$url?param1=value1&param2=$json
What is the trick here?
Note that I HAVE TO make only one call.
Thank you!

Ok, if we escape everything (using python) here's what it looks like
>>> x
'{"data": "value"}'
>>> urllib.urlencode({'param1':'value1', 'param2':x})
'param2=%7B%22data%22%3A+%22value%22%7D&param1=value1'
Or, using the curl option
curl localhost:8080 --data-urlencode 'param1={"data":"value"}'
Will send to the server
param1=%7B%22data%22%3A%22value%22%7D
You may notice that the first version has a +, which probably comes from the space in the json encoded, not sure it works or if it can be removed

Related

CURL command with --data-raw JSON, Unexpected character error

Hi I need to send JSON format through the CURL command but I am getting an error.
curl --request POST "https://app.io/api/graphql?accountId=Xzg" --header "x-api-key: WHpnV=" --header "Content-Type: application/json" --data-raw "{applications(limit: 2) {nodes {name}}}"
So the problematic part is data-raw, although I have checked it online it's Valid JSON.
--data-raw "{applications(limit: 2) {nodes {name}}}"
Response:
{"metaData":null,"resource":null,"responseMessages":[{"code":"DEFAULT_ERROR_CODE","level":"ERROR",
"message":"Unable to process JSON Unexpected character ('a' (code 97)):
was expecting double-quote to start field name","exception":null,"failureTypes":[]}]}
I tried different ways how I could change this request data but without success.
Can someone please assist me how I should change this query which was definitely generated as the proper one from some app.
Thanks!
Solution:
--data-raw '{"query": "{applications(limit: 2) {nodes {name}}}" }'

Increment bash variable while evaluating it

I'm using curl to send some json data. Part of the data is a request counter that needs to be incremented after each call.
I would like to reduce the code below by incrementing it right after evaluating it. I'm not sure how to format the variable within the json string though.
Thank you in advance!
#!/bin/bash
reqcnt=0
curl http://myurl.com --data-binary '{"requestCounter":'${reqcnt}'}'
((reqcnt++))
Expected:
#!/bin/bash
reqcnt=0
curl http://myurl.com --data-binary '{"requestCounter":'${((reqcnt++)}'}'
Edit
Taking into account the great answer by Inian, I noticed there are cases where I need to save the output of curl. For some reason the arithmetic operation is not performed on the variable in that case:
res=$(curl http://myurl.com --data-binary {"requestCounter":'"$((reqcnt++))"'}')

Pass json value to curl variable via CLI within Bash script

Using GET I need to pass a json value to a URL via the command line within a bash script.
This works:
curl -i "http://MYURL:8080/admin/rest_api/api?api=trigger_dag&dag_id=spark_submit&conf=\{\"filename\":\"myfile.csv\"\}"
If I want to expand on the json value, I would prefer to pass a variable via the URL parameter for readability. Somethig like ... but this doesn't appear to work correctly.
generate_post_data =
{
"filename": "myfile.csv"
}
curl -i "http://MYURL:8080/admin/rest_api/api?api=trigger_dag&dag_id=spark_submit&conf=${generate_post_data}"
You need to properly set the variable and you should url encode it using the --data-urlencode option.
#!/bin/bash
generate_post_data="filename=myfile.csv"
curl -G "http://MYURL:8080/admin/rest_api/api?api=trigger_dag&dag_id=spark_submit" --data-urlencode $generate_post_data
From the manpage:
--data-urlencode <data>
(HTTP) This posts data, similar to the other -d, --data options with
the exception that this performs URL-encoding.
To be CGI-compliant, the part should begin with a name followed
by a separator and a content specification. The part can be
passed to curl using one of the following syntaxes:
For more info you can use man curl and then /data-urlencode to jump to the section on it.

{"errorMessages":["Unexpected character (''' (code 39)): expected a valid value

I found "Query using POST" from here.
And tried to use curl command from command like. Installed curl by refering this for windows.
Here is my CURL string:
curl -D- -u admin:password -X POST -H "Content-Type: application/json" --data
'{"jql":"project = CI","startAt":0,"maxResults":50,"fields":["summary","status","assignee"]}'
"https://myclientname.atlassian.net/rest/api/2/search"
This is how I'm doing and getting error:
{"errorMessages":["Unexpected character (''' (code 39)): expected a valid value
(number, String, array, object, 'true', 'false' or 'null')\n
at [Source: org.apache.catalina.connector.CoyoteInputStream#1626cb2; line: 1, column: 2]"]}
Is there any problem making this curl string in windows? Please suggest? How can I correct this and get JSON object? Please note that, userID, password and client name is correct. Thanks.
Seems to be an windows issue. Do not use the ' (single-quote) character.
Instead, use " (double-quote) character for enclosing the string. Then, if you have inner quotes, use """ (3x double-quotes) to escape them.
Example: "{ """name""":"""Frodo""", """age""":123 }"
I tried the cURL you pointed to in your question, but with no luck. Also, the cURL comes with Git is not working either. However, the one I installed with CygWin works. And the same command is also working in Ubuntu. Which basically indicates that your command itself is OK.
If you are working on Windows, I recommend you to use a tool called Fiddler. It can perform almost all HTTP requests you may need. Good luck!
Update:
Here I add the steps to make HTTP POST request with Fiddler.
1) After starting Fiddler, you will see the GUI like Figure 1. The upper right panel is where you should input staff like JIRA's website, request type, and the content you want to post. To be specific, under the "Composer" tab, you need to select "POST" as your request type, and put the JIRA's URL there, keep HTTP/1.1 selected. You should put the request header under the URL bar. Now, you need to pay attention to. At least, you should input two things in HTTP header: the content type, which is "application/json", and the authorization header. The authentication is a Base64 string, you can get your Base64 string here with your "admin:password". If you want to know more about the basic authentication method, please refer Jira's website here. The lower right panel of the GUI is where you should put your post content.
2) When you get these staff ready, you can click the "Execute" button at upper right corner of the GUI. The execution result will be shown at the left panel. As Figure 2 shows, if you get a result with the status 200, congratulations, you got it. If you get other types of results, please google the error code or leave comments here.
3) Double click the result, the returned JSON content will be shown in the lower right panel like Figure 3. You can try different tab to see the returned staff. For example, if you go to the "TextView", you will get the returned JSON as pure string.
Please comment if you have any further question.
Pls verify if you have any value wrapped with single quote.
e.g
"NetworkType": 'Test'
Try this. It should work.
curl -D- -u admin:password -X POST -H "Content-Type: application/json" --data
\\"{"jql":"project = CI","startAt":0,"maxResults":50,"fields":["summary","status","assignee"]}\\"
"https://myclientname.atlassian.net/rest/api/2/search"
Don't forget to use a slash (\*{}\*}after and before json
This worked for me:
curl.exe -u elastic:Password! -k -X POST "https://localhost:9200/_security/user/kibana_system/_password?pretty" -H "Content-Type: application/json" --data '{"""password""" : """CHANGEME"""}'
Notice the format of the last parameter (--data): it uses single quotes (') as string delimiters and triple double-quotes (") inside)

Output bash variable with multiple lines to curl json

I'm trying to create a script that will use the Github API to post a comment containing the output of a command. This output has multiple lines.
Here's what I'm trying to do:
curl -H "Authorization: token oauthtoken" \
-H "Content-Type: application/json" \
-X POST -d#- \
https://api.github.com/repos/company/repo/issues/14/comments <<EOF
{
"body":"$OUTPUT"
}
EOF
How can I output the variable in such a way that it respects the multiple lines contained within? Now when I run that command, all of the newlines get squished on to one line.
I don't think that the basic cause of the problem are the newlines, the issues is that the value of $text is not properly formatted json.
Follow this simple example:
test="
Hello
World
"
curl -X POST -d '{"body": "'"$test"'"}' http://server.com/...
to see new lines working.
To make it possible to send the result of arbitrary commands using json, you need to json-encode the text before.