I would like to send a fairly large JSON file to an API using curl from ubuntu, in order to update the inventory of our machines.
Note that the aPI works fine and we already do this for windows machines using invoke-webrequest.
Windows command:
$postHash = ConvertTo-Json(#{
"hostName" = $computerName;
"macAddress" = $macAddress;
"ipAddress" = $ipAddress;
"pcModel" = $model;
"diskType" = $diskType;
"graphicsType" = $graphicsType;
"serviceTag" = $serviceTag;
"diskMode" = $diskMode;
"wolMode" = $wolMode;
"displayType" = $displayType;
"displayServiceTag" = $displayServiceTag;
"recDate" = $recDate;
"sleepState" = $sleepState;
})
Invoke-WebRequest -Method PUT -Body $postHash https://our_api.org/api/inventory/$macAddress/ -ContentType "application/json" -Headers #{"Authorization" = "Token 62d85f430210cd1a827bfdc34cd6c1fb1a64d1"} | Out-Null
Ubuntu command:
user#ubuntu:~$ echo $json
{ "pcModel": "KAT12", "displayType": "DELL U2311H", "graphicsType": "Microsoft Remote Display Adapter", "displayServiceTag": "HV8XP08Q079L", "ipAddress": "172.16.4.194", "recDate": "2022-10-06 16:57:55", "serviceTag": "18LQ9X1;Diskwear:(4.91TBW ; 15393 Hours)", "wolMode": "lanwithpxeboot;CC:101010-0118ZH;os:Ubuntu", "sleepState": "disable", "macAddress": "90:B1:1C:8E:D5:11", "hostName": "CI-KR95-05", "diskMode": "raid", "diskType": "Samsung SSD 850 PRO 512GB;TBW+Hrs:(4.91TB;15393 HrH) ;Clock:3.4GHz;Max Clock:3.67GHz(108%);RAM:32GB" }
user#ubuntu:~$ curl -k -L -X "PUT" -H "Accept: application/json" -H "Authorization: Token 62d85df90210cd1a827bc1518c4cd6c1fb1a64d1" "https://our_api.org/api/inventory/$macAddress/" -d $json
{"hostName":["This field is required."],"pcModel":["This field is required."],"diskType":["This field is required."],"macAddress":["This field is required."],"serviceTag":["This field is required."],"diskMode":["This field is required."],"wolMode":["This field is required."],"sleepState":["This field is required."],"displayType":["This field is required."],"displayServiceTag":["This field is required."],"recDate":["This field is required."],"ipAddress":["This field is required."]}curl: (3) URL using bad/illegal format or missing URL
curl: (6) Could not resolve host: "pcModel"
curl: (6) Could not resolve host: "KAT12",
curl: (6) Could not resolve host: "displayType"
curl: (6) Could not resolve host: "DELL
curl: (6) Could not resolve host: U2311H",
curl: (6) Could not resolve host: "graphicsType"
curl: (6) Could not resolve host: "Microsoft
curl: (6) Could not resolve host: Remote
curl: (6) Could not resolve host: Display
curl: (6) Could not resolve host: Adapter",
curl: (6) Could not resolve host: "displayServiceTag"
curl: (6) Could not resolve host: "HV8XP08Q079L",
curl: (6) Could not resolve host: "ipAddress"
curl: (6) Could not resolve host: "172.16.4.194",
curl: (6) Could not resolve host: "recDate"
curl: (6) Could not resolve host: "2022-10-06
curl: (3) URL using bad/illegal format or missing URL
curl: (6) Could not resolve host: "serviceTag"
curl: (3) URL using bad/illegal format or missing URL
curl: (6) Could not resolve host: ;
^^
The concern is that I get the following errors,curl: (6) and curl(3), curl 3 is probably due to the fact that we send and receive data with the mac_addresses which have special characters like ": : : : etc.". I've seen that I can put mac address inside double quotes but it doesn't change the error.
curl 6
My JSON file looks like this.
I have already looked at this post, but I dont know how to modify my json file arrordingly.
[Updated]:
If i put my json file in quotes like this:
user#ubuntu:~$ curl --request PUT --header "Accept: application/json" --header "Authorization: Token 62d85df90210cd1a827bc1518c4cd6c1fb1a64d1" "https://our_api.org/api/inventory/$macAddress" -d "$value"
curl: (3) URL using bad/illegal format or missing URL
i get the "curl: (3) URL using bad/illegal format or missing URL" error
I have checked the file format several times and it seems to be the same as the one we send with windows. Does anyone have any idea why I can't post a file with curl this way? Thanks
I found the reasons why my code was not updating the API
1 - I needed to specify the "Content-Type: "application/json" parameter to tell the API that the data sent hat JSON format otherwise I saw in verbose mode that the data has a "x-www-form-urlencoded" format that the API can't understand.
2 - The data sent with curl must accept a value with the format like this
'{"abc":"def",..,"ghi":"jkl"}'
So I ll have
json='{
"pcModel": "KAT12",
"displayType": "DELL U2311H",
"diskType": "Samsung SSD .."
}'
3 - The curl request must be sent with double quotes around the url and the data ""
So the end request looks like this:
curl -k -L -X "PUT" -H "Content-Type: application/json" \
-H "Accept: application/json" -H "Authorization: Token \
62d85df90210cd1a827bc1518c4cd6c1fb1a64d1" \
--url "https://our_api.org/api/inventory/$macAddress/" \
-d "$json"
I use Ubuntu and installed cURL on it. I want to test my Spring REST application with cURL. I wrote my POST code at the Java side. However, I want to test it with cURL. I am trying to post a JSON data. Example data is like this:
{"value":"30","type":"Tip 3","targetModule":"Target 3","configurationGroup":null,"name":"Configuration Deneme 3","description":null,"identity":"Configuration Deneme 3","version":0,"systemId":3,"active":true}
I use this command:
curl -i \
-H "Accept: application/json" \
-H "X-HTTP-Method-Override: PUT" \
-X POST -d "value":"30","type":"Tip 3","targetModule":"Target 3","configurationGroup":null,"name":"Configuration Deneme 3","description":null,"identity":"Configuration Deneme 3","version":0,"systemId":3,"active":true \
http://localhost:8080/xx/xxx/xxxx
It returns this error:
HTTP/1.1 415 Unsupported Media Type
Server: Apache-Coyote/1.1
Content-Type: text/html;charset=utf-8
Content-Length: 1051
Date: Wed, 24 Aug 2011 08:50:17 GMT
The error description is this:
The server refused this request because the request entity is in a format not supported by the requested resource for the requested method ().
Tomcat log:
"POST /ui/webapp/conf/clear HTTP/1.1" 415 1051
What is the right format of the cURL command?
This is my Java side PUT code (I have tested GET and DELETE and they work):
#RequestMapping(method = RequestMethod.PUT)
public Configuration updateConfiguration(HttpServletResponse response, #RequestBody Configuration configuration) { //consider #Valid tag
configuration.setName("PUT worked");
//todo If error occurs response.sendError(HttpServletResponse.SC_NOT_FOUND);
return configuration;
}
You need to set your content-type to application/json. But -d (or --data) sends the Content-Type application/x-www-form-urlencoded, which is not accepted on Spring's side.
Looking at the curl man page, I think you can use -H (or --header):
-H "Content-Type: application/json"
Full example:
curl --header "Content-Type: application/json" \
--request POST \
--data '{"username":"xyz","password":"xyz"}' \
http://localhost:3000/api/login
(-H is short for --header, -d for --data)
Note that -request POST is optional if you use -d, as the -d flag implies a POST request.
On Windows, things are slightly different. See the comment thread.
Try to put your data in a file, say body.json and then use
curl -H "Content-Type: application/json" --data #body.json http://localhost:8080/ui/webapp/conf
For Windows, having a single quote for the -d value did not work for me, but it did work after changing to double quote. Also I needed to escape double quotes inside curly brackets.
That is, the following did not work:
curl -i -X POST -H "Content-Type: application/json" -d '{"key":"val"}' http://localhost:8080/appname/path
But the following worked:
curl -i -X POST -H "Content-Type: application/json" -d "{\"key\":\"val\"}" http://localhost:8080/appname/path
You might find resty useful:
It's a wrapper round CURL which simplifies command line REST requests. You point it to your API endpoint, and it gives you PUT and POST commands (Examples adapted from the homepage).
resty http://127.0.0.1:8080/data #Sets up resty to point at your endpoing
GET /blogs.json #Gets http://127.0.0.1:8080/data/blogs.json
#Put JSON
PUT /blogs/2.json '{"id" : 2, "title" : "updated post", "body" : "This is the new."}'
# POST JSON from a file
POST /blogs/5.json < /tmp/blog.json
Also, it's often still necessary to add the Content Type headers. You can do this once, though, to set a default, of add configuration files per-method per-site: Setting default RESTY options
It worked for me using:
curl -H "Accept: application/json" -H "Content-type: application/json" -X POST -d '{"id":100}' http://localhost/api/postJsonReader.do
It was happily mapped to the Spring controller:
#RequestMapping(value = "/postJsonReader", method = RequestMethod.POST)
public #ResponseBody String processPostJsonData(#RequestBody IdOnly idOnly) throws Exception {
logger.debug("JsonReaderController hit! Reading JSON data!"+idOnly.getId());
return "JSON Received";
}
IdOnly is a simple POJO with an id property.
You can use Postman to convert to CURL:
Note:
The latest Postman version has some UI upgrades and now the code link is available in the sidebar.
As an example, create a JSON file, params.json, and add this content to it:
[
{
"environment": "Devel",
"description": "Machine for test, please do not delete!"
}
]
Then you run this command:
curl -v -H "Content-Type: application/json" -X POST --data #params.json -u your_username:your_password http://localhost:8000/env/add_server
I just run into the same problem. I could solve it by specifying
-H "Content-Type: application/json; charset=UTF-8"
This worked well for me.
curl -X POST --data #json_out.txt http://localhost:8080/
Where,
-X Means the HTTP verb.
--data Means the data you want to send.
You can use Postman with its intuitive GUI to assemble your cURL command.
Install and Start Postman
Type in your URL, Post Body, Request Headers etc. pp.
Click on Code
Select cURL from the drop-down list
copy & paste your cURL command
Note: There are several options for automated request generation in the drop-down list, which is why I thought my post was neccessary in the first place.
HTTPie is a recommended alternative to curl because you can do just
http POST http://example.com/some/endpoint name=value name1=value1
It speaks JSON by default and will handle both setting the necessary header for you as well encoding data as valid JSON. There is also:
Some-Header:value
for headers, and
name==value
for query string parameters. If you have a large chunk of data, you can also read it from a file have it be JSON encoded:
field=#file.txt
Using CURL Windows, try this:
curl -X POST -H "Content-Type:application/json" -d "{\"firstName\": \"blablabla\",\"lastName\": \"dummy\",\"id\": \"123456\"}" http-host/_ah/api/employeeendpoint/v1/employee
TL;DR:
Use the holy trinity, jo + curl + jq (or fx):
jo value=30 type="Tip 3" targetModule="Target 3" configurationGroup=null name="Configuration Deneme 3" description=null identity="Configuration Deneme 3" | \
curl --json #- \
-X POST \
http://localhost:8080/xx/xxx/xxxx | \
jq
This will cover necessary headers that were missing: no need to explicitly define the Content-Type and Accept headers.
The new curl way with --json
Early March 2022, curl released a new command line parameter --json with version 7.82.0. This allows for a shortcut to send through JSON and eliminating the need to define the Content-Type that you had missing and Accept headers as these are automatically assumed, hence reducing risk of mistakes:
curl --json '{"tool": "curl"}' https://example.com/
But wait... there is more. Instead of defining the json parameter as a string to the curl command line, use the nifty jo CLI tool to define JSON as series of key value pairs and pipe the output through curl. Using jo only to define your JSON, it works this way:
> jo -p value=30 type="Tip 3" targetModule="Target 3" configurationGroup=null name="Configuration Deneme 3" description=null identity="Configuration Deneme 3"
version=0 systemId=3 active=true
{
"value": 30,
"type": "Tip 3",
"targetModule": "Target 3",
"configurationGroup": null,
"name": "Configuration Deneme 3",
"description": null,
"identity": "Configuration Deneme 3",
"version": 0,
"systemId": 3,
"active": true
}
Let's showcase this now with a similar curl command of yours but without extra headers and using jo + jq for nice output:
jo value=30 type="Tip 3" targetModule="Target 3" configurationGroup=null name="Configuration Deneme 3" description=null identity="Configuration Deneme 3" | \
curl --json #- \
-X POST \
http://localhost:8080/xx/xxx/xxxx | \
jq
Example with a free API
Using a free mock API for demonstration:
> jo title="Blog Post" body="lorem ipsum" userId=33 | \
curl --json #- \
-X POST \
https://jsonplaceholder.typicode.com/posts | \
jq
The output has a pretty format thanks to jq:
{
"title": "Blog Post",
"body": "lorem ipsum",
"userId": 33,
"id": 101
}
Use -d option to add payload
curl -X POST \
http://<host>:<port>/<path> \
-H 'Accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"foo": "bar",
"lorem": "ipsum"
}'
In addition:
use -X POST to use POST method
use -H 'Accept: application/json' to add accept type header
use -H 'Content-Type: application/json' to add content type header
If you're testing a lot of JSON send/responses against a RESTful interface, you may want to check out the Postman plug-in for Chrome (which allows you to manually define web service tests) and its Node.js-based Newman command-line companion (which allows you to automate tests against "collections" of Postman tests.) Both free and open!
This worked well for me, additionally using BASIC authentication:
curl -v --proxy '' --basic -u Administrator:password -X POST -H "Content-Type: application/json"
--data-binary '{"value":"30","type":"Tip 3","targetModule":"Target 3","configurationGroup":null,"name":"Configuration Deneme 3","description":null,"identity":"Configuration Deneme 3","version":0,"systemId":3,"active":true}'
http://httpbin.org/post
Of course, you should never use BASIC authentication without SSL and a checked certificate.
I ran into this again today, using Cygwin's cURL 7.49.1 for Windows... And when using --data or --data-binary with a JSON argument, cURL got confused and would interpret the {} in the JSON as a URL template. Adding a -g argument to turn off cURL globbing fixed that.
See also Passing a URL with brackets to curl.
You could also put your JSON content in a file and pass it to curl using the --upload-file option via standard input, like this:
echo 'my.awesome.json.function({"do" : "whatever"})' | curl -X POST "http://url" -T -
This worked for me:
curl -H "Content-Type: application/json" -X POST -d #./my_json_body.txt http://192.168.1.1/json
I had the issue of:
curl -X POST http://your-server-end-point -H "Content-Type: application/json"
-d #path-of-your-json-file.json
See, I did everything right. Only one thing - I missed "#" before the JSON file path.
I found one relevant go-to document on Internet - Common Options.
Here is another way to do it, if you have dynamic data to be included.
#!/bin/bash
version=$1
text=$2
branch=$(git rev-parse --abbrev-ref HEAD)
repo_full_name=$(git config --get remote.origin.url | sed 's/.*:\/\/github.com\///;s/.git$//')
token=$(git config --global github.token)
generate_post_data()
{
cat <<EOF
{
"tag_name": "$version",
"target_commitish": "$branch",
"name": "$version",
"body": "$text",
"draft": false,
"prerelease": false
}
EOF
}
echo "Create release $version for repo: $repo_full_name branch: $branch"
curl --data "$(generate_post_data)" "https://api.github.com/repos/$repo_full_name/releases?access_token=$token"
I am using the below format to test with a web server.
use -F 'json data'
Let's assume this JSON dict format:
{
'comment': {
'who':'some_one',
'desc' : 'get it'
}
}
Full example
curl -XPOST your_address/api -F comment='{"who":"some_one", "desc":"get it"}'
This worked for me for on Windows 10:
curl -d "{"""owner""":"""sasdasdasdasd"""}" -H "Content-Type: application/json" -X PUT http://localhost:8080/api/changeowner/CAR4
I made a tool called fetcher for this. It can send requests and format curl snippets:
Here's an example:
Example output:
curl -XGET -H "Accept: application/json" -d "{\"value\":\"30\",\"type\":\"Tip 3\",\"targetModule\":\"Target 3\",\"configurationGroup\":null,\"name\":\"Configuration Deneme 3\",\"description\":null,\"identity\":\"Configuration Deneme 3\",\"version\":0,\"systemId\":3,\"active\":true}" "http://localhost:8080/xx/xxx/xxxx"
For PowerShell I've used:
curl.exe -H "Content-Type: application/json" --data "#content.json" http://localhost:8080/appname/path
Where content.json was the name of the JSON file on my local containing the request, and curl.exe instead of just curl not to use the alias for Invoke-WebRequest.
Or if you want to specify directly the JSON:
curl.exe -H "Content-Type: application/json" --data '{\"username\":\"xyz\",\"password\":\"xyz\"}' http://localhost:8080/appname/path
-H to send something like content-type or an authentication token in the header
-d here adds your data
finally add a site link
Note: Don't forget to add an authentication token (if you have) for authentication credentials
curl -X POST -H 'Content-Type: application/json' -H 'Authorization: Token 2de403987713595a7955a9b4655b9e206d4294b3' -d '{"title":"Post test with curl", "body": "test body"}' http://127.0.0.1:8000/api/v1/feeds/
You can cat the contents of a JSON file to curl via the --data-raw parameter.
curl 'https://api.com/route' -H 'Content-Type: application/json' --data-raw "$(cat ~/.json/payload-2022-03-03.json | grep -v '^\s*//')"
Note: comments in the JSON file are filtered out via grep -v '^\s*//'
You can also pass the data to curl via standard input using grep or cat.
grep -v '^\s*//' ~/.json/payload-2022-03-03.json | curl 'https://api.com/route' -H 'Content-Type: application/json' -d #-
cat ~/.json/payload-2022-03-03.json | grep -v '^\s*//' | curl 'https://api.com/route' -H 'Content-Type: application/json' -d #-
The issue is here:
HTTP/1.1 415 Unsupported Media Type
Server Login can't interpret the Media Type of this request, so it is parsing it as text/html
The media type of any resource is declared in the Content-Type
property of the request header
"accept" ... header will fail this request, so the following is required for any JSON request to be sent i.e. content-type
-H 'content-type: application/json'
Suppose data and URL are something like this
{"email": "admin#admin.com","password": "123456"}
http://localhost:5000/api/login
Then in Linux
curl http://localhost:5000/api/login -H 'content-type: application/json' -d '{"email": "user#admin.com", "password": "123456"}'
In Windows (single quotes around parameters will not work)
curl http://localhost:5000/api/login -H "content-type: application/json" -d "{\"email\": \"user#admin.com\", \"password\": \"123456\"}"
-X POST key is not required when -d {.....} is present in command.
For a PUT request:
-X PUT
Based on Anand Rockzz's answer, here is what I did to this on GitHub Actions. It was a bit tricky due to the EOF tag.
My goal was to send an HTTP call once a Vercel deployment was finished (similar to a webhook).
This real-world example might help other people.
send-webhook-callback-once-deployment-ready:
name: Invoke webhook callback url defined by the customer (Ubuntu 18.04)
runs-on: ubuntu-18.04
needs: await-for-vercel-deployment
steps:
- uses: actions/checkout#v1 # Get last commit pushed - See https://github.com/actions/checkout
- name: Expose GitHub slug/short variables # See https://github.com/rlespinasse/github-slug-action#exposed-github-environment-variables
uses: rlespinasse/github-slug-action#v3.x # See https://github.com/rlespinasse/github-slug-action
- name: Expose git environment variables and call webhook (if provided)
# Workflow overview:
# - Resolves webhook url from customer config file
# - If a webhook url was defined, send a
run: |
MANUAL_TRIGGER_CUSTOMER="${{ github.event.inputs.customer}}"
CUSTOMER_REF_TO_DEPLOY="${MANUAL_TRIGGER_CUSTOMER:-$(cat vercel.json | jq --raw-output '.build.env.NEXT_PUBLIC_CUSTOMER_REF')}"
VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK=$(cat vercel.$CUSTOMER_REF_TO_DEPLOY.staging.json | jq --raw-output '.build.env.VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK')
# Checking if a webhook url is defined
if [ -n "$VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK" ]; then
# Run script that populates git-related variables as ENV variables
echo "Running script populate-git-env.sh"
. ./scripts/populate-git-env.sh
echo "Resolved git variables:"
echo "'GIT_COMMIT_SHA': $GIT_COMMIT_SHA"
echo "'GIT_COMMIT_REF': $GIT_COMMIT_REF"
echo "'GIT_COMMIT_TAGS': $GIT_COMMIT_TAGS"
# Generates JSON using a bash function - See https://stackoverflow.com/a/57369772/2391795
# "End Of File" must be at the beginning of the line with no space/tab before or after - See https://stackoverflow.com/a/12909284/2391795
# But, when executed by GitHub Action, it must be inside the "run" section instead
generate_post_data() {
cat <<EOF
{
"MANUAL_TRIGGER_CUSTOMER": "${MANUAL_TRIGGER_CUSTOMER}",
"CUSTOMER_REF": "${CUSTOMER_REF_TO_DEPLOY}",
"STAGE": "staging",
"GIT_COMMIT_SHA": "${GIT_COMMIT_SHA}",
"GIT_COMMIT_REF": "${GIT_COMMIT_REF}",
"GIT_COMMIT_TAGS": "${GIT_COMMIT_TAGS}",
"GITHUB_REF_SLUG": "${GITHUB_REF_SLUG}",
"GITHUB_HEAD_REF_SLUG": "${GITHUB_HEAD_REF_SLUG}",
"GITHUB_BASE_REF_SLUG": "${GITHUB_BASE_REF_SLUG}",
"GITHUB_EVENT_REF_SLUG": "${GITHUB_EVENT_REF_SLUG}",
"GITHUB_REPOSITORY_SLUG": "${GITHUB_REPOSITORY_SLUG}",
"GITHUB_REF_SLUG_URL": "${GITHUB_REF_SLUG_URL}",
"GITHUB_HEAD_REF_SLUG_URL": "${GITHUB_HEAD_REF_SLUG_URL}",
"GITHUB_BASE_REF_SLUG_URL": "${GITHUB_BASE_REF_SLUG_URL}",
"GITHUB_EVENT_REF_SLUG_URL": "${GITHUB_EVENT_REF_SLUG_URL}",
"GITHUB_REPOSITORY_SLUG_URL": "${GITHUB_REPOSITORY_SLUG_URL}",
"GITHUB_SHA_SHORT": "${GITHUB_SHA_SHORT}"
}
EOF
}
echo "Print generate_post_data():"
echo "$(generate_post_data)"
echo "Calling webhook at '$VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK'"
echo "Sending HTTP request (curl):"
curl POST \
"$VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK" \
-vs \
--header "Accept: application/json" \
--header "Content-type: application/json" \
--data "$(generate_post_data)" \
2>&1 | sed '/^* /d; /bytes data]$/d; s/> //; s/< //'
# XXX See https://stackoverflow.com/a/54225157/2391795
# -vs - add headers (-v) but remove progress bar (-s)
# 2>&1 - combine stdout and stderr into single stdout
# sed - edit response produced by curl using the commands below
# /^* /d - remove lines starting with '* ' (technical info)
# /bytes data]$/d - remove lines ending with 'bytes data]' (technical info)
# s/> // - remove '> ' prefix
# s/< // - remove '< ' prefix
else
echo "No webhook url defined in 'vercel.$CUSTOMER_REF_TO_DEPLOY.staging.json:.build.env.VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK' (found '$VERCEL_DEPLOYMENT_COMPLETED_WEBHOOK')"
fi
simple is that
curl -X POST https://localhost:3000/
-H "Content-Type: application/json"
-d '{"productId": 123456, "quantity": 100}'
--json <data> Sends the specified JSON data in a POST request to the HTTP server.
curl 7.82.0+
# Send a basic JSON object
curl --json '{"name":"xyz","breed":"xyz","age":100}' http://127.0.0.1:3000/cats
# letter #, read the data from a file
curl --json #cat.txt http://127.0.0.1:3000/cats
# letter -, read the data from stdin
echo '{"name":"xyz","breed":"xyz","age":100}' | curl --json #- http://127.0.0.1:3000/cats
curl 7.82.0-
curl -X POST --header "Content-Type: application/json" --data '{"name":"xyz","breed":"xyz","age":100}' http://127.0.0.1:3000/cats
https://curl.se/docs/manpage.html#--json