How can I use libcurl to process complex form? [duplicate] - json

Question

I'm defining a PUT request with a JSON request body using libcurl in C.
This how I'm doing it:
sprintf(jsonObj, "\"name\" : \"%s\", \"age\" : \"%s\"", name, age);
struct curl_slist *headers = NULL;
curl_slist_append(headers, "Accept: application/json");
curl_slist_append(headers, "Content-Type: application/json");
curl_slist_append(headers, "charset: utf-8");
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonObj);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&chunk);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "libcrp/0.1");
res = curl_easy_perform(curl);
The request body is arriving like this:
{ '"name" : "Pedro", "age" : "22"' }
With { ' at the start and ' } at the end.
--- MORE INFO -----
if I declare this code
char* jsonObj = "{ \"name\" : \"Pedro\" , \"age\" : \"22\" }";
struct curl_slist *headers = NULL;
curl_slist_append(headers, "Accept: application/json");
curl_slist_append(headers, "Content-Type: application/json");
curl_slist_append(headers, "charset: utf-8");
curl_easy_setopt(curl, CURLOPT_URL, url);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonObj);
curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, WriteMemoryCallback);
curl_easy_setopt(curl, CURLOPT_WRITEDATA, (void *)&chunk);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "libcrp/0.1");
res = curl_easy_perform(curl);
the server receives this as the request body:
{ '{ "name" : "Pedro" , "age" : "22" }': '' }
My question is:
Is the libCurl pre-formatting/encoding automatically the Json request?
By the way, does the libCurl have some way of encoding a JSON object?
Thanks so much!

Answer 1

The problem may be with the headers. When you are configuring your curl_slist headers I think you should assign the output of curl_slist_append back to headers:
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, "charset: utf-8");

Answer 2

Firstly, let's note a few things. To start with, Daniel Stenberg is correct (as I'd hope he would be, given that he wrote the code): libcurl does not append any data to your code. I can demonstrate this with this sample program, which is like your example but with some additional setup/teardown code. This is all of the code: there is nothing else present here:
#include <curl/curl.h>
int main (int argc, char *argv[]) {
CURL *curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl == NULL) {
return 128;
}
char* jsonObj = "{ \"name\" : \"Pedro\" , \"age\" : \"22\" }";
struct curl_slist *headers = NULL;
curl_slist_append(headers, "Accept: application/json");
curl_slist_append(headers, "Content-Type: application/json");
curl_slist_append(headers, "charset: utf-8");
curl_easy_setopt(curl, CURLOPT_URL, "http://http2bin.org/put");
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "PUT");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, jsonObj);
curl_easy_setopt(curl, CURLOPT_USERAGENT, "libcrp/0.1");
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
curl_global_cleanup();
return res;
}
Packet capturing the request with Wireshark shows that this emits the following HTTP request:
PUT /put HTTP/1.1
Host: http2bin.org
User-Agent: libcrp/0.1
Accept: */*
Content-Length: 35
Content-Type: application/x-www-form-urlencoded
{ "name" : "Pedro" , "age" : "22" }
As you can see, this is exactly the JSON data you asked to send. There is no extra data here, no enclosing braces.
This means that the extra braces are being added either by your server or by some intermediate middlebox. My guess is that your server is adding it because it is forcibly trying to turn any body that is not an application/json body into one by considering the entire body a string.
The reason your server doesn't consider this a JSON body is encapsulated by another answer here: you aren't setting your headers properly. curl_slist_append returns a new struct curl_slist * that you need to assign back into your headers variable. That means you need to change these four lines:
struct curl_slist *headers = NULL;
curl_slist_append(headers, "Accept: application/json");
curl_slist_append(headers, "Content-Type: application/json");
curl_slist_append(headers, "charset: utf-8");
to these four:
struct curl_slist *headers = NULL;
headers = curl_slist_append(headers, "Accept: application/json");
headers = curl_slist_append(headers, "Content-Type: application/json");
headers = curl_slist_append(headers, "charset: utf-8");
This should convince your server that you are sending JSON data.
In the future, I recommend you get familiar with Wireshark for solving problems like this. It is extremely helpful to see the actual request you sent. Failing that, if you'd rather not, you can use CURLOPT_DEBUGFUNCTION to grab the data curl is sending to validate it.

Answer 3

I was having a similar Issue.
I discovered the -libcurl option for the curl command.
It helped a lot! Just add it to the end of your working curl command.
In the end it help me create this code:
CURLcode ret;
CURL *hnd;
struct curl_slist *slist1;
std::string jsonstr = "{\"username\":\"bob\",\"password\":\"12345\"}";
slist1 = NULL;
slist1 = curl_slist_append(slist1, "Content-Type: application/json");
hnd = curl_easy_init();
curl_easy_setopt(hnd, CURLOPT_URL, "http://u/r/l");
curl_easy_setopt(hnd, CURLOPT_NOPROGRESS, 1L);
curl_easy_setopt(hnd, CURLOPT_POSTFIELDS, jsonstr.c_str());
curl_easy_setopt(hnd, CURLOPT_USERAGENT, "curl/7.38.0");
curl_easy_setopt(hnd, CURLOPT_HTTPHEADER, slist1);
curl_easy_setopt(hnd, CURLOPT_MAXREDIRS, 50L);
curl_easy_setopt(hnd, CURLOPT_CUSTOMREQUEST, "POST");
curl_easy_setopt(hnd, CURLOPT_TCP_KEEPALIVE, 1L);
ret = curl_easy_perform(hnd);
curl_easy_cleanup(hnd);
hnd = NULL;
curl_slist_free_all(slist1);
slist1 = NULL;
Express recieves this JSON as:
{ username: 'bob', password: '12345' }
I hope this helps!

Answer 4

libcurl will send exactly the bytes you ask it to send. It has no knowledge of JSON at all.
See #Lukasa's excellent and more elaborate answer for better and more details.

Answer 5

A key part of understanding whether the system is behaving correctly is seeing what the program is actually sending over the network. So another way to check the byte stream, instead of pointing it at your server (and/or running Wireshark), is to just run a netcat instance in a separate window on the test machine:
nc -l 8080
and point the code (CURLOPT_URL) at "http://localhost:8080".
You'll need to hit Control-D in the nc window to terminate the connection so that curl completes, but you can also type a test return text beforehand, which can be useful if you're expecting some sort of reply to test against.

Answer 6

Had this same problem when posting data to node-red with json parser.
Solution for me was to treat string as a HTML
#include <curl/curl.h>
int main (int argc, char *argv[]) {
CURL *curl;
CURLcode res;
curl_global_init(CURL_GLOBAL_ALL);
curl = curl_easy_init();
if (curl == NULL) {
return 128;
}
struct curl_slist *headers = NULL;
curl_slist_append(headers, "Content-Type: application/json");
curl_easy_setopt(curl, CURLOPT_URL, #yourURL);
curl_easy_setopt(curl, CURLOPT_CUSTOMREQUEST, "POST");
curl_easy_setopt(curl, CURLOPT_HTTPHEADER, headers);
curl_easy_setopt(curl, CURLOPT_POSTFIELDS, "age=42&sex=male");
res = curl_easy_perform(curl);
curl_easy_cleanup(curl);
curl_global_cleanup();
return res;
}
Hope someone will find it helpful.

Answer 7

I used json-c for encoding and decoding jsons and it worked very well.
The documentation is also easy to understand.
https://linuxprograms.wordpress.com/2010/05/20/json-c-libjson-tutorial/