I am using the following curl command to upload data to CouchDB:
curl -d #abcd.json -H "Content-Type: application/json" -X POST http://#localhost:5984/database/_bulk_docs
The file contains multiple JSON documents and is valid JSON.
The response I get is: {"error":"bad_request","reason":"Request body must be a JSON object"}
I have studied other answers to similar questions but don't seem to be able to find the reason for the error.
(The file does not have a 'BOM' as far as I can see.)
I am running on Windows 10.
I have tried using the RESTClient addon in Firefox with the same result.
To solve this I have found that one needs an added structure in the input file, nl. an additional:
{
"docs":
before the first "[" of the first JSON document in the file(naturally with closing "}") then everything works.
Sorry for the inconvenience.
This post jogged my thinking.
Related
I am sending a POST request from curl like this:
curl -H 'Content-Type:application/json" -X POST -d '{"key":"value"}' http://localhost:3000/parsejson
However, I am getting on my Node/Express server:
{'key':'value'} // req.body
So it is unclear to me if it is the curl request or the configuration of my node server. On my node server, I'm using: bodyParser.json() and bodyParser.urlencoded()
Thank you!
First of all this line is incorrect and you cannot run curl with that:
curl -H 'Content-Type:application/json" -X POST -d '{"key":"value"}' http://localhost:3000/parsejson
I'm mentioning it because this is a question about single and double quotes and you have single and double quotes messed up in your example of how you make a request, resulting in a code that cannot possibly be executed. Since this is obviously not how you really make the request, it is not clear how you do.
Now, if you want to see what you really get in the request, then don't use a body parser (temporarily switch it off, remove or comment it out) and run req.pipe(process.stdout) to display the request body on the server. Then you will know what you are getting from the client.
Also run curl with the -v option to see what it is actually sending.
If it turns out that curl is sending a correct JSON and your server gets a correct JSON in the request body then your problem must be somewhere else than in topics that you are asking about in this question.
Of course it's impossible to tell you what is the problem in that case because you didn't include even a single line of code in your question.
Currently I am able to PUT a single json file to a document in Cloudant using this : curl -X PUT 'https://username.cloudant.com/dummydb/doc3' -H "Content-Type: application/json" -d #numbers.json.I have many JSON files to be uploaded as different documents in the same DB.How can it be done?
So you definitely want to use Cloudant's _bulk_docs API endpoint in this scenario. It's more efficient (and cost-effective) if you're doing a bunch of writes. You basically POST an array that contains all your JSON docs. Here's the documentation on it: https://docs.cloudant.com/document.html#bulk-operations
Going one step further, so long as you've structured your JSON file properly, you can just upload the file to _bulk_docs. In cURL, that would look something like this: curl -X POST -d #file.json <domain>/db/_bulk_docs ... (plus the content type and all that other verbose stuff).
One step up from that would be using the ccurl (CouchDB/Cloudant cURL) tool that wraps your cURL statements to Cloudant and makes them less verbose. See https://developer.ibm.com/clouddataservices/2015/10/19/command-line-tools-for-cloudant-and-couchdb/ from https://stackoverflow.com/users/4264864/glynn-bird for more.
Happy Couching!
You can create a for loop and create documents from each JSON file.
For example, in the command below I have 4 JSON files in my directory and I create 4 documents in my people database:
for file in *.json
> do
> curl -d #$file https://username:password#myinstance.cloudant.com/people/ -H "Content-Type:application/json"
> done
{"ok":true,"id":"763a28122dad6c96572e585d56c28ebd","rev":"1-08814eea6977b2e5f2afb9960d50862d"}
{"ok":true,"id":"763a28122dad6c96572e585d56c292da","rev":"1-5965ef49d3a7650c5d0013981c90c129"}
{"ok":true,"id":"763a28122dad6c96572e585d56c2b49c","rev":"1-fcb732999a4d99ab9dc5462593068bed"}
{"ok":true,"id":"e944282beaedf14418fb111b0ac1f537","rev":"1-b20bcc6cddcc8007ef1cfb8867c2de81"}
I'm working on a cli for the Pushbullet HTTP API using Bash scripting. Sending pushes (notes and links), as well as creating, deleting, and modifying contacts & devices are all straight forward using curl and Bash. However, sending SMS and files are a bit more complex, as both require sending more complex JSON-formatted requests to the server (multiple JSON-formatted requests, in the case of pushing files).
I've tried sending many variations on the following (both with and without escape characters), but the server keeps replying about JSON-formatting errors. The following code is based off of the example given in the Pushbullet HTTP API documentation.
curl -su $auth_id: -X POST https://api.pushbullet.com/v2/ephemerals --header "Content-Type: application/json"
--data-binary '{ "\"type"\": "\"push"\", "\"push"\": { \
"\"type"\": "\"messaging_extension_reply"\", \
"\"package_name"\": "\"com.pushbullet.android"\", \
"\"source_user_iden"\": "\"$source_idens"\", \
"\"target_device_iden"\": "\"$target_idens"\", \
"\"conversation_iden"\": "\"$sms_device"\", \
"\"message"\": "\"Hello"\" \
} }'
Using bash -x, I can see that this is (supposedly) what is being sent to the server:
--data-binary '{"type": "push", "push": {
"type": "messaging_extension_reply",
"package_name": "com.pushbullet.android",
"source_user_iden": "<source_idens>",
"target_device_iden": "<device_idens>",
"conversation_iden": "<sms_phone_number>",
"message": "Hello" } }'
In all cases, the server returns:
{"error":{"type":"invalid_request","message":"Failed to parse JSON body.","cat":"(=^‥^=)"}}
What is the appropriate formatting of a JSON request using curl to send an SMS via the Pushbullet API? Am I overlooking something obvious? I'm trying to accomplish this using only curl and Bash, I see no reason why it's not possible (maybe not the fastest or most elegant way, but certainly possible).
I found the solution to my issue so I thought I'd share it. It was actually very simple:
Because the curl command includes a JSON-formatted response with single quotes, variable expansion was not occurring. This is a limitation (or perhaps a feature) of Bash. So, even though the server responded with { } indicating no errors in the request, the requests were actually being sent without the proper values for parameters, such asuser_iden,source_user_iden, etc.
Solution:
Enclose all variable expansions inside the JSON-formatted request in a double-quote and single-quote, like so:
"'"$user_idens"'"
First I'd like to apologize for how bad the API is, especially file upload and sending SMS. I was thinking of adding multipart or base64 file uploads to /v2/pushes. I think the first one might help you with curl, not sure about the base64 one. multipart is a huge pain though, so I'd prefer to make it better than the current setup if possible, rather than about equally as bad. Suggestions are welcome.
I tried your command line and it seemed to work, so I'm not sure what is going wrong. Here's the command line I did. Perhaps your quote escaping or newlines are causing the JSON error?
curl -u <access_token> -X POST https://api.pushbullet.com/v2/ephemerals --header "Content-Type: application/json" --data-binary '{"type": "push", "push": {"type": "messaging_extension_reply","package_name": "com.pushbullet.android","source_user_iden": "iden","target_device_iden": "device_idens", "conversation_iden": "sms_phone_number","message": "Hello" } }'
I'm getting an invalid JSON error when trying to add new JSON data to a couchapp.
I have JSON data that I converted from a CSV, and have been trying to add it with _bulk_docs to a couchapp. I solved the most obvious JSON errors with a JSON validator: http://jsonlint.com/
But one of the files is still giving an error, even thought the validator says the JSON is valid.
curl -d #songs_raw2.json -H "Content-Type: application/json" -X POST http://username:password#127.0.0.1:5984/songs/_bulk_docs
{"error":"bad_request","reason":"invalid_json"}
The file is here:
https://github.com/anatighe/sacred-harp/blob/master/songs_raw2.json
Looks like your json is not valid. This doesn't work for me:
$ curl https://raw.githubusercontent.com/anatighe/sacred-harp/master/songs_raw2.json | python -m json.tool | view -
Might have to do with special characters / encoding?
It looks like all of the single quote characters are messing it up. In every place there should be a ', there appears to be a � for me. I'm on a mac, I don't know if that matters.
When I replaced the weird characters with regular single quotes the curl request worked fine.
I have a large dataset in JSON format that I would like to upload to IrisCouchDB.
I found the following instructions: http://kxepal.iriscouch.com/docs/1.3/api/database/common.html
But I am a newbie and it also seems to be for a single JSON document. Im afraid I will just create one huge entry and not multiple documents entries which is what I want.
I have NodeJS but I don't have Cradle. Will I need it in order to perform this function? Any help is greatly appreciated!
Oh, no! That's bad idea to reference on docs at my Iris host - that's was preview dev build. Please follow the official docs: http://docs.couchdb.org/en/latest/api/index.html
To update multiple document's you need to use /db/_bulk_docs resource:
curl -X POST http://localhost:5984/db/_bulk_docs \
-H "Content-Type:application/json" \
-d '{"docs":[{"name":"Yoda"}, {"name":"Han Solo"}, {"name":"Leia"}]}'
So you see the format of data you should send to CouchDB? Now it's all depending from you JSON file format. If whole file is an array of objects: just wrap his content into {"docs":..} object. If not: you have to write some small library to convert this file data into the required format.