I have used
curl -XPOST "http://localhost:9200/<my_index_name>" -d #<absolute_path_to_my_json_file>
Then when I tried to get the data using
curl -XGET "http://localhost:9200/<my_index_name>"
its giving me data only for first line of my json file. (including other stuff also - settings, mappings,alias etc).
But why is it not able to load the entire json file ?
BTW, I am using ES 2.4.0. If I have to use bulk, what is the syntax ?
Try using these
curl -XPUT "http://localhost:9200/<my_index_name>" -d #<absolute_path_to_my_json_file>
GET index does not search actually.
You have also to run something like GET index/_search.
Related
I am using the following curl command to upload data to CouchDB:
curl -d #abcd.json -H "Content-Type: application/json" -X POST http://#localhost:5984/database/_bulk_docs
The file contains multiple JSON documents and is valid JSON.
The response I get is: {"error":"bad_request","reason":"Request body must be a JSON object"}
I have studied other answers to similar questions but don't seem to be able to find the reason for the error.
(The file does not have a 'BOM' as far as I can see.)
I am running on Windows 10.
I have tried using the RESTClient addon in Firefox with the same result.
To solve this I have found that one needs an added structure in the input file, nl. an additional:
{
"docs":
before the first "[" of the first JSON document in the file(naturally with closing "}") then everything works.
Sorry for the inconvenience.
This post jogged my thinking.
Currently I am able to PUT a single json file to a document in Cloudant using this : curl -X PUT 'https://username.cloudant.com/dummydb/doc3' -H "Content-Type: application/json" -d #numbers.json.I have many JSON files to be uploaded as different documents in the same DB.How can it be done?
So you definitely want to use Cloudant's _bulk_docs API endpoint in this scenario. It's more efficient (and cost-effective) if you're doing a bunch of writes. You basically POST an array that contains all your JSON docs. Here's the documentation on it: https://docs.cloudant.com/document.html#bulk-operations
Going one step further, so long as you've structured your JSON file properly, you can just upload the file to _bulk_docs. In cURL, that would look something like this: curl -X POST -d #file.json <domain>/db/_bulk_docs ... (plus the content type and all that other verbose stuff).
One step up from that would be using the ccurl (CouchDB/Cloudant cURL) tool that wraps your cURL statements to Cloudant and makes them less verbose. See https://developer.ibm.com/clouddataservices/2015/10/19/command-line-tools-for-cloudant-and-couchdb/ from https://stackoverflow.com/users/4264864/glynn-bird for more.
Happy Couching!
You can create a for loop and create documents from each JSON file.
For example, in the command below I have 4 JSON files in my directory and I create 4 documents in my people database:
for file in *.json
> do
> curl -d #$file https://username:password#myinstance.cloudant.com/people/ -H "Content-Type:application/json"
> done
{"ok":true,"id":"763a28122dad6c96572e585d56c28ebd","rev":"1-08814eea6977b2e5f2afb9960d50862d"}
{"ok":true,"id":"763a28122dad6c96572e585d56c292da","rev":"1-5965ef49d3a7650c5d0013981c90c129"}
{"ok":true,"id":"763a28122dad6c96572e585d56c2b49c","rev":"1-fcb732999a4d99ab9dc5462593068bed"}
{"ok":true,"id":"e944282beaedf14418fb111b0ac1f537","rev":"1-b20bcc6cddcc8007ef1cfb8867c2de81"}
I am new to elasticsearch, running elasticsearch from chrome:extension- Postman.
I want to enter bulk data into it, from JSON using Bulk API.
I have seen the command :
curl -s -XPOST 'http://jfblouvmlxecs01:9200/_bulk' --data-binary #bulk.json
While using Postman, I do not know where to store the file #bulk.json,
Currently I have stored it at C:\elasticsearch-1.5.2\bin\bulk.JSON
The command I am using is http://localhost:9200/_bulk --data-binary #bulk.JSON
This is throwing following error:
"error": "InvalidIndexNameException[[_bulk --data-binary #bulk.JSON] Invalid index name [_bulk --data-binary #bulk.JSON], must not contain the following characters [\, /, *, ?, \", <, >, |, , ,]]",
"status": 400 }
Can someone please suggest, where to store the JSON file.
Or am I doing something wrong here.
You can store your file anywhere and then pass the path like this:
curl -s -XPOST 'http://jfblouvmlxecs01:9200/_bulk' --data-binary #"/blah/blah/bulk.json"
If you want to do it with postman there is an answer here that explains how
But if you want to use this for large data, I wouldn't recommend using postman or sense. Use curl directly or logstash
See also this: https://github.com/taskrabbit/elasticsearch-dump
I'm getting an invalid JSON error when trying to add new JSON data to a couchapp.
I have JSON data that I converted from a CSV, and have been trying to add it with _bulk_docs to a couchapp. I solved the most obvious JSON errors with a JSON validator: http://jsonlint.com/
But one of the files is still giving an error, even thought the validator says the JSON is valid.
curl -d #songs_raw2.json -H "Content-Type: application/json" -X POST http://username:password#127.0.0.1:5984/songs/_bulk_docs
{"error":"bad_request","reason":"invalid_json"}
The file is here:
https://github.com/anatighe/sacred-harp/blob/master/songs_raw2.json
Looks like your json is not valid. This doesn't work for me:
$ curl https://raw.githubusercontent.com/anatighe/sacred-harp/master/songs_raw2.json | python -m json.tool | view -
Might have to do with special characters / encoding?
It looks like all of the single quote characters are messing it up. In every place there should be a ', there appears to be a � for me. I'm on a mac, I don't know if that matters.
When I replaced the weird characters with regular single quotes the curl request worked fine.
I have a large dataset in JSON format that I would like to upload to IrisCouchDB.
I found the following instructions: http://kxepal.iriscouch.com/docs/1.3/api/database/common.html
But I am a newbie and it also seems to be for a single JSON document. Im afraid I will just create one huge entry and not multiple documents entries which is what I want.
I have NodeJS but I don't have Cradle. Will I need it in order to perform this function? Any help is greatly appreciated!
Oh, no! That's bad idea to reference on docs at my Iris host - that's was preview dev build. Please follow the official docs: http://docs.couchdb.org/en/latest/api/index.html
To update multiple document's you need to use /db/_bulk_docs resource:
curl -X POST http://localhost:5984/db/_bulk_docs \
-H "Content-Type:application/json" \
-d '{"docs":[{"name":"Yoda"}, {"name":"Han Solo"}, {"name":"Leia"}]}'
So you see the format of data you should send to CouchDB? Now it's all depending from you JSON file format. If whole file is an array of objects: just wrap his content into {"docs":..} object. If not: you have to write some small library to convert this file data into the required format.