Post PowerShell JSON into ElasticSearch - json

Question

I'm trying to push some data into ElasticSearch via PowerShell. I'm converting the data into JSON, then trying both Invoke-WebRequest and Invoke-RestMethod, but always get errors on malformed data or content-type not supported. I haven't created the index as I believe it will create it for me.
Anyone able to assist on what I'm missing/doing wrong?
Example code:
$data = #()
$CustomObject = [pscustomobject]#{
SqlInstance = "myserver1"
Database = "mydb"
Schema = "versioning"
Name = "DataVersionHistory"
IndexSpaceUsed = 0
DataSpaceUsed = 0
RowCount = 0
};
$data += $CustomObject;
$CustomObject = [pscustomobject]#{
SqlInstance = "myserver1"
Database = "mydb"
Schema = "versioning"
Name = "VersionHistory"
IndexSpaceUsed = 10
DataSpaceUsed = 25
RowCount = 3000
};
$data += $CustomObject;
$myJson = ConvertTo-Json -InputObject $data ;
Invoke-RestMethod -Uri http://localhost:9200/myindex/mytype/_bulk?pretty `
-Method POST -Body $myJson -ContentType "application/json"

Answer 1

Bulk request is not actual json. You should use following notation:
curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }
'

Answer 2

$data = #()
$CustomObject = [pscustomobject]#{
SqlInstance = "myserver1"
Database = "mydb"
Schema = "versioning"
Name = "DataVersionHistory"
IndexSpaceUsed = 0
DataSpaceUsed = 0
RowCount = 0
};
$data += $CustomObject;
$CustomObject = [pscustomobject]#{
SqlInstance = "myserver1"
Database = "mydb"
Schema = "versioning"
Name = "VersionHistory"
IndexSpaceUsed = 10
DataSpaceUsed = 25
RowCount = 3000
};
$data += $CustomObject
At this point you have a collection of objects contained in the array $data.
NOTE: for the bulk API the final line of the json data must end with a newline character \n. So I am using a here-string to append the newline char \n to each json element.
also notice that i removed pretty print. this is to play nice with the bulk api.
doc_as_upsert ensures that if the doc exists leave it alone, if not add as new.
$json_col = #()
$data |
ForEach-Object {
#convert object to json
$json_element = #{doc = $_; doc_as_upsert = $true}
| ConvertTo-Json -Depth 1 -Compress
#construt here string with literal \n
$json_col += #"
$json_element\n
"#
}
Invoke-RestMethod -Method Post -Uri "http://localhost:9200/myindex/mytype/_bulk?" -Body $json_col -ErrorAction Stop