indexing json with json value in elasticsearch - json

I am trying to index a document in elasticsearch. The json I have comes from the document being transformed from XML to JSON. It is valid JSON. Looks like this:
{
"shortcasename": {
"_attributes": {
"party1": "People",
"party2": "Johnson"
},
"_children": [
"People",
{
"connector": {
"_attributes": {
"normval": "v"
},
"_children": [
" v. "
]
}
},
"Johnson"
]
}
}
Elasitcsearch seems to have a problem with the shortcasename._children. The error I get is:
{
"error": {
"root_cause": [
{
"type": "mapper_parsing_exception",
"reason": "failed to parse"
}
],
"type": "mapper_parsing_exception",
"reason": "failed to parse",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "mapper [shortcasename._children] of different type, current_type [string], merged_type [ObjectMapper]"
}
},
"status": 400
}
Is there a way to get the json to be indexed the way it is?

The JSON you have has a conflict with the _children field:
{
"shortcasename": {
"_attributes": {
"party1": "People",
"party2": "Johnson"
},
"_children": [
"People",
{
"connector": {
"_attributes": {
"normval": "v"
},
"_children": [
" v. "
]
}
},
"Johnson"
]
}
}
The top-level _children field is an array containing a mix of objects ({"connector": ...}) and strings ("People", "Johnson"). Elasticsearch doesn't support that, that's why it complains that it cannot merge string and Object

Related

jmespath :select json object element based on other (array) element in the object

I have this JSON
{
"srv_config": [{
"name": "db1",
"servers": ["srv1", "srv2"],
"prop": [{"source":"aa"},"destination":"bb"},{"source":"cc"},"destination":"cc"},]
}, {
"name": "db2",
"servers": ["srv2", "srv2"],
"prop": [{"source":"dd"},"destination":"dd"},{"source":"ee"},"destination":"ee"},]
}
]
}
I try to build a JMESPath expression to select the prop application in each object in the main array, but based on the existence of a string in the servers element.
To select all props, I can do:
*.props [*]
But how do I add condition that says "select only if srv1 is in servers list"?
You can use the contains function in order to filter based on a array containing something.
Given the query:
*[?contains(servers, `srv1`)].prop | [][]
This gives us:
[
{
"source": "aa",
"destination": "bb"
},
{
"source": "cc",
"destination": "cc"
}
]
Please mind that I am also using a bit of flattening here.
All this run towards a corrected version of you JSON:
{
"srv_config":[
{
"name":"db1",
"servers":[
"srv1",
"srv2"
],
"prop":[
{
"source":"aa",
"destination":"bb"
},
{
"source":"cc",
"destination":"cc"
}
]
},
{
"name":"db2",
"servers":[
"srv2",
"srv2"
],
"prop":[
{
"source":"dd",
"destination":"dd"
},
{
"source":"ee",
"destination":"ee"
}
]
}
]
}

GraphQL Query returns null objects both in GraphiQL and App from JSON data source

I'm trying to get my mocked JSON data via GraphQL in Gatsby. The response shows the correct data, but also two null objects as well. Why is it happening?
I'm using the gatsby-transformer.json plugin to query my data and gatsby-source-filesystem to point the transformer to my json files.
categories.json
the mock file I'm trying to get to work :)
{
"categories": [
{
"title": "DEZERTY",
"path": "/dezerty",
"categoryItems": [
{
"categoryName": "CUKRIKY",
"image": "../../../../static/img/dessertcategories/cukriky.jpg"
},
{
"categoryName": "NAHODNE",
"image": "../../../../static/img/dessertcategories/nahodne.jpg"
},
]
},
{
"title": "CANDY BAR",
"path": "/candy-bar",
"categoryItems": [
{
"categoryName": "CHEESECAKY",
"image": "../../../../static/img/dessertcategories/cheesecaky.jpg"
},
{
"categoryName": "BEZLEPKOVÉ TORTY",
"image": "../../../../static/img/dessertcategories/bezlepkove-torty.jpg"
},
]
}
]
}
GraphQL query in GraphiQL
query Collections {
allMockJson {
edges {
node {
categories {
categoryItems {
categoryName
image
}
title
path
}
}
}
}
}
And the response GraphiQL gives me
{
"data": {
"allMockJson": {
"edges": [
{
"node": {
"categories": null
}
},
{
"node": {
"categories": null
}
},
{
"node": {
"categories": [
{
"categoryItems": [
{
"categoryName": "CHEESECAKY",
"image": "../../../../static/img/dessertcategories/cheesecaky.jpg"
},
{
"categoryName": "BEZLEPKOVÉ TORTY",
"image": "../../../../static/img/dessertcategories/bezlepkove-torty.jpg"
}
],
"title": "DEZERTY",
"path": "/dezerty"
},
{
"categoryItems": [
{
"categoryName": "CUKRIKY",
"image": "../../../../static/img/dessertcategories/CUKRIKY.jpg"
},
{
"categoryName": "NAHODNE",
"image": "../../../../static/img/dessertcategories/NAHODNE.jpg"
}
],
"title": "CANDY BAR",
"path": "/candy-bar"
}
]
}
}
]
}
}
}
I expected only to get the DEZERTY and CANDY BAR sections. Why are there null categories and how do I fix it?
Thanks in advance
Your JSON contains syntax errors in the objects DEZERTY and CANDY BAR. It silently fails without telling you. Try this json linter.
Error: Parse error on line 12: },
Error: Parse error on line 25: },
Try again. Your query should work now.
You should look into an IDE that highlights these types of errors and saves you time and frustration.

Expected name at xx:yy in JSON file in eclipse

"LOTGraphData": [
{
buildUnits:"1"
ScheduledUnits:2"
prodGroupId:"9288"
},
{
buildUnits:"1"
ScheduledUnits:2"
prodGroupId:"9289"
},
{
buildUnits:"1"
ScheduledUnits:2"
prodGroupId:"9280"
}
]
}
This is the end of JSON file.
You have to separate your key/value pairs by commas. And (as per your confirmation), the keys have to be wrapped in quotes. So the final result will look like:
"LOTGraphData": [
{
"buildUnits": "1",
"ScheduledUnits": "2",
"prodGroupId": "9288"
},
{
"buildUnits": "1",
"ScheduledUnits": "2",
"prodGroupId": "9289"
},
{
"buildUnits": "1",
"ScheduledUnits": "2",
"prodGroupId": "9280"
}
]

How to add a json object to multiple documents in a Elastic index using _update_by_query?

I need to update several documents in my Elasticsearch index and I tried the following using the the _update_by_query plugin.
What I need to do is to add a new field to several existing documents matching a certain condition. The new field is a nested JSON. So after adding it document source should look like
_source: {
...existing fields,
"new_field" : {
"attrName1" : "value",
"attrName2" : "value",
}
}
I tried using the _update_by_query API to get this done. But so far I only could add String fields and arrays with it. When trying to add a JSON with the following query it gives me an error.
Query
curl -XPOST "http://xxx.xxx.xxx.xxx:pppp/my_index_name/_update_by_query" -d'
{
"query": {
"bool": {
"must": [
{
"term": {
"team.keyword": "search_phrase"
}
}
]
}
},
"script" : {
"inline":"ctx._source.field_name = {\"a\":\"b\"}"
}
}'
Error
{
"error": {
"root_cause": [
{
"type": "script_exception",
"reason": "compile error",
"script_stack": [
"ctx._source.field_name = {\"a\":\"b\"}",
" ^---- HERE"
],
"script": "ctx._source.field_name = {\"a\":\"b\"}",
"lang": "painless"
}
],
"type": "script_exception",
"reason": "compile error",
"caused_by": {
"type": "illegal_argument_exception",
"reason": "invalid sequence of tokens near ['{'].",
"caused_by": {
"type": "no_viable_alt_exception",
"reason": null
}
},
"script_stack": [
"ctx._source.field_name = {\"a\":\"b\"}",
" ^---- HERE"
],
"script": "ctx._source.field_name = {\"a\":\"b\"}",
"lang": "painless"
},
"status": 500
}
So far I could only add Strings as a new field. What is the correct way to achieve this?
Instead of direct assignment, use params to achieve the same.
{
"query": {
"bool": {
"must": [
{
"term": {
"team.keyword": "search_phrase"
}
}
]
}
},
"script": {
"inline": "ctx._source.field_name = params.new_field",
"params": {
"new_field": {
"a": "b"
}
}
}
}

Sending the JSON response as array or normal object

I am implementing a restful service where I am getting the pdf names and their ids from the database in the JSON fromat. Which one of the both the convenient JSON resful service response?
First Option:
{
"results": {
"documentNames": [
"test.pdf",
"ireport-ultimate-guide.pdf",
"sending report.pdf",
"Motor Hour.pdf"
],
"documentds": [
21116,
21117,
21118,
21119
]
}
}
Second Option:
{
"results": {
"21116": "test.pdf",
"21117": "ireport-ultimate-guide.pdf",
"21118": "sending report.pdf",
"21119": "Motor Hour.pdf"
}
}
I would use this "third option": The result is a list of object.
{
"result": [{
"id": "21116",
"filename": "test.pdf"
},
{
"id": "21117",
"filename": "ireport-ultimate-guide.pdf"
},
{
"id": "21118",
"filename": "sending report.pdf"
},
{
"id": "21119",
"filename": "Motor Hour.pdf"
}
]
}
because it better models the object structure.
I would create an entity for each document that contains both name and I'd.
[
{"name": "doc_1", "id": 123},
{"name": "doc_2", "id": 456}
]