How to create index for nested data structure with list? There will be list of otherUserID's and I don't know how to index them with elasticsearch 6.5.
UserID -> OtherUserID-> name:"text" , count : "long"
You can uses nested data type for creating such a field and indexing list of object.
Refer to the example below and modify it to your needs:
Mapping:
PUT testindex
{
"mappings": {
"_doc": {
"properties": {
"nestedField": {
"type": "nested",
"properties": {
"field1": {
"type": "text",
"fields": {
"keywords": {
"type": "keyword"
}
}
},
"field2": {
"type": "integer"
}
}
}
}
}
}
}
Adding documents:
For single item in list:
PUT testindex/_doc/1
{
"nestedField": [
{
"field1": "Some text",
"field2": 10
}
]
}
For multiple items in list:
PUT testindex/_doc/2
{
"nestedField": [
{
"field1": "Some other text",
"field2": 11
},
{
"field1": "random value",
"field2": 15
}
]
}
Related
I want to create a JSON with openAPI specification. Example payload format to api/endpoint is as below,
"abc": {"name": "PA", "id": "21"}
So here is the openapi JSON format,
{
"openapi": "3.0.0",
"info": {...},
"paths": {
"api/endpoint": {
"put": {
"requestBody": {
"required": true,
"content": {
"application/json": {
"schema": {
"$ref": "#/components/schemas/putSchema"
}
}
}
}
}
}
},
"components": {
"schemas": {
"putSchema": {
"abc": { ----------> *
"type": "object",
"properties": {
"name": {...},
"id": {...}
}
}
}
}
}
}
But my payload can vary like,
"abc": {"name": "PA", "id": "21"}
"xyz": {"name": "ST", "id": "35"}
"def": {"name": "UV", "id": "94"}
Not sure how to define the JSON format for dynamic key name (here, abc/xyz/def) - (ie) what to fill in the place * in JSON).
So my query is how to specify the key in JSON whose name is not static.
In your example, putSchema is a string-to-object map, where "abc"/"xyz"/etc. is the key in the map. Maps are defined by using the additionalProperties keyword:
"putSchema": {
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"name": { ...},
"id": { ... }
}
}
}
If your payload has only one root key (e.g. just "abc" but not "abc" and "xyz" at the same time), you can add "maxProperties": 1 to your putSchema to limit the number of the root keys.
I have the following JSON format and want to create a mapping for it from Elasticsearch console :
{
"properties": {
"#timestamp" : {
"type" : "date"
},
"list": [
{
"name": "John",
"age": "37",
"title": "Tester"
}
]
}
}
There's no list or array type in ES, you simply declare objects and then you can add a list of those objects to your documents:
PUT your-index
{
"mappings": {
"properties": {
"#timestamp" : {
"type" : "date"
},
"list": {
"type": "object",
"properties": {
"name": {
"type": "text"
},
"age": {
"type": "integer"
},
"title": {
"type": "text"
}
}
}
}
}
}
I'm using django-elasticsearch-dsl-drf package and I have Postgres jsonField which I want to index. I tried to use Nestedfield in the document but without any properties since the json field is arbitrary, But I'm not able to search on that field, and I don't see anything related to that on their documentation.
Any idea how can I achieve this?
Mapping:
{
"mappings": {
"_doc": {
"properties": {
"jsondata": {
"type": "nested",
"properties": {
"timestamp": {
"type": "date"
},
"gender": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
},
"group_id": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}, ...
I want to search on that field like jsondata.gender = x
Query for jsonfield.gender = x
GET <index>/_search
{
"query": {
"nested": {
"path": "jsonfield",
"query": {
"term": {
"jsonfield.gender.keyword": {
"value": "x"
}
}
}
}
}
}
NOTE: The query has not been verified using Kibana Dev Tools.
I am trying to use nested JSON to query DB records. Here is my query -
"query": {
"nested": {
"path": "metadata.technical",
"query": {
"bool": {
"must": [
{
"term": {
"metadata.technical.key": "techcolor"
}
},
{
"term": {
"metadata.technical.value": "red"
}
}
]
}
}
}
}
Here is this part in my mapping.json -
"metadata": {
"include_in_parent": true,
"properties": {
"technical": {
"type": "nested",
"properties": {
"key": {
"type": "string"
},
"value": {
"type": "string"
}
}
}
}
}
And I have table that has column 'value' and its content is -
{"technical":
{
"techname22": "test",
"techcolor":"red",
"techlocation": "usa"
}
}
Why I can't get any results with this? FYI I am using ES 1.7. Thanks for any help.
To respect the mapping you've defined your sample document should look like this:
{
"technical": [
{
"key": "techname22",
"value": "test"
},
{
"key": "techcolor",
"value": "red"
},
{
"key": "techlocation",
"value": "usa"
}
]
}
Changing your document with the above structure would make your query work as it is.
The real mapping of this document:
{
"technical": {
"techname22": "test",
"techcolor": "red",
"techlocation": "usa"
}
}
Is more like this:
{
"include_in_parent": true,
"properties": {
"technical": {
"type": "nested",
"properties": {
"techname22": {
"type": "string"
},
"techcolor": {
"type": "string"
},
"techlocation": {
"type": "string"
}
}
}
}
}
If all your keys are dynamic and not known in advance, you can also configure your mapping to be dynamic as well, i.e. don't define any fields in the nested type and new fields will be added if not already present in the mapping:
{
"include_in_parent": true,
"properties": {
"technical": {
"type": "nested",
"properties": {
}
}
}
}
I have below mapping, I want to access a property of imageMap instead of all collection.
"imageMap": {
"properties": {
"item1": {
"type": "long"
},
"item2": {
"type": "string"
},
"item3": {
"type": "string"
}
}
}
Below is the sample data
imageMap": [
{
"item1": 20893,
"item2": "ImageThumbnail_100_By_44",
"item3": "/9j/4AAQSkZJRg"
},
{
"item1": 20893,
"item2": "ImageThumbnail_400_By_244",
"item3": "/9j/4AAQSkZJRgABAQEAYABgAAD/2w"
}
]
Below is my Query that is not working. Any help is appreciated. Thank you in advance.
Updated:
{
"_source": {
"include": [
"imageMap"
]
},
"query": {
"bool": {
"must": [
{
"term": {
"imageMap.item1": {
"value": 20893
}
},
"term": {
"imageMap.item2": {
"value": "imagethumbnail_100_by_44"
}
}
}
]
}
}
}
Expected Result is below only single element of imageMap, but i am getting array :
"_source": {
"imageMap": [
{
"item2": "ImageThumbnail_100_By_44",
"item1": 20893,
"item3": "/9j/4AAQSkZJRgABAQ"
}
]
}
If you only wan to get a single element from the imageMap array, you need to map imageMap as a nested object like this:
"imageMap": {
"type": "nested", <--- add this
"properties": {
"item1": {
"type": "long"
},
"item2": {
"type": "string"
},
"item3": {
"type": "string"
}
}
}
Then you need to wipe your index and re-build it from scratch with this new mapping.
In the end, you'll be able to retrieve only a specific element using a nested inner_hits query:
{
"_source": false,
"query" : {
"nested" : {
"path" : "imageMap",
"query" : {
"match" : {"imageMap.item2" : "ImageThumbnail_100_By_44"}
},
"inner_hits" : {}
}
}
}
Your query is not working because you are using term query which does not do any analysis on your search string. Since you have not specified any analyzer in mapping ImageThumbnail_100_By_44 is stored as imagethumbnail_100_by_44 because it is analyzed by standard analyzer
Depending on your requirement you could either map your item2 as "index : not_analyzed" and your query will work fine or you could use match query which will do analysis.
{
"_source": {
"include": [
"imageMap"
]
},
"query": {
"bool": {
"must": [
{
"match": {
"imageMap.item2": {
"query": "ImageThumbnail_100_By_44"
}
}
}
]
}
}
}
Please go through this document to have better understanding of analysis process