Elasticsearch filtering nested object - json

I have a simplified object that looks something like this:
"name" : "Partner Name",
"features" : [
{
"val" : "Family",
"key" : "Type"
},
{
"val" : "Paris",
"key" : "City"
}
],
"variants" : [
{
"name" : "Activity 1 Name",
"description" : "Quick description",
"price" : 20
}
]
I want to filter by the City and Type keys. My current query filters by price but I can't get it working for City or Type. Adding more terms to the filter array didn't do the trick.
'query':{
'filtered':{
'query':{
'query_string':{
'query':query
}
},
'filter': {
'bool':{
'filter': [{
'range': {
'variants.price': {
'gte': 0
}
}
},
{
'range': {
'variants.price': {
'lte': 50
}
}
},
{
'term': {
'active': true
}
}
]
}
}
}
}
Any help would be appreciated. Thanks!

{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"filter": [
{
"range": {
"variants.price": {
"gte": 0
}
}
},
{
"range": {
"variants.price": {
"lte": 50
}
}
},
{
"nested": {
"path": "features",
"query": {
"bool": {
"should": [
{"term":{"features.key":"type"}},
{"term":{"features.key":"city"}}
]
}
}
}
}
]
}
}
}
}
}

Related

Elasticsearch query throwing an error when using range with bool

I am querying elastic search using status field and range but getting an error:
"type": "parsing_exception","reason": "[status] query malformed, no
start_object after query name"
Query looks as below:
{
"_source": {
"includes": []
},
"query": {
"bool": {
"must": [
{
"status": "IN_PROGRESS"
},
{
"range": {
"requestDate": {
"gte": "2018-10-01T08:00:00.000Z",
}
}
}
]
}
},
"sort": {
"requestDate": {
"order": "desc"
}
}
}
The error is that you haven't specified the query type - term or match - against status field. So if status is a text datatype, you should perform a match query:
{
"_source": {
"includes": []
},
"query": {
"bool": {
"must": [
{
"match":{ "status": "IN_PROGRESS"
}},
{
"range": {
"requestDate": {
"gte": "2018-10-01T08:00:00.000Z",
}
}
}
]
}
},
"sort": {
"requestDate": {
"order": "desc"
}
}
}

elastic search nested filter

Here is the structure of a document, as returned by the Elastic Search API:
{
"process_name":"process01",
"beat":
{
"hostname":"12345","name":"blablabla"
},
}
Filtering by process_name was easy, but how can I filter by host_name, which is nested inside beat?
Failed attempt 1
{
"size":10000,
"query" : {
"bool" : {
"should": [
{ "match" : { "process_name" : "process01" } },
{ "match" : { "process_name" : "process02" } }
],
"must": [
{ "match" : { beat: { "hostname":"12345" } } }
]
}
}
}
error message 1:
(failed to deserialize object type=class com.logshero.api.SearchApiRequest):
Failed attempt 2
{
"size":10000,
"query" : {
"bool" : {
"should": [
{ "match" : { "process_name" : "process01" } },
{ "match" : { "process_name" : "process02" } }
],
"must": [
{ "match" : { "hostname":"12345" } }
]
}
}
}
error message 2:
{"hits":{"total":0,"max_score":null,"hits":[]}}
you can use the following query. You also have to make sure that beat in your mappings is defined as nested type.
{
"size": 10000,
"query": {
"bool": {
"should": [{
"match": {
"process_name": "process01"
}
}, {
"match": {
"process_name": "process02"
}
}],
"must": [{
"match": {
"beat.hostname": "12345"
}
}]
}
}
}
Thanks

How to make nested and, or and like in elasticsearch

I need to do as this sentence sql in elasticsearch but not run me
select * from user where (name like '%juan%') and position = 2 and catid in (4,3,76453,345,345,345345,345) and prof = 9
This is one of the examples I've used
{
"size": 25,
"query":
{
"filtered":
{
"filter":
{
"and":
[
{ "term": { "user":"juan" } },
{ "term": { "position":"2" } },
{
"or":
[
{ "term": { "catid":"4" } },
{ "term": { "catid":"3" } }
]
}
]
}
}
}
}
Other :
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"bool": {
"should": [{
"term": {
"name": "juan"
}
}]
}
},
{
"bool": {
"should": [{
"term": {
"position": "2"
}
}]
}
}
]
},
"or":
[
{ "term": { "catid":"4" } },
{ "term": { "catid":"3" } }
]
}
}
},
"size": 1000
}
I really need to search for word and with many variables , here I do not put all but given time are at least 10 more ..
The filters were removed in version 2.0 of Elasticsearch and you have to use the queries instead. If you want to filter, you have two options now:
Use bool query with filter clause (equivalent to a filtered query containing scoring)
Use constant_score query (equivalent to a filtered query without scoring)
Moreover, you should avoid and and or queries, as they are deprecated and will be removed soon.
Regarding your SQL query, I'd translate it like that:
"query": {
"bool": {
"filter": {
"bool": {
"must": [
{
"match": { "user": "juan" }
},
{
"term": { "position":"2" }
},
{
"terms": {
"catid": [4,3,76453,345]
}
}
]
}
}
}
}
Please note that term-query do exact matches on your fields! If e.g. your user field contains names containing of multiple words. The term-query won't return the expected result. In this case, you should use the match-query.
It seems to work
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"numeric_range": {"stocktotal": {"gte": 2,"lte": 99999999999}}
}
,
{
"term": { "cat_name":"limpiadoasabarra_derechabarra_derechaapulidoras__de__superficies" }
},
{"terms": {"supplier_id": [8634, 916]}}
]
}
}
},
"size" : 1000,
"aggs" : {
"etiquetas" : {"terms" : { "field" : "supplier_name" }, "size" : 20 },
"nombre_familia" : {"terms" : { "field" : "nombre_familia", "size" : 20 }},
"nombre_subfamilia" : {"terms" : { "field" : "nombre_subfamilia", "size" : 20 }},
"cat_name" : {"terms" : { "field" : "cat_name" }, "size" : 20}
}
}

How can I get all the documents which have at least the properties specified in an Elasticsearch query?

It is possible to select an item from the index which match multiple values for a certain sub-item? I think this is not so clear but I added more details below.
I have the following index:
{
"mappings" : {
"entity" : {
"properties" : {
"name" : {"type" : "string"},
"features" : {
"type" : "nested",
"include_in_parent" : false,
"properties" : {
"id" : {"type" : "integer"},
"value_int" : {"type" : "integer"},
"value_text" : {"type" : "string"},
"value_decimal" : {"type" : "integer"}
}
}
}
}
},
"settings" : {
"number_of_shards" : 1,
"number_of_replicas" : 0
}
}
Some items from the index
{
"name" : "Bazar",
"features" : [
{
"id" : 1,
"value_text" : null,
"value_decimal" : null,
"value_int": 51
},
{
"id" : 9,
"value_text" : "Amsterdam",
"value_decimal" : null,
"value_int": null
}
]
}
{
"name" : "Bazar Test",
"features" : [
{
"id" : 1,
"value_text" : null,
"value_decimal" : null,
"value_int": 52
},
{
"id" : 9,
"value_text" : "Leiden",
"value_decimal" : null,
"value_int": null
}
]
}
{
"name" : "Bazar no city",
"features" : [
{
"id" : 1,
"value_text" : null,
"value_decimal" : null,
"value_int": 51
},
]
}
What I need is a way to find just the items which have the features.id = 1 and features.id = 2 (ex: "Bazar" and "Bazar Test" items).
The query I got some far is
{
"query" : {
"nested" : {
"path" : "features",
"query" : {
"bool" : {
"must" : [
{ "terms" : { "features.id" : [1, 9]} }
]
}
}
}
}
}
The problem with this query is that it selects the items which have features.id = 1 OR features.id = 9 so all the items are returned.
Edit
Tried a new query
{
"query" : {
"nested" : {
"path" : "features",
"query" : {
"bool" : {
"must" : [
{ "terms" : {
"features.id" : [1, 9],
"minimum_should_match": 2
}
}
]
}
}
}
}
}
But I got no results.
Edit:
After I combined the answers, I managed to get it working.
Thank you for help :)
This is my query (a bit modified)
{
"from": 0,
"size": 20,
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"match_phrase_prefix": {
"title": {
"query": "deli",
"max_expansions": 5
}
}
},
{
"match": {
"entity_type_id": 5
}
}
]
}
},
"filter": {
"and": {
"filters": [
{
"nested": {
"path": "features",
"query": {
"bool": {
"must": [
{
"match": {
"features.id": 31
}
},
{
"match": {
"features.value_int": {
"query": [
56, 57
],
"operator": "and"
}
}
}
]
}
}
}
}
]
}
}
}
}
}
Thank you.
The match query supports a Boolean operator parameter. You should also wrap the query in a nested query, as the features field is nested in your mapping.
Try this query:
{
"query": {
"nested": {
"query": {
"match": {
"features.id": {
"query": "1 9",
"operator": "and"
}
}
},
"path": "features"
}
}
}
Nested documents are more difficult to query. This should be what you want:
{
"query": {
"filtered": {
"filter": {
"and": {
"filters": [
{
"nested": {
"path": "features",
"query": {
"term": {
"features.id": {
"value": "1"
}
}
}
}
},
{
"nested": {
"path": "features",
"query": {
"term": {
"features.id": {
"value": "9"
}
}
}
}
}
]
}
}
}
}
}

Elasticsearch aggregation

I'm looking how to merge elasticsearch hits into one. Basicly I want to do the equivalent of
SELECT COUNT(*) WHERE TOTO = 1 AND TATA = 2 AND TITI = 3
I've manage to deal with the WHERE part using filters but I get all the hits in separate entities
so what I have is
SELECT TOTO, TATA, TITI WHERE TOTO = 1 AND TATA =2 AND TATA =3
How do I manage to get only one hit containing the COUNT(*) value ?
Environment
{
"my_element": {
"mappings": {
"test": {
"properties": {
"baskets": {
"type": "nested",
"properties": {
"basket_id": {
"type": "string"
},
"num_basket": {
"type": "integer"
},
"tp_basket": {
"type": "string"
}
}
},
"test_id": {
"type": "string"
},
"test_name": {
"type": "string"
}
}
}
}
}
}
so I want to count how many test element have a basket_id of X and a num_basket of 3 (if they do that means baskets are Identical, so I want also to show baskets field)
{
"fields": [
"bucket_list",
"baskets.basket_id",
"baskets.num_basket"
],
"query": {
"filtered": {
"filter": {
"and": {
"filter": [
{
"nested": {
"path": "baskets",
"filter": {
"and": {
"filters": [
{
"or": {
"filters": [
{
"bool": {
"must": [
{
"term": {
"baskets.basket_id": "40"
}
},
{
"term": {
"baskets.num_basket": "1"
}
},
{
"term": {
"baskets.tp_basket": "1"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"baskets.basket_id": "41"
}
},
{
"term": {
"baskets.num_basket": "1"
}
},
{
"term": {
"baskets.tp_basket": "1"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"baskets.basket_id": "342"
}
},
{
"term": {
"baskets.num_basket": "1"
}
},
{
"term": {
"baskets.tp_basket": "1"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"baskets.basket_id": "342"
}
},
{
"term": {
"baskets.num_basket": "1"
}
},
{
"term": {
"baskets.tp_basket": "1"
}
}
]
}
}
]
}
}
]
}
}
}
},
{
"nested": {
"path": "baskets",
"filter": {
"and": {
"filters": [
{
"bool": {
"must": [
{
"term": {
"baskets.basket_id": "15"
}
},
{
"term": {
"baskets.num_basket": "2"
}
}
]
}
}
]
}
}
}
},
{
"nested": {
"path": "baskets",
"filter": {
"and": {
"filters": [
{
"bool": {
"must": [
{
"term": {
"baskets.basket_id": "15"
}
},
{
"term": {
"baskets.num_basket": "3"
}
}
]
}
}
]
}
}
}
}
]
}
}
}
}
}
and result
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 2,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "my_element",
"_type": "test",
"_id": "sMHPv3i4RTqNCIChGh4Iew",
"_score": 1,
"fields": {
"baskets.basket_id": [
"15",
"15",
"15"
],
"baskets.num_basket": [
2,
3,
1
]
}
},
{
"_index": "my_element",
"_type": "test",
"_id": "KL3U-g-7RtuusNV8hi9YHQ",
"_score": 1,
"fields": {
"baskets.basket_id": [
"15",
"15",
"15"
],
"baskets.num_basket": [
1,
2,
3
]
}
}
]
}
}
Probably I am answering really late, but for those who are looking for the answer. This query can be written in very simple way, as follows.
GET _count
{
"query":
{
"bool" : {
"must" : [ {
"term" : {
"basket_id" : "X"
}
}, {
"term" : {
"num_basket" : 3
}
} ]
}
}
}
The above query will give you following result.
{
"count": 6,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
}
}
You can read the count from above response, which is your desired result.
Hope this was helpful.
When the search results are returned from Elasticsearch you should see a total field - this has the count of matching documents.
for example, see "total" : 2, below:
{
"took" : 3,
"timed_out" : false,
"_shards" : {
"total" : 12,
"successful" : 12,
"failed" : 0
},
"hits" : {
"total" : 2,
"max_score" : 2.098612,
"hits" : [ {
"_index" : "mytest",
"_type" : "message",
"_id" : "P9wGgJHjQmK8GUvw8M5Q8A",
"_score" : 2.098612,
"fields" : {
"tata" : [ "1" ],
"toto" : [ "1" ],
"titi" : [ "2" ]
}
}, {
"_index" : "mytest",
"_type" : "message",
"_id" : "M26ychoyRR6HkordRdS_HA",
"_score" : 0.30685282,
"fields" : {
"tata" : [ "1" ],
"toto" : [ "1" ],
"titi" : [ "2" ]
}
} ]
}
}