I've got a set of records in one of my elastic indexes and I'm trying to execute a search query via postman, my current query looks like this and should be getting 2 results found, but getting zero. Anything wrong with it?
{
"query": {
"match": {
"vehicle.CAR_WHEEL_DESCRIPTION": "A1=BB=C2C=D35"
}
}
}
Current Response:
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}
I tried doing something like this, but I'm getting not only exact matches, but some others as well, total 12 records (should be 2, exact match records)
{
"query": {
"nested": {
"path": "vehicle",
"query": {
"bool": {
"must": [
{ "match": { "vehicle.CAR_WHEEL_DESCRIPTION":
"A1=BB=C2C=D35" } }
]
}
},
"score_mode": "avg"
}
}
}
This is my example based on your information.
PUT teste
{
"mappings": {
"properties": {
"vehicle": {
"type": "nested",
"properties": {
"CAR_WHEEL_DESCRIPTION": {
"type": "text"
}
}
}
}
}
}
POST teste/_doc
{
"vehicle": {
"CAR_WHEEL_DESCRIPTION": "A1=BB=C2C=D35"
}
}
GET teste/_search
{
"query": {
"nested": {
"path": "vehicle",
"query": {
"match": {
"vehicle.CAR_WHEEL_DESCRIPTION": {
"query": "A1=BB=C2C=D35"
}
}
}
}
}
}
Related
I am querying elastic search using status field and range but getting an error:
"type": "parsing_exception","reason": "[status] query malformed, no
start_object after query name"
Query looks as below:
{
"_source": {
"includes": []
},
"query": {
"bool": {
"must": [
{
"status": "IN_PROGRESS"
},
{
"range": {
"requestDate": {
"gte": "2018-10-01T08:00:00.000Z",
}
}
}
]
}
},
"sort": {
"requestDate": {
"order": "desc"
}
}
}
The error is that you haven't specified the query type - term or match - against status field. So if status is a text datatype, you should perform a match query:
{
"_source": {
"includes": []
},
"query": {
"bool": {
"must": [
{
"match":{ "status": "IN_PROGRESS"
}},
{
"range": {
"requestDate": {
"gte": "2018-10-01T08:00:00.000Z",
}
}
}
]
}
},
"sort": {
"requestDate": {
"order": "desc"
}
}
}
I have a problem with Elasticsearch
The following json values work in my local server but not in the remote server.
ERROR:query doesn't support multiple fields, found [date] and [price]
post.json
{
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "product:DESKTOP"
}
},
{
"range": {
"date": {
"gt": "2018-04-24",
"lte": "2018-06-24"
},
"price": {
"gt": 0,
"lte": 2000
}
}
}
]
}
},
"from": 10,
"size": 200 }
Where do I mistake? Thank you for answers
You can only specify one field per range query.
Try including two separate range queries. They'll be AND'd together, since they both show up in your must clause.
{
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "product:DESKTOP"
}
},
{
"range": {
"date": {
"gt": "2018-04-24",
"lte": "2018-06-24"
}
}
},
{
"range": {
"price": {
"gt": 0,
"lte": 2000
}
}
}
]
}
},
"from": 10,
"size": 200
}
You need multiple range queries, like this:
{
"query": {
"bool": {
"must": [
{
"query_string": {
"query": "product:DESKTOP"
}
},
{
"range": {
"date": {
"gt": "2018-04-24",
"lte": "2018-06-24"
}
}
},
{
"range": {
"price": {
"gt": 0,
"lte": 2000
}
}
}
]
}
},
"from": 10,
"size": 200
}
query name sets from 20151216 to 20151217
{
"from": 0,
"size": 200,
"query": {
"bool": {
"must": {
"range": {
"DATE": {
"from": 20151216,
"to": 2015121617,
"include_lower": true,
"include_upper": true
}
}
}
}
},
"_source": {
"includes": [
"NAME"
],
"excludes": []
}
}
Another day
{
"from": 0,
"size": 200,
"query": {
"bool": {
"must": {
"range": {
"DATE": {
"from": 20151217,
"to": 2015121618,
"include_lower": true,
"include_upper": true
}
}
}
}
},
"_source": {
"includes": [
"NAME"
],
"excludes": []
}
}
If in MYSQL I will use the following SQL to solve my problem.
SELECT NAME FROM Table1 where DATE between 20151216 and 20151217 intersect SELECT NAME FROM Table1 where DATE between 20151217 and 20151218
How does elasticsearch to find the intersection of two search results like mysql?
I'm using ELK stack and I'm trying to find out how to visualize all logs except of those from specific IP ranges (for example 10.0.0.0/8). Is there any way how to negate filter query:
{"wildcard":{"src_address":"10.*"}}
I put it to Buckets -> Split Bars -> Aggregation -> Filters and I would like to negate this query so I got all logs except of those from 10.0.0.0/8
This is the whole JSON request:
{
"query": {
"filtered": {
"query": {
"query_string": {
"query": "low_level_category:\"user_authentication_failure\" AND NOT src_address:\"10.*\"",
"analyze_wildcard": true
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"#timestamp": {
"gte": 1474384885044,
"lte": 1474989685044,
"format": "epoch_millis"
}
}
}
],
"must_not": []
}
}
}
},
"size": 0,
"aggs": {
"2": {
"date_histogram": {
"field": "#timestamp",
"interval": "3h",
"time_zone": "Europe/Berlin",
"min_doc_count": 200,
"extended_bounds": {
"min": 1474384885043,
"max": 1474989685043
}
},
"aggs": {
"3": {
"terms": {
"field": "src_address.raw",
"size": 5,
"order": {
"_count": "desc"
}
}
}
}
}
}
}
Thanks
You can input this in the Kibana search box and it should get you what you need:
NOT src_address:10.*
I have the following object whose value attribute is a nested object type:
{
"metadata": {
"tenant": "home",
"timestamp": "2016-03-24T23:59:38Z"
},
"value": {
{ "key": "foo", "int_value": 100 },
{ "key": "bar", "str_value": "taco" }
}
}
This type of object has the following mapping:
{
"my_index": {
"mappings": {
"my_doctype": {
"properties": {
"metadata": {
"properties": {
"tenant": {
"type": "string",
"index": "not_analyzed"
},
"timestamp": {
"type": "date",
"format": "dateOptionalTime"
}
}
},
"value": {
"type": "nested",
"properties": {
"str_value": {
"type": "string",
"index": "not_analyzed"
},
"int_value": {
"type": "long"
},
"key": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}
}
With this setup, I would like to perform an aggregation that performs the following result:
Perform a term aggregation on the str_value attribute of objects where the key is set to "bar"
In each bucket created from the above aggregation, calculate the sum of the int_value attributes where the key is set to "foo"
Have the results laid out in a date_histogram for a given time range.
With this goal in mind, I have been able to get the term and date_histogram aggregations to work on my nested objects, but have not had luck performing the second level of calculation. Here is the current query I am attempting to get working:
{
"query": {
"match_all": {}
},
"aggs": {
"filters": {
"filter": {
"bool": {
"must": [
{
"term": {
"metadata.org": "gw"
}
},
{
"range": {
"metadata.timestamp": {
"gte": "2016-03-24T00:00:00.000Z",
"lte": "2016-03-24T23:59:59.999Z"
}
}
}
]
}
},
"aggs": {
"intervals": {
"date_histogram": {
"field": "metadata.timestamp",
"interval": "1d",
"min_doc_count": 0,
"extended_bounds": {
"min": "2016-03-24T00:00:00Z",
"max": "2016-03-24T23:59:59Z"
},
"format": "yyyy-MM-dd'T'HH:mm:ss'Z'"
},
"aggs": {
"nested_type": {
"nested": {
"path": "value"
},
"aggs": {
"key_filter": {
"filter": {
"term": {
"value.key": "bar"
}
},
"aggs": {
"groupBy": {
"terms": {
"field": "value.str_value"
},
"aggs": {
"other_nested": {
"reverse_nested": {
"path": "value"
},
"aggs": {
"key_filter": {
"filter": {
"term": {
"value.key": "foo"
}
},
"aggs": {
"amount_sum": {
"sum": {
"field": "value.int_value"
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
}
The result I am expecting to receive from Elasticsearch would look like the following:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 7,
"max_score": 0.0,
"hits": []
},
"aggregations": {
"filters": {
"doc_count": 2,
"intervals": {
"buckets": [
{
"key_as_string": "2016-03-24T00:00:00Z",
"key": 1458777600000,
"doc_count": 2,
"nested_type": {
"doc_count": 5,
"key_filter": {
"doc_count": 2,
"groupBy": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "taco",
"doc_count": 1,
"other_nested": {
"doc_count": 1,
"key_filter": {
"doc_count": 1,
"amount_sum": {
"value": 100.0
}
}
}
}
]
}
}
}
}
]
}
}
}
}
However, the innermost object (...groupBy.buckets.key_filter.amount_sum) is having its value return 0.0 instead of 100.0.
I think this is due to the fact that nested objects are indexed as separate documents, so filtering by one key attribute's value is not allowing me to query to against another key.
Would anyone have any idea on how to get this type of query to work?
For a bit more context, the reason for this document structure is because I do not control the content of the JSON documents that get indexed, so different tenants may have conflicting key names with different values (e.g. {"tenant": "abc", "value": {"foo": "a"} } vs. {"tenant": "xyz", "value": {"foo": 1} }. The method I am trying to use is the one laid out by this Elasticsearch Blog Post, where it recommends to transform objects that you don't control into a structure that you do and to use nested objects to help with this (specifically the Nested fields for each data type section of the article). I would also be open to learn of a better way to handle this situation of not controlling the document's JSON structure if there is one so that I can perform aggregations.
Thank you!
EDIT: I am using Elasticsearch 1.5.
Solved this situation by utilizing the reverse_nested aggregation in the correct way as described here: http://www.shayne.me/blog/2015/2015-05-18-elasticsearch-nested-docs/