New to ElasticSearch.
I have documents under an index: myindex in Elastic search with mappings:
http://host:port/myindex/_mapping
{
"mappings":{
"properties": {
"en_US": {
"type": "keyword"
}
}
}
}
Let's say my 3 documents look like this:
{
"product": "p1",
"subproduct": "p1.1"
}
{
"product": "p1",
"subproduct": "p1.2"
}
{
"product": "p2",
"subproduct": "p2.1"
}
Now, I am querying using for single subproduct p1.1 with product p1 as below and it's working fine:
POST: http://host:port/myindex/_search
{
"query": {
"bool" : {
"must" : {
"term" : { "product" : "p1" }
},
"filter": {
"term" : { "subproduct" : "p1.1" }
}
}
}
}
My question is:
How I can query for 2 or more subproducts in one _search query, like suproducts p1.1 and p1.2 under product p1 ?
Query should return list of all subproduct p1.1 and subproduct p1.2 with p1 product.
Simply change the term-query in your filter-clause to a terms-query and search for multiple terms.
{
"query": {
"bool" : {
"must" : {
"term" : { "product" : "p1" }
},
"filter": {
"terms" : { "subproduct" : ["p1.1", "p1.2"] }
}
}
}
}
Related
I have the following query in MongoDB, which selects the last 10 conversations a user has participated in. They are sorted based on the last message in a conversation. I have a way to paginate between the results using $gt in the match statement. For now, this is commented out.
I limited the number of members to 3 (using slice), because a group conversation can contain, for example, 30, 40, 50, ... members.
The problem: I have a MySQL database to store relational information. So I have a table users and a table followers, to show who is following each other. I want to include this information in a way into MongoDB, so I can find 3 members I am following, to show relevant profile pictures of known people. Let's say I have a conversation with 50 members, then I want to retrieve 3 people I follow (if any), so I see familiar profile pictures. Is this possible in a way, or is my wish not possible?
query
db.getCollection('conversations').aggregate([
{
$lookup: {
foreignField: "c_ID",
from: "messages",
localField: "_id",
as: "messages"
}
},
{
"$unwind": "$messages"
},
{
"$sort": {
"messages.t": -1
}
},
{
"$group": {
"_id": "$_id",
"lastMessage": {
"$first": "$messages"
},
"allFields": {
"$first": "$$ROOT"
}
}
},
{
"$replaceRoot": {
"newRoot": {
"$mergeObjects": [
"$allFields",
{
"lastMessage": "$lastMessage"
}
]
}
}
},
{
$project: {
messages: 0
}
},
{
$match: {
"members.uID": "1",
//"lastMessage.t": { $gt: ISODate("2020-02-04 20:38:02.154Z") }
}
},
{
$sort: { "lastMessage.t": 1 }
},
{
$limit: 10
},
{
$project: {
members: {
$slice: [ {
$filter: {
input : "$members", as : "member", cond : {
$ne : ["$$member.uID" , "1"]
}
}
}, 3 ]
}
}
},
])
conversations document
{
"_id" : ObjectId("5e35f2c840713a43aeeeb3d9"),
"members" : [
{
"uID" : "1",
"j" : 1580580922
},
{
"uID" : "4",
"j" : 1580580922
},
{
"uID" : "5",
"j" : 1580580922
}
]
}
messages document
{
"_id" : ObjectId("5e35ee5f40713a43aeeeb1c5"),
"c_ID" : ObjectId("5e35f2c840713a43aeeeb3d9"),
"fromID" : "1",
"msg" : "What's up?",
"t" : 1580591922,
"d" : {
"4" : 1580592039
},
"r" : {
"4" : 1580592339
}
}
I building a standard free text search on a site that sells cars.
In the search box the user can enter a search word that are passed on to the query where it is used to match both nested and non-nested properties.
I'm using inner_hits to limit the number of variants returned by the query (in this sample variants is not remove from _source)
When matching on a nested property color the inner_hits collection contains the correct variant as expected.
However when matching on a non-nested property title the inner_hits collection is empty. I understand why it's empty.
Can you suggest a better way to structure the query?
Another option would be to always just return at least 1 variant - but how can the be achieved?
Mappings
PUT test
{
"mappings": {
"car": {
"properties": {
"variants": {
"type": "nested"
}
}
}
}
}
Insert data
PUT test/car/1
{
"title": "VW Golf",
"variants": [
{
"color": "red",
"forsale": true
},
{
"color": "blue",
"forsale": false
}
]
}
Query by color
GET test/_search
{
"query": {
"nested": {
"path": "variants",
"query": {
"match": {
"variants.color": "blue"
}
},
"inner_hits": {}
}
}
}
Color query: works as expected!
"hits" : [
{
"_source" : {
"title" : "VW Golf",
"variants" : [
{
"color" : "red",
"forsale" : true
},
{
"color" : "blue",
"forsale" : false
}
]
},
"inner_hits" : {
"variants" : {
"hits" : {
"total" : 1,
"hits" : [
{
"_nested" : {
"field" : "variants",
"offset" : 1
},
"_source" : {
"color" : "blue",
"forsale" : false
}
}
]
}
}
}
}
]
Query by brand
GET test/_search
{
"query": {
"bool": {
"should": [
{
"match": {
"title": "golf"
}
},
{
"nested": {
"path": "variants",
"query": {
"match": {
"variants.color": "golf"
}
},
"inner_hits": {}
}
}
]
}
}
}
Brand query result :-(
"hits" : [
{
"_source" : {
"title" : "VW Golf",
"variants" : [
{
"color" : "red",
"forsale" : true
},
{
"color" : "blue",
"forsale" : false
}
]
},
"inner_hits" : {
"variants" : {
"hits" : {
"total" : 0,
"hits" : [ ]
}
}
}
}
You already know it but inner_hits returns an empty array because no nested documents matched in the nested query.
A simple solution is to change the query such that the nested query will always match. This can be done by wrapping the nested query into a bool query and add a match_all query.
If you set the boost of the match_all query to 0, it will not contribute to the score. Consequently, if a nested document match it will be first.
Now the inner hits will not be empty, but there is a second problem, all the documents will match. You can either:
set a min_score with a very small value (e.g., 0.00000001) to discard document with a score of 0
duplicate the original nested query and use a minimum_should_match at 2.
{
"query": {
"bool": {
// Ensure that at least 1 of the first 2 queries will match
// The third query will always match
"minimum_should_match": 2,
"should": [
{
"match": {
"title": <SEARCH_TERM>
}
},
{
"nested": {
"path": "variants",
"query": {
"match": {
"variants.color": <SEARCH_TERM>
}
}
}
},
{
"nested": {
"path": "variants",
"query": {
"bool": {
"should": [
{
"match": {
"variants.color": <SEARCH_TERM>
}
},
{
// Disable scoring
"match_all": { "boost": 0 }
}
]
}
},
"inner_hits": {}
}
}
]
}
}
}
One way to do it is using a script_fields clause.
You would write a little script in painless that would do the following:
store the List you get from variants in a variable
then iterate over the Maps in this List
if the Map has
color blue you return the Map . (If none evaluate to true you return an empty
Map). This would create an additional field per searchresult with only those variants where the color is blue.
One important drawback is that this is a very heavy operation, especially if you have many records.
You can take this approach if it is something only you will ever do, maybe a few times a year outside peak hours. If your use case is something with regular use and to be performed by many users, I would change the mapping, return variants as a whole or choose some other solution.
Context: Querying logz.io through the API.
The following query returns results for yesterday
{
"size":10000,
"query" : {
"bool" : {
"must" : [
{ "range": { "#timestamp": { "gte": "now-1d/d", "lt": "now/d" } } }
]
}
}
}
I was hoping this one to return results for the day before yesterday
{
"size":10000,
"query" : {
"bool" : {
"must" : [
{ "range": { "#timestamp": { "gte": "now-2d/d", "lt": "now-1/d" } } }
]
}
}
}
But it doesn't return anything.
What am I missing?
You're just missing a d in the lt part
{
"size":10000,
"query" : {
"bool" : {
"must" : [
{ "range": { "#timestamp": { "gte": "now-2d/d", "lt": "now-1d/d" } } }
^
|
here
]
}
}
}
I want to convet the foll. sql query to elastic json query
select count(distinct(fk_id)),city_id from table
where status1 != "xyz" and satus2 = "abc" and
cr_date >="date1" and cr_date<="date2" group by city_id
Also is there any way of writing nested queries in elastic.
select * from table where status in (select status from table2)
The first query can be translated like this in the Elasticsearch query DSL:
curl -XPOST localhost:9200/table/_search -d '{
"size": 0,
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"status2": "abc"
}
},
{
"range": {
"cr_date": {
"gt": "date1", <--- don't forget to change the date
"lt": "date2" <--- don't forget to change the date
}
}
}
],
"must_not": [
{
"term": {
"status1": "xyz"
}
}
]
}
}
}
},
"aggs": {
"by_cities": {
"terms": {
"field": "city_id"
},
"aggs": {
"fk_count": {
"cardinality": {
"field": "fk_id"
}
}
}
}
}
}'
Using Sql API In Elastic search, we can write queries and also we can translate them to elastic query
POST /_sql/translate
{
"query": "SELECT * FROM customer where address.Street='JanaChaitanya Layout' and Name='Pavan Kumar'"
}
Response for this is
{
"size" : 1000,
"query" : {
"bool" : {
"must" : [
{
"term" : {
"address.Street.keyword" : {
"value" : "JanaChaitanya Layout",
"boost" : 1.0
}
}
},
{
"term" : {
"Name.keyword" : {
"value" : "Pavan Kumar",
"boost" : 1.0
}
}
}
],
"adjust_pure_negative" : true,
"boost" : 1.0
}
},
"_source" : {
"includes" : [
"Name",
"address.Area",
"address.Street"
],
"excludes" : [ ]
},
"docvalue_fields" : [
{
"field" : "Age"
}
],
"sort" : [
{
"_doc" : {
"order" : "asc"
}
}
]
}
Now we can use this result to query elastic search
For further details please go through this article
https://xyzcoder.github.io/elasticsearch/2019/06/25/making-use-of-sql-rest-api-in-elastic-search-to-write-queries-easily.html
I have a document indexed in my elastic search like:
{
...
purchase:{
zones: ["FR", "GB"]
...
}
...
}
I use this kind of query to find for example document with puchase's zone to GB
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"term": {
"purchase.zones": "GB"
}
}
}
}
}
But with it i get no results...
I would like to perform a query like in php in_array("GB", purchase.zones).
Any help would be very helpful.
If your "purchase" field is nested type then you have to use nested query to access the "zones".
{
"nested" : {
"path" : "obj1",
"score_mode" : "avg",
"query" : {
"bool" : {
"must" : [
{
"match" : {"obj1.name" : "blue"}
},
{
"range" : {"obj1.count" : {"gt" : 5}}
}
]
}
}
}
}
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-nested-query.html