I have some documents in my index and I wanna get notified if in any document a specific field is set to false.
Can I do this whitin Kibana alerts, or am I just better off running an exists/boolean query to check for what I need that runs everyday?
I have this query:
GET my-index/_count
{
"query": {
"bool": {
"must": [
{
"term": {
"product.is_visible": {
"value": "false"
}
}
}
]
}
}
}
I want to get notified if this query ever returns more than 0 rows. What would be a good solution?
Using Kibana is a good solution, especially since you can use the Elasticsearch query rule.
Then to receive notifications, you can add an action (e.g. send to email) to run when a rule condition is met (e.g. query hits > 0 (threshold value)).
Related
I am syncing a data set with about 300k rows to Phonograph2 and need to make those records available via REST (End Point: /phonograph2/api/search/tables).
My requests looks as following (retrieving records after a certain timestamp):
{
"tableRids": [
"ri.phonograph2.main.table.xxxxxxxxxxxxxxx"
],
"filter": {
"type" : "range",
"range": {
"field": "reco_timestamp",
"gte": "1634408219000"
}
}
}
}
The response ends with:
"nextPageToken": "xxxxxxx"
This leads me to the following questions:
How do I use the "nextPageToken" to retrieve the next set of results?
Can the consumer get a list/array of pages to consume?
Can the number of hits which are displayed until the nextPageToken is written be configured?
We discussed this with our Palantir project support and will use Objects Gateway - as suggested. Thanks for pointing us into this direction.
What is the better JSON representation for GET ALL REST API response? With Option 1, it is easier to get the details of any particular employee. With Option 2, we need to filter the complete array to get any particular employee details. So, Option 1 seems to be better option. Are there any other scenarios where Option 2 can be better?
Option 1:
{
"data":{
"1234":{
"name":"ABC",
"country":"US",
"mobile":"999999999"
},
"1235":{
"name":"BDE",
"country":"IND",
"mobile":"999999998"
}
}
}
Option 2:
{
"data":[
{
"empId":"1234",
"name":"ABC",
"country":"US",
"mobile":"999999999"
},
{
"empId":"1235",
"name":"BDE",
"country":"IND",
"mobile":"999999998"
}
]
}
For option 1, since data is a JSON object, the order is not preserved and you cannot iterate over it in order. This may or may not be an issue depending on your use case. If this data is being consumed by a front-end app and you want to display the employees in order, then option 2 would be more suitable. If ordering doesn't matter, go with option 1.
I'm trying to write a python class that takes a properly formatted Elasticsearch JSON query as a parameter. So far so good.. however, as part of that class I'd like to also take a "to" and "from" parameter to limit the date range that the query runs over.
Is there a way to combine the JSON for the DSL query along with URI parameters to pass in the date and time constraint?
I know I can limit the time using a range parameter like this:
GET /my-awesome-index*/_search
{
"query":
{
"bool":
{
"must": [{"match_all": {}}],
"filter":
[
{"range": {"date_time": {"gte": "now-24h","lte": "now"}}},
{"match_phrase": {"super_cool_field": "foo"}},
{
"bool":
{
"should":
[
{"match_phrase": {"somewhat_cool_field_1": "bar"}},
{"match_phrase": {"somewhat_cool_field_2": "boo-ta"}}
],
"minimum_should_match": 1
}
}
]
}
}
}
and that's all well and good.. but, I want to craft my class to make the timeframe a variable. I also know I can limit the timeframe by submitting the URL like this..
GET /my-awesome-index*/_search?q=date_time:[1611732033412+TO+1611777796000]
{
"query":
{
"bool":
{
"must": [{"match_all": {}}],
"filter":
[
{"match_phrase": {"super_cool_field": "foo"}},
{
"bool":
{
"should":
[
{"match_phrase": {"somewhat_cool_field_1": "bar"}},
{"match_phrase": {"somewhat_cool_field_2": "boo-ta"}}
],
"minimum_should_match": 1
}
}
]
}
}
}
However, when I submit that Elasticsearch only seems to consider the timeframe from the URI and ignores the DSL JSON entirely.
Is there a way to get Elastic to consider / concatenate the two queries into one?
I'm considering programmatically making the range query something like this
range_part = '{{"range":{{"{}":{{"gte":"{}","lte":"{}"}}}}}}'.format(field,start,end)
And then dynamically inserting into any JSON the class takes.. but that seems cumbersome since there are so many formats available for the query and finding where to put the string etc..
Your suspicion regarding q is right. Quoting from the docs:
The q parameter overrides the query parameter in the request body. If both parameters are specified, documents matching the query request body parameter are not returned.
So q and query cannot be combined.
As to "programatically making the range query" -- since you don't know in what format you'll receive the queries, the standard approach would be to traverse the query json, find/create the correct bool-must/bool-filter and set the range query there.
But let's take a step back -- maybe your class shouldn't be expecting a pre-baked JSON query in the first place. Why not pass a query config array like
[
{
"type": "match_phrase",
"field": "super_cool_field",
"value": "foo"
},
...
]
and build the query yourself? That way you have full control over what gets passed downstream to ES. Plus, adding a date range query would be piece of cake...
I am trying to create a view to group on a particular attribute inside an array. However, the below map function is not returning any result.
JSON Document Structure :
{
"jsontype": "survey_response",
"jsoninstance": "xyz",
"jsonlanguage": "en_us",
"jsonuser": "test#test.com",
"jsoncontact": "test#mayoclinic.com",
"pages": [
{
q-placeholder": "q1-p1",
q:localized": "q1-localized-p1",
q-answer-placeholder": "jawaabu121",
q-answer-localized": "localized jawaabu1"
},
{
q-placeholder": "q2-p2",
q:localized": "q2-localized-p2",
q-answer-placeholder": "jawaabu221",
q-answer-localized": "localized jawaabu2"
},
{
"q-placeholder": "q3-p3",
"q:localized": "q3-localized-p3",
"q-answer-placeholder": "jawaabu313",
"q-answer-localized": "localized jawaabu3"
}
]
}
Map Function :
function(doc, meta){
emit(doc.jsoninstance,[doc.pages[0].q-placeholder, doc.pages[0].q-localized,doc.pages[0].q-answer-placeholder,q-answer-localized]);
}
It looks like you made a typo at the end of your emit statement:
doc.pages[0].q-answer-placeholder,q-answer-localized.
Instead q-answer-localized should be changed to doc.pages[0].q-answer-localized.
Further to this it seems that you have defined a field as q-localized in your emit statement, but actually according to the sample document that you posted this should actually be q:localized, I assume that this was a mistake in the snippet of the document and not the emit statement, but if not then will also need amending.
I would imagine errors like this would be flagged up in the view engine's map-reduce error log, in future you should check this log so that you will be able to debug errors like this yourself.
The location of the mapreduce_errors log can be found in the Couchbase documentation
I'm going to implement a groupchat into my app by using Firebase. I were thinking of 2 different structures of saving data in JSON.
First structure:
Second structure:
I would like to achieve a fast query and i would like it to parse a small amount of data each time. What structure should i go for and are there maybe a better alternative then these 2?
The first solution is clearly not viable, as you would have a hard time finding all messages belonging to a given group.
The second solution is ok if every time you will query a given group's node, you are going to need also all of its messages, which is probably not what you want.
It is hard, of course, to advise on data structure without more info on your use-case, the queries you are going to make, etc., but a rather standard approach would be:
{
"users": {
"$userId"": {
// user data
}
},
"groups": {
"$groupId": {
// group data
}
},
"group_users": {
"$groupId": {
"$userId": true
// separation of list of users from the group is useful
// if you are going to query the group node not needing its full list of users
}
},
"group_messages": {
"$groupId": {
"$messageId": {
// message data
}
}
}
}