I have many documents in a bucket as
{ prop1: value, prop2: value, prop3: value}
But some documents do not have the prop3 field.
How can I retrieve all documents that DO NOT have the prop3 field, using an n1ql query?
The IS MISSING keyword is your friend here:
SELECT * FROM bucket WHERE prop3 IS MISSING;
Related
I have a json array like
{
"response":[
{
"Rate Lock":"Yes",
"Loan Amount":"1M - 2M",
"Credit Score":"800",
"Pre-Approved":"Yes",
"Mortgage Type":"15 ARM",
"Property Type":"Commercial",
"Forecasted Close Date":"2018-07-12"
}
]
}
in a column called 'custom_response'
select * from custom_deals_response where json_contains(`custom_response`,'"2018-07-12"')
returns and empty set. What am I doing wrong?
JSON_CONTAINS has 3 parameters(targer column, candidate, path(optional)). In you data structure your targer search column location inside response objects. You have to set 3 params to specific path where mySql have to search. Documentation reference
JSON_CONTAINS(target, candidate[, path])
select * from custom_deals_response
where json_contains(`custom_response`,'{"Forecasted Close Date":"2018-07-12"}','$.response');
I have a column named params in a table named reports which contains JSON.
I need to find which rows contain the text 'authVar' anywhere in the JSON array. I don't know the path or level in which the text could appear.
I want to just search through the JSON with a standard like operator.
Something like:
SELECT * FROM reports
WHERE params LIKE '%authVar%'
I have searched and googled and read the Postgres docs. I don't understand the JSON data type very well, and figure I am missing something easy.
The JSON looks something like this.
[
{
"tileId":18811,
"Params":{
"data":[
{
"name":"Week Ending",
"color":"#27B5E1",
"report":"report1",
"locations":{
"c1":0,
"c2":0,
"r1":"authVar",
"r2":66
}
}
]
}
}
]
In Postgres 11 or earlier it is possible to recursively walk through an unknown json structure, but it would be rather complex and costly. I would propose the brute force method which should work well:
select *
from reports
where params::text like '%authVar%';
-- or
-- where params::text like '%"authVar"%';
-- if you are looking for the exact value
The query is very fast but may return unexpected extra rows in cases when the searched string is a part of one of the keys.
In Postgres 12+ the recursive searching in JSONB is pretty comfortable with the new feature of jsonpath.
Find a string value containing authVar:
select *
from reports
where jsonb_path_exists(params, '$.** ? (#.type() == "string" && # like_regex "authVar")')
The jsonpath:
$.** find any value at any level (recursive processing)
? where
#.type() == "string" value is string
&& and
# like_regex "authVar" value contains 'authVar'
Or find the exact value:
select *
from reports
where jsonb_path_exists(params, '$.** ? (# == "authVar")')
Read in the documentation:
The SQL/JSON Path Language
jsonpath Type
Consider following documents and consider that is created Full-text index over following documents:
{
email : "A",
"data" : {
"dynamic_property" : "ANY_TYPE",
"dynamic_property2" : {
"property" : "searchableValue"
},
"field" : "VALUE"
}
},
{
email : "B",
"data" : {
"other_dynamic_prop" : "test-searchableValue-2",
}
},
{
email : "A",
"data" : {
"thirdDynamicProp" : {
"childProp" : "this should be searchableValue!"
}
}
}
The goal: Create N1QL query which will match all the documents with have associated given email address AND the data property contains given substring.
Basically following:
SELECT * FROM `bucket` WHERE `email` = 'A' AND `data` LIKE '%searchableValue%';
The expected result is the first and second document because matching criteria. But the query does not work because data is not a text type but is object type. If the data property would be like:
{"data" : "this should be searchableValue!" }
The query would return expected result.
The question is:
How to create such a N1QL query which would return expected result?
I know that Couchbase is not able to do compare substring in the text, but using Full-text index it should be possible since Couchbase 4.5+
Couchbase4.6 and 5.0 have more/better options (explained below). In couchbase4.5, you can use Array Indexing to solve this:
https://developer.couchbase.com/documentation/server/4.5/n1ql/n1ql-language-reference/indexing-arrays.html
https://www.couchbase.com/blog/2016/october/n1ql-functionality-enhancements-in-couchbase-server-4.5.1
For instance, using the travel-sample sample bucket, following array index, and query would do the kind of substring search you want.
create index tmp_geo on `travel-sample`(DISTINCT ARRAY x FOR x IN object_values(geo) END) where type = "airport";
select meta().id, geo from `travel-sample` where type = "airport"
and ANY x IN object_values(geo) SATISFIES to_string(x) LIKE "12%" END;
N1QL introduced a function TOKENS() in 4.6, which can help you create functional index on tokenized sub-objects (instead of array index in the above example):
https://developer.couchbase.com/documentation/server/4.6/n1ql/n1ql-language-reference/string-functions.html
https://dzone.com/articles/more-than-like-efficient-json-search-with-couchbas
And, Couchbase 5.0 developer build (https://blog.couchbase.com/2017/january/introducing-developer-builds) has N1QL function CURL(), which allows you to access any HTTP/REST endpoint as part of the N1QL query (hence, can access the FTS endpoint). See following blogs for more details & examples:
- https://blog.couchbase.com/2017/january/developer-release--curl-n1ql
- https://dzone.com/articles/curl-comes-to-n1ql-querying-external-json-data
Btw, can you clarify if you want partial tokens or only full tokens in the query?
-Prasad
Here are the specific queries based on the answer from #prasad.
Using Couchbase 4.5:
CREATE INDEX idx_email ON `bucket`( email );
SELECT *
FROM `bucket`
WHERE
`email` = 'A'
AND ANY t WITHIN `data` SATISFIES t LIKE '%searchableValue%' END;
Using Couchbase 4.6:
CREATE INDEX idx_email ON `bucket`( email );
CREATE INDEX idx_tokens ON `bucket`( DISTINCT ARRAY t FOR t IN TOKENS( `data` ) END );
SELECT *
FROM `bucket`
WHERE
`email` = 'A'
AND ANY t IN TOKENS( `data` ) SATISFIES t = 'searchableValue' END;
{
"Actor": {
"knownlanguages": [
"English"
]
}
}
This JSON is stored in JSON columntype of MySQL with name data.
My question is how to check whether knownlanguages key contains value English from JSON datatype of MySQL using query?
Easy:
SELECT * FROM table WHERE JSON_CONTAINS(json, '"English"', "$.Actor.knownlanguages")
or (depending on what you have to do):
SELECT JSON_CONTAINS(json, '"English"', "$.Actor.knownlanguages") FROM table
reference: https://dev.mysql.com/doc/refman/5.7/en/json-search-functions.html
you can use like keyword with wildcard %
SELECT * FROM table where data like "%English%"
NOTE:
this won't search in JSON field knownlanguages, but in column data
I have this queries:
select otherDocKey from bucket use keys '1234'
update bucket use keys 'hear I need the result of the first query' set ...
I want to do something like that:
update bucket use keys (select otherDocKey from bucket use keys '1234') set kuku = 3
but the response I get is:
[
{
"code": 5030,
"msg": "Missing or invalid primary key map[otherDocKey:\"56443\"] of type map[string]interface {}."
}
]
is there a way to do that in one query?
I am using couchbase version 4.5
The problem with your query is that the nested subquery returns a json result. I.e., the query:
select otherDocKey from bucket use keys '1234'
will return a result that looks like:
{"otherDocKey":"This_is_the_key_for_the_other_doc"}
But you don't want json, you just want the value from the json. For that you need to use 'select raw'. E.g.,
select raw otherDocKey from bucket use keys '1234'
That should give you a result that looks like:
["This_is_the_key_for_the_other_doc"]
When the subquery returns that kind of result, the "use keys" should work properly.