Couchbase Full Text search with combination of dynamic fields and N1QL - couchbase

Question

Consider following documents and consider that is created Full-text index over following documents:
{
email : "A",
"data" : {
"dynamic_property" : "ANY_TYPE",
"dynamic_property2" : {
"property" : "searchableValue"
},
"field" : "VALUE"
}
},
{
email : "B",
"data" : {
"other_dynamic_prop" : "test-searchableValue-2",
}
},
{
email : "A",
"data" : {
"thirdDynamicProp" : {
"childProp" : "this should be searchableValue!"
}
}
}
The goal: Create N1QL query which will match all the documents with have associated given email address AND the data property contains given substring.
Basically following:
SELECT * FROM `bucket` WHERE `email` = 'A' AND `data` LIKE '%searchableValue%';
The expected result is the first and second document because matching criteria. But the query does not work because data is not a text type but is object type. If the data property would be like:
{"data" : "this should be searchableValue!" }
The query would return expected result.
The question is:
How to create such a N1QL query which would return expected result?
I know that Couchbase is not able to do compare substring in the text, but using Full-text index it should be possible since Couchbase 4.5+

Answer 1

Couchbase4.6 and 5.0 have more/better options (explained below). In couchbase4.5, you can use Array Indexing to solve this:
https://developer.couchbase.com/documentation/server/4.5/n1ql/n1ql-language-reference/indexing-arrays.html
https://www.couchbase.com/blog/2016/october/n1ql-functionality-enhancements-in-couchbase-server-4.5.1
For instance, using the travel-sample sample bucket, following array index, and query would do the kind of substring search you want.
create index tmp_geo on `travel-sample`(DISTINCT ARRAY x FOR x IN object_values(geo) END) where type = "airport";
select meta().id, geo from `travel-sample` where type = "airport"
and ANY x IN object_values(geo) SATISFIES to_string(x) LIKE "12%" END;
N1QL introduced a function TOKENS() in 4.6, which can help you create functional index on tokenized sub-objects (instead of array index in the above example):
https://developer.couchbase.com/documentation/server/4.6/n1ql/n1ql-language-reference/string-functions.html
https://dzone.com/articles/more-than-like-efficient-json-search-with-couchbas
And, Couchbase 5.0 developer build (https://blog.couchbase.com/2017/january/introducing-developer-builds) has N1QL function CURL(), which allows you to access any HTTP/REST endpoint as part of the N1QL query (hence, can access the FTS endpoint). See following blogs for more details & examples:
- https://blog.couchbase.com/2017/january/developer-release--curl-n1ql
- https://dzone.com/articles/curl-comes-to-n1ql-querying-external-json-data
Btw, can you clarify if you want partial tokens or only full tokens in the query?
-Prasad

Answer 2

Here are the specific queries based on the answer from #prasad.
Using Couchbase 4.5:
CREATE INDEX idx_email ON `bucket`( email );
SELECT *
FROM `bucket`
WHERE
`email` = 'A'
AND ANY t WITHIN `data` SATISFIES t LIKE '%searchableValue%' END;
Using Couchbase 4.6:
CREATE INDEX idx_email ON `bucket`( email );
CREATE INDEX idx_tokens ON `bucket`( DISTINCT ARRAY t FOR t IN TOKENS( `data` ) END );
SELECT *
FROM `bucket`
WHERE
`email` = 'A'
AND ANY t IN TOKENS( `data` ) SATISFIES t = 'searchableValue' END;