Couchbase Multiple Keys - couchbase

I presume a simple question. I have the following data.
I want to search for all rows where the ID is > 2 but < 8 and the Price is > 30
I have used various versions of: startkey=["2", null] or even something like startkey=["2", "30"] just for testing.
It only ever seems to run both conditions on the first row. So if I do: startkey=["2", "30"] then I get back:
{"id":"3","key":["3","30"],"value":null},
{"id":"4","key":["4","30"],"value":null},
{"id":"5","key":["5","20"],"value":null},
{"id":"6","key":["6","60"],"value":null},
{"id":"8","key":["8","60"],"value":null}
Why is row 5 there?
I am starting to get the view that I need to handle this in the code (.net) and make multiple calls somehow... I can't seem to find anything on this that works....
Note: I have tried doing say a loop with for (i = 0; i < doc.ID.length; i++) and then using doc.ID[i] but it never returns anything....
Currently I just have
function (doc, meta) {
emit([doc.ID, doc.Price ],null);
}
Essentially I want to have a search where there are 5 input keys that a user has. So do I need to make 5 calls and then keep taking data from the previous output as the source for the next???
Other references I have looked at include: the manual
Thanks in advance,
Kindest Regards
Robin

This is a common misconception, with a compound array index key, it's still treated as a string, therefore the index key [2,10] is actually "[2,10]", and the index key [5,20], is actually "[5,20]".
So the reason that startkey=["2", "30"]shows the {"id":"5","key":["5","20"],"value":null}, row is because as a string it is > startkey.
Likewise, the Query startkey=[2,10]&endkey=[5,10] returns
{"total_rows":7,"rows":[
{"id":"2","key":[2,20],"value":null},
{"id":"3","key":[3,30],"value":null},
{"id":"4","key":[4,30],"value":null}
]
}
because startkey="[2,10]" < "[2,20]" && "[4,30]" < "[5,10]"=endkey, but "[5,20]" is not within that string Range.
Range Queries with startkey and endkey
startkey => endkey is a Range query using strcmp(), the group and group level is based on the string, where the comma is separating string tokens.
A Good Reference Link (since Couchbase Views work much like Apache CouchDB Views (inspired by them))
http://wiki.apache.org/couchdb/View_collation#Collation_Specification
Spatial View/Query
To achieve the result you are trying for, you could also write a Spatial View to have multi-dimensional Queries, numeric only. While you might not initially think of it
function (doc, meta) {
emit({
type: "Point",
coordinates: [doc.ID, doc.Price]
}, meta.id);
}
The Query would be a Bounding Box Query:
&bbox=2,0,8,30
{"total_rows":0,"rows":[
{"id":"2","bbox":[2,20,2,20],"geometry":{"type":"Point","coordinates":[2,20]},"value":"2"},
{"id":"3","bbox":[3,30,3,30],"geometry":{"type":"Point","coordinates":[3,30]},"value":"3"},
{"id":"4","bbox":[4,30,4,30],"geometry":{"type":"Point","coordinates":[4,30]},"value":"4"},
{"id":"5","bbox":[5,20,5,20],"geometry":{"type":"Point","coordinates":[5,20]},"value":"5"}
]
}
Another Query:
&bbox=2,30,8,30
{"total_rows":0,"rows":[
{"id":"3","bbox":[3,30,3,30],"geometry":{"type":"Point","coordinates":[3,30]},"value":"3"},
{"id":"4","bbox":[4,30,4,30],"geometry":{"type":"Point","coordinates":[4,30]},"value":"4"}
]
}

Related

How can I query for multiple values after a wildcard?

I have a json object like so:
{
_id: "12345",
identifier: [
{
value: "1",
system: "system1",
text: "text!"
},
{
value: "2",
system: "system1"
}
]
}
How can I use the XDevAPI SearchConditionStr to look for the specific combination of value + system in the identifier array? Something like this, but this doesn't seem to work...
collection.find("'${identifier.value}' IN identifier[*].value && '${identifier.system} IN identifier[*].system")
By using the IN operator, what happens underneath the covers is basically a call to JSON_CONTAINS().
So, if you call:
collection.find(":v IN identifier[*].value && :s IN identifier[*].system")
.bind('v', '1')
.bind('s', 'system1')
.execute()
What gets executed, in the end, is (simplified):
JSON_CONTAINS('["1", "2"]', '"2"') AND JSON_CONTAINS('["system1", "system1"]', '"system1"')
In this case, both those conditions are true, and the document will be returned.
The atomic unit is the document (not a slice of that document). So, in your case, regardless of the value of value and/or system, you are still looking for the same document (the one whose _id is '12345'). Using such a statement, the document is either returned if all search values are part of it, and it is not returned if one is not.
For instance, the following would not yield any results:
collection.find(":v IN identifier[*].value && :s IN identifier[*].system")
.bind('v', '1')
.bind('s', 'system2')
.execute()
EDIT: Potential workaround
I don't think using the CRUD API will allow to perform this kind of "cherry-picking", but you can always use SQL. In that case, one strategy that comes to mind is to use JSON_SEARCH() for retrieving an array of paths corresponding to each value in the scope of identifier[*].value and identifier[*].system i.e. the array indexes and use JSON_OVERLAPS() to ensure they are equal.
session.sql(`select * from collection WHERE json_overlaps(json_search(json_extract(doc, '$.identifier[*].value'), 'all', ?), json_search(json_extract(doc, '$.identifier[*].system'), 'all', ?))`)
.bind('2', 'system1')
.execute()
In this case, the result set will only include documents where the identifier array contains at least one JSON object element where value is equal to '2' and system is equal to system1. The filter is effectively applied over individual array items and not in aggregate, like on a basic IN operation.
Disclaimer: I'm the lead developer of the MySQL X DevAPI Connector for Node.js

Using json_extract in sqlite to pull data from parent and child objects

I'm starting to explore the JSON1 library for sqlite and have been so far successful in the basic queries I've created. I'm now looking to create a more complicated query that pulls data from multiple levels.
Here's the example JSON object I'm starting with (and most of the data is very similar).
{
"height": 140.0,
"id": "cp",
"label": {
"bind": "cp_label"
},
"type": "color_picker",
"user_data": {
"my_property": 2
},
"uuid": "948cb959-74df-4af8-9e9c-c3cb53ac9915",
"value": {
"bind": "cp_color"
},
"width": 200.0
}
This json object is buried about seven levels deep in a json structure and I pulled it from the larger json construct using an sql statement like this:
SELECT value FROM forms, json_tree(forms.formJSON, '$.root')
WHERE type = 'object'
AND json_extract(value, '$.id') = #sControlID
// In this example, #sControlID is a variable that represents the `id` value we're looking for, which is 'cp'
But what I really need to pull from this object are the following:
the value from key type ("color_picker" in this example)
the values from keys bind ("cp_color" and "cp_label" in this example)
the keys value and label (which have values of {"bind":"<string>"} in this example)
For that last item, the key name (value and label in this case) can be any number of keywords, but no matter the keyword, the value will be an object of the form {"bind":"<some_string>"}. Also, there could be multiple keys that have a bind object associated with them, and I'd need to return all of them.
For the first two items, the keywords will always be type and bind.
With the json example above, I'd ideally like to retrieve two rows:
type key value
color_picker value cp_color
color_picker label cp_label
When I use json_extract methods, I end up retrieving the object {"bind":"cp_color"} from the json_tree table, but I also need to retrieve the data from the parent object. I feel like I need to do some kind of union, but my attempts have so far been unsuccessful. Any ideas here?
Note: if the {"bind":"<string>"} object doesn't exist as a child of the parent object, I don't want any rows returned.
Well, I was on the right track and eventually figured out it. I created a separate query for each of the items I was looking for, then INNER JOINed all the json_tree tables from each of the queries to have all the required fields available. Then I json_extracted the required data from each of the json fields I needed data from. In the end, it gave me exactly what I was looking for, though I'm sure it could be written more efficiently.
For anyone interested, this is what hte final query ended up looking like:
SELECT IFNULL(json_extract(parent.value, '$.type'), '_window_'), child.key, json_extract(child.value, '$.bind') FROM (SELECT json_tree.* FROM nui_forms, json_tree(nui_forms.formJSON, '$') WHERE type = 'object' AND json_extract(nui_forms.formJSON, '$.id') = #sWindowID) parent INNER JOIN (SELECT json_tree.* FROM nui_forms, json_tree(nui_forms.formJSON, '$') WHERE type = 'object' AND json_extract(value, '$.bind') != 'NULL' AND json_extract(nui_forms.formJSON, '$.id') = #sWindowID) child ON child.parent = parent.id;
If you have any tips on reducing its complexity, feel free to comment!

How do I search for a specific string in a JSON Postgres data type column?

I have a column named params in a table named reports which contains JSON.
I need to find which rows contain the text 'authVar' anywhere in the JSON array. I don't know the path or level in which the text could appear.
I want to just search through the JSON with a standard like operator.
Something like:
SELECT * FROM reports
WHERE params LIKE '%authVar%'
I have searched and googled and read the Postgres docs. I don't understand the JSON data type very well, and figure I am missing something easy.
The JSON looks something like this.
[
{
"tileId":18811,
"Params":{
"data":[
{
"name":"Week Ending",
"color":"#27B5E1",
"report":"report1",
"locations":{
"c1":0,
"c2":0,
"r1":"authVar",
"r2":66
}
}
]
}
}
]
In Postgres 11 or earlier it is possible to recursively walk through an unknown json structure, but it would be rather complex and costly. I would propose the brute force method which should work well:
select *
from reports
where params::text like '%authVar%';
-- or
-- where params::text like '%"authVar"%';
-- if you are looking for the exact value
The query is very fast but may return unexpected extra rows in cases when the searched string is a part of one of the keys.
In Postgres 12+ the recursive searching in JSONB is pretty comfortable with the new feature of jsonpath.
Find a string value containing authVar:
select *
from reports
where jsonb_path_exists(params, '$.** ? (#.type() == "string" && # like_regex "authVar")')
The jsonpath:
$.** find any value at any level (recursive processing)
? where
#.type() == "string" value is string
&& and
# like_regex "authVar" value contains 'authVar'
Or find the exact value:
select *
from reports
where jsonb_path_exists(params, '$.** ? (# == "authVar")')
Read in the documentation:
The SQL/JSON Path Language
jsonpath Type

How should I search through this json structure?

I'm working with PHP, I have a json structure which looks like that :
{
"events": [
{
"timestamp": 1468774519,
"id": 75964,
},
{
"timestamp": 1468771410,
"id": 24891,
},
// etc
I need to fetch 5 events in a row, but starting from one specific id, so my first idea is to loop every event from the beginning and check if the id is the offset i'm looking for, and then when i get it i can loop the next 5 events.
But is there a better way to do so ? It could possibly loop through hundreds of events so maybe there's a better way to get there ? thanks
Since the ids aren't in numeric order, you can't use a binary search, so you need to use a sequential search. Here's an example in JavaScript. Also note this code assumes the id is present and there are at least four more events after it in the array.
var index = 0;
var id = 12345; // for example
var json = {...}; // whatever that object was
while(json.events[index].id!=id) {
index++;
}
// found the one, do something with the next five
for(var i=0; i<5; i++) {
var event = json.events[index+i];
// do something
}
In my opinion, you can only take one loop over the events with a filter event.id >= theId,and then check if the filtered array contains theId. if you get it, you can sort this smaller array and take the 5 events.
First I would make a key:value hash object (a lookup object), where the key would be the id from your structure, and the value would be the reference to the event. As a result, you iterate over the structure only once and then get all the events from the lookup structure, by just accessing them by their keys.
You could sort it as well(ideally, you could get it already sorted by id from your source of data) and then use a binary search algorithm.

Append value to JSON decode array parameter stored in MySQL

Im trying to work out how to append a zero to a specific JSON decoded array value for multiple records stored in a MySQL table according to some conditions.
for example, for table 'menu', column 'params'(text) have records containing JSON decoded arrays of this format:
{"categories":["190"],"singleCatOrdering":"","menu-anchor_title":""}
and column 'id' has a numeric value of 90.
my goal is to add a zero to 'categories' value in menu.params whenever (for example) menu.id is under 100.
for this records the result being
{"categories":["1900"],"singleCatOrdering":"","menu-anchor_title":""}
so im looking for a SQL Query that will search and find the occurrences of "categories": ["999"] in the Database and update the record by adding a zero to the end of the value.
this answer is partially helpful by offering to use mysql-udf-regexp but its referring to REPLACE a value and not UPDATE it.
perhaps the REGEXP_REPLACE? function will do the trick. i have never used this library and am not familiar with it, perhaps there is an easier way to achieve what i need ?
Thanks
If I understand your question correctly, you want code that does something like this:
var data = {
"menu": {
"id": 90,
"params": {
"categories": ["190"],
"singleCatOrdering": "",
"menu-anchor_title": ""
}
}
};
var keys = Object.keys(data);
var columns;
for (var ii = 0, key; key = keys[ii]; ii++) {
value = data[key];
if (value.id < 100) {
value.params.categories[0] += "0";
alert(value.params.categories[0]);
}
}
jsFiddle
However, I am not using a regular expression at all. Perhaps if you reword the question, the necessity of a regex will become clearer.