I'm a novice Couchbase user and I have a bucket which I've created that contains documents which are actually arrays in the form of:
{
"key": [
{
"data1": "somedata1"
},
{
"data2": "somedata2"
}
]
}
I want to query these documents via N1QL statements and have yet to find a solution to how to do this properly. More specifically, I would like to select fields inside each sub-document that is in an array of a certain key. For example, I would like to access: key.[1].data2 or key.[0].data1
How should I do it?
Couchbase has some reserved keywords that need to be escaped. In this case, key needs to be escaped. For example, if you're querying against my_bucket, then
SELECT my_bucket.`key`[0].data1 FROM my_bucket;
should return somedata1
Related
If I have a table called configurations where rows contain a jsonb column called data with values similar to the following:
{
"US": {
"1234": {
"id": "ABCD"
}
},
"CA": {
"5678": {
"id": "WXYZ"
}
}
}
My hope is to be able to write a query akin to the following:
select * from configurations where data->'$.*.*.id' = 'WXYZ'
(Please note: I'm aware that the SQL above is not correct, treat it as pseudo.)
Questions:
What is the correct syntax to perform the query I've written above?
What type of index would I need to create to ensure I'm not scanning the entire table using any query from my previous question?
You can turn your pseudo code into real jsonpath code:
select * from configurations where data ## '$.*.*.id == "WXYZ"'
And this can use a default gin index on "data":
create index on configurations using gin (data);
I've got JSON file that looks like this
{
"alliance":{
"name_part_1":[
"Ab",
"Aen",
"Zancl"
],
"name_part_2":[
"aca",
"acia",
"ythrae",
"ytos"
],
"name_part_3":[
"Alliance",
"Bond"
]
}
}
I want to store it in dynamoDB.
The thing is that I want a generator that would take random elements from fields like name_part_1, name_part_2 and others (number of name_parts_x is unlimited and overalls number of items in each parts might be several hundreds) and join them to create a complete word. Like
name_part_1[1] + name_part_2[10] + name_part[3]
My question is that what format I should use to do that effectively? Or NoSQL shouldn't be used for that? Should I refactor JSON for something like
{
"name": "alliance",
"parts": [ "name_part_1", "name_part_2", "name_part_3" ],
"values": [
{ "name_part_1" : [ "Ab ... ] }, { "name_part_2": [ "aca" ... ] }
]
}
This is a perfect use case for DynamoDB.
You can structure like this,
NameParts (Table)
namepart (partition key)
range (hash key)
namepart: name_part_1 range: 0 value: "Ab"
This way each name_part will have its own range and scalable. You can extend it to thousands or even millions.
You can do a batch getitem from the sdk of your choice and join those values.
REST API reference:
https://docs.aws.amazon.com/amazondynamodb/latest/APIReference/API_BatchGetItem.html
Hope it helps.
You can just put the whole document as it is in DynamoDB and then use document path to access the elements you want.
Document Paths
In an expression, you use a document path to tell DynamoDB where to
find an attribute. For a top-level attribute, the document path is
simply the attribute name. For a nested attribute, you construct the
document path using dereference operators.
The following are some examples of document paths. (Refer to the item
shown in Specifying Item Attributes.)
A top-level scalar attribute: ProductDescription A top-level list
attribute. (This will return the entire list, not just some of the
elements.) RelatedItems The third element from the RelatedItems list.
(Remember that list elements are zero-based.) RelatedItems[2] The
front-view picture of the product. Pictures.FrontView All of the
five-star reviews. ProductReviews.FiveStar The first of the five-star
reviews. ProductReviews.FiveStar[0] Note The maximum depth for a
document path is 32. Therefore, the number of dereferences operators
in a path cannot exceed this limit.
Note that each document requires a unique Partition Key.
In a table items, I have a jsonb column called users. The JSON structure of users follows the following example:
[
{
"required": 1,
"agents": {
"user1": "A",
"user2": "P",
"user3": "A"
}
},
{
"required": 3,
"agents": {
"user1": "P",
"user4": "P",
"user5": "P"
}
}
]
Note that the table items has many fields, but for the sake of simplicity, we can consider that it has only an item_id and a users field. And all answers I saw here on SO provide queries for elements of objects directly inside an array.
I also wish I could rewrite the object's structure in a better way, but it's not my decision in this case :D.
I'm new to JSON queries in postgres, so I tried to write a few queries without success.
Question:
I'm trying to find a query, that can return all items that have a key 'user4' inside the agents sub-object of any element in the array. Any suggestions?
Use the function jsonb_array_elements() and the ? operator:
select i.*
from items i
cross join jsonb_array_elements(users)
where value->'agents' ? 'user4'
See JSON Functions and Operators.
I'm currently trying to do a bit of complex N1QL for a project I'm working on, theoretically I could do all of this processing in multiple N1QL calls and by parsing the results each time, however if possible I'd like for this to contained in one call.
What I would like to do is:
filter all documents that contain a "dataSync.test.id" field with more than 1 id
Read back all other ids in that list
Use that list to get other documents containing those ids
Get the "dataSync.test._channels" field for those documents (optionally a filter by docType might help parsing)
This would probably return a list of "dataSync.test._channels"
Is this possible in N1QL? It appears like it might be but I can't get the syntax right.
My data structures look a little like
{
"dataSync": {
"test": {
"_channels": [
"RP"
],
"id": [
"dataSync_user_1015",
"dataSync_user_1010",
"dataSync_user_1005"
],
"_lastUpdatedBy": "TEST"
}
},
...
}
{
"dataSync": {
"test": {
"_channels": [
"RSD"
],
"id": [
"dataSync_user_1010"
],
"_lastUpdatedBy": "TEST"
}
},
...
}
Yes. I think you can do all these.
Initial set of IDs with filtering can be retrieved as a subquery and then you can get subsquent documents by joins.
SELECT fulldoc
FROM (select meta().id as dockey from doc where a=1) as mydoc
INNER JOIN doc fulldoc ON KEYS mydoc.dockey;
There are optimizations that can be done here. Try the sequencing first to ensure you're get the job done.
I need a little help regarding lucene index files, thought, maybe some of you guys can help me out.
I have json like this:
[
{
"Id": 4476,
"UrlName": null,
"PhoneData": [
{
"PhoneType": "O",
"PhoneNumber": "0065898",
},
{
"PhoneType": "F",
"PhoneNumber": "0065898",
}
],
"Contact": [],
"Services": [
{
"ServiceId": 10,
"ServiceGroup": 2
},
{
"ServiceId": 20,
"ServiceGroup": 1
}
],
}
]
Adding first two fields is relatively easy:
// add lucene fields mapped to db fields
doc.Add(new Field("Id", sampleData.Id.Value.ToString(), Field.Store.YES, Field.Index.NOT_ANALYZED));
doc.Add(new Field("UrlName", sampleData.UrlName.Value ?? "null" , Field.Store.YES, Field.Index.ANALYZED));
But how I can add PhoneData and Services to index so it can be connected to unique Id??
For indexing JSON objects I would go this way:
Store the whole value under a payload field, named for example $json. This field would be stored but not indexed.
For each (indexable) property (maybe nested) create an indexable field with its name as a XMLPath-like expression identifying the property, for example PhoneData.PhoneType
If is ok that all nested properties will be indexed then it's simple, just iterate over all of them generating this indexable field.
But if you don't want to index all of them (a more realistic case), how to know which property is indexable is another problem; in this case you could:
Accept from the client the path expressions of the index fields to be created when storing the document, or
Put JSON Schema into play to describe your data (assuming your JSON records have a common schema), and extend it with a custom property that would allow you to tag which properties are indexable.
I have created a library doing this (and much more) that maybe can help you.
You can check it at https://github.com/brutusin/flea-db