Couchbase N1QL query to get id of documents connected as a linked list - couchbase

I have collection that contain documents with the following structure
{
id: "doc01",
property1: "",
nextCollection: "doc02"
prevCollection: null
},
{
id: "doc02",
property1: "",
nextCollection: "doc03",
prevCollection: "doc01"
},
{
id: "doc03",
property1: "",
nextCollection: null,
prevCollection: "doc02"
},
The above code contains three documents. What I want to do is to write a N1QL query that gets all documents in the list given doc01 id.
That is I want the return type to be
[
{
id: "doc01",
property1: "",
nextCollection: "doc02"
prevCollection: null
},
{
id: "doc02",
property1: "",
nextCollection: "doc03",
prevCollection: "doc01"
},
{
id: "doc03",
property1: "",
nextCollection: null,
prevCollection: "doc02"
}
]
How to write such query?

N1QL doesn't support hierarchical queries or recursive CTE.
Instead of single N1QL statement you can achieve this with the following options
Using N1QL UDF checkout https://www.couchbase.com/blog/traverse-hierarchy-user-defined-functions-in-7-1/
From application using SDK (As you know document keys you can use collection.get() avoid N1QL all together). You can even stop when recursive list detected circular and if list is huge control how many you want in the list.
Get the first document and add to the list
In loop until nextDocument is known value
get the document of key nextDocument
add to the list

Related

How do I properly use deleteMany() with an $and query in the MongoDB shell?

I am trying to delete all documents in my collection infrastructure that have a type.primary property of "pipelines" and a type.secondary property of "oil."
I'm trying to use the following query:
db.infrastructure.deleteMany({$and: [{"properties.type.primary": "pipelines"}, {"properties.type.secondary": "oil"}] }),
That returns: { acknowledged: true, deletedCount: 0 }
I expect my query to work because in MongoDB Compass, I can retrieve 182 documents that match the query {$and: [{"properties.type.primary": "pipelines"}, {"properties.type.secondary": "oil"}] }
My documents appear with the following structure (relevant section only):
properties": {
"optional": {
"description": ""
},
"original": {
"Opername": "ENBRIDGE",
"Pipename": "Lakehead",
"Shape_Leng": 604328.294581,
"Source": "EIA"
},
"required": {
"unit": null,
"viz_dim": null,
"years": []
},
"type": {
"primary": "pipelines",
"secondary": "oil"
}
...
My understanding is that I just need to pass a filter to deleteMany() and that $and expects an array of objects. For some reason the two combined isn't working here.
I realized the simplest answer was the correct one -- I spelled my database name incorrectly.

Solr grouped query pagination not working properly. [Solr, Lucene]

I have grouped my solr documents by a field family.
the solr query for getting first 20 groups is as follows
/select?q=*:*&group=true&group.field=family&group.ngroups=true&start=0&group.limit=1
Result of this query is 20 groups as following
responseHeader: {
zkConnected: true,
status: 0,
QTime: 1260,
params: {
q: "*:*",
group.limit: "1",
start: "0",
group.ngroups: "true",
group.field: "family",
group: "true"
}
},
grouped: {
family: {
matches: 464779,
ngroups: 396324,
groups: [
{
groupValue: "__fam__ME.EA.HE.728928",
doclist: {
numFound: 1,
start: 0,
maxScore: 1,
docs: [
{
sku: "ME.EA.HE.728928",
title: "Rexton Pocket Family Hearing Instrument Fusion",
family: "__fam__ME.EA.HE.728928",
brand: "Rexton",
brandId: "6739",
inStock: false,
bulkDiscount: false,
quoteOnly: false,
cats: [
"Hearing Machine & Components",
"Health & Personal Care",
"Medical Supplies & Equipment"
],
leafCatIds: [
"6038"
],
parentCatIds: [
"6259",
"4913"
],
Type__attr__: "Pocket Family",
Type of Products__attr__: "Hearing Instrument",
price: 3790,
discount: 40,
createdAt: "2016-02-18T04:51:36Z",
moq: 1,
offerPrice: 2255,
suggestKeywords: [
"Rexton",
"Pocket Family",
"Rexton Pocket Family"
],
suggestPayload: "6038,Hearing Machine & Components",
_version_: 1548082328946868200
}
]
}
},
Just the thing to notice in this result is the value of ngroups which is 396324
But when i want to get data of last pages i would hit this query on Solr
select?q=*:*&group=true&group.field=family&group.ngroups=true&start=396320&group.limit=1
{
responseHeader: {
zkConnected: true,
status: 0,
QTime: 5238,
params: {
q: "*:*",
group.limit: "1",
start: "396320",
group.ngroups: "true",
group.field: "family",
group: "true"
}
},
grouped: {
family: {
matches: 464779,
ngroups: 396324,
groups: [ ]
}
}
}
0 results when i set start to 396320. There must be 5 documents in the result. The actual number of groups are 386887. Why is ngroups incorrect?
btw this issue is not present in my local solr server i have setup up. just shows up in solr cloud on the test env
This is a known issue with grouping across distributed nodes (which is what happens in SolrCloud mode):
Grouping is supported for distributed searches, with some caveats:
Currently group.func is is not supported in any distributed searches
group.ngroups and group.facet require that all documents in each group must be co-located on the same shard in order for accurate counts to be returned. Document routing via composite keys can be a useful solution in many situations.
The most direct solution is to use the family as a part of the routing key, ensuring that all identical family values will end up on the same node. As it seems that the number of distinct family values are very high compared to the number of nodes, this should still ensure that you have a good distribution of documents across nodes.
Depending on what you're actually trying to do, there might be other alternative solutions as well (if you just want a count, using a JSON facet might be a good solution).

Solr Query Syntax Access Field in Object

I'm looking at Data.gov which implements solr queries. When I use a test search:
http://catalog.data.gov/api/3/action/package_search?fq=crime
I see that in the resources array there are fields within objects:
resources: [
{
resource_group_id: "8803cac0-41c0-4df9-9c1a-b6bf1be234d6",
cache_last_updated: null,
revision_timestamp: "2015-04-16T07:26:28.750291",
webstore_last_updated: null,
id: "4c948605-0db4-45e8-9c06-c851f05f9f37",
size: null,
state: "active",
hash: "",
description: "",
format: "CSV",
tracking_summary: {
total: 7893,
recent: 294
},
mimetype_inner: null,
revision_id: "baaab1e5-d3da-493c-9459-13a250a8c8fc",
mimetype: "text/csv",
cache_url: null,
name: "Comma Separated Values File",
created: "2015-02-04T14:39:09.238366",
url: "https://data.cityofchicago.org/api/views/ijzp-q8t2/rows.csv?accessType=DOWNLOAD",
webstore_url: null,
last_modified: null,
position: 0,
no_real_name: true,
resource_type: null
},
Is there a way to do filter based upon a field in that object returned? For instance, say I want to format only objects with the format "CSV" to return. How would I build that query?
I have tried to do something like these to no avail:
fq=filter:CSV
fq=filter:'CSV'
fq=resources.filter:'CSV'
fq=resources[* TO *].filter:'CSV'
fq=resources{filter:'CSV'}
I've been trying to find the correct pattern in examples but am having problems.

Json array in mongoDB

I want to get objects according to an ID they have in an array in a json file in mongodb.
I tried a lot of ways to get them with no success:
db.collection.find({"Id":"2"})
db.collection.find({"Messages.Id":"2"})
db.collection.find({"Messages":{$elemMatch:{"Id":"2"}}})
db.collection.find({"Messages.Id":{$elemMatch:{"Id":"2"}}})
{
"Messages" : [
{
"text":"aaa",
"Id" : [ "1", "2" ]
},
{
"texts" : "bbb",
"Id" : [ "1", "3" ]
}
]
}
Even though that's how it's supposed to be done according to the mongodb documentation.
So I thought something was wrong with my json design (I tried changing it but that didn't help either).
Can anyone suggest to me a good design or query to get the objects with a certain id will work?
UPDATE:
I want for example that if in the query i request the id 2
only the first message and all of it will be displayed (I don't mind if the Id field wont be displayed)
{
"text":"aaa",
"Id":["1","2"]
}
To find single elements that match you will need to utilize the positional operator ($).
db.collection.find({"Messages.Id": "2"}, {"Messages.$": 1, _id: 0})
For finding multiple matches, you would use the aggregation pipeline:
db.collection.aggregate([
{ $unwind: "$Messages" },
{ $match: {"Messages.Id": "1"}},
{ $group: { _id: null, messages: { $push: "$Messages"}}}
])

How to load multidimensional / nested json into a store?

So I'm trying to load the data received from a webservice into a sencha touch 2 store.
The data is nested JSON, however it is made to include multiple dataArrays.
I am working with sencha touch 2.3.1, somewhat equal to Ext JS 4.2. I don't have that much experience with sencha yet, but I'm getting there. I decided to go for MVC, so I'd like the answers to be as close to this as possible :).
This is the example JSON I am using:
[
{
"DataCollection": {
"DataArrayOne": [
{
"Name": "John Smith",
"Age": "19"
},
{
"Name": "Bart Smith",
"Age": "16"
}
],
"DataArrayTwo": [
{
"Date": "20110601",
"Product": "Apple",
"Descr": "",
"Remark": ""
},
{
"Date": "20110601",
"Product": "Orange",
"Descr": "",
"Remark": ""
},
{
"Date": "20110601",
"Product": "Pear",
"Descr": "",
"Remark": ""
}
],
"DataArrayThree": [
{
"SomeTotalCost": "400,50",
"IntrestPercentage": "3"
}
]
}
}
]
Through only one call, I get this json. I don't want to cause any unnecessary traffic so I hope to be able to use the data somehow.
I want to be able to use each DataArray on its own.
The data gets sent to the store through its proxy:
Ext.define("MyApp.store.myDataObjects", {
extend: "Ext.data.Store",
config: {
model: "MyApp.model.myDataObject",
proxy: {
reader: {
type: "json",
rootProperty: "DataCollection"
},
type: "ajax",
api: {
read: "https://localhost/Service.svc/json"
},
limitParam: false,
startParam: false,
pageParam: false,
extraParams: {
id: "",
token: "",
filter: ""
},
writer: {
encodeRequest: true,
type: "json"
}
}
}
});
I am a bit stuck with the model here. I tried using mappings which would look like this:
config: {
fields: [ {
name: "IntrestPercentage",
mapping: "Calculation.IntrestPercentage",
type: "string"
}
]}
I tried associations as well but to no avail.
According to google chrome console, it doesn't make any objects containing data. I get only 1 object with all values "null".
My endgoal is to be able to show each dataArray in a separate table. So a table for DataArrayOne, a table for DatarrayTwo... The data itself isn't linked. They are only details that have to be shown on a view.
John Smith isn't related to the apples, as in he didn't buy. The apples are just there as an item to be shown.
The possible solutions I've seen yet not understood due to them being outdated are:
ChildStores: You have a master store that receives the data, and then
you split the data to other stores according to rootProperty. I have
no idea how to do this however and I'm not sure if it will work at
all.
Associations, in case I was doing them wrong. I don't think they
are needed because the data isn't linked to each other but it is part
of "DataCollection" though.
Could someone please post an example on how to deal with this unusual(?) kind of nested json.
Or any other solution which will lead to being able to use the 3 dataArrays at will.
Thanks in advance
The best would be to load the complete data with a separate Ext.Ajax.request and then use store.loadData in the success callback. For example:
var data = Ext.decode(response.responseText);
store1.loadData(data[0].DataCollection.DataArrayOne);
store2.loadData(data[0].DataCollection.DataArrayTwo);
store3.loadData(data[0].DataCollection.DataArrayThree);