My http request will return below data:
It returns below data:
Users.js
{
{
...
friends:[]
},
{
...
friends:[{id:xxx,...},...]
},
{
...
friends:[]
},
}
If I want to use query to get all data whose friends array is [],
should I do below query.
select * from users where (what should I write here)
If friends is a direct column in your database is JSON array. You can use JSON_LENGTH to find out the length of array.
SELECT JSON_LENGTH('[1, 2, {"a": 3}]'); // Output: 3
SELECT JSON_LENGTH('[]'); // Output: 0
You can use same concept to get data from your database.
select *
FROM users
WHERE JSON_LENGTH(friends) = 0;
If you've nested JSON and one of key is friends in that json for given column(data) then your query would be like using JSON_CONTAINS
SELECT *
FROM users
WHERE JSON_CONTAINS(data, JSON_ARRAY(), '$.friends') -- To check do we have `friends` as key in that json
and JSON_LENGTH(data, '$.friends') = 0; -- To check whether it is empty array.
Now you can convert it to sequelize query. One of the way you can use is
Model.findAll({
where: {
[Op.and]: [
Sequelize.literal('RAW SQL STATEMENT WHICH WONT BE ESCAPED!!!')
]
}
})
Make sure to update Model with your user model and query.
{
"movies": {
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson"
},
"movie2": {
"genre": "Horror",
"name": "The Shining",
"lead": "Jack Nicholson"
},
"movie3": {
"genre": "comedy",
"name": "The Mask",
"lead": "Jim Carrey"
}
}
}
I am a Firebase newbie. How can I retrieve a result from the data above where genre = 'comedy' AND lead = 'Jack Nicholson'?
What options do I have?
Using Firebase's Query API, you might be tempted to try this:
// !!! THIS WILL NOT WORK !!!
ref
.orderBy('genre')
.startAt('comedy').endAt('comedy')
.orderBy('lead') // !!! THIS LINE WILL RAISE AN ERROR !!!
.startAt('Jack Nicholson').endAt('Jack Nicholson')
.on('value', function(snapshot) {
console.log(snapshot.val());
});
But as #RobDiMarco from Firebase says in the comments:
multiple orderBy() calls will throw an error
So my code above will not work.
I know of three approaches that will work.
1. filter most on the server, do the rest on the client
What you can do is execute one orderBy().startAt()./endAt() on the server, pull down the remaining data and filter that in JavaScript code on your client.
ref
.orderBy('genre')
.equalTo('comedy')
.on('child_added', function(snapshot) {
var movie = snapshot.val();
if (movie.lead == 'Jack Nicholson') {
console.log(movie);
}
});
2. add a property that combines the values that you want to filter on
If that isn't good enough, you should consider modifying/expanding your data to allow your use-case. For example: you could stuff genre+lead into a single property that you just use for this filter.
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson",
"genre_lead": "comedy_Jack Nicholson"
}, //...
You're essentially building your own multi-column index that way and can query it with:
ref
.orderBy('genre_lead')
.equalTo('comedy_Jack Nicholson')
.on('child_added', function(snapshot) {
var movie = snapshot.val();
console.log(movie);
});
David East has written a library called QueryBase that helps with generating such properties.
You could even do relative/range queries, let's say that you want to allow querying movies by category and year. You'd use this data structure:
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson",
"genre_year": "comedy_1997"
}, //...
And then query for comedies of the 90s with:
ref
.orderBy('genre_year')
.startAt('comedy_1990')
.endAt('comedy_2000')
.on('child_added', function(snapshot) {
var movie = snapshot.val();
console.log(movie);
});
If you need to filter on more than just the year, make sure to add the other date parts in descending order, e.g. "comedy_1997-12-25". This way the lexicographical ordering that Firebase does on string values will be the same as the chronological ordering.
This combining of values in a property can work with more than two values, but you can only do a range filter on the last value in the composite property.
A very special variant of this is implemented by the GeoFire library for Firebase. This library combines the latitude and longitude of a location into a so-called Geohash, which can then be used to do realtime range queries on Firebase.
3. create a custom index programmatically
Yet another alternative is to do what we've all done before this new Query API was added: create an index in a different node:
"movies"
// the same structure you have today
"by_genre"
"comedy"
"by_lead"
"Jack Nicholson"
"movie1"
"Jim Carrey"
"movie3"
"Horror"
"by_lead"
"Jack Nicholson"
"movie2"
There are probably more approaches. For example, this answer highlights an alternative tree-shaped custom index: https://stackoverflow.com/a/34105063
If none of these options work for you, but you still want to store your data in Firebase, you can also consider using its Cloud Firestore database.
Cloud Firestore can handle multiple equality filters in a single query, but only one range filter. Under the hood it essentially uses the same query model, but it's like it auto-generates the composite properties for you. See Firestore's documentation on compound queries.
I've written a personal library that allows you to order by multiple values, with all the ordering done on the server.
Meet Querybase!
Querybase takes in a Firebase Database Reference and an array of fields you wish to index on. When you create new records it will automatically handle the generation of keys that allow for multiple querying. The caveat is that it only supports straight equivalence (no less than or greater than).
const databaseRef = firebase.database().ref().child('people');
const querybaseRef = querybase.ref(databaseRef, ['name', 'age', 'location']);
// Automatically handles composite keys
querybaseRef.push({
name: 'David',
age: 27,
location: 'SF'
});
// Find records by multiple fields
// returns a Firebase Database ref
const queriedDbRef = querybaseRef
.where({
name: 'David',
age: 27
});
// Listen for realtime updates
queriedDbRef.on('value', snap => console.log(snap));
var ref = new Firebase('https://your.firebaseio.com/');
Query query = ref.orderByChild('genre').equalTo('comedy');
query.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
for (DataSnapshot movieSnapshot : dataSnapshot.getChildren()) {
Movie movie = dataSnapshot.getValue(Movie.class);
if (movie.getLead().equals('Jack Nicholson')) {
console.log(movieSnapshot.getKey());
}
}
}
#Override
public void onCancelled(FirebaseError firebaseError) {
}
});
Frank's answer is good but Firestore introduced array-contains recently that makes it easier to do AND queries.
You can create a filters field to add you filters. You can add as many values as you need. For example to filter by comedy and Jack Nicholson you can add the value comedy_Jack Nicholson but if you also you want to by comedy and 2014 you can add the value comedy_2014 without creating more fields.
{
"movies": {
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson",
"year": 2014,
"filters": [
"comedy_Jack Nicholson",
"comedy_2014"
]
}
}
}
For Cloud Firestore
https://firebase.google.com/docs/firestore/query-data/queries#compound_queries
Compound queries
You can chain multiple equality operators (== or array-contains) methods to create more specific queries (logical AND). However, you must create a composite index to combine equality operators with the inequality operators, <, <=, >, and !=.
citiesRef.where('state', '==', 'CO').where('name', '==', 'Denver');
citiesRef.where('state', '==', 'CA').where('population', '<', 1000000);
You can perform range (<, <=, >, >=) or not equals (!=) comparisons only on a single field, and you can include at most one array-contains or array-contains-any clause in a compound query:
Firebase doesn't allow querying with multiple conditions.
However, I did find a way around for this:
We need to download the initial filtered data from the database and store it in an array list.
Query query = databaseReference.orderByChild("genre").equalTo("comedy");
databaseReference.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
ArrayList<Movie> movies = new ArrayList<>();
for (DataSnapshot dataSnapshot1 : dataSnapshot.getChildren()) {
String lead = dataSnapshot1.child("lead").getValue(String.class);
String genre = dataSnapshot1.child("genre").getValue(String.class);
movie = new Movie(lead, genre);
movies.add(movie);
}
filterResults(movies, "Jack Nicholson");
}
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
Once we obtain the initial filtered data from the database, we need to do further filter in our backend.
public void filterResults(final List<Movie> list, final String genre) {
List<Movie> movies = new ArrayList<>();
movies = list.stream().filter(o -> o.getLead().equals(genre)).collect(Collectors.toList());
System.out.println(movies);
employees.forEach(movie -> System.out.println(movie.getFirstName()));
}
The data from firebase realtime database is as _InternalLinkedHashMap<dynamic, dynamic>.
You can also just convert this it to your map and query very easily.
For example, I have a chat app and I use realtime database to store the uid of the user and the bool value whether the user is online or not. As the picture below.
Now, I have a class RealtimeDatabase and a static method getAllUsersOnineStatus().
static getOnilineUsersUID() {
var dbRef = FirebaseDatabase.instance;
DatabaseReference reference = dbRef.reference().child("Online");
reference.once().then((value) {
Map<String, bool> map = Map<String, bool>.from(value.value);
List users = [];
map.forEach((key, value) {
if (value) {
users.add(key);
}
});
print(users);
});
}
It will print [NOraDTGaQSZbIEszidCujw1AEym2]
I am new to flutter If you know more please update the answer.
ref.orderByChild("lead").startAt("Jack Nicholson").endAt("Jack Nicholson").listner....
This will work.
[
{
"RollNo":1,
"name":"John",
"age":20,
"Hobby":"Music",
"Date":"9/05/2018"
"sal":5000
},
{
"RollNo":2,
"name":"Ravi",
"age":25,
"Hobby":"TV",
"Date":"9/05/2018"
"sal":5000
},
{
"RollNo":3,
"name":"Devi",
"age":30,
"Hobby":"cooking",
"Date":"9/04/2018"
"sal":5000
}
]
Above is the JSON file i need to insert into a MongoDB. Similar JSON data is already in my mongoDB collection named 'Tests'.I have to ignore the records which is already
in the mongoDB based on a certain condition.
[RollNo in mongoDB == RollNo in the json need to insert && Hobby in mongoDB ==Hobby in the json need to insert && Date in mongoDB == Date in the json need to insert].
If this condition matches, i need to igore the insertion,else need to insert the data into DB .
I am using nodejs. Anyone please help me to do it.
If you are using mongoose then use upsert.
db.people.update(
{ RollNo: 1 },
{
"RollNo":1,
"name":"John",
"age":20,
"Hobby":"Music",
"Date":"9/05/2018"
"sal":5000
},
{ upsert: true }
)
But to avoid inserting the same document more than once, only use upsert: true if the query field is uniquely indexed.
The easiest and safest way to do this is by using a compound index.
You can create a compound index like this:
db.people.createIndex( { "RollNo": 1, "Hobby": 1, "Date" : 1 }, { unique: true } )
Then the duplicated inserts will produce an error which you need to process in your code.
(Couchbase 4.5) Suppose I have the following object stored in my couchbase instance:
{
parentArray : [
{
childArray: [{value: 'v1'}, {value:'v2'}]
},
{
childArray: [{value: 'v1'}, {value: 'v3'}]
}
]
}
Now I want to select the distinct elements from childArray, which should return an array equal to ['v1', 'v2', 'v3'].
I have a couple solutions to this. My first thought was to go ahead and use the UNNEST operation:
SELECT DISTINCT ca.value FROM `my-bucket` AS b UNNEST b.parentArray AS pa UNNEST pa.childArray AS ca WHERE _class="someclass" AND dataType="someDataType";
With this approach I get a polynomial explosion in the number of scanned elements (due to the unnest'ing of two arrays), and the query takes a bit of time to complete (for my real data on the order of 24 seconds). When I remove unnest, and simply query for distinct elements on the top-level elements (those adjacent to parentArray), it takes on the order of milliseconds.
Another solution is to handle this in the application code, by simply iterating through the returned values and finding the distinct values my-self. This approach is bad, because it brings too much data into the application space.
Any help please!
Thank you!
UPDATE: Looks like without a "WHERE" clause using the "UNNEST" statements the performance is fast. So do I need Array Indexes here?
UPDATE: Nevermind about the previous update, since there is no index elements in the where clause. Also, but I do notice that if I remove the UNNEST OR the WHERE then the query is fast. Moreover, looking at the explain and adding a GSI for compound index (_class, dataType) I can see "IndexScan" on the provided index.
INSERT INTO default values("3",{ "parentArray" : [ { "childArray": [{"value": 'v1'}, {"value":'v2'}] }, { "childArray": [{"value": 'v1'}, {"value": 'v3'}] } ] });
SELECT ARRAY_DISTINCT(ARRAY v.`value` FOR v WITHIN parentArray END) FROM default;
OR
SELECT ARRAY_DISTINCT(ARRAY_FLATTEN(
ARRAY ARRAY v.`value` FOR v IN ca.childArray END FOR ca IN parentArray END,
2)) FROM default;
You can add where clause. If this requires across the documents use the following.
INSERT INTO default values("4",{ "parentArray" : [ { "childArray": [{"value": 'v5'}, {"value":'v2'}] }, { "childArray": [{"value": 'v1'}, {"value": 'v3'}] } ] });
SELECT ARRAY_DISTINCT(ARRAY_FLATTEN(ARRAY_AGG(ARRAY v.`value` FOR v WITHIN parentArray END),2)) FROM default;
SELECT ARRAY_DISTINCT(ARRAY_FLATTEN(ARRAY_AGG(ARRAY_FLATTEN(ARRAY ARRAY v.`value` FOR v IN ca.childArray END FOR ca IN parentArray END,2)),2)) FROM default;
I'm currently trying to do a bit of complex N1QL for a project I'm working on, theoretically I could do all of this processing in multiple N1QL calls and by parsing the results each time, however if possible I'd like for this to contained in one call.
What I would like to do is:
filter all documents that contain a "dataSync.test.id" field with more than 1 id
Read back all other ids in that list
Use that list to get other documents containing those ids
Get the "dataSync.test._channels" field for those documents (optionally a filter by docType might help parsing)
This would probably return a list of "dataSync.test._channels"
Is this possible in N1QL? It appears like it might be but I can't get the syntax right.
My data structures look a little like
{
"dataSync": {
"test": {
"_channels": [
"RP"
],
"id": [
"dataSync_user_1015",
"dataSync_user_1010",
"dataSync_user_1005"
],
"_lastUpdatedBy": "TEST"
}
},
...
}
{
"dataSync": {
"test": {
"_channels": [
"RSD"
],
"id": [
"dataSync_user_1010"
],
"_lastUpdatedBy": "TEST"
}
},
...
}
Yes. I think you can do all these.
Initial set of IDs with filtering can be retrieved as a subquery and then you can get subsquent documents by joins.
SELECT fulldoc
FROM (select meta().id as dockey from doc where a=1) as mydoc
INNER JOIN doc fulldoc ON KEYS mydoc.dockey;
There are optimizations that can be done here. Try the sequencing first to ensure you're get the job done.