Related
{
"movies": {
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson"
},
"movie2": {
"genre": "Horror",
"name": "The Shining",
"lead": "Jack Nicholson"
},
"movie3": {
"genre": "comedy",
"name": "The Mask",
"lead": "Jim Carrey"
}
}
}
I am a Firebase newbie. How can I retrieve a result from the data above where genre = 'comedy' AND lead = 'Jack Nicholson'?
What options do I have?
Using Firebase's Query API, you might be tempted to try this:
// !!! THIS WILL NOT WORK !!!
ref
.orderBy('genre')
.startAt('comedy').endAt('comedy')
.orderBy('lead') // !!! THIS LINE WILL RAISE AN ERROR !!!
.startAt('Jack Nicholson').endAt('Jack Nicholson')
.on('value', function(snapshot) {
console.log(snapshot.val());
});
But as #RobDiMarco from Firebase says in the comments:
multiple orderBy() calls will throw an error
So my code above will not work.
I know of three approaches that will work.
1. filter most on the server, do the rest on the client
What you can do is execute one orderBy().startAt()./endAt() on the server, pull down the remaining data and filter that in JavaScript code on your client.
ref
.orderBy('genre')
.equalTo('comedy')
.on('child_added', function(snapshot) {
var movie = snapshot.val();
if (movie.lead == 'Jack Nicholson') {
console.log(movie);
}
});
2. add a property that combines the values that you want to filter on
If that isn't good enough, you should consider modifying/expanding your data to allow your use-case. For example: you could stuff genre+lead into a single property that you just use for this filter.
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson",
"genre_lead": "comedy_Jack Nicholson"
}, //...
You're essentially building your own multi-column index that way and can query it with:
ref
.orderBy('genre_lead')
.equalTo('comedy_Jack Nicholson')
.on('child_added', function(snapshot) {
var movie = snapshot.val();
console.log(movie);
});
David East has written a library called QueryBase that helps with generating such properties.
You could even do relative/range queries, let's say that you want to allow querying movies by category and year. You'd use this data structure:
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson",
"genre_year": "comedy_1997"
}, //...
And then query for comedies of the 90s with:
ref
.orderBy('genre_year')
.startAt('comedy_1990')
.endAt('comedy_2000')
.on('child_added', function(snapshot) {
var movie = snapshot.val();
console.log(movie);
});
If you need to filter on more than just the year, make sure to add the other date parts in descending order, e.g. "comedy_1997-12-25". This way the lexicographical ordering that Firebase does on string values will be the same as the chronological ordering.
This combining of values in a property can work with more than two values, but you can only do a range filter on the last value in the composite property.
A very special variant of this is implemented by the GeoFire library for Firebase. This library combines the latitude and longitude of a location into a so-called Geohash, which can then be used to do realtime range queries on Firebase.
3. create a custom index programmatically
Yet another alternative is to do what we've all done before this new Query API was added: create an index in a different node:
"movies"
// the same structure you have today
"by_genre"
"comedy"
"by_lead"
"Jack Nicholson"
"movie1"
"Jim Carrey"
"movie3"
"Horror"
"by_lead"
"Jack Nicholson"
"movie2"
There are probably more approaches. For example, this answer highlights an alternative tree-shaped custom index: https://stackoverflow.com/a/34105063
If none of these options work for you, but you still want to store your data in Firebase, you can also consider using its Cloud Firestore database.
Cloud Firestore can handle multiple equality filters in a single query, but only one range filter. Under the hood it essentially uses the same query model, but it's like it auto-generates the composite properties for you. See Firestore's documentation on compound queries.
I've written a personal library that allows you to order by multiple values, with all the ordering done on the server.
Meet Querybase!
Querybase takes in a Firebase Database Reference and an array of fields you wish to index on. When you create new records it will automatically handle the generation of keys that allow for multiple querying. The caveat is that it only supports straight equivalence (no less than or greater than).
const databaseRef = firebase.database().ref().child('people');
const querybaseRef = querybase.ref(databaseRef, ['name', 'age', 'location']);
// Automatically handles composite keys
querybaseRef.push({
name: 'David',
age: 27,
location: 'SF'
});
// Find records by multiple fields
// returns a Firebase Database ref
const queriedDbRef = querybaseRef
.where({
name: 'David',
age: 27
});
// Listen for realtime updates
queriedDbRef.on('value', snap => console.log(snap));
var ref = new Firebase('https://your.firebaseio.com/');
Query query = ref.orderByChild('genre').equalTo('comedy');
query.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(DataSnapshot dataSnapshot) {
for (DataSnapshot movieSnapshot : dataSnapshot.getChildren()) {
Movie movie = dataSnapshot.getValue(Movie.class);
if (movie.getLead().equals('Jack Nicholson')) {
console.log(movieSnapshot.getKey());
}
}
}
#Override
public void onCancelled(FirebaseError firebaseError) {
}
});
Frank's answer is good but Firestore introduced array-contains recently that makes it easier to do AND queries.
You can create a filters field to add you filters. You can add as many values as you need. For example to filter by comedy and Jack Nicholson you can add the value comedy_Jack Nicholson but if you also you want to by comedy and 2014 you can add the value comedy_2014 without creating more fields.
{
"movies": {
"movie1": {
"genre": "comedy",
"name": "As good as it gets",
"lead": "Jack Nicholson",
"year": 2014,
"filters": [
"comedy_Jack Nicholson",
"comedy_2014"
]
}
}
}
For Cloud Firestore
https://firebase.google.com/docs/firestore/query-data/queries#compound_queries
Compound queries
You can chain multiple equality operators (== or array-contains) methods to create more specific queries (logical AND). However, you must create a composite index to combine equality operators with the inequality operators, <, <=, >, and !=.
citiesRef.where('state', '==', 'CO').where('name', '==', 'Denver');
citiesRef.where('state', '==', 'CA').where('population', '<', 1000000);
You can perform range (<, <=, >, >=) or not equals (!=) comparisons only on a single field, and you can include at most one array-contains or array-contains-any clause in a compound query:
Firebase doesn't allow querying with multiple conditions.
However, I did find a way around for this:
We need to download the initial filtered data from the database and store it in an array list.
Query query = databaseReference.orderByChild("genre").equalTo("comedy");
databaseReference.addValueEventListener(new ValueEventListener() {
#Override
public void onDataChange(#NonNull DataSnapshot dataSnapshot) {
ArrayList<Movie> movies = new ArrayList<>();
for (DataSnapshot dataSnapshot1 : dataSnapshot.getChildren()) {
String lead = dataSnapshot1.child("lead").getValue(String.class);
String genre = dataSnapshot1.child("genre").getValue(String.class);
movie = new Movie(lead, genre);
movies.add(movie);
}
filterResults(movies, "Jack Nicholson");
}
}
#Override
public void onCancelled(#NonNull DatabaseError databaseError) {
}
});
Once we obtain the initial filtered data from the database, we need to do further filter in our backend.
public void filterResults(final List<Movie> list, final String genre) {
List<Movie> movies = new ArrayList<>();
movies = list.stream().filter(o -> o.getLead().equals(genre)).collect(Collectors.toList());
System.out.println(movies);
employees.forEach(movie -> System.out.println(movie.getFirstName()));
}
The data from firebase realtime database is as _InternalLinkedHashMap<dynamic, dynamic>.
You can also just convert this it to your map and query very easily.
For example, I have a chat app and I use realtime database to store the uid of the user and the bool value whether the user is online or not. As the picture below.
Now, I have a class RealtimeDatabase and a static method getAllUsersOnineStatus().
static getOnilineUsersUID() {
var dbRef = FirebaseDatabase.instance;
DatabaseReference reference = dbRef.reference().child("Online");
reference.once().then((value) {
Map<String, bool> map = Map<String, bool>.from(value.value);
List users = [];
map.forEach((key, value) {
if (value) {
users.add(key);
}
});
print(users);
});
}
It will print [NOraDTGaQSZbIEszidCujw1AEym2]
I am new to flutter If you know more please update the answer.
ref.orderByChild("lead").startAt("Jack Nicholson").endAt("Jack Nicholson").listner....
This will work.
I've been trying to build an excel sheet about all the papers published by staffs and students of my university. I used Scopus API to retrieve all the information like Author, Title and publish dates and it worked perfectly.
Since the retrieved data was a JSON File I had to convert it to Excel, So I chose OpenRefine and when I converted the file it created multiple rows if the paper had more than one writers
For example Like
Sample Scopus
And My JSON response looks like
{
abstracts-retrieval-response: {
coredata: {
citedby-count: 0,
prism:volume: 430-431,
prism:pageRange: 240-246,
prism:coverDate: 2018-03-01,
dc:title: Solving the 3-COL problem by using tissue P systems without environment and proteins on cells,
prism:aggregationType: Journal,
prism:doi: 10.1016/j.ins.2017.11.022,
prism:publicationName: Information Sciences
},
authors: {
author: [
{
ce:given-name: Daniel,
preferred-name: {
ce:given-name: Daniel,
ce:initials: D.,
ce:surname: Díaz-Pernil,
ce:indexed-name: Díaz-Pernil D.
},
#seq: 1,
ce:initials: D.,
#_fa: true,
affiliation: {
#id: 60033284,
#href: http://api.elsevier.com/content/affiliation/affiliation_id/60033284
},
ce:surname: Díaz-Pernil,
#auid: 16645195100,
author-url: http://api.elsevier.com/content/author/author_id/16645195100,
ce:indexed-name: Diaz-Pernil D.
},
{
ce:given-name: Hepzibah A.,
preferred-name: {
ce:given-name: Hepzibah A.,
ce:initials: H.A.,
ce:surname: Christinal,
ce:indexed-name: Christinal H.
},
#seq: 2,
ce:initials: H.A.,
#_fa: true,
affiliation: {
#id: 60100082,
#href: http://api.elsevier.com/content/affiliation/affiliation_id/60100082
},
ce:surname: Christinal,
#auid: 57197875639,
author-url: http://api.elsevier.com/content/author/author_id/57197875639,
ce:indexed-name: Christinal H.A.
},
{
ce:given-name: Miguel A.,
preferred-name: {
ce:given-name: Miguel A.,
ce:initials: M.A.,
ce:surname: Gutiérrez-Naranjo,
ce:indexed-name: Gutiérrez-Naranjo M.
},
#seq: 3,
ce:initials: M.A.,
#_fa: true,
affiliation: {
#id: 60033284,
#href: http://api.elsevier.com/content/affiliation/affiliation_id/60033284
},
ce:surname: Gutiérrez-Naranjo,
#auid: 6506630834,
author-url: http://api.elsevier.com/content/author/author_id/6506630834,
ce:indexed-name: Gutierrez-Naranjo M.A.
}
]
}
}
}
So how do I combine all the authors into a single cell according to the Title?
After importing the JSON into OpenRefine, you need to organise the project into Records. See http://kb.refinepro.com/2012/03/difference-between-record-and-row.html for an explanation of the difference between 'rows' and 'records' in OpenRefine.
To get the project into records you need to move a column containing information that will only appear once in each record (e.g. the title column - which maybe labelled something like "_ - abstracts-retrieval-response - coredata - dc:title" based on the JSON you've pasted here) to the start of the project. See http://kb.refinepro.com/2012/06/create-records-in-google-refine.html for more information on creating records in OpenRefine.
Once you have done this, switch to the 'records' view (click the 'records' link towards the top left of the data table) and then do as #Ettore-Rizza mentions in his comment - pick the column containing the names you want to use (e.g. "_ - abstracts-retrieval-response - authors - author - _ - ce:indexed-name" column) and use the Edit cells -> Join Multi-valued Cells option from the drop down menu at the top of the column.
Because each author related to the article is described in the JSON with multiple fields including various name forms plus a URL) you'll need to either remove the other columns containing author info, or merge the multiple values into a single field using the 'Join Multi-value cells option on all the affected columns (unless you need to retain this information, it is much easier to remove the unwanted columns)
Once this is done, and assuming there are no other fields which have repeated data in the record, you should have a single row per title.
I am going to implement the REST base CRUD modal in my my app.I wan to display the list of product data with edit and delete link
Product
id, title, unit_id, product_type_id, currency_id,price
Q1: what should be json response look like?
There are two formats comes in my mind to place the data in Json as a response of REST Get call
[
{
id:1,
title:"T-Shirt",
unit_id:20,
unit_title: "abc"
product_type_id:30,
product_type_title:"xyz"
currency_id: 10,
currency_name: "USD"
min_price:20
},
{...}
]
and the another one is
[
{
id:1,
title:"T-Shirt",
unit: {
id: 20,
title: "abc"
},
product_type: {
id: 30,
title: "xyz"
},
currency_id: {
id:10,
name: "USD"
},
min_price:20
},
{...}
]
what is the better and standard way to handle the above scenario?
Furthermore, let suppose I have 10 more properties in product table which will never display on list page. but i needed it when user going to edit the specific item.
Q2: Should I the load all data once at the time of displaying product list and pass the data to edit component.
or
Load only the needed propeties of product table and pass the id to produt edit component and a new REST GET call with id to get the properties of product.
I am using React + Redux for my front end
Typically, you would create additional methods for API consumers to retrieve the values that populate the lists of currency, product_type and unit when editing in a UI.
I wouldn't return more data than necessary for an individual Product object.
We are stuck with filling in form from JSON data and need help. The component is about selecting the ward in the district of the given city.
The data structure is a tree of Cities, Districts, and Wards with approximately following structure (everything is wrapped in GeoJSON):
// Cities: '/api/cities/berlin'
{
features: [
{
type: "Feature",
properties: {
slug: "berlin",
name: "Berlin",
districts: [
{name: "Neukölln", slug: "neukolln", ...}
]
},
geometries: {...}
}
]
}
// Districts: '/api/cities/berlin/districts/neukolln'
{
features: [
{
type: "Feature",
properties: {
slug: "neukolln",
name: "Neukölln",
wards: [
{name: "Britz", slug: "britz", ...}
]
},
geometries: {...}
}
]
}
// Wards: '/api/cities/berlin/districts/neukolln/wards'
{
features: [
{
type: "Feature",
properties: {
slug: "britz",
name: "Britz",
},
geometries: {...}
}
]
}
In the view, three are three dropdown boxes for selecting City, District and Ward, thus, when User select City, then District dropdown is filled from the properties.districts field of the JSON response.
Same is applied for the Districts dropdown: wards are filled in from the properties.wards
When the page is loaded it already has an injected JSON of all available Cities (and, accordingly their districts)
What strategy would you advise on:
1) how to get currently selected city and hit server for next administrative divisions? I.e. when a user selects District, get its slug and query server for the wards?
2) how to fill subsequent select from the response or injected JSON on the page? I.e. when user select another City, fill District select box with respective Districts?
Here's how I have done something similar in the past (in elm v0.17.0):
dropdown : List City -> Html Msg
dropdown cities =
div []
[ div []
[ span [] [ text "dropdown label name" ]
, selectOptions cities
]
]
selectOptions : List City -> Html Msg
selectOptions cities =
select [ on "change" ( Json.map (\city -> GetDistrictsMsg city) targetValue ) ]
(List.map setOption cities)
setOption : City -> Html Msg
setOption city =
option [ value city.name ]
[ text city.name ]
And you will repeat the same for districts to get wards.
Start-up
If selected city is known, before the start-up, you can pass in as a flag to Html.App.programWithFlags
The same thing you can do to the list of cities.
Please see the http example, it covers most of the stuff.
If you want to send xhr request on start-up, you might use a little neat trick for that:
init : String -> (Model, Cmd Msg)
init topic =
( Model topic "waiting.gif"
, getRandomGif topic
)
Where getRandomGif will execute the xhr request on start-up, assuming that you have gotten some data for that from passing them as flags or from user input.
On every FetchSucceed, you should send the next xhr to grab the data for the next step.
The flow
Please consider this flow chart, illustrating the flow of your multi step form. Dashed arrows point to steps, where you can restart the cycle, if you want to change the city/district at some point.
Caching layer is optional, Elm offers a variety of data structures for that.
I'm trying to take data in from a JSON file and link it to my geoJSON file to create a choropleth map with the county colours bound to the "amount" value but also I would like a corresponding "comment" value to be bound to a div for when I mouseover that county.
My code at http://bl.ocks.org/eoiny/6244102 will work to generate a choropleth map when my counties.json data is in the form:
"Carlow":3,"Cavan":4,"Clare":5,"Cork":3,
But things get tricky when I try to use the following form:
{
"id":"Carlow",
"amount":11,
"comment":"The figures for Carlow show a something." },
I can't get my head around how join the "id": "Carlow" from counties.json and "id": "Carlow" path created from ireland.json, while at the same time to have access to the other values in counties.json i.e. "amount" and "comment".
Apologies for my inarticulate question but if anyone could point me to an example or reference I could look up that would be great.
I would preprocess the data when it's loaded to make lookup easier in your quantize function. Basically, replace this: data = json; with this:
data = json.reduce(function(result, county) {
result[county.id] = county;
return result;
}, {});
and then in your quantize function, you get at the amounts like this:
function quantize(d) {
return "q" + Math.min(8, ~~(data[d.id].amount * 9 / 12)) + "-9";
}
What the preprocessing does is turn this array (easily accessed by index):
[{id: 'xyz', ...}, {id: 'pdq', ...}, ...]
into this object with county keys (easily accessed by county id):
{'xyz': {id: 'xyz', ...}, 'pdq': {id: 'pdq', ...}, ...}
Here's the working gist: http://bl.ocks.org/rwaldin/6244803