How to create JSON tree and nested nodes in Firebase using AngularFire2 - json

There is no proper documentation for AngularFire2 v5, so I don't know where am i supposed to learn from, but here it's only for Firestore. I don't use firestore, I use Firebase Realtime Database. https://github.com/angular/angularfire2/tree/master/docs
However, I'm pretty new to it, and I'm looking how can I store data in a collection, that would look something like this:
nepal-project-cffb1:
users:
userid:
email: "lajmis#mail.com"
score: 1
My function now looks like this:
this._db.list("users/" + "id").set({
email: this.user,
score: this.score
})
but it doesn't work, I get Expected 2 arguments, but got 1. error. There are a bunch of syntaxes, like .ref .list and i don't know which one to use.

There is an entire section of the AngularFire guide dedicated to RTDB, so I'm confused by the statement that there is no real documentation. Typically you would use valueChanges() to create a list observer as described here for reading, and set() is covered here.
So as that doc illustrates, you need to pass the key of the record to be modified, and also the data to be changed when you call it.
this._db.list("users").set("id", {...});
But if you aren't monitoring the collection, you can also just use db.object directly:
this._db.object("users/id").set({...});
One aspect that may not be immediately intuitive with AngularFire is that it's mainly an adapter pattern intended to take away some of the boilerplate of ferrying data between your Angular model and your Firebase servers.
If you aren't syncing data (downloading a local copy) then there's no reason to use AngularFire here. You can simply call the Firebase SDK directly and avoid the overhead of subscribing to remote endpoints:
firebase.database().ref("users/id").set({...});

Related

Notify nearby users in cloud functions using geofirestore

I have 2 firestore collections - crews/{crew}/clients and crews/{crew}/pros. If a new client registers and a new document is created, I want to search collection pros for pros working the matching sector and living within 5 km (of the new client), and send notification to the pros filtered. In order to implement that in cloud functions,
I installed geofirestore using npm, saved crews/{crew}/pros like this;
https://i.stack.imgur.com/YwAFO.png
but after executing this function, I have error message on cloud functions console like this;
Error: Registration token(s) provided to sendToDevice() must be a non-empty string or a non-empty array
Is there anything wrong with my firestore data structure? Thank you.
I found that this data structure was correct, because I could get notification to work with this data structure. I tried another structure like this;
https://i.stack.imgur.com/vtB5X.jpg
However, it gave me another error.

Data Studio connector making multiple calls to API when it should only be making 1

I'm finalizing a Data Studio connector and noticing some odd behavior with the number of API calls.
Where I'm expecting to see a single API call, I'm seeing multiple calls.
In my apps script I'm keeping a simple tally which increments by 1 every url fetch and that is giving me the correct number I expect to see with getData().
However, in my API monitoring logs (using Runscope) I'm seeing multiple API requests for the same endpoint, and varying numbers for different endpoints in a single getData() call (they should all be the same). E.g.
I can't post the code here (client project) but it's substantially the same framework as the Data Connector code on Google's docs. I have caching and backoff implemented.
Looking for any ideas or if anyone has experienced something similar?
Thanks
Per the this reference, GDS will also perform semantic type detection if you aren't explicitly defining this property for your fields. If the query is semantic type detection, the request will feature sampleExtraction: true
When Data Studio executes the getData function of a community connector for the purpose of semantic detection, the incoming request will contain a sampleExtraction property which will be set to true.
If the GDS report includes multiple widgets with different dimensions/metrics configuration then GDS might fire multiple getData calls for each of them.
Kind of a late answer but this might help others who are facing the same problem.
The widgets / search filters attached to a graph issue getData calls of their own. If your custom adapter is built to retrieve data via API calls from third party services, data which is agnostic to the request.fields property sent forward by GDS => then these API calls are multiplied by N+1 (where N = the amout of widgets / search filters your report is implementing).
I could not find an official solution for this either, so I invented a workaround using cache.
The graph's request for getData (typically requesting more fields than the Search Filters) will be the only one allowed to query the API Endpoint. Before starting to do so it will store a key in the cache "cache_{hashOfReportParameters}_building" => true.
if (enableCache) {
cache.putString("cache_{hashOfReportParameters}_building", 'true');
Logger.log("Cache is being built...");
}
It will retrieve API responses, paginating in a look, and buffer the results.
Once it finished it will delete the cache key "cache_{hashOfReportParameters}building", and will cache the final merged results it buffered so far inside "cache{hashOfReportParameters}_final".
When it comes to filters, they also invoke: getData but typically with only up to 3 requested fields. First thing we want to do is make sure they cannot start executing prior to the primary getData call... so we add a little bit of a delay for things that might be the search filters / widgets that are after the same data set:
if (enableCache) {
var countRequestedFields = requestedFields.asArray().length;
Logger.log("Total Requested fields: " + countRequestedFields);
if (countRequestedFields <= 3) {
Logger.log('This seams to be a search filters.');
Utilities.sleep(1000);
}
}
After that we compute a hash on all of the moving parts of the report (date range, plus all of the other parameters you have set up that could influence the data retrieved form your API endpoints):
Now the best part, as long as the main graph is still building the cache, we make these getData calls wait:
while (cache.getString('cache_{hashOfReportParameters}_building') === 'true') {
Logger.log('A similar request is already executing, please wait...');
Utilities.sleep(2000);
}
After this loop we attempt to retrieve the contents of "cache_{hashOfReportParameters}_final" -- and in case we fail, its always a good idea to have a backup plan - which would be to allow it to traverse the API again. We have encountered ~ 2% error rate retrieving data we cached...
With the cached result (or buffered API responses), you just transform your response as per the schema GDS needs (which differs between graphs and filters).
As you start implementing this, you`ll notice yet another problem... Google Cache is limited to max 100KB per key. There is however no limit on the amount of keys you can cache... and fortunately others have encountered similar needs in the past and have come up with a smart solution of splitting up one big chunk you need cached into multiple cache keys, and gluing them back together into one object when retrieving is necessary.
See: https://github.com/lwbuck01/GASs/blob/b5885e34335d531e00f8d45be4205980d91d976a/EnhancedCacheService/EnhancedCache.gs
I cannot share the final solution we have implemented with you as it is too specific to a client - but I hope that this will at least give you a good idea on how to approach the problem.
Caching the full API result is a good idea in general to avoid round trips and server load for no good reason if near-realtime is good enough for your needs.

Storing data in FIWARE Object Storage

I'm building an application that stores files into the FIWARE Object Storage. I don't quite understand what is the correct way of storing files into the storage.
The code python code snippet below taken from the Object Storage - User and Programmers Guide shows 2 ways of doing it:
def store_text(token, auth, container_name, object_name, object_text):
headers = {"X-Auth-Token": token}
# 1. version
#body = '{"mimetype":"text/plain", "metadata":{}, "value" : "' + object_text + '"}'
# 2. version
body = object_text
url = auth + "/" + container_name + "/" + object_name
return swift_request('PUT', url, headers, body)
The 1. version confuses me, because when I first looked at the only Node.js module (repo: fiware-object-storage) that works with Object Storage, it seemed to use 1. version. As the module was making calls to the old (v.1.1) API version instead of the presumably newest (v.2.0), referencing to the python example, not sure if that is an outdated version of doing it or not.
As I played more with the module, realised it didn't work and the code for it was a total mess. So I forked the project and quickly understood that I will need rewrite it form the ground up, taking the above mention python example from the usage guide as an reference. Link to my repo.
As of writing this the only methods that aren't implement is the object storage (PUT) and object fetching (GET).
Had some addition questions about the Object Storage which I sent to fiware-lab-help#lists.fiware.org, but haven't heard anything back so asking them here.
Haven't got much experience with writing API libraries. Should I need to worry about auth token expiring? I presume it is not needed to make a new authentication, every time we interact with storage. The authentication should happen once when server is starting-up (we create a instance) and it internally keeps it. Should I implement some kind of mechanism that refreshes the token?
Does the tenant id change? From the quote below is presume that getting a tenant I just a one time deal, then later you can use it in the config to make less authentication calls.
A valid token is required to access an object store. This section
describes how to get a valid token assuming an identity management
system compatible with OpenStack Keystone is being used. If the
username, password and tenant details are known, only step 3 is
required. source
During the authentication when fetching tenants how should I select the "right" one? For now i'm just taking the first one similar as the example code does.
Is it true that a object storage container belongs to only a single region?
Use only what you call version 2. Ignore your version 1. It is commented out in the example. It should be removed from the documentation.
(1) The token will be valid for some period of time. This could be an hour or a day, depending on the setup. This period of time should be specified in the token that is returned by the authentication service. The token needs to be periodically refreshed.
(2) The tenant id does not change.
(3) Typically only one tenant id is returned. It is possible, however, that you were assigned more than one id, in which case you have to pick which one you are currently using. Containers typically belong to a single tenant and are not shared between tenants.
(4) Containers are typically limited to a single region. This may change in the future when multi-region support for a container is added to Swift.
Solved my troubles and created the NPM module that works with the FIWARE Object Storage: https://github.com/renarsvilnis/fiware-object-storage-ge

Create an API directly from a data source

I need to create my own REST API.
I just saw strongloop and loopback and I thought it will be perfect for my project.
In fact, I was able to get mysql connected using strongloop. However, I had to create something called a "model" and I did it. But, it was like creating a new model from scratch and use it for persisting on the Datasource.
Instead, what I was looking for, is to get a REST API directly from my model on the DB.
I mean taking the models from each table on the DB and then set them up as web services.
Is that possible?
I am newbie on these technologies, although I think it is an interesting question.
Thanks
I'm not sure of a Node tool to do what you're asking, but in other languages / databases you have some choices!
The only one I'm really familiar with is postgrest.
postgrest: You import your data into a Postgres database (similar to MySQL), and it generates a REST API on top of your tables instantly. Bam. Done. I've used this and it was amazingly awesome. You can also deploy it directly on Heroku.
StrongLoop actually has a "discovery" tool for just this purpose! Read up on that page, but here's the basic code to do it. Just drop this code in a file inside /server/boot/ (the docs are wrong on that, it must be in the directory I mention). Of course, you'll need to tailor it for your use case:
var loopback = require('loopback');
var ds = loopback.createDataSource('mysql', {
"host": "yourhost",
"port": 1234,
"database": "foobar",
"username": "someuser",
"password": "somepass"
});
// Discover and build models from a given table
ds.discoverAndBuildModels('PERSON', {visited: {}, associations: true},
function (err, models) {
// Now we have a list of models keyed by the model name
// You only need the rest of this if wanted to inspect what came in...
// For example, you could find the first record from the table
// and verify info or something.
models.Person.findOne({}, function (err, person) {
if(err) {
// handle this if need be...
console.error(err);
return;
}
// Some code using `person`
});
});
Good luck!
I was supposing "discovery tool" was to find out pattern, cluster or whatever else thing into the data. But following
the recomendation of #jakarella I went with more depth.
It was even easier than that, because you can do everything via StrongLoop Arc (GUI). I always prefer cli
but to have a general look of it, sometimes is better to start with the GUI if you don't have too much idea about the subject.
Anyway,
first of all you would need to connect your datasource before (installing previously the driver).
After that, through StrongLoop Arc you can do "discover the models" choosing your tables (taking care of run Arc and restarting every time
you get a new model) and that's it, you get an API over your Datasource
(for testing go to the explorer).
I described before generally the main tasks but if anyone needs more detail pls let me know. Hope this helps to anyone else who is looking to do
something similar
thanks guys for the interest.

Can I put a map in my http api for extensibility?

My http api uses JSON to pass parameters, it looks like:
{
param1: xxx
param2: xxx
param3: xxx
}
However, my system is a plugin system that each plugin needs to have its own parameters in the JSON body and all plugins will cooperate with each other to produce final result.
for example, let's say the api is
CreateACar {
name: xxx
description: xxx
model: xxxx
}
the base api has three fields for basic meta data. And the system has plugins like:
CarColorPlugin: needs parameters as
{
doorColor: xxx
roofColor: xxx
decoratorColor: xxx
}
TirePlugin: needs parameters as
{
tireSize: xxx
tireBrand: xxx
}
WindShieldPlugin: needs parameters as
{
brand: xxx
needRearWindShield: true or false
}
you can imagine this kind of plugins as many as possible. Now the problem is all plugins need api CreateACar to carry their information and sometimes later a new plugin may join the system so CreateACar must be extensible for future needs.
Now I am considering to put a map in JSON body and passing api CreateACar to all plugins so they can fetch parameters by themselves.
However, this design looks a little ugly to me. I have been researching for a while, the projects having beautiful api usually have limited business domain. For projects having broad unanticipated business domain usually use extensible data structure like XML in API body, however, all these API I have seen so far are mess, especially these without good documentation.
Yes - presumably many of your requests will ask for one car, not all of them, though.
Here's my overall design suggestion:
Use the Java SPI system to handle your plugins. Define a plugin interface that includes a method to identify the plugin's key ("color", "wipers", and so on) and one that takes a Map and returns a plugin data object. Collect all of the implementations in a Map<String,Plugin>.
Write a getPlugins() method on your Car class that does not have a backing field but instead collates the information from all of the plugins applied to the Car in a nested Map.
Write a setPlugins() method that takes the nested map, iterates over the keys, looks up the appropriate factory plugin by name, and rehydrates the plugin data object from the JSON object data.