In the docs it states that Services only emit events when a Service method modifies data. This is the case in all examples I have seen, where a client modifies that data from the browser itself and it gets automatically updated in other clients (like a chat webapp). But what if my data is modified externally outside of Feathers? Will I be able to use Feathers so that the data is updated in all clients?
In my specific case my data is actually stored in a MongoDB database which gets updated externally and autonomously. I want my web application to use MongoDB Change Streams to listen to changes on the MongoDB database (I already know how to do this) and then I want Feathers to take care of sending updates to all my clients in real-time.
In the example chat app, this would be equivalent to having a bot that also writes messages directly to the database outside of Feathers, and this messages should also be broadcasted to clients in real-time.
Is my use-case a good fit for Feathers? Any hint on how should I approach it?
Watching changefeeds has been done for feathers-rethinkdb here. Something similar could be done for MongoDB but there are several challenges discussed in this issue.
If your MongoDB collection only gets updated externally you could also create a simple pass through service like this:
app.use('/feed/messages', {
async create(data) {
return data;
},
async remove(id) {
return { id };
},
async update(id, data) {
return data;
},
async patch(id, data) {
return data;
}
});
Which is then called by the changefeed watcher and will automatically take care of updating all the clients through its events.
Is there any support for Couchbase Sync Gateway's "Channels" in Pouch DB?
I'd like to be able to have uses see a subset of the overall data and if they create new data to be able to share whom they share it with.
Is that possible with PouchDB? Or would I have to interact with the server directly or use couchbase lite for mobile devices?
Just a little update: This is now possible, PouchDB (since version V3.4.0) is now compatible with sync gateway.
See tutorial here: http://blog.couchbase.com/first-steps-with-pouchdb--sync-gateway-todomvc-todolite
Here is the solution to make the pouch db client work with Couchbase Sync Gateway over user's channels:
var sync = function () {
var opts = {
live: true,
retry: true,
//-- from here
filter: "sync_gateway/bychannel",
query_params: {
"channels": channels
}
//-- to here
};
database.sync(syncServer, opts);
}
The key here is you just pass the filter & query_params as is, the Sync Gateway anyway has the ability to understand this filter.
PouchDB is modeled after CouchDB, which doesn't have the concept of channels, so there are no plans to implement it in PouchDB.
However, one easy way to solve your problem is to sync PouchDB to a CouchDB, then sync that to Couchbase Sync Gateway. The reason you will need CouchDB as an intermediary is that there are a few issues with direct PouchDB <-> Couchbase Sync Gateway syncing, although hopefully they should be resolved soon (see e.g. this and this).
PouchDB syncing through channels / filtered replication
Here's a concrete example to use channels.
var db = new PouchDB("yep");
db.sync(new PouchDB("http://localhost:4984/beer-sample/"), {
live: true,
retry: true,
filter: "sync_gateway/bychannel",
query_params: {
channels: "channel-1,channel-2,channel-3,bar"
}
})
filter: "sync_gateway/bychannel"
Passes the name of filter to apply to the source documents, currently the only supported filter is "sync_gateway/bychannel", this will replicate documents only from the set of named channels.1
query_params.channels
Instead of passing array we separate them by commas.2
Sync Function Example
And in the Sync Gateway your sync function might look like this (It was my intention to keep the sync function as stupid as possible so at one glance you can understand how we used the channels above in PouchDB):
function sync(doc, oldDoc) {
if (doc.type == "beer") {
channel("channel-1");
} else if (doc.type == "soap") {
channel("channel-2");
} else if (doc.type == "sweets") {
channel("channel-3");
} else if (doc.type == "bar") {
channel(doc.type);
}
}
6 years too late though... But it's better late than never!
I'm trying to get dojo to show Json data that comes from a remote web service. I need to be clear though - the web server hosting the html/dojo page I access isn't the same server as the one that's running the web service that returns the json data - the web service server just can't serve html pages reliably (don't ask!!).
As a test I move the page into the same web server as the web service and the below works. As soon as I move it back so that the html/dojo is served from Apache (//myhost.nodomain:82 say) and the web service sending the json is "{target:http://myhost.nodomain:8181}", then it stops working.
I've used FFox to look at the network & I see the web service being called ok, the json data is returned too & looks correct (I know it is from the previous test), but the fields are no longer set. I've tried this with DataGrid and the plain page below with the same effects.
Am I tripping up over something obvious???
Thanks
require([
"dojo/store/JsonRest",
"dojo/store/Memory",
"dojo/store/Cache",
"dojox/grid/DataGrid",
"dojo/data/ObjectStore",
"dojo/query",
"dojo/domReady!"
],
function(JsonRest, Memory, Cache, DataGrid, ObjectStore, query) {
var myStore, dataStore, grid;
myStore = JsonRest(
{
target: "http://localhost:8181/ws/job/definition/",
idProperty: "JOB_NAME"
}
);
myStore.query("JOB00001"
).then(function(results) {
var theJobDef = results[0];
dojo.byId("JOB_NAME").innerHTML = theJobDef.JOB_NAME;
dojo.byId("SCHEDULED_DAYS").innerHTML = theJobDef.SCHEDULED_DAYS;
});
}
);
Its true what Frans said about the cross domain restriction but dojo has this link to work around the problem.
require(["dojo/request/iframe"], function(iframe){
iframe("something.xml", {
handleAs: "json"
}).then(function(xmldoc){
// Do something with the XML document
}, function(err){
// Handle the error condition
});
// Progress events are not supported using the iframe provider
});
you can simply use this and the returned data can be inserted into a store and then into the grid.
Are you familiar with the Same Origin Policy:
http://en.wikipedia.org/wiki/Same-origin_policy
Basically it restricts websites to do AJAX requests to other domains than the html page was loaded from. Common solutions to overcome this are CORS and JSON-P. However, remember that these restrictions are made for security reasons.
I'm currently creating my first Chrome extension, so far so good.
It's just a little test where I run multiple timers.
But obviously all my timers reset when I open and close the extension.
So to keep all my timers running, I would have to same them somehow when I close the extension and make them run in the background page.
When I open the extension again, those timers should be send back to the open page.
How would you handle this?
I already have an array of all my timers, what would be the best option for me>
A background page runs at all times when the extension is enabled. You cannot see it, but it can modify other aspects of the extension, like setting the browser action badge.
For example, the following would set the icon badge to the number of unread items in a hypothetical service:
function getUnreadItems(callback) {
$.ajax(..., function(data) {
process(data);
callback(data);
});
}
function updateBadge() {
getUnreadItems(function(data) {
chrome.browserAction.setBadgeText({text:data.unreadItems});
});
}
Then, you can make a request and schedule it so the data is retrieved and processed regularly, you can also stop the request at any time.
var pollInterval = 1000 * 60; // 1 minute
function startRequest() {
updateBadge();
window.setTimeout(startRequest, pollInterval);
}
function stopRequest() {
window.clearTimeout(timerId);
}
Now just load it...
onload='startRequest()'
Also, HTML5 offline storage is good for storing data and constantly update it...
var data = "blah";
localStorage.myTextData = data;
I have read some posts about this topic and the answers are comet, reverse ajax, http streaming, server push, etc.
How does incoming mail notification on Gmail works?
How is GMail Chat able to make AJAX requests without client interaction?
I would like to know if there are any code references that I can follow to write a very simple example. Many posts or websites just talk about the technology. It is hard to find a complete sample code. Also, it seems many methods can be used to implement the comet, e.g. Hidden IFrame, XMLHttpRequest. In my opinion, using XMLHttpRequest is a better choice. What do you think of the pros and cons of different methods? Which one does Gmail use?
I know it needs to do it both in server side and client side.
Is there any PHP and Javascript sample code?
The way Facebook does this is pretty interesting.
A common method of doing such notifications is to poll a script on the server (using AJAX) on a given interval (perhaps every few seconds), to check if something has happened. However, this can be pretty network intensive, and you often make pointless requests, because nothing has happened.
The way Facebook does it is using the comet approach, rather than polling on an interval, as soon as one poll completes, it issues another one. However, each request to the script on the server has an extremely long timeout, and the server only responds to the request once something has happened. You can see this happening if you bring up Firebug's Console tab while on Facebook, with requests to a script possibly taking minutes. It is quite ingenious really, since this method cuts down immediately on both the number of requests, and how often you have to send them. You effectively now have an event framework that allows the server to 'fire' events.
Behind this, in terms of the actual content returned from those polls, it's a JSON response, with what appears to be a list of events, and info about them. It's minified though, so is a bit hard to read.
In terms of the actual technology, AJAX is the way to go here, because you can control request timeouts, and many other things. I'd recommend (Stack overflow cliche here) using jQuery to do the AJAX, it'll take a lot of the cross-compability problems away. In terms of PHP, you could simply poll an event log database table in your PHP script, and only return to the client when something happens? There are, I expect, many ways of implementing this.
Implementing:
Server Side:
There appear to be a few implementations of comet libraries in PHP, but to be honest, it really is very simple, something perhaps like the following pseudocode:
while(!has_event_happened()) {
sleep(5);
}
echo json_encode(get_events());
The has_event_happened function would just check if anything had happened in an events table or something, and then the get_events function would return a list of the new rows in the table? Depends on the context of the problem really.
Don't forget to change your PHP max execution time, otherwise it will timeout early!
Client Side:
Take a look at the jQuery plugin for doing Comet interaction:
Project homepage: http://plugins.jquery.com/project/Comet
Google Code: https://code.google.com/archive/p/jquerycomet/ - Appears to have some sort of example usage in the subversion repository.
That said, the plugin seems to add a fair bit of complexity, it really is very simple on the client, perhaps (with jQuery) something like:
function doPoll() {
$.get("events.php", {}, function(result) {
$.each(result.events, function(event) { //iterate over the events
//do something with your event
});
doPoll();
//this effectively causes the poll to run again as
//soon as the response comes back
}, 'json');
}
$(document).ready(function() {
$.ajaxSetup({
timeout: 1000*60//set a global AJAX timeout of a minute
});
doPoll(); // do the first poll
});
The whole thing depends a lot on how your existing architecture is put together.
Update
As I continue to recieve upvotes on this, I think it is reasonable to remember that this answer is 4 years old. Web has grown in a really fast pace, so please be mindful about this answer.
I had the same issue recently and researched about the subject.
The solution given is called long polling, and to correctly use it you must be sure that your AJAX request has a "large" timeout and to always make this request after the current ends (timeout, error or success).
Long Polling - Client
Here, to keep code short, I will use jQuery:
function pollTask() {
$.ajax({
url: '/api/Polling',
async: true, // by default, it's async, but...
dataType: 'json', // or the dataType you are working with
timeout: 10000, // IMPORTANT! this is a 10 seconds timeout
cache: false
}).done(function (eventList) {
// Handle your data here
var data;
for (var eventName in eventList) {
data = eventList[eventName];
dispatcher.handle(eventName, data); // handle the `eventName` with `data`
}
}).always(pollTask);
}
It is important to remember that (from jQuery docs):
In jQuery 1.4.x and below, the XMLHttpRequest object will be in an
invalid state if the request times out; accessing any object members
may throw an exception. In Firefox 3.0+ only, script and JSONP
requests cannot be cancelled by a timeout; the script will run even if
it arrives after the timeout period.
Long Polling - Server
It is not in any specific language, but it would be something like this:
function handleRequest () {
while (!anythingHappened() || hasTimedOut()) { sleep(2); }
return events();
}
Here, hasTimedOut will make sure your code does not wait forever, and anythingHappened, will check if any event happend. The sleep is for releasing your thread to do other stuff while nothing happens. The events will return a dictionary of events (or any other data structure you may prefer) in JSON format (or any other you prefer).
It surely solves the problem, but, if you are concerned about scalability and perfomance as I was when researching, you might consider another solution I found.
Solution
Use sockets!
On client side, to avoid any compatibility issues, use socket.io. It tries to use socket directly, and have fallbacks to other solutions when sockets are not available.
On server side, create a server using NodeJS (example here). The client will subscribe to this channel (observer) created with the server. Whenever a notification has to be sent, it is published in this channel and the subscriptor (client) gets notified.
If you don't like this solution, try APE (Ajax Push Engine).
Hope I helped.
According to a slideshow about Facebook's Messaging system, Facebook uses the comet technology to "push" message to web browsers. Facebook's comet server is built on the open sourced Erlang web server mochiweb.
In the picture below, the phrase "channel clusters" means "comet servers".
Many other big web sites build their own comet server, because there are differences between every company's need. But build your own comet server on a open source comet server is a good approach.
You can try icomet, a C1000K C++ comet server built with libevent. icomet also provides a JavaScript library, it is easy to use as simple as:
var comet = new iComet({
sign_url: 'http://' + app_host + '/sign?obj=' + obj,
sub_url: 'http://' + icomet_host + '/sub',
callback: function(msg){
// on server push
alert(msg.content);
}
});
icomet supports a wide range of Browsers and OSes, including Safari(iOS, Mac), IEs(Windows), Firefox, Chrome, etc.
Facebook uses MQTT instead of HTTP. Push is better than polling.
Through HTTP we need to poll the server continuously but via MQTT server pushes the message to clients.
Comparision between MQTT and HTTP: http://www.youtube.com/watch?v=-KNPXPmx88E
Note: my answers best fits for mobile devices.
One important issue with long polling is error handling.
There are two types of errors:
The request might timeout in which case the client should reestablish the connection immediately. This is a normal event in long polling when no messages have arrived.
A network error or an execution error. This is an actual error which the client should gracefully accept and wait for the server to come back on-line.
The main issue is that if your error handler reestablishes the connection immediately also for a type 2 error, the clients would DOS the server.
Both answers with code sample miss this.
function longPoll() {
var shouldDelay = false;
$.ajax({
url: 'poll.php',
async: true, // by default, it's async, but...
dataType: 'json', // or the dataType you are working with
timeout: 10000, // IMPORTANT! this is a 10 seconds timeout
cache: false
}).done(function (data, textStatus, jqXHR) {
// do something with data...
}).fail(function (jqXHR, textStatus, errorThrown ) {
shouldDelay = textStatus !== "timeout";
}).always(function() {
// in case of network error. throttle otherwise we DOS ourselves. If it was a timeout, its normal operation. go again.
var delay = shouldDelay ? 10000: 0;
window.setTimeout(longPoll, delay);
});
}
longPoll(); //fire first handler