Can anyone give me a brief rundown of how Facebook and Google Talk work, are there persistent connections involved similar to the classic Java based chat systems whereby a server manages the connections and directs messages to the necessary destination or are they stateless? I'd like to create something similar to these but I'm not sure where to start and if I need to have custom services running on a server I may have to rethink my approach.
I'm not after a full-blown explanation but I am interested to know if there's a stateless approach that doesn't require services running on a server. If Html5 is required that's ok.
Both use the Jabber protocol: http://www.jabber.org/
Related
Project background: Building an API driven Learning Management System. The back-end system will be receiving data from multiple systems and interfaces: web, mobile, VR.
Looking at API Gateways to front our APIs. Preferably an Open Source API gateway but need to be sure that the support and service is available. Tried out Tyk.io and it feels like it might be the way to go. Been reading other StackOverflow threads around this and looks like TYK's gateway fairs quite well against the likes of Kong and WSO2.
Main areas of consideration for us are:
Rate-limiting
Open ID Connect authentication
Analytics
Scalability
Hybrid model of hosting - combination of on-prem and cloud depending on compliance requirements of educational institutes (Probably rules of AWS' gateway)
It would be really helpful if anyone who is using or has used TYK.io for their production projects can share their experience, especially for enterprise clients/projects.
Full disclosure: I work for Tyk, so of course think that Tyk is the best fit for your project ;)
Seriously, though - Tyk can do all those things you’re after. Here are some links to the documentation for each item that is big on your list:
Rate-limiting
Open ID Connect authentication
Analytics
Scalability
Hybrid model of hosting
You can also post on the Tyk community for help, if you haven’t already, or search to see what else others have said.
The Tyk Open Source API Gateway will do everything you need, even outputting analytics to difference sources, like ElasticSearch, Mongo or just CSV.
In addition, you can also use our API Management Platform to control your open source gateway. The Tyk API Management platform includes a Dashboard with analytics and out-of-the-box developer portal. Tyk is free to use, under a developer license, to manage a single gateway node, ideal if you are doing a POC.
Hope this helps and please keep in touch to let us know more about your use case.
I'm planning to build an application that runs on WEB(AngularJS), Android(Native), IOS(Native).
I have experience with MongoDB, but I found CouchBase which sounds really good for me.
I read documentation and I found out I need to use sync_gatway to sync my mobile databases with main database server and reverse, until now everything is fine.
I also need to use "channels" to share records with multiple users.
The problem comes when I need to implement this for web application.
In their documentation on "Working with web applications" they explain how "bucket shadowing" is working, but they also say:
Bucket shadowing is meant to enable sync for existing Couchbase Server
apps. If you are creating a new app with both mobile and web clients,
we recommend starting with the Sync Gateway REST APIs, and connecting
backend services using the Changes Worker Pattern.
After reading Sync Gateway REST API I found out I'm limited to facebook and persona authentication. So I can't use my own authentication mechanism?
Also, there is nothing specified in REST API about channels?
Is there any example project or more documentation about this? I couldn't find anything :(
If someone has experience with this, please explain how this works.
Thanks
There is also Custom (Indirect) Authentication available on Sync Gateway, which you can use for any type of auth you need.
But you have to hide Sync Gateway's Admin API under your backend layer.
As for the channels: it is responsibility of Sync Function to route different documents to necessary channels based on Document data.
Here is a good video that describes how to build production architecture around Couchbase Lite.
I'm probably late for the party - but as of today I'd recommend taking a look in the PouchDB project for the WEB AngularJS side - they match pretty well and will sync with Couchbase.
Regarding authentication, I just released an article on that topic, find it here. Hope this helps somebody
I've been looking into sockjs-tornado recently and am working on a chat function for a social networking site. I'm trying to get a feel for common methods used in building scalable multiroom chat functionality. I'll outline a couple of the methods I've thought of and I'd like to get feedback. What methods are used in the real world? What are the advantages and disadvantages to these methods?
Prereqs:
running tornado
using sockjs-tornado lib
sockjs-client lib for js
Everything else is open.
Methods I've considered:
For loop
This seems like the simplest way to go. You create a user class that subscribes to certain room classes. The user sends a message class that contains a room id and the server redirects the message in the loop only to users that have subscribed to that room. This seems to me to be by far the worst because the complexity is obviously at least linear. (Imagine 500 users connected at once to 5 chat rooms each.)
Multi-tasking/multiple server instances
This also seems like a bad idea because you could have 500 server instances running at any time on... different ports? I'm really not sure on the implementation of this method.
Native support
Now granted, a lot of libraries have this built in such as socketio. However that's not an option due to the sole node.js support. (I'm on tornado server.) Socks in particular does not have built in support for multiple "rooms".
Conclusion
I'm looking for resources/case studies, and industry standards. Any help would be appreciated.
I would just use a message queue server like RabbitMQ with a fanout exchange as each "chat room".
You can see an example of using a fanout exchange in Python here.
The Pika AMQP library works with Tornado, too.
The advantage with using a message queueing system is that you can have users connected to different Tornado processes on different servers while still being in the same "room", giving you high availability on the HTTP layer.
RabbitMQ also has HA capabilities (although not the greatest).
I'm planning to setup an scalable architecture capable of providing web services on a REST interface where JSON will be sent as a result.
The web services will be quite simple for a CRUD web 2.0 app.
I think javascript (nodejs + mongodb) is a good choice for the following reasons:
Easy to find javascript developers
Good performance
Easy to scale
Shared logic/language or possible reuse of code between database query language, back-end and web-client.
There are testing and logging frameworks for node
By the examples I've seen node seems light in terms of the lines of code needed to implement web services.
Questions:
I think of scaling a node app which supplies a web service as having
a central node which will be routing/balancing charge to each of the
node instances. Which will also help doing seamless updates, is
there any piece of software already implemented which can fit that
task?
Please point all the disadavantages or other advantages you find in this back-end stack
If you feel this question opens too much of a debate and doesn't fit the stackoverflow policy please indicate a forum where I could get feedback.
Any other good persistence choices other than MongoDB? Mainly this choice comes from the javascript query language and JSON schemas.
Regarding your "router" piece:
Since your REST api will be composed of HTTP requests, it's common practice to use a high-speed proxy such as NGINX or HAProxy to distribute requests among the many servers who actually perform the work (in your case, NodeJS servers). This generally works well and allows easy scaling and failover.
I want to implement a discrete remote authentication server that handles login for many sites. Somewhat similar to OpenID.
Basically, I have site-1 and site-2 and they're both reliant on the same user database, which is on a separate auth-site. So, auth-site handles user authentication for them, and during this process, makes information on the authenticating user available to the requesting system.
Each site can be on a completely separate domain name, on completely separate machines.
This is all via HTTP(S), there can be no direct database access.
There's one last quirk: once an user has logged in to site-1, when accessing any other site reliant on auth-site, the site must treat the user as already authenticated.
This whole business must be entirely fuss-free to the end-user. It should work like a simple everyday login form.
As a concrete example, say we're talking about stackoverflow.com and serverfault.com, and they both authenticate via authentic-overflow-server-stack.com. Again, once logged in to either site, I can go to the other and do my business without logging in again.
What I'd like to know are the general interaction mechanism between the sites behind this scenario.
In my particular setup, I'm using Rails, but I'm not looking for code[1], just general best practice and guidance, so feel free to answer in pseudo-code or any generally readable language. OTOH, bear in mind that I'll have decent MVC, REST, and meta-programming in my toolkit.
[1]: unless you happen to know an existing tiny neat free MIT/BSD-licensed app/plugin/generator that handles this.
It sounds like (especially with the emphasis on fuss-free), you want something like what the Wikimedia Foundation is doing. Basically, you log on to en.wikipedia.org, then that server communicates with other servers (e.g. en.wikinews.org) and gets authentication tokens. Finally, those tokens are embedded into images, e.g. http://en.wikinews.org/wiki/Special:AutoLogin?token=xxxxxxxxxxxxxxx , and when your browser visits that url (img src) it gets a authentication cookie for Wikinews. Of course, the source code is available for your reivew at http://www.mediawiki.org/wiki/Extension:CentralAuth .
OpenID is also a good choice, but it does require that the user "consciously" visit two domains. An example of one entity with two domains doing this is Canonical. E.g., if you go to https://help.ubuntu.com/community/UserPreferences they will redirect you to Launchpad (https://login.launchpad.net/+openid) for authentication.
Note that Wikipedia is doing this over http, but you can do it all https to ensure the img src tokens aren't intercepted.
Looks like CAS is good enough for me, and has ruby implementations, along with dozens of other lesser languages, e.g. one that rhymes with femoral bone rage.
http://code.google.com/p/rubycas-server/
http://code.google.com/p/rubycas-client/
It sounds like you want to actually use the OpenID protocol itself. There's no reason you can't restrict the authentication provider to only your own server, and do some shortcuts that make the authentication process transparent. Also, the OpenID protocol supports what you describe about logging into one implies logging in to all services.