I have a Nodejs application which is using Mysql as a database, express and passport to manage user authentication. There can be 20-30 users connected to my Nodejs application at once time.
Now, there are certain pages in my application where multiple users can work on the same stuff at once. So if one user changes the value of the field, the other user will also see that change. As of right now to achieve this I am just using a Setinterval function that is running every 5 seconds with an ajax request post to the Nodejs server and then redraw the user field if necessary. This is working fine till now, but now I have decided, I wanted other pages in my application that I want to work this way. This means there will be multiple post backs happening to my Nodejs server every seconds to run mysql query. I am kind of new to Nodejs and I am not sure if this is an optimal way to handle this situation.
I was wandering if there is a way to send new field data to client, without client request and redrawing the DOM for them.
There are two solutions built-into the browser for the server to send data directly to a connected client.
webSocket connections
Server sent events
With each of these technologies, the client establishes one of these two types of connections on any given web page and then the server is able to send the client data whenever it wants.
webSockets are two way communication channels. Server sent events are one-way (data sent from server to client). Server sent events were designed to be a bit more efficient, but are more limited in what they can do.
It's important to realize that the lasting connection between client and server is only for the duration of that current page in the browser. If the end-user switches to another web page (even another page on your site), then the browser will close your current connection. If that new web page wants a similar connection, then it establishes a new connection on the new page.
With these types of connections from browser to server, your server then keeps track of each connection and some identifying information for each connection (like a username or userID). Then, when something changes in the data on the server, your server can figure out which clients should be notified of that change and send that new data over their connection. The client then receives that data and updates the visuals of the webpage using Javascript (displaying new data, updating status, etc...).
FYI, there is also a popular library called socket.io that works on top of webSocket and adds a number of useful features outlined here (such as connection failure detection, auto-reconnect, message passing layer, etc...). You would use the socket.io library in both client and server to add these features.
Related
I'm building a mobile app where a pull-down gesture on the UI initiates an update of existing data/posts (also retrieves new posts if there are any, but that's not the point here). The server is stateless meaning there is no sessions.
If the posts have been updated in the database, how do I let the front-end know which posts need to be updated? Only way I could think of is to send a list of ids of all retrieved posts to the server, and have it check if any of the posts have been modified since the time fetched.
This however seems quiet inefficient as the users might have stacked up hundreds of posts in some extreme cases, and it's most likely that only few or none of the posts need to be updated. Issuing hundreds of db requests could be a huge overhead.
There are at least 2 ways of doing this
Long Polling
client requests server for new information.
the server keeps the connection open until there is some new data to send.
once the server gets data, it sends this to the client and connection is closed.
the client then sends a new request for information.
This is a continuous process.
WebSockets
Create a websocket connection, keep it open.
The server pushes any updates as and when they come.
Problems with both situations
May take a significant amount of time to have production ready implementations.
Both them will require the server to be aware of any change in the database. This can be tricky as well
I am trying to implement a websocket realtime chat system in nodejs. I already have a login system in place and the websocket chat system. The only thing I'm stuck on is saving the messages to MySQL so that the user can access their chat history. The problem is that I don't want to send a web socket message to the recipient then have the MySQL insert fail for whatever reason.
A possible solution:
The obvious solution is to save the message data in MySQL then send the web socket message to the recipient. If there is an error, send that back to the sender instead of sending the message. Wouldn't this defeat the purpose of web sockets though, if the server has to wait for a MySQL insert success confirmation?
Assuming this is a big deal because you want to avoid a small amount of latency waiting for a database save (which I think is not actually a problem to do that)..Send the message. On the screen, display 'sending message..'
On the server, initiate the asynchronous database save.
When the other user's client receives the message, send print an
acknowledgement on the sender's screen
When the database save on the server completes, you dont have to do anything, unless it gives an error in which case you can inform the user there was an error saving the message and then retry the save.
Some systems like Google Hangouts during voice/video don't even try to save chat history.
You are only really going to have a problem if your database server runs out of space or goes down, in which case you have bigger problems and your error messages will clue the users in.
Or you can wait to save in the database, it isn't a huge amount of time to wait. Also, websockets and socket.io always go to the server anyway. If you were using WebRTC Data Channels, that is a peer 2 peer protocol that would not involve the server.
I have a realtime HTML5 canvas game that runs off a node backend. Players are connected via Websocket (socket.io). The problem is sometimes I need to deploy new code (hotfixes for instance) and restart the server but I don't want to disconnect players.
My idea for this was to divide the websocket server and application server into separately deployable components and add a message queue in the middle to decouple the 2 components. That way if the application server was rebooting there would just be a short delay while the messages bunch up but nothing would be lost. Is this a good strategy? Is there an alternative?
It's very possible for websocket based applications to be restarted without the user noticing anything (that's the case for my chat server for example).
To make that possible, the solution isn't to have a websocket application isolated and never restarted. In fact this would be very optimistic (are you sure you could ensure its API is never changed ?).
A solution is
to ensure the client reconnects if disconnected (this is standard if you use socket.io for websocketing)
to make the server ask the client its id (or session id) on client initiated reconnection
to persists the state of the application. This is usually done with a database. If your server has no other state than the queue between clients (which is a little unlikely) then you might look for an existing persistent queue implementation or build your own over a fast local storage (redis comes to mind)
I am basically writing an almost purely clientside application (there is a webserver which can be used to store some persistent data, but its easier to forget about it), but as part of this I was looking to add some functionality akin to hosting a game.
The scenario would be 1 person would host the game via their browser (open a TCP socket awaiting connections), then X other people would connect to that server and join. The server would be in charge of receiving and sending data between clients.
So in this scenario is it possible to host a websocket server within a webpage?
I was looking at trying to do something peer to peer style, but I don't think it is currently supported, but its not a major problem as its only going to be for sending small amounts of text and some update messages between clients.
The WebSocket browser API is client only (for the foreseeable future).
In some sense, WebRTC us peer-to-peer, but even if the WebRTC API adds the ability to send arbitrary data, you still need a STUN/TURN server to establish the initial connection.
First let me explain the data flow I need
Client connects and registers with server
Server sends initialization JSON to client
Client listens for JSON messages sent from the server
Now all of this is easy and straightforward to do manually, but I would like to leverage a server of some sort to handle all of the connection stuff, keep-alive, dead clients, etc. etc.
Is there some precedent set on doing this kind of thing? Where a client connects and receives JSON messages asynchronously from a server? Without using doing manual socket programming?
A possible solution is known as Comet, which involves the client opening a connection to the server that stays open for a long time. Then the server can push data to the client as soon as it's available, and the client gets it almost instantly. Eventually the Comet connection times out, and another is created.
Not sure what language you're using but I've seen several of these for Java and Scala. Search for comet framework and your language name in Google, and you should find something.
In 'good old times' that would be easy, since at the first connection the server gets the IP number of the client, so it could call back. So easy, in fact, that it was how FTP does it for no good reason.... But now we can be almost certain that the client is behind some NAT, so you can't 'call back'.
Then you can just keep the TCP connection open, since it's bidirectional, just make the client wait for data to appear. The server would send whatever it wants whenever it can.... But now everybody wants every application to run on top of a web browser, and that means HTTP, which is a strictly 'request/response' initiated by the client.
So, the current answer is Comet. Simply put, a JavaScript client sends a request, but the server doesn't answer for a looooong time. if the connection times out, the client immediately reopens it, so there's always one open pipe waiting for the server's response. That response will contain whatever message the server want's to send to the client, and only when it's pertinent. The client receives it, and immediately sends a new query to keep the channel open.
The problem is that HTTP is a request response protocol. The server cannot send any data unless a requests is submitted by the client.
Trying to circumvent this by macking a request and then continously send back responses on the same, original, requests is flawed as the behavior does not conform with HTTP and it does not play well with all sort of intermediaries (proxies, routers etc) and with the browser behavior (Ajax completion). It also doesn't scale well, keeping a socket open on the server is very resource intensive and the sockets are very precious resources (ordinarly only few thousand available).
Trying to circumvent this by reversing the flow (ie. server connects to the client when it has somehting to push) is even more flawed because of the security/authentication problems that come with this (the response can easily be hijacked, repudiated or spoofed) and also because often times the client is unreachable (lies behind proxies or NAT devices).
AFAIK most RIA clients just poll on timer. Not ideal, but this how HTTP works.
GWT provides a framework for this kind of stuff & has integration with Comet (at least for Jetty). If you don't mind writing at least part of your JavaScript in Java, it might be the easier approach.