This can be any type of data feed, let's just say for this example stock market data since that's a popular one. But I'm talking about real time data feeds. So it continuously sends data.
I'm trying to understand how this is implemented. Does this happen in some way over http? I just don't get how the clients hook up to the server and the server sends continuous data, can anyone clue me into how this works technically? How might a client hook up to the server? Let's take C# or Java or something. Does this happen over http or maybe some other type of way? please go into details.
Thanks
It's not really any different from normal HTTP traffic, just longer.
What happens when you open a website? (very rough overview)
Your computer contacts a server, establishes a connection.
Server starts sending data split into packets to you.
Your computer receives the packets, possibly out-of-order and some with significant delay, and re-assembles them into a website.
Your computer sends an acknowledgment for received packets.
The server will retransmit any packets it hasn't received an acknowledgment for for a significant amount of time, assuming they were dropped on the way.
In between receiving packets and/or acknowledgments, both wait.
When all data necessary for displaying the site is transferred, your computer thanks the server for its time and disconnects.
If at any point in this process either party stops responding for a significant amount of time, either side may drop the connection, assuming technical difficulties.
What's happening with "streaming data feeds"? (even rougher overview)
A client contacts a server, establishing a connection.
The server starts sending data split into packets to the client.
The client receives the packets, possibly out-of-order and some with significant delay, and re-assembles them.
The client sends an acknowledgment for received packets.
The server will retransmit any packets it hasn't received an acknowledgment for for a significant amount of time, assuming they were dropped on the way.
In between receiving packets and/or acknowledgments, both wait.
The only difference is that the client doesn't hang up on the server, because it's still expecting data and that both aren't as quick to drop the connection.
The thing is that web servers (for web sites) are specialized for the task of delivering small snippets of data to many people, so they're quick to hang up on you once all data has been sent. Your server script can simply not quit though, and the connection will stay alive. Here's a tiny PHP script that will demonstrate that:
while (true) {
echo '.';
sleep(1);
}
This will send a new . every second indefinitely (note that the web server needs to be configured appropriately to not terminate the script and to immediately send the output).
Try the Wikipedia article about TCP/IP for the basics and this article about long-polling/HTTP streaming for concrete examples.
Related
I have an domain name to test.
Ping is ~20 ms.
'HTTP HEAD' is ~500 ms.
Why there are so big difference between them? Is this a server-side problem? Isn't there are too big difference? 25 times.
Well, for one, ping goes over a different protocol, ICMP. The server itself directly responds to pings. HTTP is a different protocol handled by an additional application, a web server, that must be running on the server (ping is built-in to the OS). Depending on how heavy the web server is, it could take a significant amount of time more, relative to something like a ping. Also, HEAD is sent along with a particular URL. If that URL is handled by something like ASP.NET instead of just the web server directly, then there's additional processing that must be done to return the response.
Ping is usually implemented as an ICMP echo request. A simpler datagram protocol: You send a packet, the server replies with the corresponding packet and that's about it.
HTTP HEAD is still HTTP: a TCP connection must be established between both ends and the HTTP server must reply with the headers for your request. It's obviously fast but not as simple as sending a single packet response.
If you're testing a domain, ping is a more adequate tool, while HTTP HEAD is a tool better suited to test an HTTP server.
If I'm not mistaken, a ping request is handled on the network driver level, and is extremely fast as a result (sometimes it's handled by the hardware itself, skipping software processing altogether). It will portray network latency fairly well.
An HTTP HEAD request must visit the web server, which is a user-level program, and requires copying bits of data a couple times, and web server code to parse the request, etc. The web server then has to generate the HTTP response headers for the request. Depending on the server and the requested page, this can take a while, as it has to generate the requested page anyway (It just sends you the headers only, and not the page content.)
When you run ping it responds much quicker because is it is designed to respond immediately. It shows you approximate latency, so if you get consistent results using ping you cannot get lower latency than that.
When you run HTTP HEAD you are actually making a request to a specific page, it is processed, executed rendered and only head is returned. It has much more overhead compared to ping, that's why it is taking much longer.
or any other type of realtime data feed from server to client... I'm talking about a bunch of realtime data from server to client. i.e., an informational update every second.
Does the server magically push the data to the client, or does the client need to continuously poll the server for updates? And under what protocol does this usually work? (http, socket communication, etc?)
In server-side financial applications used by brokers/banks etc. market data (quotes,trades etc) is transmitted over TCP via some application-level protocol which most probably won't be HTTP. Of course, there's no polling. Client is establishing TCP connection with server, server pushes data to client. One of common approaches to distribute market data is FIX.
Thomson-Reuters have bunch of cryptic proprietary protocols dating from mainframe days to distribute such data.
HTTP can be used for SOAP/RESTful to transmit/request data of not-so-large volume, like business news.
UPDATE Actually, even FIX is not enough in some cases, as it has big overhead because of it's "text" nature. Most brokers and exchanges transmit high-volume streams, such as quotes, using binary-format protocols (FAST or some proprietary).
In a simple case:
Create a server with a listening socket.
On the client, connect to the server's socket.
Have the client do a while(data = recv(socket)) (pseudocode)
When the server has something exciting to tell the client, it simply send(...)s on the socket.
You can even implement this pattern over HTTP (there is no real upper time limit to an HTTP socket). The server need not even read from the socket - it can be trying to write to the firehose only.
Usually a TCP socket is employed - messages arrive in order, and are best-effort. If latency is more important and dropped or out of order is not an issue, UDP can be used.
I have one app. that consists of "Manager" and "Worker". Currently, the worker always initiates the connection, says something to the manager, and the manager will send the response.
Since there is a LOT of communication between the Manager and the Worker, I'm considering to have a socket open between the two and do the communication. I'm also hoping to initiate the interaction from both sides - enabling the manager to say something to the worker whenever it wants.
However, I'm a little confused as to how to deal with "collisions". Say, the manager decides to say something to the worker, and at the same time the worker decides to say something to the manager. What will happen? How should such situation be handled?
P.S. I plan to use Netty for the actual implementation.
"I'm also hoping to initiate the interaction from both sides - enabling the manager to say something to the worker whenever it wants."
Simple answer. Don't.
Learn from existing protocols: Have a client and a server. Things will work out nicely. Worker can be the server and the Manager can be a client. Manager can make numerous requests. Worker responds to the requests as they arrive.
Peer-to-peer can be complex with no real value for complexity.
I'd go for a persistent bi-directional channel between server and client.
If all you'll have is one server and one client, then there's no collision issue... If the server accepts a connection, it knows it's the client and vice versa. Both can read and write on the same socket.
Now, if you have multiple clients and your server needs to send a request specifically to client X, then you need handshaking!
When a client boots, it connects to the server. Once this connection is established, the client identifies itself as being client X (the handshake message). The server now knows it has a socket open to client X and every time it needs to send a message to client X, it reuses that socket.
Lucky you, I've just written a tutorial (sample project included) on this precise problem. Using Netty! :)
Here's the link: http://bruno.linker45.eu/2010/07/15/handshaking-tutorial-with-netty/
Notice that in this solution, the server does not attempt to connect to the client. It's always the client who connects to the server.
If you were thinking about opening a socket every time you wanted to send a message, you should reconsider persistent connections as they avoid the overhead of connection establishment, consequently speeding up the data transfer rate N-fold.
I think you need to read up on sockets....
You don't really get these kinds of problems....Other than how to responsively handle both receiving and sending, generally this is done through threading your communications... depending on the app you can take a number of approaches to this.
The correct link to the Handshake/Netty tutorial mentioned in brunodecarvalho's response is http://bruno.factor45.org/blag/2010/07/15/handshaking-tutorial-with-netty/
I would add this as a comment to his question but I don't have the minimum required reputation to do so.
If you feel like reinventing the wheel and don't want to use middleware...
Design your protocol so that the other peer's answers to your requests are always easily distinguishable from requests from the other peer. Then, choose your network I/O strategy carefully. Whatever code is responsible for reading from the socket must first determine if the incoming data is a response to data that was sent out, or if it's a new request from the peer (looking at the data's header, and whether you've issued a request recently). Also, you need to maintain proper queueing so that when you send responses to the peer's requests it is properly separated from new requests you issue.
I have a client/server connection over a TCP socket, with the server writing to the client as fast as it can.
Looking over my network activity, the production client receives data at around 2.5 Mb/s.
A new lightweight client that I wrote to just read and benchmark the rate, has a rate of about 5.0Mb/s (Which is probably around the max speed the server can transmit).
I was wondering what governs the rates here, since the client sends no data to the server to tell it about any rate limits.
In TCP it is the client. If server's TCP window is full - it needs to wait until more ACKs from client came. It is hidden from you inside the TCP stack, but TCP introduces guaranteed delivery, which also means that server can't send data faster than rate at which client is processing them.
TCP has flow control and it happens automatically. Read about it at http://en.wikipedia.org/wiki/Transmission_Control_Protocol#Flow_control
When the pipe fills due to flow control, the server I/O socket write operations won't complete untill the flow control is releaved.
The server is writing data at 5.0Mb/s, but if your client is the bottleneck here then server has to wait before the data in "Sent Buffer" is completely sent to client, or enough space is released to put in more data.
As you said that the light weight client was able to receive at 5.0Mb/s, then it will be the post-receiving operations in your client that you have to check. If you are receiving data and then processing it before you read more data, then this might be the bottleneck.
It is better to receive data asynchronously, and as soon as one receive is complete, ask the client sockets to start receiving data again, while you process the received data in a separate thread pool thread. This way your client is always available to receive incomming data, and server can send it at full speed.
First let me explain the data flow I need
Client connects and registers with server
Server sends initialization JSON to client
Client listens for JSON messages sent from the server
Now all of this is easy and straightforward to do manually, but I would like to leverage a server of some sort to handle all of the connection stuff, keep-alive, dead clients, etc. etc.
Is there some precedent set on doing this kind of thing? Where a client connects and receives JSON messages asynchronously from a server? Without using doing manual socket programming?
A possible solution is known as Comet, which involves the client opening a connection to the server that stays open for a long time. Then the server can push data to the client as soon as it's available, and the client gets it almost instantly. Eventually the Comet connection times out, and another is created.
Not sure what language you're using but I've seen several of these for Java and Scala. Search for comet framework and your language name in Google, and you should find something.
In 'good old times' that would be easy, since at the first connection the server gets the IP number of the client, so it could call back. So easy, in fact, that it was how FTP does it for no good reason.... But now we can be almost certain that the client is behind some NAT, so you can't 'call back'.
Then you can just keep the TCP connection open, since it's bidirectional, just make the client wait for data to appear. The server would send whatever it wants whenever it can.... But now everybody wants every application to run on top of a web browser, and that means HTTP, which is a strictly 'request/response' initiated by the client.
So, the current answer is Comet. Simply put, a JavaScript client sends a request, but the server doesn't answer for a looooong time. if the connection times out, the client immediately reopens it, so there's always one open pipe waiting for the server's response. That response will contain whatever message the server want's to send to the client, and only when it's pertinent. The client receives it, and immediately sends a new query to keep the channel open.
The problem is that HTTP is a request response protocol. The server cannot send any data unless a requests is submitted by the client.
Trying to circumvent this by macking a request and then continously send back responses on the same, original, requests is flawed as the behavior does not conform with HTTP and it does not play well with all sort of intermediaries (proxies, routers etc) and with the browser behavior (Ajax completion). It also doesn't scale well, keeping a socket open on the server is very resource intensive and the sockets are very precious resources (ordinarly only few thousand available).
Trying to circumvent this by reversing the flow (ie. server connects to the client when it has somehting to push) is even more flawed because of the security/authentication problems that come with this (the response can easily be hijacked, repudiated or spoofed) and also because often times the client is unreachable (lies behind proxies or NAT devices).
AFAIK most RIA clients just poll on timer. Not ideal, but this how HTTP works.
GWT provides a framework for this kind of stuff & has integration with Comet (at least for Jetty). If you don't mind writing at least part of your JavaScript in Java, it might be the easier approach.