We're designing a communications system for supporting asynchronous near-realtime server pushing of events/messages to thousands of clients acrros JVM, .Net and mobile platforms. We have a mixed model (both publish-subscribe and request-response) and binary data.
We are using Netty and Google Protocol Buffers and we need to encapsulate all this in http(s).
We were looking into the Bayeux Protocol and its Netty implementation and were inclined to use a modified version of it with protobuf codecs instead of JSONs' or roll our own starting from the WebSockets example (client and server). For performance concerns we would stay out of Base64 encodings an the like.
Are there any implementations that we are missing? How feasible would it be to do a protobuf-based Bayeux protocol? Any pointers are greatly appreciated.
How about STOMP over Web Sockets?
Related
I don't like tools that do many things at once. So GRPC seems to me overhead, it's like kubernetes.
GRPC is the tool that combines actually two things: extended Protobuf (Service support) and HTTP2.
I read a lot of articles saying that using GRPC is awesome for performance. And there are two reasons
protobuf is used, it's smaller than json or xml.
GRPC uses HTTP2 for transport protocol
Here is main part: protobuf and HTTP2 are independent projects, tools, whatever. With that understanding i can say that GRPC is nothing but combining several different tools, like kubernetes combines docker and orchestration tools.
So my questions is: What's actual advantages of using GRPC vs HTTP2 with any payload (CSV, XML, JSON, etc).
Let's skip part about serialization because as i mentioned protobuf is independent library from grpc
As you pointed out, gRPC and Protobuf are often conflated. While, in the vast majority of cases, gRPC will be using protobuf as an IDL and HTTP/2 as the transport, this is not always the case.
So then, what value does gRPC provide on its own? For starters, it provides battle-tested implementations of each of those transports, along with first class support for the protobuf IDL. Integrating these things is not trivial. gRPC packages all of them into one nice pluggable box so you don't have to do the legwork.
It also provides you with functionality that HTTP/2 on its own does not. Pluggable authorization/authentication, distributed tracing instrumentation, debugging utilities, look-aside load balancing (including upcoming support for the xDS protocol), and more are provided.
In general, supposing your server, written in Javascript in the node.js environment, is assembling a response to an end user from services provided by other servers, is there any reasonable alternative to having them all talk JSON?
RPC calls usually use XDR, which is binary, or JSON, which is ascii. There are handy libraries to take care of all the boring sockets stuff for you for these cases.
Node.js tools to use RPC are available for RPC in JSON, but is there any easy way to access RPC calls in binary formats such as XDR, ASN.1 PER, or Avro Apache from node.js. Or to do something similar?
I am not asking what is the best way, or what can be done, because anything can be done with enough hard work, but rather what ways have standards to avoid reinventing the wheel, and libraries to do as much of the work as possible.
avsc implements Avro's RPC specification, you can use it to expose "type-safe" services over any transport (e.g. HTTP, TCP, WebSockets). The quickstart guide goes through a simple example that should help you get started. (Disclaimer: I wrote it.)
I am really a beginner in delphi, as the client request the project to be done in delphi, so I have to ask to get some help as possible as I can. This project will be involved in HTML5 and websockets as well, so there are some questions I need your advices.
Sorry let me explain more, this application have two parts mainly, one is HTML5 client web app (this app will be loaded from web server, and run locally in browser), and the other is its service located in server. The HTML5 web app is doing some product editing job, as well send its final output to server, during this process, there will be many communications between client app and service, and I planed to use Websockets.
I did plan this application in HTML5 and Node.js before, but as client send request to ask for the Delphi XE as primary development tool also it will be compiled to DLL (ISAPI, have to be) at last. For consistency of UI, I also was required to use uniGUI (http://www.unigui.com/demo)for Delphi. Now I have these questions:
Is there any other way to do this application with HTML5 as web front exclude uniGUI in Delphi?
What good libraries support Websockets in Delphi?
What book you can recommend me to read, as I don't know Delphi at all.
Thanks.
1: yes, make a "single page app" (static html + js) and communicate with server using xml or json.
At least with indy 10 it is quite straight forward to implement static file serving (response.contentstream := tfilestream.create(sfilename) or something like that)
2: yes, if you search for it on google you will find some :)
WebSocket server implementations for Delphi
btw, I'm busy with delphi socket.io implementation, which makes it easier to use websockets instead of plain low level tcp-like websockets itself
Is there any other way to do this application with HTML5 as web front exclude uniGUI in Delphi?
First concept : HTML5 applications are a bunch of HTML files and JS's wich can be processed by browsers.
Second concept : You need an application that can bind HTTP queries and return files from the first concept.
Third concept (paradigm) : With first two concepts, a third concept arises. Dynamic HTML. What is dynamic HTML? HTML that generates at runtime at the backend server and/or at frontend with JS.
Solution :
Yes, with Delphi as a backend server you can manipulate and return HTML5 views, if you know (MVC paradigmn for example) you can program an HTTPServer application that binds HTTP request from network and responses your HTMLs and JS.
What good libraries support Websockets in Delphi?
You need a multilayer library. The best library I know is mORMot. mORMot is like a swiss razor for Object Pascal.
You can create an HTTP server with websocket published interfaces. mORMot's Site
What book you can recommend me to read, as I don't know Delphi at all.
I have more than 10 years with Delphi and Free Pascal and I didn't read a book at all. But I read a lot from internet and a lot of computer architecture and another languages like C and C++.
If you like Delphi there are tons of documentation via Internet. I cannot recomend you a book because I will lie you.
I'm looking lua memcached client that supports memcached binary protocol.
Could you recommend something?
There is a client that supports memcached binary protocol.
lua-resty-couchbase - Lua couchbase client driver for the ngx_lua based on the cosocket API.
Not sure about binary protocol, but check this out: https://github.com/silentbicycle/lua-memcached
Also, this one may serve as a reference (it is for ngx_lua, so it is unlikely that you'll be able to immediately reuse it out of the box): https://github.com/agentzh/lua-resty-memcached
I did not try either of the modules, please do leave feedback here in comments if you will try them.
I've publish lua binary client:
http://github.com/kolchanov/Lua-couchbase
from my understanding obited can do both comet and xmpp ? it is better than using typical comet library dwr right?
Orbited is a library to create 'realtime' applications that run in a web browser (it implements a technique known as Comet). More specifically is provides code and strategies to implement many protocols (xmpp, irc, stomp, amqp, etc) on top of what is known as a "web socket" - an object that abstracts away the many peculiarities of actually implementing Comet. Furthermore, Orbited describes, and in some cases provides, the back-end components (asynchronous webservers) that need to know and work with the front-end Comet clients.
Here is a little more detail on Comet and how Orbited plays in:
What makes Comet more complex to
implement (than making an Ajax app,
say), is that you need both the client
(a Comet javascript client) and the
server (an asynchronous webserver like
Twisted) working in conjunction, and
this is what you get with Orbited -
both the javascript client and the
server.