Related
I'm looking for a way to implement basic Publish / Subscribe between applications written in different languages, to exchange events with JSON payloads.
WebSocket seems like the obvious choice for the transport, but you need an (arguably small) layer on top to implement some of the plumbing:
aggreeing on messages representing the pubsub domain "subscribe to a topic", "publish a message"
aggreeing on messages for the infra ("heartbeat", "authentication")
I was expecting to find an obvious standard for this, but there does not seem to be any.
WAMP is often refered to, but in my (short) experience, the implementations of server / clients libraries are not great
STOMP is often refered to, but in my (even shorter) experience, it's even worse
Phoenix Channels are nice, but they're restricted to Phoenix/Elixir world, and not standard (so the messages can be changed at any phoenix version without notice.)
So, is everyone using MQTT/WS (which require another broker components, rather than simple servers ?) Or gRPC ?
Is everyone just re-implementing it from scratch ? (It's one of those things that seems easy enough to do oneselves, but I guess you just end up with an half-baked, poorly-specified, broken version of the thing I'm looking for...)
Or is there something fundamentally broken with the idea of serving streams of data from a server over WS ?
There are two primary classes of WebSocket libraries; those that implement the protocol and leave the rest to the developer, and those that build on top of the protocol with various additional features commonly required by realtime messaging applications, such as restoring lost connections, pub/sub, and channels, authentication, authorization, etc.
The latter variety often requires that their own libraries be used on the client-side, rather than just using the raw WebSocket API provided by the browser. As such, it becomes crucial to make sure you’re happy with how they work and what they’re offering. You may find yourself locked into your chosen solution’s way of doing things once it has been integrated into your architecture, and any issues with reliability, performance, and extensibility may come back to bite you.
ws, faye-websockets, socket.io, μWebSockets and SocketCluster are some good open-source options.
The number of concurrent connections a server can handle is rarely the bottleneck when it comes to server load. Most decent WebSocket servers can support thousands of concurrent connections, but what’s the workload required to process and respond to messages once the WebSocket server process has handled receipt of the actual data?
Typically there will be all kinds of potential concerns, such as reading and writing to and from a database, integration with a game server, allocation and management of resources for each client, and so forth.
As soon as one machine is unable to cope with the workload, you’ll need to start adding additional servers, which means now you’ll need to start thinking about load-balancing, synchronization of messages among clients connected to different servers, generalized access to client state irrespective of connection lifespan or the specific server that the client is connected to – the list goes on and on.
There’s a lot involved when implementing support for the WebSocket protocol, not just in terms of client and server implementation details, but also with respect to support for other transports to ensure robust support for different client environments, as well as broader concerns, such as authentication and authorization, guaranteed message delivery, reliable message ordering, historical message retention, and so forth. A data stream network such as Ably Realtime would be a good option to use in such cases if you'd rather avoid re-inventing the wheel.
There's a nice piece on WebSockets, Pub/Sub, and all issues related to scaling that I'd recommend reading.
Full disclosure: I'm a Developer Advocate for Ably but I hope this genuinely answers your question.
I'm attempting to make a mail client that is HTML5/JS only where users would have to define their mail server and credentials.
I've surfed google leading to dead ends and figured that this is the next best place (or superuser.com).
I was thinking of using HTML5 WebSocket if I could to make the connection to the server but I don't know enough of smtp, imap, or pop3's architecture to understand how javascript could pass the args and perform what I want it to do.
PHP cannot be involved in this project otherwise I wouldn't be asking the question.
Any help in locating this would be fantastic.
Nothing is impossible. Except this.
In standard HTML or JavaScript, you can't make raw-socket connections, which is what you'd need to speak any of the protocols you've listed. The WebSocket API doesn't help you, because the server must also speak WebSockets, and mail servers don't. This is actually all a good thing, though. Imagine if you visited a random website and it telnetted to your home router, setting it on fire and burning down your house. That's what websites would be able to do if they could initiate plain TCP socket connections.
You have a few options I can think of, neither of which involves building a webpage. (And to be clear, you didn't say your project had to be a webpage; you merely said HTML/JS, so these are indeed legitimate options.)
First is something like node-webkit. As the readme says, "You can write native apps in HTML and Javascript with node-webkit." You'll still have to distribute it as a native app, because that's what it is.
Second is a Chrome app, specifically using the chrome.socket API. But have fun writing a TLS layer over those sockets, which you'll find is a requirement for almost any mail server these days. If you succeed in doing that, you'll be able to distribute your mail client in the Chrome Web Store, where I assure you an email client will be quite popular.
Third, write a webserver that operates locally, so your users will visit something like http://localhost:9999/mail in their browser. This will be a real pain to distribute, but you can use almost any technology you want.
I'm sure there's a fourth valid option, and someone else can chime in on that one.
Best of luck.
My company writes software that installs on client machines to perform point-of-sale transactions. The software interfaces with a variety of external peripherals (receipt printers, bar code scanners, credit-card readers, etc). We do this with a WinForms app that we created in Visual Studio using the Microsoft OPOS library, which in turn communicates with our cloud server.
There are obvious inefficiencies in this model, primarily with updates. I'm researching other ways to communicate with these peripherals over the web, preferably via web browser. So far as I can tell, Java is one of the only technologies out there that can do what we're looking for (via applet), and I assume Adobe Flash can as well (via the Air platform). These are viable, but not preferable because we want to run our software on web-enabled mobile devices.
Does anybody have suggestions for other ways to communicate with external peripherals over the web?
UPDATE (Jan 16th, 2019): The Credential Management API has been announced. It's currently only supported on Chrome and Opera but it's looking promising. Google Developers wrote an article elaborating on the spec.
UPDATE (Dec 28th, 2016): Another couple years gone, and another update. This one will be more focused on two new developments than anything else. See the new "WebUSB & Web BlueTooth" section under "Full Device API". But the answer remains the same.
UPDATE (Nov 3rd, 2014): It's been just over two years since the original post was made, but the answer remains mostly the same for now. We are, however, closer to your original goal in several areas.
ORIGINAL ANSWER:
There would be a number of ways to go about this.
Background
The HTML5 specification has entered into the "Recommendation" state. This means that HTML5 is pretty much set for what it looks like. However, I will be using HTML5 in the same way that every marketing person in the world has decided is best. That is, I will not be talking about HTML. Well, I will, in so far as you will utilize it from an HTML page, but not really. What I'll actually be discussing is JavaScript (JS) and that's a horse of a different color. But for all intents and purposes, we're putting it all under the same heading as HTML5, which has been decided to mean "shiny and new" now.
Also, the items which I am discussing will vary in support. Some are very browser dependent projects (like Chromium specific implementations), and some are more standards driven projects that may not have browsers implementing or experimenting with them yet. I'll try to distinguish between the two as I go along.
Full Device API
Status: Incoming, but not ready
Being able to access devices from the browser is making slow but steady progress. Right now, many modern browsers have access to some of the more common devices like the camera or gamepads, but they are all high level APIs. Browser vendors, the standards groups, and lots of companies involved with the web are all trying to make webapps just as powerful as your local applications.
But the APIs you are looking for are still in progress and a ways off. For your particular case, and for the more general case of connecting your webapp to most devices, we're still a few years away from something we can use. If you want to see what awesome things are coming up in that field, here are just a select few items that may help you directly:
Web Near Field Communication (NFC) API
This one unfortunately may be dead in the water for now. But it looks like originally some folks at the W3C (mostly Intel it looks like) were looking at adding a NFC API to the web.
Media Capture Streams
The WebRTC group is working on programmatic access to media streams like the camera which would allow to integrate things like barcode scanning or other features. This has reached CR status and is available in browsers, but is less helpful on its own.
Web Bluetooth
If you had bluetooth capable tools, this API would help you connect with them from computers and devices that were able to listen and connect. The primary driver for this at the moment seems like it is the Chrome team, including an experimental implementation, but I wouldn't consider it anywhere ready to use yet (See "WebUSB & Web BlueTooth" section).
WebUSB
This would allow full access to low level USB information including listing devices and interacting with them. Same as Web BlueTooth, this seems to be current Chrome pet project, but I also wouldn't rely on it (See "WebUSB & Web BlueTooth" section).
Network Service Discovery
If you have other devices or items on the network which broadcast and use HTTP, this API would allow you to discover and interact with these services. No browser implementation, but it is in a working draft for the W3C.
Originally, Mozilla was pushing a number of these forward because of Boot2Gecko (or Firefox OS). However, with that project officially cancelled, we aren't seeing much progress from them in these areas now.
Members of the Chrome team, however, seem to have decided to dive in and start not only working towards these, but putting them live in browsers. Which leads us to...
WebUSB & Web BlueTooth
Like sausages, it's better to not know how Web Standards are made
-Abraham Lincoln (probably)
There's been a little bit of buzz in this area as it looks like the Chrome team snuck in these as experimental features and developed their own specification for it. Which is great! Just maybe not in the way that you were hoping for.
Each browser vendor and W3C contributor group has their own style and makes contributions towards the specs in their own way. The result is usually a fairly decent specification that the browsers have agreed upon. But getting from nothing to something is... messy. Real messy. And is quite a process a lot of times. It doesn't always result in a good spec (yeah, I'm talking about you Florian compromise...) but even when it does, it takes a while.
However, It seems like Google developed this version of the spec all on their own. And, in my experience, Google's approach to the specs is always a little... well... setting my personal opinions aside we'll say "gung-ho". They tend to just dive right into the deep end. And that seems to be what they've done here.
I highly doubt these specs or implementations will look anything like this when they become standards. And there's nothing wrong with that. That's part of the process. But I wouldn't go relying on this implementation or developing any code or products against it. This is an unprecedented feature on the web and all the browser vendors are gonna want a big say in this.
That said, this is actually good. One of the things Google often does (for better or worse) with situations like this is forces the conversation and it can push things along. And having a feature shipped in the browser, even an experimental feature, can turn up the heat on the demand for it. So we may see more progress in this area soon.
PhoneGap Apache Cordova. You know, for your phone
Status: Not fully featured and phone only
Apache Cordova, previously Adobe PhoneGap, is a way to write your program in HTML, CSS, and JS that allows you to access lower level functionality on things like phones, and compile across devices. This would be a way to implement your program, but it would be a phone application, not necessarily a desktop one. An option to consider, and something I figured I would mention.
Cordova implements a few of the above features already, but doesn't have some of the more powerful ones like NFC or BlueTooth.
The Native-App solution (for Windows 8)
Status: Possible, but OS specific and desktop app
Windows 8 offers the ability to build applications in HTML and JS. This would allow you to easily access lower level functionality on the OS via their API. From the looks of it, it is pretty extensive and you can do a lot. You mentioned cross OS support, however, and this obviously limits you to one OS.
It's so Flash-y!
Status: Dying/Dead, not possible as a web app
Flash won't have direct access to the system through the web. You could create an AIR application, but that will sort of defeat the purpose of having it web based. In addition, Flash support on mobile, and on the web it would seem, is on the decline.
NodeJS
Status: Can be a bit of a pain and only possible as a desktop app
NodeJS and JS applications have sort of been a hot topic the last couple years. I didn't discuss it in my original post because I felt it wasn't quite there yet. However, things have progressed and it is much closer to being ready for this sort of thing, and has the support and power of a growing user base. That said, for your particular case, I wouldn't recommend using it. It would have to be local on the users machine, and because of how NodeJS (and similar engines) are at the moment, it would require a lot of extra configuration and setup that would complicate things a bit.
So you could build an app using HTML, CSS and JS with NodeJS or similar engines and have low level access to what you need, but it has to be local, and it would take more work than I'm sure you want to do every time you'd like to install it for a customer.
... Now where was I?
So where does that leave us? Well, simple: if you want a single language/set of code as your code base, HTML/CSS/JS aren't a great option... yet. But they could be some day. For now, your options are limited to what you feel is best for your customer. Java is a stable option you listed, but obviously comes with its own drawbacks. As the web develops, I think we'll see a lot of really cool things coming out of the new functionality, but we've got a ways to go still.
More reading:
Brian.IO: Beyond HTML5
HTML5 Apps on Windows 8
Wikipedia list of projects built using JS
This is possible, but it would have to be done indirectly. In theory, you could write a socket-server in a low level language, which gets I/O, and sends the I/O through the socket (relaying, I guess). HTML5 uses WebSockets, or some equivalent to communicate with this socket-server.
Now it can be achieved with WebUSB API.
It is available in Chrome since version 54.
It is a W3C editor's draft so we can expect (hope) that it will be adopted by other browser vendors...
I've been thinking about this a lot lately... have a POS app mostly written in VB6, considering what to do next. HTML5 is an option and I was thinking I'd use VSPE to get the serial stuff into the JS.
http://www.eterlogic.com/Products.VSPE.html
Love this product! Works very well for getting serial traffic where you need it, so I think it would work well, at least as a proof-of-concept to get you going. You'll want to use a combination of "connector" types along with the "tcpclient" and "tcpserver".
Just for the record, a method that works well in 2016 (since chrome 26), but is to be withdrawn over the next 2 years is to deploy your html5 as a chrome app and use chrome.usb (or chrome.serial or chrome.bluetooth).
I am currently using chrome.usb and planning to migrate to a web app using WebUSB API (see Supersharp's answer), which I hope will be adopted by the time Google discontinue chrome apps 🤞.
I was a big fan of AIM and live chat/buddy lists back in the day. With the rise of HTML5 and its use becoming more common in modern browsers, I'd like to develop an HTML5 messaging system.
What technologies do I have to look up? At the start, I won't care about the design (CSS), just functionality.
I'll most likely have a standard registration and store users in a MySQL Database.
Additionally, "friends" will also easily be stored in a database, populating a user's buddy list based on which user ID's he/she has marked as "friend".
The actual server and client connectivity is what most interests me. Is this technology available for HTML5 yet? Point me in the right direction and I'll be good to go!
For the chat, you would probably like to look in to Websockets (as you talk about HTML5).
There are also examples like this where NodeJS is used. To use node, you would have to run a node-server. For examples and more info: nodejs.org
I think the websockets API will be your first port of call for a messaging app in HTML5. You'll be wanting the server to notify the client rather than the client poll or rely on callbacks and this would be the start i think.
However, I don't think this is very well supported in even the most modern browser. In fact i believe firefox and opera have pulled support because of security concerns.
I haven't done any work in this myself but just though it looked interesting stuff. So I guess I just wish you luck with your dev. Exciting cutting edge stuff I think.
I really need your help with this. We are planning on developing a real-time web application. We look at different libraries and concepts and a little confused.
What we need is: clients connect to websites and send data(usually an integer + client machine name) whenever they want (usually 1-5 seconds). Also, the same clients must receive data(the data received from other clients) from the server in a real-time mode. (maximum 0.5 seconds). Also, this data must be stored in the database.
We were thinking about using different technologies, but cannot decide which one to use.
We need this web application to be supported on Iphones and Android Phones (maybe blackberry).
and, of course desktop browsers.
Pooling seems not a very good Idea in this situation, due to highloads.
Html 5 web sockets kinda new, and probably not supported by all browsers.
Have anyone used nodejs ?
or twisted matrix: http://twistedmatrix.com/trac/?
or orbited(cannot post more than one link)?
or tornado?
Or XMPP(Jabber. I did not find good examples.)?
or something else?
What technology is the best to use in this type of project? Also, we would probably prefer the technology that has some community support and free to use.
Thanks a lot!
There is a lot of things to consider here. I would say that HTML 5 is not an option, simply due to support across platform.
Running with NodeJS is most likely possible, but the communication methods are really complicated. Pushing data to a page isn't really something that HTML/web apps are designed to do....
To get a valid answer you are going to need to get someone to come in and sit with you to really iron out details and implementation.
When you say that clients "connect to a website", do you really need it to be a website? It sounds like all the client is sending is a number and for that you don't need a website. Just pick the language of your choice, open up a socket, and go from there.
Are you streaming data to be visualized? You might want to take a look at graphite (and/or "pyped" which is part of graphite).
What kind of data? What is the purpose?
For real-time you're not going to get a web site unless you use some type of RIA but even then, it isn't going to be enough. Services aren't going to be good enough either. You're going to end up doing some type of polling which will only ever be psuedo-real-time unless you do duplex mode which wont be supported on most of the platforms you want to support.
sockets are the way to go but that requires a client for each platform you want to handle. Maybe you should rethink your requirements.