what is the difference between MQ and MQTT? - message-queue

I am beginner to MQTT , but what is confusing me is it's basic definition
MQTT is a publish-subscribe based "light weight" messaging protocol'.
Based on my previous understanding on MQ (Message Queue), both MQ and MQTT sounds very same to me. Can any one elaborate what the real difference between MQ and MQTT is and their use case? Thanks!!

MQTT was initially called "Message Queue Telemetry Transport". You can find an in-depth explanation how it evolved here: http://www.hivemq.com/mqtt-essentials-part-1-introducing-mqtt/

MQTT is a protocol, it is open source. A lot of vendors are using this protocol for messaging.
MQ is an over-arching team that just means messaging. There are lot of different protocols, AMQP is one of them as well.
Additionally, IBM had a product called MQ.

MQTT is an OASIS standard protocol that is not only used for MQ messaging but it can also be used for the Internet of Things IOT and M2M (machine to machine) communication.
Here is a slide presentation that you may find useful.

Related

Is there a standard PubSub protocol over WebSocket?

I'm looking for a way to implement basic Publish / Subscribe between applications written in different languages, to exchange events with JSON payloads.
WebSocket seems like the obvious choice for the transport, but you need an (arguably small) layer on top to implement some of the plumbing:
aggreeing on messages representing the pubsub domain "subscribe to a topic", "publish a message"
aggreeing on messages for the infra ("heartbeat", "authentication")
I was expecting to find an obvious standard for this, but there does not seem to be any.
WAMP is often refered to, but in my (short) experience, the implementations of server / clients libraries are not great
STOMP is often refered to, but in my (even shorter) experience, it's even worse
Phoenix Channels are nice, but they're restricted to Phoenix/Elixir world, and not standard (so the messages can be changed at any phoenix version without notice.)
So, is everyone using MQTT/WS (which require another broker components, rather than simple servers ?) Or gRPC ?
Is everyone just re-implementing it from scratch ? (It's one of those things that seems easy enough to do oneselves, but I guess you just end up with an half-baked, poorly-specified, broken version of the thing I'm looking for...)
Or is there something fundamentally broken with the idea of serving streams of data from a server over WS ?
There are two primary classes of WebSocket libraries; those that implement the protocol and leave the rest to the developer, and those that build on top of the protocol with various additional features commonly required by realtime messaging applications, such as restoring lost connections, pub/sub, and channels, authentication, authorization, etc.
The latter variety often requires that their own libraries be used on the client-side, rather than just using the raw WebSocket API provided by the browser. As such, it becomes crucial to make sure you’re happy with how they work and what they’re offering. You may find yourself locked into your chosen solution’s way of doing things once it has been integrated into your architecture, and any issues with reliability, performance, and extensibility may come back to bite you.
ws, faye-websockets, socket.io, μWebSockets and SocketCluster are some good open-source options.
The number of concurrent connections a server can handle is rarely the bottleneck when it comes to server load. Most decent WebSocket servers can support thousands of concurrent connections, but what’s the workload required to process and respond to messages once the WebSocket server process has handled receipt of the actual data?
Typically there will be all kinds of potential concerns, such as reading and writing to and from a database, integration with a game server, allocation and management of resources for each client, and so forth.
As soon as one machine is unable to cope with the workload, you’ll need to start adding additional servers, which means now you’ll need to start thinking about load-balancing, synchronization of messages among clients connected to different servers, generalized access to client state irrespective of connection lifespan or the specific server that the client is connected to – the list goes on and on.
There’s a lot involved when implementing support for the WebSocket protocol, not just in terms of client and server implementation details, but also with respect to support for other transports to ensure robust support for different client environments, as well as broader concerns, such as authentication and authorization, guaranteed message delivery, reliable message ordering, historical message retention, and so forth. A data stream network such as Ably Realtime would be a good option to use in such cases if you'd rather avoid re-inventing the wheel.
There's a nice piece on WebSockets, Pub/Sub, and all issues related to scaling that I'd recommend reading.
Full disclosure: I'm a Developer Advocate for Ably but I hope this genuinely answers your question.

Fiware Lightweight M2M agent: how to send measures from devices

I am investigating how to use the Fiware's Lightweight M2M agent.
I read: https://fiware-iotagent-lwm2m.readthedocs.io/en/latest/deviceProvisioning/index.html and this other helped so much: Setting up LWM2M device communicates with IDAS
But I didn't find how is the message's coap format waited, and where fiware-service and fiware-service-path are placed.
I get with the Lightweight M2M client I can test the client (but it is a kind of blackbox), and I would like understand how is the format for develop my own client/device.
Wikipedia
LwM2M is specified by the OMA. It offers a resource-model and supports two encoding modes TLV and CBOR (at least, the newer spec).

Send Message To Another User

I was wondering how to send messages to another user using Progress 4gl. We are trying to cut down on the PA speaker where I work and I want some way to notify a certain user/users of some predefined messages. I'm not sure if this is even possible with Progress, or if there is a message queue that can be used. If anyone has any samples, ideas or has done this before, please let me know. Thanks!!
You might find this helpful:
Following on from presentations in Boston and Finland, dot.r is
pleased to announce the open source Stomp project, available
immediately.
Download from either http://www.dotr.com or
https://bitbucket.org/jmls/stomp , the dot.r stomp programs allow you
to connect your progress session to any other application or service
that is connected to the same message broker.
Open source , free message brokers that support Stomp are :
Fuse (http://fusesource.com/products/fuse-mq-enterprise/)
[a Progress company now owned by Red Hat inc]
Fuse MQ Enterprise is a standards-based, open source messaging platform
that deploys with a very small footprint. The lack of license
fees combined with high-performance, reliable messaging that can be
used with any development environment provides a solution that
supports integration everywhere
ActiveMQ (http://activemq.apache.org/)
Apache ActiveMQ (tm) is the most popular and powerful open source messaging
and Integration Patterns server. Apache ActiveMQ is fast,
supports many Cross Language Clients and Protocols, comes with
easy to use Enterprise Integration Patterns and many advanced features
while fully supporting JMS 1.1 and J2EE 1.4.
Apache ActiveMQ is released under the Apache 2.0 License
RabbitMQ
RabbitMQ is a message broker. The principal idea is pretty simple: it
accepts and forwards messages. You can think about it as a post
office: when you send mail to the post box you're pretty sure that Mr.
Postman will eventually deliver the mail to your recipient. Using this
metaphor RabbitMQ is a post box, a post office and a postman.
The major difference between RabbitMQ and the post office is the fact
that it doesn't deal with paper, instead it accepts, stores and
forwards binary blobs of data - messages.
Please feel free to log any issues on the
https://bitbucket.org/jmls/stomp issue system, and fork the project in
order to commit back all those new features that you are going to add
...
dot.r Stomp uses the permissive MIT licence
(http://en.wikipedia.org/wiki/MIT_License)
Have fun, enjoy !
Julian
I tried it -- the code is dead-simple to install and run. And peeking inside the source is a pleasure.
ApacheMQ is pretty much painless to get going. This is a really, really easy way to get going with messaging.
If you are an old character-based fossil (such as myself) you might want to skip the GUI samples. You can send a message with:
/* stompQOut.p
*
*/
dotr.Stomp.Helper.SendMessage:ToQueue("myQueue","a test message")
And receive messages with:
/* stompQIn.p
*
*/
define variable stompClient as dotr.Stomp.StompClient no-undo.
define variable msgTxt as character no-undo format "x(60)".
stompClient = new dotr.Stomp.StompClient().
stompClient:Subscribe( this-procedure ).
stompClient:SubscribeToQueue( "myQueue" ).
pause 0 before-hide.
wait-for close of this-procedure.
procedure NewStompMessage:
define input parameter stompMessage as dotr.Stomp.StompMessage no-undo.
message string( stompMessage:Body ).
end.

Good examples of applications using Message Queues

I would like to learn more about programming messaging applications and using message queues. Things like qpid, Amazon Queues, etc. Can you point me to some apps (preferably C++, open source if possible) so I can learn more.
Also, can you tell me the general guidelines one would use to decide whether ot not to use message queues. I am interesting in leanring about this just for the 'coolness' factor but I think it might be beneficial for me in the future.
RabbitMQ is the Message Queue that I am the most familiar with. It implements AMQP just like qpid. AMQP is a widely accepted wire protocol and there are many client libraries available such as C++, Java, Ruby, .Net, Python, etc.
If the distributed service is flaky and is not always online, you can use a rock solid message queue to persist the messages. The messages are then delivered when the distributed service comes back to life.
If the distributed service has low latency and the client service generates more requests than the distributed service can handle, a queue will hold the messages until the distributed service can process them while allowing the client to process uninhibited.
If requests need to be distributed to multiple services, an exchange will take care the important details such as delivering the message to each distributed service once and only once.
I don't recommend using message queues when you need a synchronous call to a remote service. Message queues are inherently asynchronous.
Not a programming language, but a useful tool when you're programming with message queues is Promela/Spin. It's designed to identify deadlocks or other concurrency issues that might arise from a distributed system. Certainly looking at the sort of issues it can help identify will give you an idea of the sort of problems you might encounter.
Not sure what its written in but Eclipse IDE seems to have some sort of message queue system. Whenever its busy (read, lagging... alot) you see a message saying "performing blank before user operation". Eclipse is Open Source. I am not sure where to download the source code.. but I believe it comes with the installation - http://www.eclipse.org/downloads/. latest is 3.6 (helios)

Enterprise Service Bus Terminology

Can anyone explain at a beginner-intermediate level the terminology of "bus", "transport" and "endpoint" in the context of an enterprise service bus? I'm a C# developer with a few years experience now, but only just starting working with an ESB.
It seems that the "bus" is effectively a queue to which you can send and receive messages. I'm fine with that. However I'm working on some existing code using NServiceBus and I think if I grokked the "endpoint" and "transport" terminology I'd make a massive leap forward in my understanding.
Let me try to clarify those terms to you:
Bus in context of ESB architecture should not be considered as simple queue for message dispatching. To allow integration of different services, ESB provides much more. Important additional functionalities of ESB:
Routing. Messages can be routed to different services, depending on message content or endpoint specification.
Message Transformations/Mediations between different formats
Transport protocol conversion. ESB should be able to seamlessly integrate applications
that use different transport protocols (JMS, HTTP/S, pure TCP, etc.)
Message enhancement. Messages can be enriched with missing data before further processing.
Security
Management and Monitoring
Those functionalites are provided by services that operate within ESB. Services connect to each other via endpoints - uniform, unique "addresses". Messages dispatched between endpoints are using unified transport (method/protocol that encapsulates message's payload). Application that natively use different transport, need to connect to ESB via suitable adapter - service that will provide necessary transport conversion. This way applications that use ESB are decoupled from each other and don't need to provide conversions themselves.
Of course, those are only very brief descriptions of terms. Remember, Enterprise Service Bus is only catch-term for specific kind of architecture (or concept), but it is not standardized in any way. So specific implementations can be very different from each other.
If you are interested in standardized ESB, you can take a look at JBI (Java Bussiness Integration). There are several open-source implementations of JBI avalable, among them Apache ServiceMix, Mule, OpenESB. Very good introduction to ESB technologies is presented in "Open Source ESBs in Action" book published by Manning.
I would recommend looking at resources related to Enterprise Application Integration (EAI), which revolves around the ESB and various models and patterns used to integrate solutions. Think of it is a GoF for ESB architectures:
http://www.enterpriseintegrationpatterns.com/
and
http://www.enterpriseintegrationpatterns.com/toc.html
All of these patterns would give you an idea of what people use ESB's to achieve and the patterns are useful for providing common pitfalls of do-it-yourself ESB integration. I've learned an immense amount through that book and through people that source from it.