Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I have scenario for which I am looking for a message queue service which supports below:
Ease to Use
Very high in performance
Message once read shouldn't be available for other consumers.
Should have capability to delete the message once read.
Message once published should not get dropped.
The scenario which I have is described below:
There are many publishers.
There will be many consumers.
Queuing server and consumers residing on same machine, but publishers are residing on different machines.
Please let me know best queuing service apart from Rabbitmq and sqs satisfying above points
I would recommend Apache Kafka: http://kafka.apache.org/
If you want to know a comparison between Kafka and RabbitMQ you should read this article: http://www.quora.com/RabbitMQ/RabbitMQ-vs-Kafka-which-one-for-durable-messaging-with-good-query-features
Also, you should take a look to this: ActiveMQ or RabbitMQ or ZeroMQ or
Kafka as far I know is mainly meant for real data propagation and I think my requirement doesn't require something like kafka. I have used SQS but the only problem I have with SQS is high latency. Publisher pushed message to queue and consumer keep on polling for new message, this implementation is hitting me with very high latency. My requirement is simple as follows:
Queue service should have high availability and reliability like SQS
Latency should be very high lets say not more than 10ms. (here 10ms includes publishing and receiving the message).
Also my message size is very small say not more than 20-30 bytes.
I have thought of using redis, in which I will be pushing the messages to a list and workers will keep on popping them back to back till list becomes empty, but I have not done any benchmarking on that. So here I really need suggestion so I go in right direction.
Thanks,
For some of my system integration projects I met the MQ-tasks. Several rich costumers wants the production solutions like IBM WebSphere MQ, but I think it's too much expressive and difficult.
I found and used the simple and stable analog: e-mail server.
All integrated systems got the local e-mail boxes. Messages are e-mail, with command-code in subject and json in attachments. E-mail server listen and dispatch all queues, to recipients or to groups of them. E-mail protocols are stable and all developers know a lot of tools to work with it. Sysadmins and testers use the simple e-mail clients for testing and auditing. All e-mail servers have a logging tools.
It's best and easy solution, and I suggest it for most of integration projects.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed last year.
Improve this question
I am currently facing an issue sending emails from sendgrid to oulook or hotmail users:
550 5.7.1 Unfortunately, messages from [XX.XX.XX.XX] weren't sent.
Please contact your Internet service provider since part of their
network is on our block list (S3140). You can also refer your provider
to http://mail.live.com/mail/troubleshooting.aspx#errors.
Sengrid support keeps saying that it is my responsability to maintain dedicated IP reputation. The thing is that this IP was added to my account some weeks ago and I am not able to build any reputation since all emails are dropped by Microsoft. Other providers like Yahoo or Google are handling correctly our emails.
Also, checking IP in block lists, it is not listed in any of SPAM lists.
Can someone help me here?
Many thanks in advance.
Best regards,
Eduardo
From what you've shared, it sounds like you are using a dedicated IP address. The error you're seeing is due to you sending to a Microsoft owned email servers from an IP address they are unfamiliar with. Unfortunately Microsoft tends to auto-block new IPs, not just IPs with a poor sending reputation/history.
Are you sending email from new IP addresses?
IP addresses not previously used to send email typically don't have any reputation built up in our systems. As a result, emails from new IPs are more likely to experience delivery issues. Once the IP has built a reputation for not sending spam, EOP will typically allow for a better email delivery experience.
New IPs that are added for domains that are authenticated under existing SPF records typically experience the added benefit of inheriting some of the domain's sending reputation. If your domain has a good sending reputation new IPs may experience a faster ramp up time. A new IP can expect to be fully ramped within a couple of weeks or sooner depending on volume, list accuracy, and junk email complaint rates.
I recommend completing the Hotmail Blacklist Removal form. I also highly recommend this RackAid post about troubleshooting Hotmail delivery issues, which includes this gem of a pro-tip:
Usually, within 24-48 hours, you should receive notice of their decision. Typically, you receive one of two replies:
Not qualified for mitigation
We have implemented mitigation
If you receive the not qualified response, reply to the email. Ask for clarification as to why your IP did not qualify. You will usually receive a response in 24-48 hours. The response comes from a real tech who will work with you to resolve the issue.
If you IP qualifies for mitigation, you should see the Hotmail email block removed in 24 hours.
You may also want to use Microsoft's delisting portal, although I have seen the most benefit for SG users coming from using the "Hotmail" support form linked above.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm new with FIWARE and I'm disoriented by the large amount of information related to this platform, the amount of components called Generic Enablers that exists. I don't know where to start.
Any advice?
Thanks!
The best place to start would be the FIWARE Developer's Website - this gives you an overview as to what FIWARE is and how the various enablers fit together
Core Context Management holds the state of the system and allows other elements to subscribe to changes of state
IoT and Robots provides sensor data and transforms it into the NGSI standard interface
Processing/Analysis/Visualization provides dashboards and complex event processing
The pillar on the right - API management adds security, as well as interface to systems to publish data externally.
Deployment tools helps the instantiation of a FIWARE system easier
If you are looking for more information, the Tour Guide describes each enabler in depth and holds a series of link to Videos, User Guides, presentation and so on.
If you learn-by-doing the Tutorials will present a series of exercises to build up a "Powered by FIWARE" application - it also describes how various Generic enablers fit together. The first six tutorial concentrate on the use of the Orion Context Broker which forms the central block of FIWARE.
Basically the the NGSI v2 standard provides a common interface to allow a series of disparate building blocks to talk to each other using a common language, these are things you're probably going to need in a generic Smart application, but are not unique to your application - you would be providing the "special sauce" by either creating custom sensors sending context data into the system (i.e. the block at the bottom), or complex processing algorithms which are reading and altering the context state (i.e. the block at the top).
If you want to speed development you can buy-not-build and use the existing free open source generic enablers. The whole system is modular so its easier to experiment and add and refactor things as necessary.
In addition to Jason Fox answer,
Every year the FIWARE foundation organizes summits, that are pretty good. There are tutorials and hands on sessions, and all slides I think that are available here. It could also be a good starting point
On the other hand, most of the FIWARE software components (GEs), are available in Docker HUB. Therefore, if you feel comfortable using Docker you could set up a bunch of FIWARE GEs in a few minutes.
Finally, there is this FIWARE-IOT-Stack website, where you can find a kind of FIWARE architecture for IoT. I do not know if it's official, but in my case it was very useful.
Regards!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm using Angular 4 Typescript and MySQL to create a chat system. I can post messages to my server.
But, I'm having a hard time of understanding how to retrieve the data. When a user subscribes to the chat channel, I don't want him/her to receive all the chat logs at once. Only the most recent. How would I go about this?
Firstly, you need the server to talk to the client instead of HTTP polling. HTML5 has server-sent events (SSE) but there's a much better way.
Websockets is the way forward. This way the server can broadcast new chat messages to all clients. npm's socket.io is a great websockets implementation. Moreover, if websockets is not available on the browser (think crappy browsers) it falls back to HTTP polling and failing that it has further fallbacks for really crappy browsers.
Very happily for you, the socket.io hello world app is a chat app. You can see it here and follow the simple tutorial - https://socket.io/get-started/chat/
You can integrate socket.io into Angular very easily. As events are received by the client from the server you can emit these events to an RxJS Subject and anything can subscribe to this subject.
See this Angular info on using RxJS Subjects for inter-component communication - https://angular.io/guide/component-interaction#parent-and-children-communicate-via-a-service
OK, that's the easy part done. What about getting the messages before the user joined the chat channel. Server side, you should look at message queuing. I'm not greatly experienced here but techs like RabbitMQ are what you need. That should do it but, on the client-side, I would also take a look at RxJS Replay Subjects (a different type of Subject) which can emit events that happened before subscription.
TIP: If you are really going pro you should be using a client-side Angular store such as ngrx. Free tutorials here tend to be rubbish. I would recommend investing in the ngrx tutorial on Udemy. ngrx is what the pro's use.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
I'd like some advice as to the best server-side code that can handle real time data from devices and make decisions based on inputs. A simple example: Suppose I have a web-enabled thermometer, running a light TCP/IP client stack. When the temperature gets to 30 degrees, I want the device to contact the server, and then I want the server to send me an email. I also want the server to be able to send a command to turn on a heater.
The issue at hand here is the ability to start a TCP message from the server, and get through an assortment of arbitrary firewalls and routers, all the way down to the client device. I know that there are 'workarounds' like polling the server for updates, or 'long polling' where I call up to the server, and keep a connection open in case it has something to send. The problem here is bandwidth. Messages are rare, but important, so the headers and handshaking make up 98% of the traffic.
I've been reading up on WebSockets, and it seems like they are exactly what I need, especially when paired with HTML5.
Does anyone know of a ready-to-go server software package that could run on a cloud server, and push data down to my devices using some standardized methods? I really don't want to reinvent the wheel here, and I can't believe I'm the first to try this. I see a few folks doing it with their own proprietary solutions, but I'm more interested in buying a one-stop package.
WebSocket is a valid choice for connecting embedded devices to backend infrastructure due to it's low overhead, low latency and compatibility with Web and general network infrastructure. There is a broad range of server implementations available, i.e. Jetty, node.js based etc.
As an example, here is a demo connecting an Arduino device to a WebSocket server and a browser client showing real-time data in a chart:
https://github.com/tavendo/AutobahnPython/tree/master/examples/wamp/serial2ws
http://www.youtube.com/watch?v=va7j86thW5M
The technology used there, AutobahnPython, is a Python/Twisted based WebSocket implementation that
provides server and client implementation
directly runs on embedded devices like RasperryPi
makes it easy to access sensors connected via serial or CANbus (since Twisted supports that very well)
provides RPC and PubSub messsaging patterns on top of WebSocket
The tech is open-source, so you can roll your own solution. If you look for help/services to get it done for you, contact me;) We also provide Tavendo WebMQ, a virtual appliance (VMware, EC2) which adds features, management UI etc and also includes a REST API to push data to WebSocket clients.
Disclaimer: I am author of Autobahn and work for Tavendo.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have been writing software for several decades now and these days everything is web.
Before the web we had Client Server apps that were basically thick client applications that spoke directly to the database. They had some disadvantages, such as deployment was cumbersome, Did not scale because DB handled all traffic. Of course back then distribution of apps was limited to being on a desktop on a corporate network. The benefits to these apps were that they had fewer layers and were quick to develop.
There are times when the requirements call for an app behind a firewall with a dedicated database and a relatively small amount of clients. I suggest (sometimes on StackOverflow) the old Client/Server type architecture and everybody looks at me like I have 3 legs and 6 arms.
With modern technologies that allow automatic deployments of apps and the tools we have today. Is there a reason this technology is not viable ? Is it that the new generation of developers only know web stuff ?
I can think of at least two large-ish markets where client-server is still big:
Online games and virtual worlds, such as Battlefield or Second Life. Usually you need a thick client plus a connection to a shared server.
Custom-made scientific software. Complex technical or scientific software, especially if it needs an interactive graphical UI that does direct manipulation, is sometimes written in this fashion too.
I'm sure thick clients are still being developed, even today.
Having said that, choosing a web-based architecture is not about the "new generation of developers" only knowing web stuff, you do get a lot of advantages if you can make your application web-based:
Deployment is dead simple. Even with things like ClickOnce, automatic updates, etc, nothing beats simply refreshing the page to get the latest version
You can use something like Silverlight to get 99% of the benefits of a desktop application (in terms of the ability to run code on the client)
Web applications can be made available remotely much more easily than desktop applications (a lot of companies have remote workers these days, setting up a VPN is a pain if all you want to do is access payroll (or whatever))
But at the end of the day, it's all about the right tool for the job. Web applications don't help when you want to write plugins for Office (Word, Outlook, etc), they don't help if you have to control custom hardware (POS terminals, etc - although you could write that into the server in some cases...), and probably a few more cases as well.
We have some Flex apps that communicate with XML based web services that are pretty close to old school Client Server apps. But rather than using SQL, they speak a custom XML language and render SOAP responses.
We currently develop and deploy numerous client/server applications annually. The development is simple and automated. We are not limited to the database technologies we are able to deploy. Client/server deployments are faster for calculations, form updates and reporting. The Web/Cloud based applications are less responsive than an application running on a client station (thick client).
This is because of the distribution of cpu load. Whereas a server side application requires the server to perform all calculations the client side can run this on the local machine. As any system gets more complex the moments that a user has to wait for results increases. These moments of employee time are more expensive as they involve more of the paid employees. These moments add up within an organization as a great many "man hours" over a year.
The problems with updates are solved within our development tool set. Just as when you may open your favorite browser it notices that the version you are using is not the most recent we embed that same process within our client/server applications. In fact we don't give them a choice to update. Since updates may, many times, require database changes we force the update to happen before the user is allowed to run the software.
To improve visibility of the information contained with our custom client/server systems we offer custom developed web sites that have specific applications such as field dispatch or customer support forum integration into the desktop client/server applications. From my perspective I see a complete integration of client server and responsive web applications taking a better position in the years to come.