Netflix Suro key features - message-queue

I'm trying to wrap my head around Netflix Suro, but there isn't much information about it in the net.
What are the key features of the product and how is it different from Apache Kafka or RabbitMQ?
For me it looks exactly like many others queue servers with some pre-built connectors to 3rd party storages/queues. I think I may really missing a point here. Maybe it's a perfect instrument for cloud based infrastructure or offers no-brain integration with other Netflix infrastructure when others cannot.
Can somebody shed some light on this piece of technology?

Recently Netflix has archived this project, judging by their last commit.
It seems that they have replaced it completely with Kafka in their data pipeline stack.

Not much difference, just a business decision I guess
some pointers:
https://github.com/Netflix/suro/wiki/FAQ

Related

Enterprise Service Bus and BPM

I am a Java professional. I have a requirement to implement Enterprise Service Bus(ESB). There are quite a few ESB providers are available in the market.I am confusing a bit among those.
Please suggest which one is the best to implement ESB and BPM. I am expecting to have an open source solution with rich documentation and easy to understand with number of examples and scenarios so that we can start from scratch and implement it without any hassle.
Yes the number of options can be confusing. It's highly depends on your use case and what you're trying to achieve. Without much context, I can only share with you the recent tutorial I wrote around Bonita (A Bpm platform) and Apache Camel (A lightweight ESB): http://community.bonitasoft.com/blog/enhance-your-bonita-applications-apache-camel
In this tutorial, I explain how to trigger new process instances on new incoming emails event.
Cheers
WSO2 has a good option with their Enterprise Integratir. This combines ESB, BPM and related functionality.
Check here for more: http://wso2.com/integration
However this is one of those questions that is purely opinion based.

Data integration vs service integration, when each one fits in a business workflow

An interesting question is how pentaho data integration fits and perhaps would be useful in an environment that involves BPM (bonita software) and ESB Enterprise Service Bus (Mule).
I didn't find any documentation about it. Maybe I`m misunderstood these two conceptions but I really would like to know how and when I can use these two approaches.
To be more clear, how I can use pentaho data integration to improve the business workflow and be a tool to work together with an ESB platform ?
It sounds like a very generic question about how to do system integrations.
You will have your high level (business perspective) business processes guiding your company, probably gathering data from and showing business data through Pentaho and the ESB will be in charge of handling how the systems used by the business processes communicates with each other.
I wrote some time ago these slides for jBPM5 but I think it will help you to understand how all these technologies fits:
http://www.slideshare.net/salaboy/jbpm5-community-training-module-25-bpm-for-developers
Cheers
There is an integration of Mule and PDI, but it doesnt appear to have been used much. see here: http://jira.pentaho.com/browse/PDI-7416
There is also an enormous overlap in the tools. Obviously Mule contains ETL functionality - and similarly PDI can do ESB like operations. So there is good sense in integrating and using the best of both!
Certainly mule/ESB seems to be where it's at with the whole "data in motion" concept.

What level of technical skills/expertise is required for implementing CAS SSO?

I am in a process of integrating several websites/content management tools to try out some of my ideas. Over the past couple of months, I have discovered tons of very helpful stuff, and it's great. I'm setting everything up just fine. To name a few it's: phpwebsite, moodle, livezilla, etc.
The problem is that I am doing everything myself and do lack technical knowledge.
I do have a strong programming background from way back then, which is no longer applicable. However, I seem to be managing do dig up HTML, PHP and JavaScript codes more or less OK, and things move forward pretty well.
Now it came to the need to implement SSO between a few of my systems. I like what I read about CAS, but the more I read the scarier it sounds:). I feel that I lack way too much technical expertise to be able to implement it myself. It looks like that it is not just simple logica installation and configuration as with most of the things I delt in a past two month, but kind overly complicated.
Should I risk it? What problems am I gonna face?
All the discussions I have been able to find so far are way to technical, not user friendly at all.
Please help me to build up some courage:)
Thanks,
Oleg
Looks like you're doing very well, there are many people ready to help you. Take a chance and you will learn a lot in the process.
You can at least remove the burdeon of installing the CAS server by using the cloud provider: http://www.casinthecloud.com (free servers are available for tests). For all the integrated applications, it's still up to you, but it should be in your technology using the right CAS client.
There is demo implementation here,
Hope that helps

What repository products are available for keeping track of web services?

My organization is starting to take SOA seriously but before we jump in one of the components we seem to be missing is a rock solid repository for tracking these services across the enterprise. Can anyone suggest a product that they have worked with? If the product is also an ESB please mention that in your answer.
You might like to take a look at IBM's WebSphere Service Registry and Repository. It does what you describe (with governance abilities as well), and integrates nicely with IBM's ESB products (although it not one itself).
Please feel free to get in touch if you want to ask any questions.
Disclaimer: I work for IBM as a WebSphere Consultant. However, I am not speaking for them in an official capacity.
UDDI functionality ships with Windows Server if all you need is a registry.
I also have worked for IBM and I would stay away from WSRR - it is buggy, immature, expensive and overly complex. I would not recommend it.

What are some viable alternatives to BizTalk Server? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
In evaluating different systems integration strategies, I've come across some words of encouragement, but also some words of frustration over BizTalk Server.
What are some pros and cons to using BizTalk Server (both from a developer standpoint and a business user), and should companies also consider open source alternatives? What viable alternatives are out there?
EDIT: Jitterbit seems like an interesting choice. Open Source and seems to be nicely engineered. Anyone on here have any experience working with it?
BizTalk Server's key benefit is that it provides a lot of 'plumbing' around deployment, management, performance, and scalability. Through Visual Studio, it also provides a comprehensive framework for developing solutions, often with relatively little code.
The frustration and steep learning curve that others mention often comes from using BizTalk for the wrong purpose and from a misunderstanding about how to work with BizTalk and message-oriented systems in general. The learning curve is not as steep as most people suggest - the essential part of the underlying learning actually focuses on changing thinking from a procedural approach to a stateless message-based approach.
A drawback people often cite is cost. The sticker price can seem to be quite high; however, this is cheap in comparison to the amount you'd spend on developing and supporting features on your own.
Before you consider alternatives, or even consider BizTalk server, you should consider your organization's approach to integration and it's long term goals. BizTalk Server is great in cases where you want to integrate systems using a hub and spoke model where BizTalk orchestrates the activities of many applications.
There are other integration models too - one of the more popular ones is a distributed bus (don't confuse this with the term "Enterprise Service Bus" or ESB). You can also get BizTalk to work as a distributed bus and there are alternative solutions that provide more direct support. One of the alternate solutions is an open source solution called nServiceBus.
When considering whether to use a commercial product like BizTalk, verses something else (open source or developed in house), also consider maintenance and enhancements and the availability of the necessary skill-set in the marketplace.
I wrote some articles that go into more detail about the points I discussed here - here are the links:
Why BizTalk?
Top 10 BizTalk Mistakes
Extensibility Features in BizTalk Server
Open Source Integration with nServiceBus
My experience with BizTalk was basically a frustrating waste of time.
There are so many edge cases and weird little business logic tweaks you have to make when you are doing B2B data integration (which is probably the hardest part of any enterprise application) that you just need to roll your own solution.
How hard is it to parse data files and convert them to a different format? Not that hard. Unless you're trying to inject a bloated middleware system like Biztalk into the middle of it.
As a BizTalk consultant I have to agree at least partly with Eric Z Beard, there are a lot of edge cases that take up alot of time. But quite a few scenarios are handled extremly smooth as well, so it all depends IMO. But when you (Eric) call BizTalk bloated I have to disagree! We've found that the performance and reliability is excellent, it's flexible and comes with a lot of good adapters out of the box.
BizTalk needs to be used correctly,
I am a BizTalk developer and my experience with BizTalk is quite good.
Its reliable, performant, scalable, contains a lot of built in architectural patterns and build in components to make integration easy and fast, you get security, retries, secondary transports, validation, transformation etc... and what ever you dont have build in with BizTalk you can easily customized with .NET code, its basically a hard earned integration system and you get all this in one box.
BUT you need to know how to implement BizTalk correctly, not once I came across solutions that where implemented and often also architected incorrectly.
but the real benefit of BizTalk is that you can implement small solutions and scale up whilst most other integration systems from big vendors will only sell a whole integration pack which can cost much more.
BizTalk is considered the most complicated server from the house of Microsoft.
So any body saying BizTalk is not good dosent know BizTalk period.
We evaluated BizTalk at our company and were really disappointed.
We are using IBM WebSphere Transformation Extender (which has lots of (other) problems, too) and the mapping tool of BizTalk is a joke in comparison to WTX.
The graphical tool is not really usable for complex mappings (we have schemas with a few hundred fields in repeating groups) and if you do more than the usual "concat first name and last name to name" mappings, you will be tired of the graphical approach (for example the arguments of the functoids in the graphical mapper are not labeled and the order in which you connect fields to these arguments is important).
The XSLT-Mapper was usable but not really convincing, and even the microsoft rep told us to use a tool like XMLSpy for XSLT and load the resulting XSL file into BizTalk.
A third approach to mapping is to use C#-Code for the mapping, which was not acceptable for us as a general approach (we don't want to teach everyone C#).
In addition to the mapping tool we did not like the deployment in BizTalk. In order to deploy your process, you need to make lots of settings in different tools and places. We had hoped to find a mechanism like a WAR file for Java Web Applications in BizTalk, so that you can give one archive for your whole process solution to your administrator and he can deploy it.
We've been using BizTalk since version 2004, and now have a mix of versions 2006 R2 and 2004 running. I found that the learning curve was quite severe, and development time for solutions is not always quick. Those are definitely shortcomings. Where BizTalk really excels is in its fault tolerance, gauranteed delivery, and performance. You can rest assured that data will not get lost. Retry functionality and fault tolerance robustness is baked in so generally speaking if systems are down BizTalk will handle that and successful delivery will occur once systems come back on line. All these issues such as downtime, etc that are important in an integration scenario are handled by BizTalk.
Further, generally speaking when developing solutions BizTalk abstracts the communication protocols and data formats of the native systems by dealing with everything as xml, so when developing solutions, you typically don't have to wrote code specific to those systems, you use the BizTalk xml framework.
In the last year, we've implemented a java open source engine called Mirth for our HL7 routing. I found that for HL7 purposes, the HL7 adaptor for BizTalk is a challange to work with. Management dicated that we use Mirth for HL7 routing. Where BizTalk falls down in terms of learning curve, Mirth makes up. It is far easier to develop a solution. The problem with mirth is that it doesn't really have any gauranteed delivery. Most of the adaptors (except for hl7) have no retry functionality so if you wanted that you'd have to write your own. Second, Mirth can lose date if it goes down. I would call it very easy to use (although there is no documentation) but I'd be hard pressed to call it an enterprise solution. I'm going to check out jitterbit which was mentioned by someone else.
We used BizTalk for a couple of years, but gave it up for our own custom framework that allowed more flexibility.
There is always Sun's (now Oracle) OpenESB framework. Its generally speaking a smaller, lighter version of Biztalk but with roughly all the same features.
You do get to write more code with it, though.
Its Open Source as well.
In the OSS space (though I've never used them as a BizTalk replacement personally - this is anecdotal) you can use one of the Java/J2EE Messaging engines such as OpenMQ (which is the Sun enterprise one rebadged and without support). If you need Orchestration / Choreography (i.e. SOA/ESB pieces) on top of this, you could look into something like Apache Mule
My experience with BizTalk and doing B2B integrations is that most organizations do not truly do schema first design or fully understand xml standards for that matter. Most tend to weave objects and hope they materialize into meaninful schemas. In an enterprise environment, this is backwards.
BizTalk does have a learning curve, but once you get it you are rewarded with durability, performance, true scalability, and extensibility. Like most have said though, it best to make sure it meets your needs and contort your needs to BizTalk.
In the past I have worked with BizTalk 2004 through 2009, and another product called webMethods.
I have no direct experience with JitterBit, but I have heard very good things from coworkers.
I came across Apatar (unable to post url, but Google finds it) while looking for a solution cheaper than BizTalk. I have yet to try this out.
My last company had many problems with BizTalk being too complex and ridged, but I can’t help but think this was mainly down to the implementation the consultant did.