Data integration vs service integration, when each one fits in a business workflow - integration

An interesting question is how pentaho data integration fits and perhaps would be useful in an environment that involves BPM (bonita software) and ESB Enterprise Service Bus (Mule).
I didn't find any documentation about it. Maybe I`m misunderstood these two conceptions but I really would like to know how and when I can use these two approaches.
To be more clear, how I can use pentaho data integration to improve the business workflow and be a tool to work together with an ESB platform ?

It sounds like a very generic question about how to do system integrations.
You will have your high level (business perspective) business processes guiding your company, probably gathering data from and showing business data through Pentaho and the ESB will be in charge of handling how the systems used by the business processes communicates with each other.
I wrote some time ago these slides for jBPM5 but I think it will help you to understand how all these technologies fits:
http://www.slideshare.net/salaboy/jbpm5-community-training-module-25-bpm-for-developers
Cheers

There is an integration of Mule and PDI, but it doesnt appear to have been used much. see here: http://jira.pentaho.com/browse/PDI-7416
There is also an enormous overlap in the tools. Obviously Mule contains ETL functionality - and similarly PDI can do ESB like operations. So there is good sense in integrating and using the best of both!
Certainly mule/ESB seems to be where it's at with the whole "data in motion" concept.

Related

Training and experimentation system for developing software interfacing with SAP ERP

I am trying to find a good place to start for a potentially SAP related project.
Suppose I wanted to develop stand-alone software that could connect with an SAP ERP (S4/HANA) system and analyze material management and accounting data. What resources would I need to accomplish that? Where could I get an SDK/the proper training for this? How would I get a system to experiment on? I've tried Googling this question but there doesn't seem to be clear authoritative information on this and what I find also seems to be mostly about SAP Business One, which is a different product.
Insights would be much appreciated!
Thank you in advance,
Joshua
You must use REST/OData to connect to ECC and Hana S/4.
Currently, SAP has around 800+ REST APIs for SD/MM/PP/FICO, and almost every quarter, new APIs are added to Hana S/4.
You can check all the available APIs # https://api.sap.com/package/SAPS4HANACloud/all
You can check out the demo below if you want to view an application that provides similar functionalities.
https://demo.inoerp.com:8090
Select Hana SandBox. This is a single app that provides similar functionalities for Oracle, SAP, Dynamics etc.
If you need an API that is not yet published. You can also create your own APIs using ABAP
Regarding technology - You can use anything Java(Springboot/ADF), C# .net, node.js. You will be fine as long as you stay away from the SAP technology stack (i.e., ABAP and UI5 :-))

Enterprise Service Bus and BPM

I am a Java professional. I have a requirement to implement Enterprise Service Bus(ESB). There are quite a few ESB providers are available in the market.I am confusing a bit among those.
Please suggest which one is the best to implement ESB and BPM. I am expecting to have an open source solution with rich documentation and easy to understand with number of examples and scenarios so that we can start from scratch and implement it without any hassle.
Yes the number of options can be confusing. It's highly depends on your use case and what you're trying to achieve. Without much context, I can only share with you the recent tutorial I wrote around Bonita (A Bpm platform) and Apache Camel (A lightweight ESB): http://community.bonitasoft.com/blog/enhance-your-bonita-applications-apache-camel
In this tutorial, I explain how to trigger new process instances on new incoming emails event.
Cheers
WSO2 has a good option with their Enterprise Integratir. This combines ESB, BPM and related functionality.
Check here for more: http://wso2.com/integration
However this is one of those questions that is purely opinion based.

Difference between JBPM and BPEL/ESB

what is difference between JBPM and BPEL(and ESB)?
Would you please explain them?
RGDS
I am not much familiar with JBPM. But it seems to be a Business work flow which can work with java services or basically java based process. Not only with web services.
BPEL is a standard to write work follows with web services. Always BPEL language used to integrate the web services and define processes based on that. Here is an sample I have written for that[1].
ESB is primarily used for mediation and transform messages. When you integrate different types of systems, the message flow between them may vary. So people can use ESB as a mediator. And also some ESBs provides service integration as well. WSO2 ESB[2] is such an ESB you can use.
[1] http://wso2.org/library/articles/2011/05/integrate-business-rules-bpel
[2] http://wso2.org/library/esb
Exactly, ESB + BPEL is a technical solution for an integration problem. If you want to use jBPM5 just to do integrations thats fine and you probably will use jBPM5 with an ESB for all your mediation and transformation of your messages. The power of BPMN2, a standard notation to describe business processes will help you to describe more high level/business oriented scenarios than just simple system integrations. The concept of human interaction is heavily embedded in the language and in the jBPM5 infrastructure. Think about the fact that your models (business processes) can be shared and understood by business/non technical people and they will be able to validate, improve and change those definitions when the business reality changes.
Hope it helps!
jBPM is BPMN based. This is a java based solution to your workflow problem.
BPEL is also solves the workflow problem, but the approach is entirely different. It is web service based.
BPEL from a syntax perspective is more complex than BPMN but is considered more extensive.
The right comparison should actually be between BPMN and BPEL I guess.
Similarity
>Both can be used for orchestration
difference in terms of technology.
JBPM has BPMN2.0 Notation for Workflow designer and workflow XML it generate is BPM2.0 compliance(which means you can import it in any BPMN2.0 tool) .It is assumed to be Product Analyst friendly whereas BPEL has its own specifications and its considered more developer oriented
BPM should only be used where there is a human task otherwise ESB fulfills everything from orchestration to transformation to Rules to CEP

Anyone have special issue using WSO2? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
Anyone have any special issue when using WSO2. I'm still looking forward should I use WSO2 or Talend for Integration framework. Anyone have experience using one of them or both?
Selecting an enterprise service bus (ESB) for integration work needs to be done based on few important factors.
Performance - as the ESB becomes the central hub, it better perform
Kinds of services to be integrated - do you have all adopters that you need supported by the ESB, like SAP, FIX, HL7 etc.
Integration language - for e.g. do you need to write Java code or is it easy to deal with using config languages like XML
Support for monitoring and management
Development tools
WSO2 ESB is strong in all these fronts, as proven with eBay case study and more over it is open source. You should compare the equivalent aspects in Talend before making a choice.
We tried WSO2 stack and found we were spending way too much time in just trying to get all the parts and pieces to work together and get installed. Then once we got it up and running, we couldn't figure out how to expose simple database calls as RESTful services. After a few weeks we gave up and tried Talend ESB. We found a demo and lots of others using it and great videos. We got installed and our hello world rest service up in less than 1 hour.
So if you are making a smaller solution, I strongly recommend Talend so far. It also handles load balancing and high performance; however, we haven't gotten that far. BTW, at work I use Oracle SOA suite, and I am a SOA Architect with more than 5 major SOA platforms under my belt such as TIBCO, WebMethods, and GenTran.
Just my humble 2 cents.
Ebay the esb casestudy is a case study which shows how eBay using WSO2 ESB to process more 1 Billion transactions per day...
I faced some compatibility issues while using with CXF client stack.
some of the WSO2 ESB / Synapse examples doesn’t work with CXF client while they work fine with Axis2 client. You have to make sure you are using WSO2 ESB supported namespaces for WS-Security , WS-Addressing m WS-Trust etc on client side. As WSO2 ESB/ Synapse is based on Axis2, If you services and/or clients are based on CXF or Sun Metro, it may be easy and less problematic using Talend or Fuse ESBs.
Being based on WSO2 carbon platform WSO2 esb provides a lot of security features such as authentication, authorization, confidentiality etc .. as well.
Following articles shows how to do the service integration and apply security with WSO2 ESB.
WSO2 ESB by Example - Service Chaining,
Integrate Business Rules with BPEL,
Securing Web Service Integration
Basically when comes to opensource ESBs available, wso2 offers you comprehensive package which you can utilize to solve your day today business use cases, by glance wso2 esb comes with variety of features such as.
Connecting Anything to Anything
Routing, Mediation & Transformation
Message, Service, API & Security Gateway
High Performance, High Availability, Scalability & Stability
pls visit:
http://wso2.org/library/articles/2012/03/performance-benchmarking-wso2-esb-different-message-transfer-mechanisms
Lightweight, Developer Friendly and Easy to Deploy
Manage & Monitor
for further details please visit to http://wso2.com/products/enterprise-service-bus/

What are some viable alternatives to BizTalk Server? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 8 years ago.
Improve this question
In evaluating different systems integration strategies, I've come across some words of encouragement, but also some words of frustration over BizTalk Server.
What are some pros and cons to using BizTalk Server (both from a developer standpoint and a business user), and should companies also consider open source alternatives? What viable alternatives are out there?
EDIT: Jitterbit seems like an interesting choice. Open Source and seems to be nicely engineered. Anyone on here have any experience working with it?
BizTalk Server's key benefit is that it provides a lot of 'plumbing' around deployment, management, performance, and scalability. Through Visual Studio, it also provides a comprehensive framework for developing solutions, often with relatively little code.
The frustration and steep learning curve that others mention often comes from using BizTalk for the wrong purpose and from a misunderstanding about how to work with BizTalk and message-oriented systems in general. The learning curve is not as steep as most people suggest - the essential part of the underlying learning actually focuses on changing thinking from a procedural approach to a stateless message-based approach.
A drawback people often cite is cost. The sticker price can seem to be quite high; however, this is cheap in comparison to the amount you'd spend on developing and supporting features on your own.
Before you consider alternatives, or even consider BizTalk server, you should consider your organization's approach to integration and it's long term goals. BizTalk Server is great in cases where you want to integrate systems using a hub and spoke model where BizTalk orchestrates the activities of many applications.
There are other integration models too - one of the more popular ones is a distributed bus (don't confuse this with the term "Enterprise Service Bus" or ESB). You can also get BizTalk to work as a distributed bus and there are alternative solutions that provide more direct support. One of the alternate solutions is an open source solution called nServiceBus.
When considering whether to use a commercial product like BizTalk, verses something else (open source or developed in house), also consider maintenance and enhancements and the availability of the necessary skill-set in the marketplace.
I wrote some articles that go into more detail about the points I discussed here - here are the links:
Why BizTalk?
Top 10 BizTalk Mistakes
Extensibility Features in BizTalk Server
Open Source Integration with nServiceBus
My experience with BizTalk was basically a frustrating waste of time.
There are so many edge cases and weird little business logic tweaks you have to make when you are doing B2B data integration (which is probably the hardest part of any enterprise application) that you just need to roll your own solution.
How hard is it to parse data files and convert them to a different format? Not that hard. Unless you're trying to inject a bloated middleware system like Biztalk into the middle of it.
As a BizTalk consultant I have to agree at least partly with Eric Z Beard, there are a lot of edge cases that take up alot of time. But quite a few scenarios are handled extremly smooth as well, so it all depends IMO. But when you (Eric) call BizTalk bloated I have to disagree! We've found that the performance and reliability is excellent, it's flexible and comes with a lot of good adapters out of the box.
BizTalk needs to be used correctly,
I am a BizTalk developer and my experience with BizTalk is quite good.
Its reliable, performant, scalable, contains a lot of built in architectural patterns and build in components to make integration easy and fast, you get security, retries, secondary transports, validation, transformation etc... and what ever you dont have build in with BizTalk you can easily customized with .NET code, its basically a hard earned integration system and you get all this in one box.
BUT you need to know how to implement BizTalk correctly, not once I came across solutions that where implemented and often also architected incorrectly.
but the real benefit of BizTalk is that you can implement small solutions and scale up whilst most other integration systems from big vendors will only sell a whole integration pack which can cost much more.
BizTalk is considered the most complicated server from the house of Microsoft.
So any body saying BizTalk is not good dosent know BizTalk period.
We evaluated BizTalk at our company and were really disappointed.
We are using IBM WebSphere Transformation Extender (which has lots of (other) problems, too) and the mapping tool of BizTalk is a joke in comparison to WTX.
The graphical tool is not really usable for complex mappings (we have schemas with a few hundred fields in repeating groups) and if you do more than the usual "concat first name and last name to name" mappings, you will be tired of the graphical approach (for example the arguments of the functoids in the graphical mapper are not labeled and the order in which you connect fields to these arguments is important).
The XSLT-Mapper was usable but not really convincing, and even the microsoft rep told us to use a tool like XMLSpy for XSLT and load the resulting XSL file into BizTalk.
A third approach to mapping is to use C#-Code for the mapping, which was not acceptable for us as a general approach (we don't want to teach everyone C#).
In addition to the mapping tool we did not like the deployment in BizTalk. In order to deploy your process, you need to make lots of settings in different tools and places. We had hoped to find a mechanism like a WAR file for Java Web Applications in BizTalk, so that you can give one archive for your whole process solution to your administrator and he can deploy it.
We've been using BizTalk since version 2004, and now have a mix of versions 2006 R2 and 2004 running. I found that the learning curve was quite severe, and development time for solutions is not always quick. Those are definitely shortcomings. Where BizTalk really excels is in its fault tolerance, gauranteed delivery, and performance. You can rest assured that data will not get lost. Retry functionality and fault tolerance robustness is baked in so generally speaking if systems are down BizTalk will handle that and successful delivery will occur once systems come back on line. All these issues such as downtime, etc that are important in an integration scenario are handled by BizTalk.
Further, generally speaking when developing solutions BizTalk abstracts the communication protocols and data formats of the native systems by dealing with everything as xml, so when developing solutions, you typically don't have to wrote code specific to those systems, you use the BizTalk xml framework.
In the last year, we've implemented a java open source engine called Mirth for our HL7 routing. I found that for HL7 purposes, the HL7 adaptor for BizTalk is a challange to work with. Management dicated that we use Mirth for HL7 routing. Where BizTalk falls down in terms of learning curve, Mirth makes up. It is far easier to develop a solution. The problem with mirth is that it doesn't really have any gauranteed delivery. Most of the adaptors (except for hl7) have no retry functionality so if you wanted that you'd have to write your own. Second, Mirth can lose date if it goes down. I would call it very easy to use (although there is no documentation) but I'd be hard pressed to call it an enterprise solution. I'm going to check out jitterbit which was mentioned by someone else.
We used BizTalk for a couple of years, but gave it up for our own custom framework that allowed more flexibility.
There is always Sun's (now Oracle) OpenESB framework. Its generally speaking a smaller, lighter version of Biztalk but with roughly all the same features.
You do get to write more code with it, though.
Its Open Source as well.
In the OSS space (though I've never used them as a BizTalk replacement personally - this is anecdotal) you can use one of the Java/J2EE Messaging engines such as OpenMQ (which is the Sun enterprise one rebadged and without support). If you need Orchestration / Choreography (i.e. SOA/ESB pieces) on top of this, you could look into something like Apache Mule
My experience with BizTalk and doing B2B integrations is that most organizations do not truly do schema first design or fully understand xml standards for that matter. Most tend to weave objects and hope they materialize into meaninful schemas. In an enterprise environment, this is backwards.
BizTalk does have a learning curve, but once you get it you are rewarded with durability, performance, true scalability, and extensibility. Like most have said though, it best to make sure it meets your needs and contort your needs to BizTalk.
In the past I have worked with BizTalk 2004 through 2009, and another product called webMethods.
I have no direct experience with JitterBit, but I have heard very good things from coworkers.
I came across Apatar (unable to post url, but Google finds it) while looking for a solution cheaper than BizTalk. I have yet to try this out.
My last company had many problems with BizTalk being too complex and ridged, but I can’t help but think this was mainly down to the implementation the consultant did.