I have a BizTalk 2010 solution that polls a database table, retrieves unprocessed records, does a transform with a map and call a 3rd party service. The happy path is working.
Here is the workflow:
Receive Location/Port from GetUnprocessedCustomers stored procedure: Poll SQL Server 2008 with WCF-SQL adapter by calling stored proc that returns unprocessed customers (WHERE IsProcessed = 0)
SendPort to 3rd Party web service: Filtered to ReceivePortName == with an outbound map to convert message returned from the above mentioned stored proc to service schema
SendPort to UpdateIsProcessed stored proc: Filtered to MessageType == with an outbound map to convert service response to stored procedure call that will update IsProcessed = 1.
I would like to catch the following possible exceptions, create a message with exception info and call an internal exception handling service.
Database or stored procedure is not accessible.
3rd party service is not accessible.
I was able to handle number 2 above by enabling Failed Message routing. Thanks to Greg.Forsythe.
My question is:
How can I create a generic solution that will capture all exceptions and send exception info (such as exception datetime, message, stack trace etc.) to an internal service?
I was able to get an answer on the BizTalk General Forum. The short answer is "There is no way to catch all exceptions". Hit the link for some options.
Related
I searched but didn't find anything specific to the issue I was having. My apologies if I overlooked it.
I have a scenario where I'm pulling data from an oData source and everything works fine. I have the oData source in a loop where I loop over each of our different companies and pull the data for that company. I'm doing this to reduce the volume of data that is being returned in each oData call.
Everything works fine as long as there is data being returned from the oData call. In the attached photo you can see that the call is being made and data is being returned.
But when the oData Service is called with parameters/filters that return no data I get the following and that is when the VS_NEEDSNEWMETADATA error is thrown. The image below shows what I get when no data is returned.
So the issue isn't that I have invalid metadata due to changes made to the service (adding/removing fields). It's that nothing is being returned so there is no Metadata. Now it's possible that this is an issue with the system that I'm pulling the oData from (SAP S4) and they way that system surfaces the oData call when there is no data??
Either way, trying to figure out a way to handle this within SSIS. I tried to set validate external metadata = false but the package still fails. I could also fix this by excluding those companies in the script but then once data does exists I'd have to remember to update the scripts and redeploy.
Ideas suggestions?
You've hit the nail on the head except your metadata is changing in the no data case because the shape is different - at least from the SSIS engine perspective.
You basically need to double invoke the OData source. First invocation will simply identify is data returned. If that evaluation is true, only then does the Data Flow start running.
How can you evaluate whether your OData will return data? You're likely looking at a Script Task and C#/VB.NET and hopefully the SAP data returns don't vary between this call and next call.
If that's the case, then you need to define an OnError event handler for the Data Flow, mark the System scoped variable Propagate to False and maybe even force a success status to be returned to get the loop to continue.
Here is an image of a common structure we have in a few of our BizTalk orchestrations:
So when we GET information from an API and the account doesn't exist, we terminate the orchestration. This works fine and I can see the orchestration doing this and terminating when it should, but what I don't understand is why do I see the suspended message from the GET in the console? Since the exception handling works, shouldn't this stop an error showing in the console?
As a way around this, I've considered using a pipeline component to check the response message, and if it contains what would be ignored, just return null in place of the message. Would this have the desired effect? I'm more interested as to why I see the suspended messages in the console.
Yes, this is a known issue with the WCF-WebHttp adapter, that has to do with the fact that it throws it back as a SOAP formatted error, but without setting the MessageType context property (see my blog post and look for Bug: BizTalk WCF-WebHttp adapter does not set the message type on error). So although the exception is thrown back at the Orchestration and can be handled there, the message is not as BizTalk does not know what type it is and it suspends.
A workaround we use is
To set Enable Routing for failed messages on the send port
To have a send port that subscribes to all the response messages from that send port using the BTS.SPName = xxxx filter, and send port sends it to a custom NULL adapter (throw the message away), and yes, your Orchestration will still get the good response as well as the exceptions.
Update the send port that handles routing errors from other send ports that we do want to go to our exception handing to exclude those send ports which we are handling via an Orchestration.
I am new to SSIS. I want to basically create tasks in SSIS to do the following
Create a task to pull data from a table
Create a task to call Webservice. Basically I need to call the webservice couple of times to get the job done. Following are the steps.
a. Call a webservice to get a session token
b. Call a webservice to get a session service token by passing session token that you get in step 1
c. Call a webervice method by passing the service token as well as uploading the data.
d. Call a webservice to end the session
What is the best possible way to do it?
You create a data flow task for 1 to get the data. Then for 2, you can do that in a script task for a and b, you can use either c# or vb. This is pretty much just programming. Then step c, it's not very clear what you do here with the data. Potentially you use a data flow task connected to the script task you created for step a and b. In the data flow you do the uploading. Then you create another script task for step d and it connects to the data flow task you created for step c.
I have implemented a WCF service (written in C#) that serves as a backend to a Webapp. The WCF is supposed to call stored procedures from a SQL database and return the data via JSON so it can be displayed in the WebApp. My issue is that the WCF service sometimes will return an empty JSON array. If I use the same parameters and execute the stored procedure using SQL Management Studio I get hundreds of rows returned. How can I debug this issue? I've been looking at Chrome's dev tools and Firebug and they both show that the request is being responded too (it's just empty).
Thanks for the help.
There's three main places you can debug:
Use the WCF test client to call the service directly rather than from front-end code.
Use SQL Server profiler to make sure your service is calling the sprocs with the parameters you expect.
Attach the debugger to your WCF service and step through the code.
Attach a debugger to the WCF service and see what happens to the result set provided by the DB.
With the default behaviour when a message is not routed the message is lost.
You could create a queue that receives all messages by using the # for the routing key in the binding. Then create a process that handles all the non routed messages. The process will have to connect to the queue and receive all messages and somehow know whether they have been routed or not. What you will need to do is call the management plugin cli to return all the bindings for an exchange. Parse that result to get you this list of the bindings for the exchange and ignore any incoming message which matches the bindings. Then you can just process the ones that never got routed in the first place. You could even read them back to another queue for a worker process to consume.
have a look at this for information on the management plugin cli.
If you prefer to use the rabbitmqctl you could use
sudo rabbitmqclt report
to get a report that would need to be parsed to get all the bindings. See here