I wanted to figure out if a SSIS package would run without any problems by implementing a Health Check script task at the beginning of my package. I did this by iterating through the connection managers in the package and calling the acquireconnection method. If there is an exception acquiring connection, I handle it using catch block and assume that this error would not cause the package to fail. The package shows a successful run but an error gets logged in the “Execution Results/Progress” tab of the SSIS and leads to the failure of the corresponding ExecutePackage Task of the Parent package.
If I throw an exception explicitly within the try block, the parent package doesn’t fail. This failure seems to be happening only when an exception gets thrown by the execution of the acquire connection method.
I am looking understand why this happens and if there is a way I can prevent the parent package from failing whenever acquire connection call throws an exception. I am also open to an alternate approach to figure out if all connection managers within a package will work alright.
Related
Managing Exceptions in Indy TIdTCPServer and TIdTCPClient
I have read in other post of this forum and in other places that in Indy components the exceptions (descendant from EIdException) are internally managed.
For this reason, in the TCPServer Execute method there is no need to use a try - catch block.
But what happens if in the TCPServer Execute method another kind of exception occurs?
In my server application, I have put a TIdTCPServer component on the main form, if something bad happens inside its Execute method and I don't manage it, the server stops.
Obviously I need the server running so I use a try - catch block and if there is an exception (any kind of exception), I restart the server.
I don't know if this is the best thing to do.
Sometimes, with the server application running in the IDE, I got the error
Project CallMonitor.exe raised exception class EIdSocketError with message 'Socket Error # 10054 - Connection reset by peer.'.
probably caused by the client running on the same computer (not in the IDE).
The application execution is blocked and If I break, code is stopped in IdStack file where there is the big READ ME!!! about exception 10038.
I press F9 and the application continues running.
For me it is not very clear what happens. My try - catch block is effective in this situation? Or is it useless or harmful?
I have to filter Indy exceptions and other exceptions?
On the client side, in my application there is a TIdTCPClient object in the main form and the connection to the server is managed in the main thread. Then there is another thread to manage communication with the server. In the communication thread Execute method I have a try - catch block and a loop in which I send a request to the server every 2 seconds or when the user asks for data and I decode the server answer.
If there is a EIdReadTimeout exception, i terminate the thread and I restart it. For other exception, I terminate the thread, I disconnect/connect the TIdTCPClient and I restart the thread.
I think this is a different situation because in the server the exception was managed inside the TIdTCPServer thread while in the client the exception is managed in a thread that just uses the TCPClient->Socket to communicate with the server, is it right?
Any answer/comment/suggestion is appreciated.
in the TCPServer Execute method there is no need to use a try - catch block. But what happens if in the TCPServer Execute method another kind of exception occurs?
ANY uncaught exception that is allowed to escape from the OnConnect or OnExecute event back into TIdTCPServer will cause the calling TIdContext thread to stop running. It will close its associated client Connection during its cleanup, firing the OnDisconnect event. And then the OnException event will be fired afterwards.
In my server application, I have put a TIdTCPServer component on the main form, if something bad happens inside its Execute method and I don't manage it, the server stops.
The server as a whole does not stop. Only the calling TIdContext thread is stopped.
Obviously I need the server running so I use a try - catch block and if there is an exception (any kind of exception), I restart the server.
It is not necessary to restart the whole server on any exception. Only on exceptions that invalidate/corrupt something that your app needs to function properly.
Sometimes, with the server application running in the IDE, I got the error
Project CallMonitor.exe raised exception class EIdSocketError with message 'Socket Error # 10054 - Connection reset by peer.'.
That is a perfectly normal socket error. That will not kill your whole server.
If you go into your IDE's debugger settings, there is an option to ignore Indy "silent" exceptions (derived from EIdSilentException, such as EIdConnClosedGracefully). Or, you can also tell the debugger specific exception types to ignore, such as EIdSocketError.
The application execution is blocked ... I press F9 and the application continues running.
Exactly. Not a fatal error. The blockage is only in the debugger, it is letting you examine the exception and decide what to do with it.
For me it is not very clear what happens. My try - catch block is effective in this situation?
The IDE debugger catches exceptions before your code does. When you press F9 to continue execution, the exception will be passed back to your code, and the appropriate catch will handle it normally.
Or is it useless or harmful?
No.
I have to filter Indy exceptions and other exceptions?
IF you catch exceptions at all, handle the ones you need, and then you should re-throw any Indy-specific exceptions you caught (all Indy exceptions are derived from EIdException for easy identification), let the server handle them. If you don't, then you should disconnect the calling Connection yourself and exit the event handler gracefully. Either way, the server will then cleanup the rest as needed.
On the client side, in my application there is a TIdTCPClient object in the main form and the connection to the server is managed in the main thread. Then there is another thread to manage communication with the server.
I would move the connection management into the worker thread as well. Connect, communicate, disconnect, repeat if needed, should all be in one thread.
In the communication thread Execute method I have a try - catch block and a loop in which I send a request to the server every 2 seconds or when the user asks for data and I decode the server answer. If there is a EIdReadTimeout exception, i terminate the thread and I restart it.
If an actual read operation times out, the state of the communication is unknown and likely unrecoverable, as you don't know which byte was the one that timed out. So you should disconnect and re-connect, not just restart the thread. The only time you wouldn't need a full re-connect is if you are handling the timeout wait yourself in between messages, and know the communication hasn't been corrupted. In which case, simply restarting the thread without a full re-connect is not likely to solve the timeout condition. It would be simpler to just re-try the wait operation again.
For other exception, I terminate the thread, I disconnect/connect the TIdTCPClient and I restart the thread.
If the connect/disconnect logic were moved into the thread, they could be done in a loop, then there would be no need to restart the thread itself. Threads are expensive to create/destroy (from the OS's perspective), so try to reuse threads when possible.
I think this is a different situation because in the server the exception was managed inside the TIdTCPServer thread while in the client the exception is managed in a thread that just uses the TCPClient->Socket to communicate with the server, is it right?
Yes.
I am getting the below error logged into the log file, when executing the
SSIS sub-package seperately.
Error :
Setting the end of rowset for the buffer failed with error code
0xC0047020.
When the main package is executed, the sub-package called within it doesn't throw any error. The reason for executing the sub-package is to just check the data getting populated as part of the process instead of the waiting for the complete process to run. Also the sub-package doesn't have any dependency on the main package.
The only thing which is shared is the connection info, which is used across packages.
I think the problem is due to too much logging of the info. in the log file. But not able to identify the property to be set in SSIS.
Pls. advise.
SSIS package is scheduled to run at a particular time. If files before the scheduled time, the load goes fine, but when file arrived late, the package fails showing error message
Process cannot access the file because it is being used by another process
Trap that specific error in the error handler, and if it occurs, have the job/package wait a while and try again until it no longer gets the error.
I'm trying to implement the SSIS package failure handler. I've added onError Event handler on whole package and failure operation the task where i'm expecting an error in the control flow.
The main task of the package is to pick up some data from the db,
the error handler is Send Mail Task sending email message with error description.
Now, I'm trying to emulate a common error: I make some changes in the SQL query (for example, set incorrect name for one of the columns) and try to run the package.
What i see is that package fails on validation stage and does not even proceed to execution, saying
Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occured. Error Code 0x80040E14. An OLE DB record is available. Source "Microsoft SQL Server Native Client 11.0"
Hresult 0x80040E14 Description "Statement(s) could not be prepared"
Description "The multipart identifier "MyTable.SomeWrongField" could not be bound".
Error: "OLE DB Source" failed validation and returned validation status "VS_BROKEN".
It doesn't even procceed to execution, failing right after validation.
How to set the error handle so it handles EVERY error, even the issue like this (incorrect sql request)?
Thanks.
A Validation error is the SSIS equivalent of a syntax error. There is no opportunity for recovery as the execution is dead in the water. OnTaskFailed and OnError won't catch this type of error (which is one of the reasons we no longer implement error notification from within our packages themselves)
I have an SSIS package that within a data flow task fetches a lot of data using an OLEDB connection.
When i run the package from my local machine it sometimes fails with the following error (snippet):
Warning: 0x80019002 at OnError: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. [..]
Error: 0xC0202009 at DFT Transform, SRC BSASREL1 [1]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80004005.
An OLE DB record is available. Source: "Microsoft OLE DB Provider for SQL Server" Hresult: 0x80004005 Description: "[DBNETLIB][ConnectionRead (recv()).]Generel netværksfejl. [..]
Error: 0xC0047038 at DFT Transform, SSIS.Pipeline: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on component "SRC BSASREL1" (1) returned error code 0xC0202009.
The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component,
but the error is fatal and the pipeline stopped executing.
If I deploy the package to the server and run it with an agent job nothing goes wrong.
The error is periodical which makes it hard for me to debug....
Have anyone else had similar errors or do anyone have ideas of how to solve this?
EDIT: It seems that the problem is solved. We haven't had connection problems since we disabled TCP Chimney.
I don't think there's enough information here to isolate the root cause of the error. That's not SSIS' fault. There are error returned from various providers and SSIS simply "forwards" them. There are likely clues in the error message pointing to the cause.
I once spent a lot of time trying to fix a connectivity issue in SSIS and SQL Server. The root cause turned out to be one of the DCs had gone offline...
Andy
Does it run for a long time? Is the server where you deploy the package the same server where you read the data from?
If so, your problem might be external. I have similar issues with some heavy data transfer packages i run. Sometimes they fail if the server is too loaded with other processes. If that is the case, it is an external problem.
My advice is that you try to pinpoint the source of the error (trying to overload a test server while you run the package, or look for timeouts on the connection) and circumvent the limitation through a retry mechanism, or by running the package on lower traffic times.
well, 0xC0202009 is a DTS_E_OLEDBERROR error (http://msdn.microsoft.com/en-us/library/ms345164.aspx) and dbnetlib.dll process "has the ability to send keep-alive TCP/IP packets to Microsoft SQL Server in order to maintain the connection" (this is from BOL)
I would say something related timeouts.
There is a command timeout propertie on the OLE DB Source component. Can you check the value?
Or maybe on the connectionstring on the Connection manager
Nobody's done anything with this post in two years, but I ran into this error and found that the size of the data set determined whether this error was thrown or not. Oddly, when running the SSIS program through the IDE, I didn't get the error. It's only in production mode that I found this error occurring.
The solution, I found, was to break up my data sets (which were being written to XML) using something like this to limit the amount of data returned:
select * from (
select *, row_number() over (order by a.pkey asc) 'ranker' from sometable
) where Ranker <500000
Sucks, but that's what I found worked.