How to override dtsx connection setting in sql server job step - ssis

I have a Web Service Task in a dtsx package developed in Visual Studio. It has an httpconnection with a Timeout setting of 30 seconds. The package is included as a step in a sql server (2008 r2) agent job. When I deployed the package, I set it up to be stored in SQL Server.
I would like to be able to change just the Timeout setting in the sql job step, but I'm not sure how to do this or even if it's possible. At the moment I'm changing the setting within VS then redeploying the package each time.
Can anyone give me any help on how to do this? Which tab of the job step should this be set on?

One thing to be aware of, there is the timeout property on the HTTP Connection Manager but that's for controlling the actual connection to the web service. It does not control the actual time for invoking a particular method, if that makes sense.
I had a 2005 package that consumed a web service for cleaning addresses. The webservice was hosted internally so the HTTP Connection was as LAN speeds, no issue there. The service itself could standardize one address pretty quick. When I need to bulk clean a few hundred thousand, then it takes a not insignificant amount of time. The XML task has a built in, as of 2008 R2, unchangable default timeout of 6 minutes. That's not so handy if you need it to be 3601 seconds or never time out. I'm having trouble finding documentation calling that out but you can verify the behaviour by ginning up a service that sleeps for 6+ minutes.
Our resolution was to use a script task to handle the actual service call so that we could override the Timeout property for the service call.
Public Sub Main()
Dim url As String
Dim inboundFile As String
Dim success As Boolean
Dim timeoutMs As Integer
' 1 hour = 60min * 60 sec * 1000 milliseconds
timeoutMs = 60 * 60 * 1000
inboundFile = CStr(Dts.Variables("NetworkShareInput").Value)
url = CStr(Dts.Variables("WebService").Value)
Try
Dim svc As New AddressCleanerService(url)
' Explicitly provide a timeout for the web service connection
svc.Timeout = timeoutMs
svc.Credentials = System.Net.CredentialCache.DefaultCredentials
success = svc.CleanBulkAddresses(inboundFile)
Catch ex As Exception
Dts.Events.FireError(0, "Address cleaning", "Something failed in the address component stuff", String.Empty, 0)
Dts.Events.FireError(0, "Address cleaning", ex.ToString(), String.Empty, 0)
End Try
If (success) Then
Dts.TaskResult = ScriptResults.Success
Else
Dts.TaskResult = ScriptResults.Failure
End If
End Sub

One way to do it is to use expressions and pass the timeout value from sql agent job. Below are highlevel steps:
Create a variable in your SSIS package to hold the timeout value.
In the properties window of the HTTP connection, click on the expressions eclipse button.
Expand Property dropdown in the property expression editor. Select Timeout.
And use the timeout variable you created earlier. Something like: #[User::Timeout]
In SQL Agent, use command line as job type, and use DTEXEC to execute the SSIS package.
In the DTEXEC command you can pass values to variables. Below is a commad example:
dtexec /f C:\SSIS\Package.dtsx /set \package.variables[Timeout].Value;45
So, when you want to change the timeout value simple change it in the SQL Agent job instead of redeploying the package.

First, if you still have control over the source code, I would point you to package configurations. Then you can edit these settings in an XML file or a data table.
Assuming you don't, you can push some values into the package using the "Set Values" tab of the job step. The hard part is getting the property path correct. Again, using Visual Studio and the package configurations feature, you should be able to find the right name.
Try this for the property path: \Package.Connections[myHttpConnection].Properties[Timeout].Value

Related

SSIS Connection String

I have a local SQL DB and an Azure SQL DB. In my data flow I am trying to pass data from local (ole db source) to Azure (ole db destination).
I am unable to save the password for the connection string so I have parameterized the connection string of the data flow task but I seem unable to work out how to get the destination to use it?
I know this isn't best practice but I just want to prototype a few things. Is this possible?
Thanks
Double click project params and add your connection string as a parameter. Give it any name you want, select string then paste your Azure connection string that you have copied directly from you Azure portal connection string option.
This creates a project level conn string parameter.
Right click connection managers and create a new ADO.Net connection manager. This should be fairly straightforward.
Once completed, select this connection manager and add an expression to it from the property window:
In the Property Expression Editor: (1) Select a property called ConnectionString. (2) Select the elipsis beside expression. From the pop up open the project parameters in the left hand area and you will see the connection parameter you created earlier. Drag this into the expression text area. Evaluate the expression to check it works.
Click OK
You should now be able to use this as an Azure connection without getting any errors

How can SQL Server 2012 (or SSIS) notify NServiceBus upon completion of a task?

We have some very long running ETL packages (some run for hours) that need to be kicked off by NServiceBus endpoints. We do not need to keep a single transaction alive for the entire process, and can break it up into smaller transactions. Since an NServiceBus handler will wrap itself in a transaction for the entirety, we do not want to handle this in a single transaction because it will time out--let alone create issues with locking in the DBMS.
My current thoughts are that we could spawn another process asynchronously, immediately return from the handler, and publish an event upon completion (success or failure). I have not found a lot of documentation on how to integrate the new NServiceBus 4.0 SQL Server Broker support with the traditional MSMQ transport. Is that even possible?
What is the preferred way to have a long running process in SQL Server 2012 (or an SSIS package) notify NServiceBus subscribers when it completes in an asynchronous manner?
It looks like it is possible to do a http request from SSIS, see How to make an HTTP request from SSIS?
With that in mind you can use send a message to NServiceBus via the Gateway (the Gateway is just an HttpListener) to your Publisher to tell it to publish a message informing all the subscribers that the long running ETL package has completed.
To send a message to the gateway you need to do something like:
var webRequest = (HttpWebRequest)WebRequest.Create("http://localhost:25898/Headquarters/");
webRequest.Method = "POST";
webRequest.ContentType = "text/xml; charset=utf-8";
webRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)";
webRequest.Headers.Add("Content-Encoding", "utf-8");
webRequest.Headers.Add("NServiceBus.CallType", "Submit");
webRequest.Headers.Add("NServiceBus.AutoAck", "true");
webRequest.Headers.Add("NServiceBus.Id", Guid.NewGuid().ToString("N"));
const string message = "<?xml version=\"1.0\" ?><Messages xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\" xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\" xmlns=\"http://tempuri.net/NServiceBus.AcceptanceTests.Gateway\"><MyRequest></MyRequest></Messages>";
using (var messagePayload = new MemoryStream(System.Text.Encoding.UTF8.GetBytes(message)))
{
webRequest.Headers.Add(HttpRequestHeader.ContentMd5, HttpUtility.UrlEncode(Hasher.Hash(messagePayload))); //Need to specify MD5 hash of the payload
webRequest.ContentLength = messagePayload.Length;
using (var requestStream = webRequest.GetRequestStream())
{
messagePayload.CopyTo(requestStream);
}
}
using (var myWebResponse = (HttpWebResponse) webRequest.GetResponse())
{
if (myWebResponse.StatusCode == HttpStatusCode.OK)
{
//success
}
}
Hope this helps!
There is actually a task in SSIS 2012 for placing messages in an MSMQ, the Message Queue Task. You just point it to your MSMQ connection and can use an Expression to customize your message with the package name, success/failure, row counts, etc.
Depending on how many packages we're talking about and how customized you want the messages to be, your best bet is to write a standalone utility to create messages in whatever format you desire, and then use an Execute Process Task to invoke that utility with whatever parameters from the package you want to pass in to be formatted into the message.
You could also use that same codebase and just create a custom SSIS task (a lot easier than it sounds.)
One thought I had to help adhere to the DRY principle would be to use a Master SSIS package.
In my mind, it would look something like an Execute Package Task with an X connected to that. Configure the package to take as a parameter a Package Name. Configure the Execute Package Task to use the Parameter for determining what package to call.
The X would probably be a Script Task but perhaps as #Kyle Hale points out, it might be the Message Queue Task. I leave that decision to those more versed in NServiceBus.
The important thing in my mind, is to not add this logic into every package as that'd be a maintenance nightmare.

CurrentProject.OpenConnection Fails in MS Access 2000

I support a LOB application written in MS Access VBA with a SQL Server back end. One feature of the application is the ability to open a second instance of the application, allowing the users to view/modify two records at the same time.
The first time I open the application it connects and everything works fine. However when I attempt to open the second instance I get the following error message:
-2147467259 - Method "OpenConnection" of object _CurrentProject failed.
This is the line of code executing when the error occurs:
CurrentProject.OpenConnection strConnection
I have stepped through the code and verified that strConnection is the same connection string in both the first and second instances of the application
I'm running out of things to look for. Any ideas are greatly appreciated!
UPDATE: It appears that something is not allowing the second MSACCESS.EXE instance to use the same connection string. My connection string is below, with database and server substituted for the actual database and server.
PROVIDER=SQLOLEDB.1;INTEGRATED SECURITY=SSPI;PERSIST SECURITY INFO=FALSE;INITIAL CATALOG=database;DATA SOURCE=server
Try
MultipleActiveResultSets=True
(http://msdn.microsoft.com/en-us/library/h32h3abf(v=vs.110).aspx)
Would it be better to open a new form from the same application?
dim frm as ShowCar_Form
frm.Show

SSIS Package won't do anything when Run through SQL Server Job

I have a simple SSIS Package, which has
a Excute SQL Task control on the Control Flow, which fetches some value from the database
In the DataFlow, am using a Script Component, which based on values given by 'Excute SQL Task', does this:
public override void CreateNewOutputRows()
{
try
{
string loginURL = "http://maps.googleapis.com/maps/api/geocode/xml?address=" + Variables.ProjectAddress + "&sensor=true";
WebClient client = new WebClient();
string downloadString = client.DownloadString(loginURL);
XmlDocument xml = new XmlDocument();
xml.LoadXml(downloadString);
///// setting output buffer variables
}
catch(Exception ex)
{
}
}
so basically am requesting a web service for latitude and longitude inside the package.
The retrieved values are then updated into the database:
Everything works fine, when I run the package from the Visual Studio SSIS project console.
But when I try to run the package through a SQL Server 2008 R2 Job, nothing happens. Job Executes successfully but no rows are updated(or inserted).
I tried importing the package into SQL MSDB and setting the protection level to all the items in the dropdown one by one as given here
...and then running this imported package from SQL Job. Still...nothing happened.
Does anyone know whats wrong?? How do I deal with following facts:
It has to do with permission of the sql user to make a web service request. How do I configure that out?
2.it has to do with the configuration file of imported ssis package. What should I look for?
Help me out please:
I hope I have given all the required info to look into the problem
is the job on an SQL Server Instance on your computer? I ask because it may be firewall or permission issues from the SQL Server to the computer you have the web service.
Also I advise removing that try catch and enabling package configurations so you can see if it is trowing an error
Regarding protection level, if you are using EncryptSensitiveWithUserKey the package wont load the database sensitive information (login and password) unless it is on the computer you developed it. Same thing applies to EncryptAllWithUserKey but in this case it wont even open the package

Execution 'iwy2vpzo52pmp555ftfn4455' cannot be found (rsExecutionNotFound)

Some users get the following error when running reports.
• Execution 'iwy2vpzo52pmp555ftfn4455' cannot be found (rsExecutionNotFound)
They run fine in the morning.
Any suggestions?
Thank you
I can help.
The problem is that the ReportViewer control uses Session to store the currently executing report. Once you navigate away from the reports, the item still remains and eventually loses its "execution context", which is the way Report Server caches reports.
Therefore, before browsing a report, you should attempt to clear out the Session of these reports, so that there are NO cached reports in the Session, and the ReportViewer control can work properly.
You will also find that sometimes when accessing Session.Keys.Count, this error can occur, as again, the execution context has failed.
Make sure you do this on the page showing the report!!
The 2 options are:
if (!IsPostBack)
{
HttpContext.Current.Session.Clear();
ReportViewer1.ServerReport.ReportServerUrl = new Uri(ReportServerUrl, System.UriKind.Absolute);
ReportViewer1.ServerReport.ReportPath = ReportPath;
System.Collections.Generic.List<ReportParameter> parameters = new System.Collections.Generic.List<ReportParameter>();
....
ReportViewer1.ServerReport.SetParameters(parameters);
ReportViewer1.ServerReport.Refresh();
}
Or
for (int i = 0; i < HttpContext.Current.Session.Keys.Count; )
{
if (HttpContext.Current.Session[i].ToString() == "Microsoft.Reporting.WebForms.ReportHierarchy")
HttpContext.Current.Session.RemoveAt(i);
else
i++;
}
I am using SSRS 2017 and was running into this issue when trying to load a report into my MVC project using URL Access. The issue for me had to do with session.
To check this for yourself, you can try deleting the RSExecutionSession cookie and reload your report. Unfortunately, this is only a temporarily fix.
If this does work, try adding rs:ClearSession=true to your query string.
You can read about this setting here.
Look for a trailing space on the report path. This was the cause for me.
On the web.config's impersonation, use identity
impersonate="true"
userName="xxxxx"
password="xxxxx"
instead of : !--<identity impersonate="true"
Hope it helps
If you're running SQL Server Express edition, the SQL Server Agent isn't running to clean up your old SSRS sessions. You'll need to run a job on SSRS DB to clean up the old sessions.
My report took 10 seconds to run and 2 seconds to export - so it wasn't to do with the session expiry length.
I'd get the error when exporting a report to excel into my app an hour after I exported the report.
This error was causing my application to display a run time error.
I added this to the Global.asax class to resolve the error. Tried Server.Clear but got nothing. Session.Clear got rid of the error completely.
Sub Application_Error(ByVal sender As Object, ByVal e As EventArgs)
If ex.InnerException IsNot Nothing Then
If ex.InnerException.ToString.Contains("The report execution") AndAlso
ex.InnerException.ToString.Contains("rsExecutionNotFound") Then
Session.Clear()
Return
End If
End If
End Sub
While it may not be 100% applicable to the question above, I haven't been able to find any other resolution.