I want to develop SSIS package in which I want to copy files from FTP location. This location will be configurable. I want that if FTP location is down due to any reason or not available then it will be able take files from some alternate FTP location. Please let me know how this can be achieved.
Precedence Constraint approach
Add an FTP task, Primary FTP, and then add a second FTP, Alt FTP. Connect the second to the first through a OnFailure constraint.
Loop approach
Create an integer SSIS Variable, FTPTryCount assign it to zero.
Add a string Variable, FTPConnectionString with EvaluateAsExpression = true and use an expression like
(#[User::FTPTryCount] ==0)?"PrimaryFTPString":"AlternateFTPString"
If the value of FTPTryCount is zero, then we'll use the first connection string. Otherwise, we use the second string. Make those value connection strings for your system.
Configure the FTP Connection Manager to use the string from #[User::FTPConnectionString]
Add a boolean Variable FileFound and set it to false.
Add a For Loop. Terminal condition is #[User::FTPTryCount] > 1 || #[User::FileFound]
If we find the file or have tried twice, we want to exit the loop.
Inside your FTP Loop, you'll have an FTP Task and a Script Task. The Script Task will be in the OnSuccess path and all it does is set the FileFound to true.
Script approach
Assuming the variables from the Loop approach exist, add a script task that will set the #[User::FTPTryCount] check the FTP Connection manager's connection string and then attempt to open a connection to it. Wrap it all in a try/catch block. If the connection opens, FTPTryCount remains 0. If the catch block fires, then you set it to 1.
From the Script Task, add your FTP task. Configure the FTP Connection Manager to use the string from #[User::FTPConnectionString]
Error handling regardless of approach
The FTP task will fail if it can't connect to the remote server (or if no file is found). You'll need to handle this otherwise it's no good.
This is all high level, I have no access to an FTP site so you're on your own for physical implementation of the design but any of the above should work fine.
Related
is there any way using SSIS (or any MSSQL Server features) to automatically run a stored procedure and have the output saved as an Excel files (or even flat file) and then have the newly created file sent to people via email?
sorry im a complete newbie to SSIS.
In broad strokes, you'll have an SSIS package with 2 tasks and 3 connection manager.
The first task is a Data Flow Task. Much as the name implies, the data is going to flow here - in your case, from SQL Server to Excel.
In the Data Flow task, add an OLE DB Source to the data flow. It will ask what Connection Manager to use and you'll create a new one pointed at your source system. Change the source from the Table Selector to a Query and then reference your stored procedure EXECUTE dbo.ExportDaily'
Hopefully, the procedure is nothing more than select col1, col2, colN from table where colDate = cast(getdate() as date) Otherwise, you might run into challenges for the component to determine the source metadata. Metadata is the name of the game in an SSIS data flow. If you have trouble, the resolution is version dependent - pre 2012 you'd have a null operation select as your starting point. 2012+ you use the WITH RESULT_SETS to describe the output shape.
With our source settled, we need to land that data somewhere and you've indicated Excel. Drag an Excel destination onto the canvas and again, this is going to need a connection manager so let it create one after you define where the data should land. Where you land the data is important. On your machine, C:\user\pmaj\Documents is a valid path, but when this runs on a server as ServerUser1... Not so much. I have a pattern of C:\ssisdata\SubjectArea\Input & Output & Archive folders.
Click into the Columns tab, and there's nothing to do here as it auto-mapped source columns to the destination. Sort the target column names by clicking on the header. A good practice is to scroll through the listing and look for anything that is unmapped.
Run the package and confirm that we have a new file generated and it has data. Close Excel and run it again. It should have clobbered the file we made. If it errors (and you don't have your "finger" on the file by having it open in Excel, then you need to find the setting in the Excel destination that says overwrite existing file)
You've now solved the exporting data to Excel task. Now you want to share your newfound wisdom with someone else and you want to use email to do so.
There are two ways of sending email. The most common will be the Email task. You'll need to establish a connection to your SMTP server and I find this tends to be more difficult in the cloud based world - especially with authentication and this thing running as an unattended job.
At this point, I'm assuming you've got a valid SMTP connection manager established. The Send Email Task is straightfoward. Define who is receiving the email, the subject, body and then add your attachment.
An alternative to the Send Mail Task, is to use an Execute SQL Task. The DBAs likely already have sp_send_dbmail configured on your server as they want the server to alert them when bad things happen. Sending your files through that process is easier as someone else has already solved the hard problems of smtp connections, permissions, etc.
EXECUTE msdb.dbo.sp_send_dbmail
#profile_name = 'TheDbasToldMeWhatThisIs'
, #recipients ='pmaj#a.com;billinkc#b.com'
, #subject = 'Daily excel'
, #body = 'Read this and do something'
, #file_attachments = 'C:\ssisdata\daily\daily.xlsx';
Besides using existing and maintained mechanism for mailing the files, Execute SQL Task is easily parameterized with the ? place holder so if you need to change profile as the package is deployed through dev/uat/prod, you can create SSIS Variables and Parameters and map values into the procedure's parameters and configure those values post deployment.
I am trying to set up deployment process for single package usinig project deployment so VS2012. I found that to change dynamicaly connection string on the server I can parametrize connections so I did this and created enviroments and I run my package with inviroment which has connnections strings as parameters and all seems to be fine, but why on connection manager I can still see some old setup made while developing? How can I remove it ?
By Parameterize, I assume you're using the Configuration section to globally configure a project/package or on a per-execution basis. This is in contrast to using project/package Parameters
I have created an SSIS Environment variable named ConnectionStrings in my deployment folder and it has two values: ServerName and CatalogName.
I right clicked on my project, DeployMe, and selected Configure. In your screenshot, you have clicked on the specific package and selected Configure. That or you manually changed the Scope drop down.
I first click on the References and add a pointer to my Environment
Back to the Parameters tab, I click over to Connection Managers and I'm going to configure the CM_Project connection manager's ServerName property to use my environment variable's ServerName value. Clear right?
After configuring the ServerName, I also configured the InitialCatalog property but instead of using my Environment Variable's value, I used the "Edit Value" option (above) to set it. The net result is that my properties now look like this.
The underscore indicates it's set from an environment variable
The bold text indicates it's set manually.
Now when I go to run my package, via Agent or manual execution, the first thing it's going to prompt me for is an environment reference. I've lost my bolding for the InitialCatalog but the underlining remains for ServerName property. None-the-less, both are different values and were I to execute it, they would pick up the correct values.
All that said, I find it far easier to just store the whole ConnectionString value. You will observe, if you take this route, that the values displayed for ServerName would show your design-time values but that's fine because the ConnectionString as a whole will override the individual values at run-time.
I know this is a generic answer but I'm hoping I've hit on what you're missing step-wise.
I have an SSIS 2008 package.
I have 3 user variables in the package. One is for an the environment, one is for the path for an executable, and the other is part of a message for an email.
I have a Script Task that sets the variable for the path (strAppPath) based on the environment variable.
strAppPath is used in an expression for the Executable property of an Execute Process Task. The job fails stating that the executable path for the Execute Process Task is not set.
I'm assuming that it is checking this path before the Script Task sets the variable.
Is there a way to work around this?
Right click on your Execute Process Task and select Properties. In the properties window, you will have a DelayValidation option that is currently set to False Flip that to True.
What is happening is that when the package starts, it goes through a validation phase to ensure everything is kosher before it begins (no need to start processing if something is broken). In your case, that full validation is not desired as the Execute Process Task won't be valid until right before it's time to run. The validation will occur, just that it is delayed until it is time for the task to begin. Make sense?
I'm trying to set up a 'master' SSIS Package in SQL Server 2008 to run other 'child' packages. When developing the 'child' packages we have all the packages sitting on disk so we can easily debug them, so we use file connectors during development and can monitor the progress nicely.
When we deploy, we deploy the child packages to SSIS on SQL Server and then go through and change all the Execute Package Task's to use a location value of 'SQL Server' and set the PackageName. Once done, we deploy the 'master'.
What I'd like to do is use an Expression on the Execute Package Task to set the connection properties so we can configure this dependent on the environment. We have already setup a SQL Server configuration database using a view which checks the host name of the query and returns different values dependent on the query.
You have options. You're in the right frame of mind using expressions, but you might benefit from using configurations as well.
To use expressions, you would need to use a Script Task or Execute SQL Task to return back the list of files you want to work through.
You would either have to assign each returned value to it's own variable that is passed into the expression, or use a FOR EACH loop and work through a list, assigning the location of the child package each time.
The other option is to use configurations. My preference is to use a configuration table inside SSIS. If you have the same list of packages in each environment, you could either pass in the root directory and have an expression use that:
#[User::RootPackagePath] + "\PackageName.dtsx"
Or, you could simply have one record for each child package in the configuration table and that would be passed into the package.
Edit based on comments:
I was successfully able to configure a package to change via configurations to call a package from the file system then SQL.
I only needed to pass the Connection and PackageName for each. With SQL, it want a name from the connection manager (.\SQL2008R2 in my case) and the package name (\Package1). For the file system, PackageName is blanked out and the connection is a FileConnection in the connection manager.
You will have to keep both in the package, but you switch between the two.
This was posted on server fault, but in retrospect that may not be the best spot for it, so I've moved it here.
I have a package that runs fine on my dev machine but in production (and also on another dev desktop) the Config seems to stop working. I'm using SQL Server Table configuration to pull a value from a table and populate a variable. My Database connection string is passed in at runtime so this should be taken care of in the production environment. There are no errors, but the value is not retrieved and the variable remains blank (it is actually a path, so I get a file not found as the path is blank).
The package works on my machine. It retrieves the correct path from the table and populates the variable as expected. Why this stops working with no error elsewhere is a mystery. Is there someplace that configuration needs to be 'switched on', apart from the checkbox on the front page of the config wizard.
Try Event Handlers
1. Select Event Handler OnVariableValueChanged
2. Add a script task to the pane
3. Pass in System::VariableName,System::VariableValue as InputVariables
4. Add the following to the code:
Public Sub Main()
Dts.Events.FireInformation(0, "VariableChangedEventLog", "Variable " & Dts.Variables("System::VariableName").Value.ToString() & " Changed to:- " & Dts.Variables("System::VariableValue").Value.ToString(), "", 0, True)
Dts.TaskResult = Dts.Results.Success
End Sub
This will print out the variables and will at least give you an indication of whether the correct values are passed into the package.
I have the problem that my package receives the correct configuration file, but then is not passing this value onto my child packages. May well post a link to a new post on this.