I'm trying to set up a 'master' SSIS Package in SQL Server 2008 to run other 'child' packages. When developing the 'child' packages we have all the packages sitting on disk so we can easily debug them, so we use file connectors during development and can monitor the progress nicely.
When we deploy, we deploy the child packages to SSIS on SQL Server and then go through and change all the Execute Package Task's to use a location value of 'SQL Server' and set the PackageName. Once done, we deploy the 'master'.
What I'd like to do is use an Expression on the Execute Package Task to set the connection properties so we can configure this dependent on the environment. We have already setup a SQL Server configuration database using a view which checks the host name of the query and returns different values dependent on the query.
You have options. You're in the right frame of mind using expressions, but you might benefit from using configurations as well.
To use expressions, you would need to use a Script Task or Execute SQL Task to return back the list of files you want to work through.
You would either have to assign each returned value to it's own variable that is passed into the expression, or use a FOR EACH loop and work through a list, assigning the location of the child package each time.
The other option is to use configurations. My preference is to use a configuration table inside SSIS. If you have the same list of packages in each environment, you could either pass in the root directory and have an expression use that:
#[User::RootPackagePath] + "\PackageName.dtsx"
Or, you could simply have one record for each child package in the configuration table and that would be passed into the package.
Edit based on comments:
I was successfully able to configure a package to change via configurations to call a package from the file system then SQL.
I only needed to pass the Connection and PackageName for each. With SQL, it want a name from the connection manager (.\SQL2008R2 in my case) and the package name (\Package1). For the file system, PackageName is blanked out and the connection is a FileConnection in the connection manager.
You will have to keep both in the package, but you switch between the two.
Related
SSIS package parameters can be configured in 2 places.
Under Integration services catalog - on the project (right click, configure).
Under the SQL agent job step package configuration.
Parameters configured at project level automatically show up at the job level, and also the job level picks up any parameter changes made at the project level.
Where as if change is made to the parameter at the job level, then this doesn't update at the project level.
Suppose you change the value of the parameter at job level, then even if you reset the value to the original value (to match with the project level value), then any subsequent changes to this parameter at the project level doesn't update the value at the job level. Is there any way to re-establish the link between the project level and job level parameter?
I think I understand the question but let me know if I've missed the mark.
I have a project parameter, MagicNumber.
In the SSIS project in VS, it's -1.
I deploy to the SSISDB to a folder called Demo and Configure the deployed Project to have a value of 0. If I right click and run that package or create a job to run the package, it's going to use value of 0.
In a SQL Agent job, I then re-define MagicNumber to be 1 for all executions of that job. Even if I then go back into Configure the project to have 2 as my MagicNumber, all instances of this job will use 1 because we've specified that we are overriding the configured value.
If I created Job2 which runs the same package, it will use 0 and then 2 as MagicNumber because it's picking up the override from the Project Configuration.
Were you to deploy the same original .ispac file to a different folder, Pristine and ran that package, it's going to use -1 because that's the default configured value.
We put the first layer of configuration on all runs of the package from our project in the Demo folder by specifying MagicNumber is 0 and then 2. But the job allows us to add yet another layer of configuration by using a job specific level of configuration, 1.
If you decide 1 should be the default value, you update the Configuration for the project in the Demo folder. If you do nothing, then the SQL Agent job will still use the locally provided value so you then need to edit the job to remove the locally configured value and use the Project scoped configuration value.
I have just started looking at using the SSIS 2012 Project Deployment. One thing I have noticed is that once the project is deployed all the properties relating to the project level connection manager get deployed too and hence need to be configured (Linked to SSIS Environment Variables).
Was just wondering whether the SSIS environment is smart enough to infer the ConnectionString property from the InitialCatalog and ServerName properties. That way I would only need InitialCatalog and ServerName Environment Variables and not a third variable containing the actual connection string which would result in a duplicate configuration.
Please see screenshot below showing an example of this.
If I got your question right, you want to set the initial catalog and server name property paramatically.
Here are the steps to do it: [Applicable for SSIS 2005 and above]
Step-1: Create two variable in SSIS package (Package scope) and name them DbName, ServerName
Step-2: In connection manager, create a OLEDB connection and test connection.
Step-3: Now, right click on the connection manager and click properties.
Step-4: In properties, click expression. You will see below window. In this window select initial catalog
Step-5: Now click on expression and drag drop DbName (i.e. your catalog name) variable from upper left window to expression text box and evaluate it.
Step-6: Do the same for Server name. and you are done
Remember: If your initial catalog and database parameter name are not
correct then you may face connection issue.
Other way, Specifically for SSIS 2012 and above
Now, as you have mentioned SSIS2012. Let;s utilize its feature of Project parameter
Create project level variables, You can either create two variable names [DBName, ServerName] or complete connection string.
Create OLEDB connection in connection manager and right click on the connection. Then select, "Parameterize"
Based on your parameter, you can set parameter to properties like Initial catalog, Server Name, Connection string etc. Or you can create project level parameter from here by selecting appropriate values, as shown below
Read this link for more details
I tested several combinations of ConnectionString, InitialCatalog and ServerName. It looks like servername (DataSource) and initialcatalog have to be in the connection string and the initialcatalog has to have a value. If I left servername blank it worked as long as DataSource was in the connection string.
I want to develop SSIS package in which I want to copy files from FTP location. This location will be configurable. I want that if FTP location is down due to any reason or not available then it will be able take files from some alternate FTP location. Please let me know how this can be achieved.
Precedence Constraint approach
Add an FTP task, Primary FTP, and then add a second FTP, Alt FTP. Connect the second to the first through a OnFailure constraint.
Loop approach
Create an integer SSIS Variable, FTPTryCount assign it to zero.
Add a string Variable, FTPConnectionString with EvaluateAsExpression = true and use an expression like
(#[User::FTPTryCount] ==0)?"PrimaryFTPString":"AlternateFTPString"
If the value of FTPTryCount is zero, then we'll use the first connection string. Otherwise, we use the second string. Make those value connection strings for your system.
Configure the FTP Connection Manager to use the string from #[User::FTPConnectionString]
Add a boolean Variable FileFound and set it to false.
Add a For Loop. Terminal condition is #[User::FTPTryCount] > 1 || #[User::FileFound]
If we find the file or have tried twice, we want to exit the loop.
Inside your FTP Loop, you'll have an FTP Task and a Script Task. The Script Task will be in the OnSuccess path and all it does is set the FileFound to true.
Script approach
Assuming the variables from the Loop approach exist, add a script task that will set the #[User::FTPTryCount] check the FTP Connection manager's connection string and then attempt to open a connection to it. Wrap it all in a try/catch block. If the connection opens, FTPTryCount remains 0. If the catch block fires, then you set it to 1.
From the Script Task, add your FTP task. Configure the FTP Connection Manager to use the string from #[User::FTPConnectionString]
Error handling regardless of approach
The FTP task will fail if it can't connect to the remote server (or if no file is found). You'll need to handle this otherwise it's no good.
This is all high level, I have no access to an FTP site so you're on your own for physical implementation of the design but any of the above should work fine.
I need to use a variable I get in a Script Task in one package in a Script Task in another package. How can I make a variable with a scope that spans packages? Are there Project variables?
Prior to SQL Server Integration Services 2012, the only way to share a value between packages was to use Parent/Child configuration. You could actually share a value between them without using configurations but it was janky as all get out.
If you have no need of bi-directional communication, then you could have package A (one that computes the value in script task) start package B and use the /SET properties to assign a value to the variable
dtexec /file PackageB.dtsx /Set \Package.Variables[User::SharedVariable].Properties[Value];\"I was computed\"
In a SQL Server 2012 project deployment model, the Configuration concept has been replaced with Parameters. This is a bit more straight forward as you define Parameters and specify whether they are required. The Execute Package Task provides a nice mapping mechanism between local variables and expected Parameters.
In SSIS 2005 and 2008:
Declare a variable - say p as int - at the package level
Call your child package from the parent package.
Now, if you have a script task in the Child package, you can access the variable p like this:
1. Pass ReadOnly variable p in the Script Task Editor of Child package
2. To access the parent variable: Dts.Variables["p"].Value;
Notice, that I have not use "User::p" in any of the above two steps. I find this method straight-forward.
Make sure you do not declare a variable p at the child package level.
So, how does this method work? Think of the basic concept of the scope of a variable. The script task will keep "going up" to find the variable p. Going up means - it will first try to find it at task level, then container level, then package level, then finally at parent package level. (This is a simple explanation - technically each of these levels are containers.)
In SSIS 2012, you can also use parameters to pass the variable in ReadOnly mode. The method described above can also be used in SSIS 2012 with added advantage of being able to overwrite the value of the parent variable.
I am running an SSIS package in a SQL Server agent job. I have several variables I need to set for production as I develop with development settings and example of this is a log variable which passes a path to use for a text file connection string. I used the "Set Values" UI dialog to set this value and the resulting Command Line for this job step looks like this ...
/SQL "\MyFolderInMSDB\MyPackage" /SERVER "mw-test" /CHECKPOINTING OFF /SET "\package.variables[log].Value";"C:\Logs\Imports\mylog.log" /REPORTING E
There does not seem to be an error trying to set the variable but when I run the job it is logging to the variable defined in the task when I developed it and not the path I am sending in as a variable above. Anyone seen something like this?
Assuming you have pasted your SQL Agent job step information and not retyped it, the problem is most likely due to case sensitivity. Stuff in SSIS is case sensitive so variable names of "log" is different from "Log".
/SQL "\MyFolderInMSDB\MyPackage" /SERVER "mw-test" /CHECKPOINTING OFF /SET "\Package.Variables[User::log].Properties[Value]";"C:\Logs\Imports\mylog.log" /REPORTING E
#Bill's example above also runs into a case sensitivity with the provided path.
Try this: http://www.buildingmeaning.com/?p=171
Looks like you are very close. Probably just changing
"\package.variables[log].Value"
to
"\package.variables[log].properties[value]"
I had to mark the ConfiguredValue as being read from the Executable value in SSIS package and config
\Package\Variable Container.Executables\Variables[User::run_type].Properties[Value]
Otherwise, the static value from the config was being read every time.