1)
I have a SSIS package and am using package parameter to configure the connections to point to dev. db . Now if i use the environment variables to pass a different connection value pointing to test, after i deploy to the SSIS catalog and further schedule it using the SQL Agent Job.
What connection info. will be taken at runtime , when the job is scheduled to run. Will it be dev or a test conn??
2)
I have a SSIS package and am using project parameter to configure the connections to point to dev. db . Now if i use the environment variables to pass a different connection value pointing to test, after i deploy to the SSIS catalog and further schedule it using the SQL Agent Job.
What connection info. will be taken at runtime , when the job is scheduled to run. Will it be dev or a test conn??
Answer on the first question depends on whether you are using SSIS Package Configurations or not. Possibly not, since you are talking about environment variables, but if you still do and use it together with Environment Variables - see Microsoft article on that, and note this behavior changes with SSIS 2008 version.
If you are using SSIS Catalog and Package/Project variables, then mind this simple rule - more specific wins. In your case, the following precedence will take place, going to the next value if former is missing:
Value specified at package start, either with execution or dtexec parameters.
Value mapped with Environment Variable.
Vales specified at Package design time.
Here is another Microsoft article on package parameter value mapping.
I've inherited some SSIS Packages that need to be modified for a SQL 2008 R2 to SQL 2012 migration. Unfortunately, the "configuration" was done rather poorly.
We have a global XML Config file for all SSIS Packages, and just a few variables in the XML Config file--Server Name, ODS Server Name, and Environment (Development, Integration, PreProd, and Production).
To "configure", they write code in the SSIS Package: If 'Development' Then ... Else If 'Integration' Then ...
In order to change the "configuration", one has to change the code in the Package.
I have unsuccessfully tried to negotiate a change, but no one is budging so the one XML Config file remains.
If I can add a second XML Config file, with my Package specific variables that need to be configured in each Environment, that's what I will do. However, I have not found a way to do this. Is is possible?
My second choice is put variables in a SQL Server table.
I think your best bet is to replace all of your script tasks with OLE DB Commands to call a stored procedure which would take the following parameters: Environment, ConnectionName, PackageName, and ServerType(standard or ODS). The output of this stored procedure would be the ServerName, which would be assigned to a variable. This variable could be then be used to set the server name for the connection. The stored procedure could depend on table(s) or global XML file(s). I would suggest tables. Either way, the logic in the package would be minimal and would allow you to implement in whatever way you see fit.
What is the difference between Variables and Parameters in SSIS Denali?
If there is any difference then What is that which Variables cannot do that Parameters can do ? or vice versa.
When should one go with SSIS Parameters and Variables?
I tried searching on Google, but I failed to get some information.
Thanks In Anticipation!
I think a little bit background will be beneficial to understand the Parameter concept. Here I will explain it in the context of comparing with Variables. To fully grasp the Parameter concept, you might need to look up for the new Project Deployment Model, Environment, Build Configuration as well..
Usage Of Variable
With SSIS prior 2012, if we need to pass any external values to the package before the execution (as we all do all the time), I normally use configuration file (or a couple of other ways). Say we have a file server, which will be used to access a shared file, I will use variable to store the server name, and expose this variable to the configuration file. If the actual file server is changed (dev env to test env etc.), we just need the change the value of that variable in the configuration file and SSIS package remains intact.
Everything looked good, but there are a couple of things that I always ask myself why and could not figure out why:
100% of the time when I am exposing variables to configuration file, I just expose the "Value" properties. Why does SSIS allow to expose all the other variable properties?
Why does SSIS not have "private" variable? By "private" I mean when I chose the variables to configure, the "private" ones just did not get shown on the pick list. The SSIS package could have dozens of variables, for the internal value-holders, what's the point to expose them? Why I have to scroll all the way to find the only one I need to expose?
New Project Deployment Model
SSIS 2012 introduces a new deployment model, Project Deployment Model. For short, this model deploys SSIS project as a single unit to SQL Server SSIS catalog, and package configuration is NOT available in this model (it is available in the old model referenced as Package Deployment Model, with SSIS 2012 you can choose which one to use, 2012 default to the new model).
If we want the pass some values into the SSIS packages, we have to pass them in via Parameters, and use SSIS catalog in SSMS to configure the value for the parameters(only the value, nothing else we can configure). Parameters and connection managers are exposed automatically in SSIS catalog which can be configured, nothing else previously available via configuration files can be configured in Project Deployment Model (The world is much cleaner). Inside SSIS package, parameters can be used in the same way as variables in terms of building up expressions. However, parameters can NOT be modified within the SSIS package, which makes perfect sense. (Why do we need to change a value which is passed in from external? If we have to, pass the value to an variable, and do the changes there..)
Sum Up
Parameter is only available in the Project Deployment Model, and it provides the only mechanism for passing values from external to SSIS packages in this model. If we think SSIS pacakge as an OO class, Parameters could be thought as public properties, which externals can access and assign value to it (the class itself can/will use it, but cannot modify it). Where Variables could be thought as private variables, which are used internally, external world does not need to know anything about it.
For the old Package Deployment model, there is no Parameter, and the world remains the same.
FYI, in short, variable's value can be changed during the runtime, but parameter cannot. Parameter can help you do the project deployment and you can set it up in SSISDB catalog, while variable cannot.
There are many differences between Variables and parameters, few of them mentioned below:
Variables values can change at run time but parameters value can't change .
Variables can be used only with in the package we can't use it for other package with in the solution but we can use Parameters for multiple package (package exist with in the Solution Explorer).
The variables & parameters are similar to that are in java,
we pass/through some values to certain method/task in the form of parameters and we use them in that particular task we cant change those values since they are external things for that method similarly in SSIS the Project Parameters are used to set certain variables or connections dynamically in the package. where as variables are limited internal to the package level.
It works like this:
say you have a project parameter called ServerName:
Lets say you deploy an SSIS package into two integration catalog environments, one which is configured for prod server and another which is configured for test server:
Then your ServerName 'parameter' will be set in prod with prod server address and in test environment to contain test server address.
If any variable in your ssis package needs a runtime value(say the variable is used to set a connection at run time for prod or test servers respectively) then the variable will use the parameter from above to find the right server to connect to.
So parameters are usually needed in environment specific scenarios.
There are two types of parameters based on how you've configured your solution in Visual Studio: Project parameters or Package parameters. Project parameters are accessible to all packages in the project.
Parameters are using send data from outside of the package like usernames, passwords or connectionstrings etc. Variables are using inside of the package. It means you can define a variable in one of your SSIS package and use it in package level.
Parameters in SSIS are like Global Constants in Programming, so if they be applied to the Project can be used anywhere. Their
highest access would be the whole project.
Variables as they are named, can be assigned, from a Query or even a Parameter, etc. Also their highest level access would the
package.
I ran into a problem with some of our packages. The basic layout is the connection strings are stored in a SSIS Configuration table in the database.
I've noticed lately that the server name for my oledb connections seem to remain static. I have seen where I move the package from one environment to another and the validation fails. If I change the password to what is used in the first environment it validates.
Is there some other property besides the connection string that I need to store as well? I'm not using variables or expressions.
Has anyone seen this before? The server version in question is 2008 R2.
This is likely because it's trying to validate using the connection string stored in the package itself and not the one provided by the configuration file. I know of two ways to get around this problem:
Each connection manager and data flow task has a property called DelayValidation. When set to true, that property will prevent SSIS from trying to validate the connections and data flow tasks until after the configuration has changed the connection string. Trouble is, the default value is false, and you have to go through and set it on every single connection manager and data flow task. You'll also need to remember to to flip the setting every time you create a new one.
You can either manually or programmatically change the actual value of the connection string in the SSIS package to match the configuration file before you deploy it. That admittedly does seem to defeat the purpose of having a configuration file in the first place, but it does ensure that it will work. This is the option I often end up taking. Before I move from my test environment to production, I pop open the package in a text editor and do a find/replace of the connection string. I've determined that to be safe for my packages, but, as with any hacky solution like that, your mileage may vary.
Is there any way in SSIS to send mail dynamically with dynamic subject, dynamic recipients, dynamic body itself? It is possible with database tables but I dont want to use database tables and stored procedures in my package.
SSIS is an in-memory ETL (Extract, Transform and Load) tool that is powered by the .NET framework (and some COM hackery). It does have a Send Mail Task which is capable of sending emails.
To make them as dynamic as you describe, you'll want to use a combination of Expressions, as Aryadev linked, coupled with Variables.
Most every thing in SSIS exposes a set of Properties that support configuration so what I generally suggest is that people create variables and perform all their expressions on the variables and then assign a variable to a property's expression. While it seems like more work, when you have to troubleshoot a package you'll be grateful. You can set a breakpoint in a package and inspect the current value of a Variable but you can't examine the run-time value of a property.
Finally, unless you're doing ETL stuff in addition to email, I'd skip the overhead of SSIS in favor of just using sp_send_dbmail in a TSQL space or call into the .NET smtp library.
Yes. Look into SSIS Expressions.