What is the difference between Variables and Parameters in SSIS Denali?
If there is any difference then What is that which Variables cannot do that Parameters can do ? or vice versa.
When should one go with SSIS Parameters and Variables?
I tried searching on Google, but I failed to get some information.
Thanks In Anticipation!
I think a little bit background will be beneficial to understand the Parameter concept. Here I will explain it in the context of comparing with Variables. To fully grasp the Parameter concept, you might need to look up for the new Project Deployment Model, Environment, Build Configuration as well..
Usage Of Variable
With SSIS prior 2012, if we need to pass any external values to the package before the execution (as we all do all the time), I normally use configuration file (or a couple of other ways). Say we have a file server, which will be used to access a shared file, I will use variable to store the server name, and expose this variable to the configuration file. If the actual file server is changed (dev env to test env etc.), we just need the change the value of that variable in the configuration file and SSIS package remains intact.
Everything looked good, but there are a couple of things that I always ask myself why and could not figure out why:
100% of the time when I am exposing variables to configuration file, I just expose the "Value" properties. Why does SSIS allow to expose all the other variable properties?
Why does SSIS not have "private" variable? By "private" I mean when I chose the variables to configure, the "private" ones just did not get shown on the pick list. The SSIS package could have dozens of variables, for the internal value-holders, what's the point to expose them? Why I have to scroll all the way to find the only one I need to expose?
New Project Deployment Model
SSIS 2012 introduces a new deployment model, Project Deployment Model. For short, this model deploys SSIS project as a single unit to SQL Server SSIS catalog, and package configuration is NOT available in this model (it is available in the old model referenced as Package Deployment Model, with SSIS 2012 you can choose which one to use, 2012 default to the new model).
If we want the pass some values into the SSIS packages, we have to pass them in via Parameters, and use SSIS catalog in SSMS to configure the value for the parameters(only the value, nothing else we can configure). Parameters and connection managers are exposed automatically in SSIS catalog which can be configured, nothing else previously available via configuration files can be configured in Project Deployment Model (The world is much cleaner). Inside SSIS package, parameters can be used in the same way as variables in terms of building up expressions. However, parameters can NOT be modified within the SSIS package, which makes perfect sense. (Why do we need to change a value which is passed in from external? If we have to, pass the value to an variable, and do the changes there..)
Sum Up
Parameter is only available in the Project Deployment Model, and it provides the only mechanism for passing values from external to SSIS packages in this model. If we think SSIS pacakge as an OO class, Parameters could be thought as public properties, which externals can access and assign value to it (the class itself can/will use it, but cannot modify it). Where Variables could be thought as private variables, which are used internally, external world does not need to know anything about it.
For the old Package Deployment model, there is no Parameter, and the world remains the same.
FYI, in short, variable's value can be changed during the runtime, but parameter cannot. Parameter can help you do the project deployment and you can set it up in SSISDB catalog, while variable cannot.
There are many differences between Variables and parameters, few of them mentioned below:
Variables values can change at run time but parameters value can't change .
Variables can be used only with in the package we can't use it for other package with in the solution but we can use Parameters for multiple package (package exist with in the Solution Explorer).
The variables & parameters are similar to that are in java,
we pass/through some values to certain method/task in the form of parameters and we use them in that particular task we cant change those values since they are external things for that method similarly in SSIS the Project Parameters are used to set certain variables or connections dynamically in the package. where as variables are limited internal to the package level.
It works like this:
say you have a project parameter called ServerName:
Lets say you deploy an SSIS package into two integration catalog environments, one which is configured for prod server and another which is configured for test server:
Then your ServerName 'parameter' will be set in prod with prod server address and in test environment to contain test server address.
If any variable in your ssis package needs a runtime value(say the variable is used to set a connection at run time for prod or test servers respectively) then the variable will use the parameter from above to find the right server to connect to.
So parameters are usually needed in environment specific scenarios.
There are two types of parameters based on how you've configured your solution in Visual Studio: Project parameters or Package parameters. Project parameters are accessible to all packages in the project.
Parameters are using send data from outside of the package like usernames, passwords or connectionstrings etc. Variables are using inside of the package. It means you can define a variable in one of your SSIS package and use it in package level.
Parameters in SSIS are like Global Constants in Programming, so if they be applied to the Project can be used anywhere. Their
highest access would be the whole project.
Variables as they are named, can be assigned, from a Query or even a Parameter, etc. Also their highest level access would the
package.
Related
1)
I have a SSIS package and am using package parameter to configure the connections to point to dev. db . Now if i use the environment variables to pass a different connection value pointing to test, after i deploy to the SSIS catalog and further schedule it using the SQL Agent Job.
What connection info. will be taken at runtime , when the job is scheduled to run. Will it be dev or a test conn??
2)
I have a SSIS package and am using project parameter to configure the connections to point to dev. db . Now if i use the environment variables to pass a different connection value pointing to test, after i deploy to the SSIS catalog and further schedule it using the SQL Agent Job.
What connection info. will be taken at runtime , when the job is scheduled to run. Will it be dev or a test conn??
Answer on the first question depends on whether you are using SSIS Package Configurations or not. Possibly not, since you are talking about environment variables, but if you still do and use it together with Environment Variables - see Microsoft article on that, and note this behavior changes with SSIS 2008 version.
If you are using SSIS Catalog and Package/Project variables, then mind this simple rule - more specific wins. In your case, the following precedence will take place, going to the next value if former is missing:
Value specified at package start, either with execution or dtexec parameters.
Value mapped with Environment Variable.
Vales specified at Package design time.
Here is another Microsoft article on package parameter value mapping.
I am using project deployment model to deploy the SSIS(2012) solution. I use a parent package to execute other child packages. What is the best way (or best practice) to pass parameters to child packages?
When should I use Project level Parameters and Master package parameter bindings? What are the Pros and cons with the approach ? Please advise. Thanks!
My personal approach is to use Project Level Parameters for parameters which are common to several if not all of the packages inside your project. For instance, server and database names. This way, I only need to update them in one place.
I would use the package level parameters for parameters which are needed by that package only and would have values specific to that package only.
By the way, if a parameter is set at Project level (Project.params), then you don't need to pass it from Master to child anymore. Child packages can directly use them. You would pass parameters only if the parameter value is generated only at run-time by the parent package.
It is not directly related but I have documented a solution to automated SSIS 2012 deployment and configuration management which you might be interested with. It's a 3-part series. Please find the link to the first one below:
http://vaniecastro.com/2015/07/19/automate-ssis-2012-project-deployment-and-configuration-management-using-powershell-part-1/
I've inherited some SSIS Packages that need to be modified for a SQL 2008 R2 to SQL 2012 migration. Unfortunately, the "configuration" was done rather poorly.
We have a global XML Config file for all SSIS Packages, and just a few variables in the XML Config file--Server Name, ODS Server Name, and Environment (Development, Integration, PreProd, and Production).
To "configure", they write code in the SSIS Package: If 'Development' Then ... Else If 'Integration' Then ...
In order to change the "configuration", one has to change the code in the Package.
I have unsuccessfully tried to negotiate a change, but no one is budging so the one XML Config file remains.
If I can add a second XML Config file, with my Package specific variables that need to be configured in each Environment, that's what I will do. However, I have not found a way to do this. Is is possible?
My second choice is put variables in a SQL Server table.
I think your best bet is to replace all of your script tasks with OLE DB Commands to call a stored procedure which would take the following parameters: Environment, ConnectionName, PackageName, and ServerType(standard or ODS). The output of this stored procedure would be the ServerName, which would be assigned to a variable. This variable could be then be used to set the server name for the connection. The stored procedure could depend on table(s) or global XML file(s). I would suggest tables. Either way, the logic in the package would be minimal and would allow you to implement in whatever way you see fit.
I'm struggling to find an easy and simple way to validate the parameters of my SSIS package.
I feel this is the basic feature any tool allowing parameters should provide : checking those values are in a specific "allowed" range and not allowing a user to pass any value (or forget to provide a mandatory value).
Especially in SSIS where the variables values can be replaced easily (throuh parameters when calling or XML configuration files).
I found some "solutions" involving VB6 scripting or constraints on tasks. But all of this feels like workarounds.
What is the best practice for checking the variables/parameters values before executing the main SSIS package tasks ?
Example: my package can process data for 3 entities of my company : 'NHY', 'JIO', 'NTL' and 2 modes 'Q' and 'M'. The "caller" specifies which data he wants to process by passing the value when calling the SSIS package. But if the user specifies an entity that doesn't exist or a mode not supported, I want the package to fail immediately.
Example of command line that should fail BEFORE doing anything:
/FILE "Path\To\File\MyPackage.dtsx" /CHECKPOINTING OFF /REPORTING EWCDI /SET EntityCode;ZZZZZ /SET ProcessMode;Z
I'm having this exact same problem I've been looking aroung but this is the only place I've seen the same issue and it is not resolved.
Does anyone knows what the problem might be?
I checked in Visual Studio and my SSIS version is 11.0.2100.60 (not a trial, not a beta).
EDIT: These are the steps I'm taking and the issue
First I choose a Connection Manager, right-click, select properties and click in Expressions option
Then in Property Expression Builder choose Connection String property and click in Expression option
Finally, in the Expression Builder dialog there is no option for variables, in every page I've read says that there should be a Variables node in there
Am I missing something?
SSIS 2012 has introduced the concept of Project level connection managers. What I see on the referenced post on the MSDN forums it the user has created a project level flat file connection manager and is unable to configure it with a local variable. Assuming that is the problem, my answer follows.
An SSIS project is generally more than one package. To simplify lives, the SSIS team now allows for the sharing of common resources across projects, connection managers being one of those resources.
Logically, if a thing is shared across a project, how can something that only exists in one file configure that resource? That configuration change would only work when Package1 is executing. When Package2 fires, unless the same variable and same expression was applied to the shared resource, you would experience different outcomes. That'd be a maintenance nightmare, which you might already experience if you don't have strong configuration practices.
If I create a Flat File Connection Manager at the project level, I can only reference variables that are also at the project level. Except there are not variables at the project level. Instead, they are called Parameters.
To that end, I created a Parameter called SomeProjectParameter
I then created a package, Package1.dtsx, and added 2 Flat File Connection Managers: FlatFileConnectionManagerLocal and FlatFileConnectionManagerProject
Instead that package, I also created a variable called SomeLocalVariable.
This screenshot shows me applying an expression to the ConnectionString property of FlatFileConnectionManagerLocal. There you can see that both the package variable, SomeLocalVariable is available as well as SomeProjectParameter
Now, if I try to apply an expression to the project's connection manager, you will only have project parameters available to you.
It's interesting to note that you can't apply an expression to a project level Connection Manager outside of the context of an SSIS package. There's simply no editor available to you until you have an open SSIS package. But, once applied, all the packages in the project will be similarly configured.
Quirk of the IDE I suppose. Also, don't be alarmed by the lack of color in these screenshots, I'm running with the 2012 version of SSDT.
i had the same problem and it was because the flat file source was set as a project source so i had to convert it into a package connection.
Right-click on the source and choose convert to package connection
Maybe this will help.
The Microsoft SSIS tutorial have these confusing statements:
In the Connection Managers pane, right-click Sample Flat File Source Data, and select Properties.
In the Properties window make sure the PackagePath starts with \Package.Connections. If not, in the
Connection Managers pane, right-click Sample Flat File Source Data, and select Convert to Package Connection.
\Package.Connections was initially displayed in the properties window, although I had not yet converted to Package Connection.
After reading this thread, I right clicked on Sample Flat File Source Data and then Convert to Package Connection and was able to complete the lesson. Thanks for the answers!