I have a SSIS package built in Business Intellegience Development Studio which have both source and destination database specified. Now I want to use some variables to parameterize the database connections and run the package in a command line.
I try to replace the database name and sql server instance with my variable #[User::SourceDb]. #[User::SourceHost], but it failed to connect to the database.
Is that possible to paramterize the database and is there anything wrong with my variable useage? Thanks in advance!
There is a nice post here that details one way of doing this.
You will need to use a ConnectionManager and set the ConnectionString property of that from a Configuration Package.
The ConnectionString property is a fully qualified database connection string, like
Data Source=localhost;Initial Catalog=SSISConfig;Provider=SQLNCLI.1;Integrated Security=SSPI;Auto Translate=False;
There are a few gotchas and annoyances with using configuration packages so you may have to fiddle around a bit. I'd be more detailed but it has been about a year since I last worked on an SSIS project.
If you have any specific problems, please come back and comment about what you've hit. I'll try and refire the old memories.
You need to define "expressions" in the connection manager which then substitute into server/database.
You can't use variables directly quite often in SSIS
See "Using Variables in Packages" whcih links to Using Property Expressions in Packages.
And after a quick Bingle: http://sqlrs.blogspot.com/2006/03/using-expression-variables-in-ssis.html
You can right-click on any connection on the project or package and "Parameterize" all the fields from connection, in project or in package parameters. Connection string differs for OLE DB connection to .NET Providers(SqlClient Data Provider) type, so be careful. We have manage to have one centralized place for connection in Project.params
Related
I have several hundred SSIS pkgs, and I need to change one of the connection mangers per package from a SQL Server connection to another DB platform (ODBC). I found this link and think I can make it work. But I wondered if it was possible to use a tool like notepad++ or the like to open all of them at the same time and do a global find/replace with the connection manager. This would also need to be done to translate all the TSQL to the new DB platform SQL.
I hesitate to post because I feel I'm missing something simple, but after reading a ton of articles, I need a second set of eyes because mine are glazed over.
SSIS 2012 Solution.
Goal: Import data from 47 MS Access Databases into corresponding SQL Server databases. This means 47 packages. I want to avoid 47 different sets of Connection Managers and parameterize out what makes sense on the project level so common data is easily configurable.
I added project-level Parameters to hold the SQL Server, SQL UserName, SQL Password, and the Path to the Access Databases. These will be mapped out to the environment to allow changing the destination SQL Server easily as well as the source directory for the Access DBs.
I added two project-level OLEDB Connection Managers - one for SQL 2012 (NativeClient 11.0), one for Access (MSJET 4.0) (they're access 2000 db's). They are set to use an initial database configuration that I hope will be overridden by the individual packages.
For each package, I want to specify the databases being used, so I create a Package Variable (SQLDatabase and AccessDatabase)
In the Connection Managers, I add Expressions to include the Package Variable - e.g. for InitialCatalog on the SQL Connection Manager = #[User::SQLDatabase] and the ServerName in the Access CM combines the project path + the #[User::AccessDatabase]
WHAT WORKS:
I can update the Package Variable values and see them reflected in the Properties of the Connection Managers.
I can go into the Data Flow and add an OLE DB Source, select the SQL Connection Manager, and successfully receive a list of tables from that database.
WHAT FAILS:
When I click Preview, or Columns, after selecting a table I get a big error that reads:
"Error at Package1: The variable 'User::SQLDatabase' was not found in the Variables collection. The variable might not exist in the correct scope."
After clicking OK to that error, then the Preview or Column window shows up with no further problems.
However, after clicking OK on the OLE DB Source Editor, there is a big red X sitting on the box indicating the same error as above.
MY CONFUSION:
If I open the Variables for the package, the variable is sitting right there, scoped to Package1. If I update the value it and save, the Connection Manager updates itself accordingly, so the Expression mapping seems to be okay.
It's as if this is only partially working, and I'm unsure where the flag is to make the rest of it work, or where I failed in my configuration. Can you not set an expression in a project-level CM to use a package-level variable? If that's the case, I'm at a loss on how to correctly set this up to avoid package-level CMs.
Any help appreciated. Please be gentle if I've been an idiot somewhere.
If the connection managers are at project level, then you cannot use package level variables to set their value. If you go to the expression builder for Initial Catalog then you can see that only your project level variables are available. Hence you can use your project level variable to set it instead. If you really want to use the package level variable then you would need to create a connection manager per package which would be a pain.
I'm having this exact same problem I've been looking aroung but this is the only place I've seen the same issue and it is not resolved.
Does anyone knows what the problem might be?
I checked in Visual Studio and my SSIS version is 11.0.2100.60 (not a trial, not a beta).
EDIT: These are the steps I'm taking and the issue
First I choose a Connection Manager, right-click, select properties and click in Expressions option
Then in Property Expression Builder choose Connection String property and click in Expression option
Finally, in the Expression Builder dialog there is no option for variables, in every page I've read says that there should be a Variables node in there
Am I missing something?
SSIS 2012 has introduced the concept of Project level connection managers. What I see on the referenced post on the MSDN forums it the user has created a project level flat file connection manager and is unable to configure it with a local variable. Assuming that is the problem, my answer follows.
An SSIS project is generally more than one package. To simplify lives, the SSIS team now allows for the sharing of common resources across projects, connection managers being one of those resources.
Logically, if a thing is shared across a project, how can something that only exists in one file configure that resource? That configuration change would only work when Package1 is executing. When Package2 fires, unless the same variable and same expression was applied to the shared resource, you would experience different outcomes. That'd be a maintenance nightmare, which you might already experience if you don't have strong configuration practices.
If I create a Flat File Connection Manager at the project level, I can only reference variables that are also at the project level. Except there are not variables at the project level. Instead, they are called Parameters.
To that end, I created a Parameter called SomeProjectParameter
I then created a package, Package1.dtsx, and added 2 Flat File Connection Managers: FlatFileConnectionManagerLocal and FlatFileConnectionManagerProject
Instead that package, I also created a variable called SomeLocalVariable.
This screenshot shows me applying an expression to the ConnectionString property of FlatFileConnectionManagerLocal. There you can see that both the package variable, SomeLocalVariable is available as well as SomeProjectParameter
Now, if I try to apply an expression to the project's connection manager, you will only have project parameters available to you.
It's interesting to note that you can't apply an expression to a project level Connection Manager outside of the context of an SSIS package. There's simply no editor available to you until you have an open SSIS package. But, once applied, all the packages in the project will be similarly configured.
Quirk of the IDE I suppose. Also, don't be alarmed by the lack of color in these screenshots, I'm running with the 2012 version of SSDT.
i had the same problem and it was because the flat file source was set as a project source so i had to convert it into a package connection.
Right-click on the source and choose convert to package connection
Maybe this will help.
The Microsoft SSIS tutorial have these confusing statements:
In the Connection Managers pane, right-click Sample Flat File Source Data, and select Properties.
In the Properties window make sure the PackagePath starts with \Package.Connections. If not, in the
Connection Managers pane, right-click Sample Flat File Source Data, and select Convert to Package Connection.
\Package.Connections was initially displayed in the properties window, although I had not yet converted to Package Connection.
After reading this thread, I right clicked on Sample Flat File Source Data and then Convert to Package Connection and was able to complete the lesson. Thanks for the answers!
I ran into a problem with some of our packages. The basic layout is the connection strings are stored in a SSIS Configuration table in the database.
I've noticed lately that the server name for my oledb connections seem to remain static. I have seen where I move the package from one environment to another and the validation fails. If I change the password to what is used in the first environment it validates.
Is there some other property besides the connection string that I need to store as well? I'm not using variables or expressions.
Has anyone seen this before? The server version in question is 2008 R2.
This is likely because it's trying to validate using the connection string stored in the package itself and not the one provided by the configuration file. I know of two ways to get around this problem:
Each connection manager and data flow task has a property called DelayValidation. When set to true, that property will prevent SSIS from trying to validate the connections and data flow tasks until after the configuration has changed the connection string. Trouble is, the default value is false, and you have to go through and set it on every single connection manager and data flow task. You'll also need to remember to to flip the setting every time you create a new one.
You can either manually or programmatically change the actual value of the connection string in the SSIS package to match the configuration file before you deploy it. That admittedly does seem to defeat the purpose of having a configuration file in the first place, but it does ensure that it will work. This is the option I often end up taking. Before I move from my test environment to production, I pop open the package in a text editor and do a find/replace of the connection string. I've determined that to be safe for my packages, but, as with any hacky solution like that, your mileage may vary.
I am working on a winforms application using LINQ to SQL - and am building the app using a SQL Express instance on my workstation. The final installation of the project will be on a proper SQL Server 2005.
The database has the same name, and all tables are identical but the hostname is different.
The only way I have found to make my app work from one machine to the next is to re-open the code in Visual Studio, delete all of the objects referring to the SQL express instance from my .mdbl, save the project, connect to the other server, drag all of the references back on, and rebuild the application for release once more.
an answer here suggested that one can simply modify the app.config/web.config file - whih would work for asp.net, but this is a winforms application. What's the correct way of making LINQ to SQL apps use a new database without having to re-open the app in visual studio?
If I understand your problem correctly, you simply change the database's connection string in your app.config / web.config.
Edit, post clarification: You have the connection strings stored somewhere. They might be in the app.config of your server. Still, you get them from somewhere and that somewhere may be in an app.config. Use that then :)
One good solution is to add another connection to the dbml file itself. You can get to this by right-clicking on the field of the design surface and selecting properties. From there, you can add another connection string. Instead of deleting everything and redragging, just change the string and recompile.
But if you want to get fancy-schmancy, you can have the program auto-detect whether it is being run locally or not, using this neat utility function: detect local
And go from there to set the appropriate connection string based on the results.
A more useful answer...
app.config ends up as appname.exe.config when it has been built.
rather than opening Visual Studio and modifying app.config, you can simply edit the appname.exe.config file, and restart the app.
I believe you can store the connection information in an app.config file and retrieve it from there. Here is a post about doing that with LINQ to SQL. Once you deploy it to a production server, you can just edit the XML to change the data source.