I generated a SSIS packed by using the SQL Server export wizard utility.
I extract records from an Oracle 11g database and migrate them to SQL Server 2008 R2.
Since I have many oracle databases (test, production) with the same schema, I would like to reuse the created package also for the other environments. The package would be executed via command line and ideally by changing the source and target connection strings according to the different systems.
Is it possible to use the package .config file in order to change the source and target connection?
If so, which are the "keywords" to ensure that a specific connection strig is used for the proper target database?
When you create a SSIS package using wizard ,select the option Don't save sensitive while saving the package
Once the package is saved ,Log in to integration service expand MSDB and then Maintenance Plan folder where you will find your SSIS package .Right Click on the Package and select Run Package
In the connection Manager tab select your datasources .
Then click on command line option .You will get the entire command to execute your package where you can modify the connection strings which you have selected above .
The command might look like
/DTS "\MSDB\PackageName" /SERVER "ServerName"
/CONNECTION DestinationConnectionOLEDB;"\"Data Source=localhost;
Initial Catalog=DBName;Provider=SQLNCLI10;Integrated Security=SSPI;
Auto Translate=false;\"" /CONNECTION SourceConnectionOLEDB;
"\"Data Source=localhost;Initial Catalog=DBName;Provider=SQLNCLI10;
Integrated Security=SSPI;Auto Translate=false;\""
/CHECKPOINTING OFF /REPORTING V
Now in order to execute the package using command prompt
just add dtexec along with the above command.
Change the connection string to execute for different servers
Related
I have a SSIS package using BIDS 2008 which is for SQL Server 2008 R2.
I was successfully able to deploy the package along with config file on a shared folder on the server . A SQL Agent job is configured to execute the package . It works fine on the DEV. While doing package deployment on the QA environment I noticed that I need to change the server names from the connection strings specified in the config file as well as amend the package .
My worry is the next time I do enhancement to the package I would need to reconfigure package file which is error prone.
Looking for dynamic configuration solution for various environments.
You may create Environment Variables on server and assign value as DEV/QA/UAT/PROD.
Configure your package to read this values so that you don't have to change during deployment.
Example :
%Environment%\MyPackageStore\ProjectName\Package.dtsx
you can fetch value and assign to variable at package level to build a connection string :
If you want to Configure SQL Agent Job, here is the example :
I am trying to set up a new SQL Agent job on a SQL2012 server. At the point of defining the job step type = SSIS package, the sql agent setup crashes and I get the following error:
the type initializer for ‘’ threw an exception. (SQLManagerUI) Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt. (DTEParseMgd)
Other posts suggest that the only workaround is to install SQL2014 or 2016, or return to SQL2008. None of these options are possible for us.
We have to move away from SQL2008 for security reasons. We cannot move to SQL2014 because we use access adp project front ends which need the SQLOLEDB data provider which is obsolete in SQL2014.
Does anyone know if Microsoft have released a service patch for SQL2012 which fixes this issue for good?
That memory error sounds like a server issue, but until you track that down, here's a workaround: set the job step type to "Operating System (CmdExec)", and use DTEXEC, for example:
DTExec /DTS "\<ssisPkgName>" /DECRYPT <pkgPass> /MAXCONCURRENT " -1 " /CHECKPOINTING OFF
Note that if the SSIS package requires 32-BIT execution (true for exporting to Excel, for example), then use the DTEXEC utility in "Program Files (x86)" by fully qualifying it. Example, where the SQL Server application was installed on an "E:" drive, and where SQL Server 2014 is being used:
E:\Program Files (x86)\Microsoft SQL Server\120\DTS\Binn\DTExec.exe" /DTS "\<ssisPkgName>" /DECRYPT <pkgPass> /MAXCONCURRENT " -1 " /CHECKPOINTING OFF
If your SSIS packages are in the file system (as ".dtsx" files), then replace "/DTS" with "/FILE".
If your SSIS packages were placed in SSISDB (using the "project deployment model", which is available starting with SQL Server 2012, instead of the older "package deployment model"), then replace "/DTS" with "/ISSERVER"
Lastly, consider your job step's "Run as": if your job step's "Run as" was already set to a proxy, then you already made that proxy "active" to the subsystem "SQL Server Integration Services Package". Now, to do command lines like the above, check that proxy's properties, and make sure it is also "active" to the subsystem "Operating system (CmdExec)". (If you're not using a proxy, then you may need to add "Operating system (CmdExec)" for the SQL job agent... but a proxy is better for security: why grant stuff to the agent, that you will only use in specific circumstances?)
(P.S. side benefit: with the above, I find it easier to get logging from the SSIS package.)
I have many packages that are using the package configuration with the following way:
-ALL Packages have the XML configuration file that has only one proporty defined. The ConnectionString of the SQL Server connection that holds the configuration table for the rest of the properties
-A SEPARATE SQL Server package configuration for each connection manager in the package.
-Finally i have an SQL Server configuration for all the properties that are specific to this package.
I attach a pic of what i mean:
Yellow is the XML config with the connectionstring, Blue the connectionamangers and purple the package specific.
So with this setup i can:
Change the xml file location and just point all the setup in another sql server or another database.
Or create different configuration filters in the same config table and try to go into the package and change the filter.
With all the above the problem is that if i do anything from within VS, i am loosing the password in the connectionstring because i am not using the encrypt property. And i dont want to use it...
What are my options? Just go in .dtsx with notepad and chage what i want BEFORE i open the package or before i deploy?
-I dont want to use EncryptSensitiveWithPassword, so:
When i go to package configuration and try to change the ConfoigurationFilter to point to another setting then i am getting to the screen to select the property (connectionstring) and when i finish the DATABASE record for the setting is cleared from the Password= that i have put previously.
So i short what i want:
-No EncryptSensitiveWithPassword in my packages.
-Being able to change configuration from within VS WITHOUT resetting the connectionstring string.
The recommended way for setting this up would be to store the file location of the dtsconfig file in an environment variable. Then change the dtsconfig to use the environment variable rather than a hardcoded location.
So the nuances of that scenario are this:
The password gets blanked out when you resave the xml file (as you pointed out in your question). This is what it is, and it is one of the reasons I never use them.
A process (devenv.exe) will cache the values of the environment variables on start up. This means you need to restart visual studio if you make a change to the value of the environment variable.
The same issue above applies to the integration services service. This will need to be restarted after you add environment variables. Or when you run your packages, the values will not be found.
The idea is that your dev machine points to a dev instance. Then as you migrate the packages to new environments - QA, Prod, each server has it's own environment variable pointing to its respective dtsconfig file.
As a side note, a similar pattern which avoids the password obliteration would be to add a sql connection manager which points to the server which will load the rest of the configurations. Then set the connection string of this connection manager with an environment variable. The advantage is that you don't have to go copying config files around. This works best with integrated security so you are not storing credentials in an environment config. If you want to be more cryptic about it, you could use a registry entry.
I converted a big project from SSIS2008 -> SSIS2014
During conversion i also erased all connection managers and recreated them based on the packages data sources.
Now i want to test my packages on another server , so i changed my 2 data sources only.
When i open the packages can see on the connection-string the old server.
If i dont do anything else and press start, what will happen?
Will the package update itself automatically and get the connection string of the data source or not?
If it doesn't then what is the point to have the option "create connenction manager from data source" and assign a DataSourceID to the package connection manager?
I am in package deployment model.
I have an SSIS package that takes data from a MySQL database and puts it in to a SQL Server table. The connection to MySQL is ADO.net. The package runs fine in BIDS.
I would like to deploy the package to be run as a Job on SQL Server 2008. The MySQL connection requires sensitive data to be either stored within the package or held in some external configuration source. If it is stored within the package it will either by encrypted with a key specific to my windows user profile or must be protected via a password.
I have tried three methods of setting up a SQL Server Agent job to execute this pacakge, and all fail:
Using a Password
I set the ProtectionLevel of the package to EncryptSensitiveWithPassword and supply a password for the package. I then save a copy of the package to the msdb. I can then connect to SSIS and run the package, at which point I am prompted for a password.
When I try to schedule this as a job in SQL Server I am prompted to enter the password by clicking on the Configurations tab of the Job Step Properties and I can see that the /DECRYPT switch has been added to the Command Line tab. The Run As property is set to the SQL Server Agent Service Account, which is mapped to a database login that has sysadmin server role. When I attempt to start this job I get the following error:
Executed as user: DOMAIN\UserROLE. Microsoft (R) SQL Server Execute Package Utility Version 10.0.5500.0 for 64-bit Copyright (C) Microsoft Corp 1984-2005. All rights reserved. Started: 12:31:30 Error: 2012-07-03 12:31:31.20 Code: 0xC00291EC Source: Get Data Execute SQL Task Description: Failed to acquire connection "DATA_SOURCE". Connection may not be configured correctly or you may not have the right permissions on this connection. End Error DTExec: The package execution returned DTSER_FAILURE
It seems that using this method the package password itself is not stored anywhere for the job to access it.
Using an XML configuration file
I then created a copy of the package and set the ProtectionLevel to DontSaveSensitive and I set up a package configuration to store the MySQL credentials in an XML file. I then save this to the msdb and then when I run the package from
Integration Services I can provide a path to the config file, and all is well. However, once again when I set up a job for the package I point to a configuration file on the server but the job fails with the same error. I am wondering whether this is because the SQL Server Agent does not have an associated windows account that would allow it to read from the XML file stored in the file system.
Using a SQL Server table configuration
This time I created a copy of the package and set the ProtectionLevel to ServerStorage. I then created a [SSIS Configurations] table in the target SQL Server database. I modified the entries in this table to contain the password for the MySQL data source. Again, the package runs successfully from Integration Services, but fails with the same error as previously when run from a job. I have checked that the SQL Server Agent is able to read from the [SSIS Configurations] table ok - I set up a standalone job to test this.
Does anyone have any suggestions about what else I should do here?
I am thinking of giving up on SQL Server Agent Jobs for this and going back to having the package scheduled simply by the windows scheduler but thought before throwing in the towel I would try here first.
I am using the method of encrypting the password by setting EncryptSensitiveWithPassword as you discribed above and don't see anything you do wrong. Remember to use option keep protection level of the original package when importing the package with integration services, but it looks like you're doing that, too.
Try setting the role SQLAgentOperatorRole to your SQL Server Agent Service Account, you didn't mention that and i am not sure if you did. This could possibly solve your permission problems. (liked to answer by commenting this, but i lack of permissions here, too :)