Changing DelayValidation on multiple SSIS packages - ssis

I have a project with about 150 packages in SSIS 2012. One of them is the entry point and the rest are called somewhere down the line from the main. Each of them creates its own table in the destination if one does not exist. I've found that I need to set DelayValidation to True for this to work, but loathe the job of doing it 150 times. Is there a way to have this property propagate or anything other than making a change to all the packages?

You can make this change by using the .NET library. Below is a PowerShell script that looks at all the packages in a given folder. If the DelayValidation property is false, then it will change it to True and save the package.
[Reflection.Assembly]::LoadWithPartialName("Microsoft.SQLServer.ManagedDTS") | Out-Null
$folder = "C:\sandbox\StackOverflow\StackOverflow\obj\Development"
$app = New-Object Microsoft.SqlServer.Dts.Runtime.Application
foreach($current in (Get-ChildItem $folder -Filter "*.dtsx").FullName)
{
Write-Host $current
$package = $app.LoadPackage($current, $null)
if ($package.DelayValidation -eq $false)
{
$package.DelayValidation = $true
$app.SaveToXml($package, $null)
}
}
Caveats
Never run automated code without testing that you have version control.
This only changes the Package's DelayValidation. Each Container and Task have their own DelayValidation property that may need to be updated
2017+ note
A user noted
SaveToXml function needs 3 parameter in SQL Server 2017.
That would make this SaveToXml look like
$app.SaveToXml($current, $package, $null)
The Documentation indicates this has been so since 2016.
They further noted,
As a Developer without installing SQL Server like me, need located your Microsoft.SQLServer.ManagedDTS.dll file explicitly
That would replace the first line's LoadWithPartialName to LoadFile with an explicit path like
([Reflection.Assembly]::LoadFile("C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Common7\IDE\CommonExtensions\Microsoft\SSIS\140\Binn\Microsoft.SqlServer.ManagedDTS.dll") | Out-Null)
Note the exact path will vary based on your installed tooling. I tend to use the dir /s /b (directory, search subfolders, bare format) command to find things so
C:\>cd "\Program Files (x86)"
C:\Program Files (x86)>dir /s /b Microsoft.SqlServer.ManagedDTS.dll
C:\Program Files (x86)\Microsoft SQL Server Management Studio 18\Common7\IDE\CommonExtensions\Microsoft\SSIS\150\Binn\Microsoft.SqlServer.ManagedDTS.dll
C:\Program Files (x86)\Microsoft SQL Server Management Studio 18\Common7\IDE\PublicAssemblies\SSIS\150\Microsoft.SqlServer.ManagedDTS.dll
Here we can see I have two ManagedDTS.dll both for the 150 (2019?) release of SQL Server

I've found the quicker way, which is finding DelayValidation">0< and replacing with DelayValidation">-1< in the code of each individual package. It is quicker than going item by item and alter the properties, but you still have to do it for each package.

Setting DelayValidation">0 to DelayValidation">-1 works in SSIS 2008 and SSIS 2008 R2, but it doesn't exists with SSIS 2012. By default, components in SSIS 2012 doesn't have the item - DTS:DelayValidation. Once you've set the component's DelayValidation to True, the following is added - DTS:DelayValidation="True". I'm using Visual Studio Ultimate 2012.

Related

Unable to cast COM object of type 'System._COMObject' to interface type

I have 3 SSIS packages. Two out of the 3 SSIS packages work perfectly, the third. Which is a copy of the 2nd one, except changing connection strings keeps throwing the problem:
Unable to cast COM object of type 'System.__ComObject' to interface type 'Microsoft.SqlServer.Dts.Pipeline.Wrapper.IDTSObject100'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{D4E5AF42-7999-473C-8082-6EFC676953C4}' failed due to the following error: The application called an interface that was marshalled for a different thread. (Exception from HRESULT: 0x8001010E (RPC_E_WRONG_THREAD)).
It has been doing this ever since I copied the package over. I have followed through online guides and run the regsvr32 dts.dll and this said it was successful but nothing has changed. I still get the error, why is this?
The SSIS package seems to validate my containers, it takes a long time to do this compared to the other two and then fails later on in the validation throwing the above error message.
You can't just copy the package, but you need additionally to change the name and generate new GUID (which identifies the COM object) for this package. Check this MSDN article for more info.
Here is the workaround: Solution Explorer -> right click project ->properties->debugging->Run64bitRuntime->set to false.
I hope it helps others. The solution worked for me:
I registered the assembly using the gacutil.exe. My SSIS project target server version was SQL 2012. So, I was using the DTSPipelineWrap.dll version 11.0. I opened the "Developer Command Prompt for VS2015" in Administrator mode, then typed the following command: C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\gacutil.exe /i "C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.DTSPipelineWrap.dll"
Then from my SSIS script task, References--> I removed the existing DTSPipelineWrap which was using the assembly from the location "C:\Program Files (x86)\Microsoft SQL Server\120\SDK\Assemblies\Microsoft.SqlServer.DTSPipelineWrap.dll". Then in the References folder, right click-->Add Reference-->Browse find the assembly from the location "C:\Windows\Microsoft.NET\assembly\GAC_MSIL\Microsoft.SqlServer.DTSPipelineWrap\v4.0_11.0.0.0__89845dcd8080cc91\Microsoft.SQLServer.DTSPipelineWrap.dll"
Click Ok. Rebuild the code and executed the package. It has worked for me.
In my case this is was a validation overload/timeout, the package contained references to hundreds of tables.
Opening the offending data flow forced re-validation and cleared the error.
Project --> Properties --> Configuration Properties --> Debugging
and then change Run64BitRuntime to False under Debug Options
In case this might help someone: I got this error due to timeout issues caused by a poorly optimised query. There was nothing inherently wrong with the SSIS package and it ran fine once i fixed the problem in the DB Source.

Pass value to variable in executing SSIS package

I have a SSIS package to loading some data based on the month and I want to call the package via windows batch file. Here is what's in the cmd file which is working fine now:
CD /D C:\Program Files\Microsoft SQL Server\100\DTS\Binn
DTExec.exe /f "E:\APAutomation\SSIS\AP\ActualDataImport_Console_PL.dtsx" /SET \Package.Variables[User::ActualMonth].Properties[Value]; "9"
Now I replace the 9 with a variable so to make it less hardcoded
CD /D C:\Program Files\Microsoft SQL Server\100\DTS\Binn
set ActualMonth = 9
DTExec.exe /f "E:\APAutomation\SSIS\AP\ActualDataImport_Console_PL.dtsx" /SET \Package.Variables[User::ActualMonth].Properties[Value]; "%ActualMonth%"
however, this won't work: this is the error msg:
C:\Program Files\Microsoft SQL Server\100\DTS\Binn>DTExec.exe /f "E:\APAutomatio
n\SSIS\AP\ActualDataImport_Console_PL.dtsx" /SET \Package.Variables[User::Actual
Month].Properties[Value]; ""
Microsoft (R) SQL Server Execute Package Utility
Version 10.50.1600.1 for 64-bit
Copyright (C) Microsoft Corporation 2010. All rights reserved.
Argument ""\Package.Variables[User::ActualMonth].Properties[Value];"" for option
"set" is not valid.
C:\Program Files\Microsoft SQL Server\100\DTS\Binn>pause
Press any key to continue . . .
I also tried:
set ActualMonth = "9"
DTExec.exe /f "E:\APAutomation\SSIS\AP\ActualDataImport_Console_PL.dtsx" /SET \Package.Variables[User::ActualMonth].Properties[Value]; %ActualMonth%
it threw me the same error msg. Pls help!
I think your specific problem is the spaces...
use this
set ActualMonth=9
instead of this:
set ActualMonth = 9
What you're doing is basically setting a package config from a batch file. It might work better if you set up the package to read from a package config file and put your month number in there.

dtexec error "The connection xxx is not found" with Project deployment model

When running DTEXEC I am getting "The connection xxxx is not found".
I beleieve this is because the connection managers are located at Project level and not within the package itself.
When running DTEXECUI - these connection managers are not displayed.
Is the only way to move them into the package - seems a bit weird as what is the point of allowing them a project level if you then have to move them to use them with DTEXEC.
Thanks
Here is the command line syntax you asked for:
C:\Users\Administrator>dtexec /FILE "\"F:\SSIS Projects\HESA\HESA\01 - Upload Metadata Files To Oracle.dtsx\"" /SET "\Package.Variables[User::varYear.Properties
[Value]";"1999" /CHECKPOINTING OFF /REPORTING EW /CONSOLELOG SMT
Your assumption that
the connection managers are located at Project level and not within the package itself
is exactly the problem. But there is a solution:
build the project to get a .ispac file
instead of invoking dtexec with /FILE you have to invoke it with /Project and /Package, like this:
/Project "path to you .ispac file, resulting from building the project"
/Package "Name of your package.dtsx"
Please, be aware that if you provide the whole path to your .dtsx package the execution will fail with very criptic SQLDUMPER error messages.

Problems using SQL Server 2008 R2 PowerShell extensions outside of SQLPS

I would like to have the SQL Server PowerShell extensions available to me whenever I start PowerShell by loading the snap-ins in my profile.ps1 script. I found an article here with a script example that shows how to do this, and this works fine on my 32-bit Windows XP box.
Unfortunately, on my 64-bit Windows 7 machine, this blows up. If I try to launch this script with the 64-bit PowerShell, I get:
Add-PSSnapin : No snap-ins have been registered for Windows PowerShell version 2.
At C:\Users\xxxx\Documents\WindowsPowerShell\profile.ps1:84 char:13
+ Add-PSSnapin <<<< SqlServerCmdletSnapin100
+ CategoryInfo : InvalidArgument: (SqlServerCmdletSnapin100:String
[Add-PSSnapin], PSArgumentException
+ FullyQualifiedErrorId : AddPSSnapInRead,Microsoft.PowerShell.Commands.AddPSSnapinCommand
If I run this instead in a 32-bit PowerShell, I get:
Get-ItemProperty : Cannot find path 'HKLM:\SOFTWARE\Microsoft\PowerShell\1\ShellIds \Microsoft.SqlServer.Management.PowerShell.sqlps' because it does not exist.
At C:\Users\xxxx\Documents\WindowsPowerShell\profile.ps1:39 char:29
+ $item = Get-ItemProperty <<<< $sqlpsreg
+ CategoryInfo : ObjectNotFound: (HKLM:\SOFTWARE\...owerShell.sqlps:String) [Get-ItemProperty], ItemNotFoundException
+ FullyQualifiedErrorId : PathNotFound,Microsoft.PowerShell.Commands.GetItemPropertyCommand
I'd like to be able to run this in a 64-bit PowerShell if possible. To this end, I tracked down what I thought was the Powershell extension dlls and in a 64-bit Administrator elevated PowerShell I ran:
cd "C:\Program Files (x86)\Microsoft SQL Server\100\Tools\Binn"
installutil Microsoft.SqlServer.Management.PSProvider.dll
installutil Microsoft.SqlServer.Management.PSSnapins.dll
No dice. Although installutil seemed to indicate success, I still get the "No snap-ins have been registered for Windows PowerShell version 2" error message when I run the script.
Anyone have any suggestions as to where I go from here?
I've used this script without issue on x64 machines. The problem with the x86 invocation is that the script looks for registry keys which on an x64 instance are only accessible from x64 PowerShell. For the x64 invocation you could try registering the snapins since that is the error message you're receiving. Run as administrator...
Change this:
cd $sqlpsPath
Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100
to this:
cd $sqlpsPath
$framework=$([System.Runtime.InteropServices.RuntimeEnvironment]::GetRuntimeDirectory())
Set-Alias installutil "$($framework)installutil.exe"
installutil Microsoft.SqlServer.Management.PSSnapins.dll
installutil Microsoft.SqlServer.Management.PSProvider.dll
Add-PSSnapin SqlServerCmdletSnapin100
Add-PSSnapin SqlServerProviderSnapin100
An even better solution is not use add-pssnapin instead turn sqlps into a module. I have blog post here:
http://sev17.com/2010/07/10/making-a-sqlps-module
Update for SQL Server 2012 - now ships a sqlps module you can install instead of the above blog: http://www.microsoft.com/en-us/download/details.aspx?id=35580
I realise this is a bit of an older question but with a stock standard Windows and SQL Server 2012 install you can just directly use the command Invoke-Sqlcmd without loading anything beforehand as it will auto import the sqlps module. However letting it do that will often cause issues so import the module yourself with the lines below in the same place in your code as you used to use the add-pssnapin commands
$cur = Get-Location
Import-Module 'sqlps' –DisableNameChecking
Set-Location $cur
Similar to that posted on this MS web forum.
The import-module line above changes the current path to something that makes UNC path strings like "\\server\share\path\filename.ext" not work with lots of cmd-lets. So we store the current path before and change it back after the import-module command.
It's possible the snapin assemblies are compiled for x86 only due to dependencies on native 32bit SMO COM objects. If it was possible to run them in a 64bit shell, I'm pretty sure MS would have shipped both x86 and x64 management shells.

Override SSIS configuration setting on command line?

I'm trying to run an SSIS package from the SQL Server Management Studio, and am having trouble overriding a configuration setting. In my case, it's the location of a flat file. The command I'm using is:
declare #returncode INT
exec #returncode = xp_cmdshell 'dtexec
/SQL "\ImportData"
/SERVER "myserver"
/CONNECTION "ImportData flatfile connection";"C:\files\ballot.dat"
/MAXCONCURRENT " -1 "
/CHECKPOINTING OFF /REPORTING E'
As you can see above, I'm trying to run this using c:\files\ballot.dat as the flat file in question. When doing so however, SSIS reverts to using the setting stored in its configuration file, which points to a different location (and ballot.dat file) on the hard drive.
Is there a way to override that when calling the package from the command line? Thanks for your suggestions.
What you can do is to add an SSIS Package configuration XML file. In this configuration you can specify all connection managers (just include the connection string). save this file as c:\otherconfig.xml or something like that. Edit the file, you should see your connection listed, and you can edit the connection string.
When running the package with dtexec you should be able to run it with that config file using /configuration.
Note also that there are lot of changes from 2005 to 2008 in how connections and package configurations are handled. See http://msdn.microsoft.com/en-us/library/bb500430.aspx for more details.
You need to set the *full" connection string, not just the file name...
/CONNECTION "ImportData flatfile connection";"Provider=...;Data Source=C:\files\ballot.dat"