I created a SSIS package pro-grammatically in C-sharp. Now I am running the package by creating it and saving it as FileName.dtsx and executing it as package.execute();
Now what I want to do is run the package in the memory itself without saving it. Is this possible to do that.
Tried a lot but cannot figure it out how to do that.
Don't save it?
static void Amarnath()
{
Microsoft.SqlServer.Dts.Runtime.Package p = null;
p = new Microsoft.SqlServer.Dts.Runtime.Package();
p.Execute();
}
Related
How to Encrypt the password to Excel using SSIS package?
I thought, we can use script task to achieve this. I tried the below code by using the Spire.xls nugget package but the respective nugget dll reference are not adding to the solution.
Script task Code:
using Spire.Xls;
namespace ProtectExcel
{
class Program
{
static void Main(string[] args)
{
//Load Workbook
Workbook book = new Workbook();
book.LoadFromFile(#"C:\Test\Test.xlsx");
//Protect Workbook
book.Protect("vinay-123");
//Save and Launch
book.SaveToFile(#"C:\Test\ProtectExcel.xlsx", ExcelVersion.Version2010);
//System.Diagnostics.Process.Start("ProtectExcel.xlsx");
}
}
}
I tried the same code in c# console application it worked well but not working in SSIS script task.
Can anyone help me on this how to fix this or any other approach??
Script task Code:
using Spire.Xls;
namespace ProtectExcel
{
class Program
{
static void Main(string[] args)
{
//Load Workbook
Workbook book = new Workbook();
book.LoadFromFile(#"C:\Test\Test.xlsx");
//Protect Workbook
book.Protect("vinay-123");
//Save and Launch
book.SaveToFile(#"C:\Test\ProtectExcel.xlsx", ExcelVersion.Version2010);
//System.Diagnostics.Process.Start("ProtectExcel.xlsx");
}
}
}
Excel should be protected with password by using SSIS.
In SSIS, the dll has to be deployed to GAC to refer in the Script task code. Refer to the post on deploying to GAC for SSIS script task reference. For workaround on avoiding deploying to GAC, refer here
I would suggest you to generate exe out of the console application & use Execute Process task to call the console application with parameters, as explained here
It seems like a very common issue with SSIS packages is releasing a package to Production that ends up with running the wrong connectionstring parameters. This could happen by making any one of many mistakes or ommisions. As a result, I find it helpful to dump all ConnectionString values to a log file. This helps me understand what connectionstrings were actually applied to the package at run time.
Now, I am considering having my packages check to see if every connnection object in my package had its connectionstring overriden by an entry in the config file and if not, return a warning or even fail the package. This is to allow easier configuration by extracting all environment variables to a config file. If a connectionstring is never overridden, this risks that a package, when run in production, may use development settings or a package, when run in a non production setting when testing, may accidentily be run against production.
I'd like to borrow from anyone who may have tried to do this. I'd also be interested in suggestions on how to accomplish this with minimal work.
Thx
Technical question 1 - what are my connection string
This is an easy question to answer. In your package, add a Script Task and enumerate through the Connections collection. I fire the OnInformation event and if I had this scheduled, I'd be sure to have the /rep iew options in my dtexec to ensure I record Information, Errors and Warnings.
namespace TurnDownForWhat
{
using System;
using System.Data;
using Microsoft.SqlServer.Dts.Runtime;
using System.Windows.Forms;
/// <summary>
/// ScriptMain is the entry point class of the script. Do not change the name, attributes,
/// or parent of this class.
/// </summary>
[Microsoft.SqlServer.Dts.Tasks.ScriptTask.SSISScriptTaskEntryPointAttribute]
public partial class ScriptMain : Microsoft.SqlServer.Dts.Tasks.ScriptTask.VSTARTScriptObjectModelBase
{
public void Main()
{
bool fireAgain = false;
foreach (var item in Dts.Connections)
{
Dts.Events.FireInformation(0, "SCR Enumerate Connections", string.Format("{0}->{1}", item.Name, item.ConnectionString), string.Empty, 0, ref fireAgain);
}
Dts.TaskResult = (int)ScriptResults.Success;
}
enum ScriptResults
{
Success = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Success,
Failure = Microsoft.SqlServer.Dts.Runtime.DTSExecResult.Failure
};
}
}
Running that on my package, I can see I had two Connection managers, CM_FF and CM_OLE along with their connection strings.
Information: 0x0 at SCR Enum, SCR Enumerate Connections: CM_FF->C:\ssisdata\dba_72929.csv
Information: 0x0 at SCR Enum, SCR Enumerate Connections: CM_OLE->Data Source=localhost\dev2012;Initial Catalog=tempdb;Provider=SQLNCLI11;Integrated Security=SSPI;
Add that to ... your OnPreExecute event for all the packages and no one sees it but every reports back.
Technical question 2 - Missed configurations
I'm not aware of anything that will allow a package to know it's under configuration. I'm sure there's an event as you will see in your Information/Warning messages that a package attempted to apply a configuration, didn't find one and is going to retain it's design time value. Information - I'm configuring X via Y. Warning - tried to configure X but didn't find Y. But how to have a package inspect itself to find that out, I have no idea.
That said, I've seen reference to a property that fails package on missed configuration. I'm not seeing it now, but I'm certain it exists in some crevice. You can supply the /w parameter to dtexec which treats warnings as errors and really, warnings are just errors that haven't grown up yet.
Unspoken issue 1 - Permissions
I had a friend who botched an XML config file as part of their production deploy. Their production server started consuming data from a dev server. Bad things happened. It sounds like you have had a similar situation. The resolution is easy, insulate your environments. Are you using the same service account for your production class SQL Server boxes and dev/test/uat/qa/load/etc? STOP. Make a new one. Don't allow prod to talk to any boxes that aren't in their tier of service. Someone bones a package and doesn't set a configuration? First of all, you'll catch it when it goes from dev to something-before-production because that tier wouldn't be able to talk to anything else that's not that level. But if you're in the ultra cheap shop and you've only got dev and prod, so be it. Non-configured package goes to prod. Prod SQL Agent fires off the package. Package uses default connection manager and fails validation because it can't talk to the dev sales database.
Unspoken issue 2 - template
What's your process when you have a new package to build? Does your team really start from scratch? There are so many ways to solve this problem but the core concept is to define your best practices for Configuration, Logging, Package Protection Level, Transaction levels, etc into some easily consumable form. Maybe that's 3 starter packages: one for raw acquisition, maybe one stages and conforms the data and the last one moves data from conformed into the final destination. Teammates then simply have to pick one to start from and fill in the spots that need it. If they choose to do their own thing, that's the stick you beat them with when their package fails to run in production because they didn't follow the standard path.
There are other approaches here. If you're a strong .NET crew, you can gen your template packages that way. At this point, I create my templates with Biml and use that to drive basic package creation.
If I am understanding you correctly the below solution should work.
My suggestion to you is to turn on the Do not save sensitive option for the ProtectionLevel property at the top level of the package.
This will require you to use package configurations for every connection, otherwise it will not have the credentials to make a connection.
Task: Loop thru these excel files and insert data into SQL table but in the process i get an error and i don't know which it errored on.
My understanding is SSIS doesn't loop thru file in an random order but i get an error about CANNOT ACQUIRE CONNECTION FROM CONNECTIONMANAGER. Excel Source failed validation and returned error code.. I did set 64bitruntime to False. This happened on VS 2008/SQL Server 2008 R2 on Windows 7 OS. Initially i was able to run the whole process successfully on Windows XP- VS2008 /SQL Server 2008 R2.
Problem: How do i know which file system is going to iterate next if i have 70 odd files in a folder. The thing i get an error and i'm not sure which file SSIS is working on. However i do see files are executed and data is in SQL.
Let me know how to find which file SSIS is currently working or the next one it will work on.
You could add a Script Task and log the variable used in the foreach loop.
Add the variable as readonly variable in the script task editor and then add something like this in the main method (C#):
public void Main()
{
bool fireAgain = true;
Dts.Events.FireInformation(0, "Logging FELC variable", "File: " + Dts.Variables["User::FilePath"].Value.ToString(), string.Empty, 0, ref fireAgain);
Add a script task inside your ForEach container, immediately before you do the Excel processing. In the script task, add the variable you configured in your ForEach loop to hold the filename to the Read Only Variables. In the script itself, call the FireInformation event, which will add an informational message to the progress log in SSIS. In the FireInformation call, pass the value of your filename variable as the message argument.
This will let you see each file being processed, and which one it was processing when it failed.
FireInformation help: http://msdn.microsoft.com/en-us/library/microsoft.sqlserver.dts.runtime.idtscomponentevents.fireinformation.aspx
In SSIS Script task I am able to load the package by providing the full path as following, but on the server I don't know the exact path after deployment, Is there any way to load the package and its configuration file.
public void Main()
{
Dts.TaskResult = (int)ScriptResults.Success;
Microsoft.SqlServer.Dts.Runtime.Application app = new Microsoft.SqlServer.Dts.Runtime.Application();
Package package = app.LoadPackage(#"C:\tfs01\AURA\DB\Main\Src\ReportSolution\AURA_ETL\AURA_ETL\DTS_PatientModel.dtsx", null);
}
As always, the documentation is your friend.
Instead of calling LoadPackage, you need to use LoadFromSqlServer. LoadPackage pulls from disk, LoadFromSqlServer talks to the msdb.
Note that if you are working with the 2012 project deployment, it's an entirely different mechanism.
I have a parent package that needs to execute the same child package multiple times. To make things more fun, each instance needs to have a different value defined for the parent parameter passed to the child package.
I've created a script task using the following script:
Microsoft.SqlServer.Dts.Runtime.Application App = new Microsoft.SqlServer.Dts.Runtime.Application();
Package pkg = new Package();
try
{
pkg = App.LoadPackage(#"\\server\SSIS Packages\ChildPackage.dtsx", null);
pkg.Variables["ChildVariableName"].Value = Dts.Variables["AParentVariableName"].Value;
pkg.Execute();
Dts.TaskResult = (int)ScriptResults.Success;
}
catch (Exception ex)
{
Dts.Events.FireError(0, "Run child pkg for parent task", ex.Message, string.Empty, 0);
Dts.TaskResult = (int)ScriptResults.Failure;
}
Problem is, my packages are stored in the SSIS package store of my SQL 2008 R2 server and I can't figure out how to reference them; every code sample I've seen is for a physical location. This is on an HA cluster so having a physical location for the package will be difficult to maintain.
So I either need to (a) figure out how to change the value of ParentVariable every time an Execute Package task is kicked off for this child package or (b) figure out how to reference the proper package inside the SSIS package store, at which point I can safely pass the proper value. Anyone have any ideas?
Instead of App.LoadPackage method, you would use the LoadFromSqlServer method
app.LoadFromSqlServer("\OptionalFolderButSlashRequired\ChildPackage", "server", null, null, null);
The documentation on Application and Package usually have examples in the methods I've needed to use.