I need to execute 29 ssis packages. So planning to create one master package which will execute all these packages. I don't know how to implement this. Can you please explain in brief. Thanks in Advance !
This article does a pretty good job on giving a high level summary of what a master package does, which is basically a package executing other packages in the control flow: http://dichotic.wordpress.com/2006/10/30/creating-the-master-package/
This article goes over logging, variable scope, transactions, etc. from a fairly high level : http://74.125.95.132/search?q=cache:zMWg5KkFGZcJ:dmachicago.com/WhitePapers/SSIS%2520Master%2520Control%2520Package.doc+ssis+master+package+template&cd=3&hl=en&ct=clnk&gl=us
Alternativly, just use the execute package task. You have to deploy your ssis packages to the server first for this to work.
Related
I have about 112 packages. Actually these packages are executing sequentially using package master.
I need to execute these packages parallel. I create package master and add all these packages manually but it's not easy for support and maintenance. Is there a way or an idea how to extract all packages names from table parameter for example, then execute them in parallel automatically and not add packages manually?
You can run an execute SQL in a foreach loop (in SSIS) or you can run in a CURSOR in SSMS.
This is a assuming you deploy to SSISDB.
BEGIN
DECLARE #execution_id BIGINT
EXEC [SSISDB].[catalog].[create_execution]
#package_name=N'[Package Name].dtsx'
, #project_name=N'[ProjectName]'
, #folder_name=N'[FolderName]'
, #use32bitruntime=False
, #reference_id=NULL
, #execution_id=#execution_id OUTPUT
-- Execute the package
EXEC [SSISDB].[catalog].[start_execution] #execution_id
END
You can also add parameters if needed.
This is a good article about different applications: https://www.timmitchell.net/post/2016/11/28/a-better-way-to-execute-ssis-packages-with-t-sql/
Note: The default is run async
SSIS does not have support for such scenario out of the box. There are some approaches to do this, but with some limitations.
Async Out of main process Execution
The approach described in #KeithL's answer is pretty straightforward, but has certain drawbacks:
Each child package started in its own execution. You can end up having started all 112 packages experiencing resource shortage. SSIS Engine does not orchestrate overall resources in this case.
Package started in its own execution. It does not inherit parent package params or connection managers. You have to set it anew from calling SQL code or environment variable mappings.
From support point of view - you get 113 independent executions in the SSIS log. Not easy to link it especially on high-load system.
In-process execution
Alternative - handle this parallel invocation scenario with SSIS Task Package Run. From SSIS design approach, you have to describe all packages being run in the master package. Constructing the master package by hand is not practical, so you can use BIML. BIML can build your master package based on some metadata; it is really powerful but requires learning and practice.
Limited parallel execution
Create a For Loop block in SSIS with control variable i and loop exist condition i==0. In the block - set i = select count(*) from <package_queue> with (updlock, readpast) with SQL Task and process to dummy Expression Task if i==0; this is a loop break path. Otherwise, with path condition i>0 read one line from the package queue removing if from the queue, and start selected package execution with Package Run task.
Quite complex, but this is not the end of the story. Make several copies of the Copy Loop block in the SSIS Master package. Thus you will get a framework for concurrent execution with limited concurrency.
I have an issue that is bothering me, and I was thinking maybe someone might be able to help.
The thing is, I have a project made up by multiple packages that are executed within a master package.
When i run the master package, the execution begins, but when it hits the first package that it needs to execute, it just stops.
The output shows me the following things:
SSIS package "Master_Package.dtsx" starting.
SSIS package "Master_Package.dtsx" finished: Canceled.
If i enter that package and run the data flow in it, it works perfectly, but if i run the entire master package, or i just execute that package alone within the master package page, it just stops.
Any help would be gladly appreciated.
Thank you for your time!
Calin
Silly and obvious question: Are you using the execute package task on your master package? If so, are you referencing the right child package at the correct location?
It is a straight forward call from Master to Child even if you are passing parameters within the packages and should be no reason for child package to abend unless you are not calling it correctly.
I was wanting to know if there was a way to run an SSIS package, besides in the IDE to monitor the progress by step.
To explain this a little better when you run a DTS package that has say 5 steps in it,the process shows you each step, which step it is currently running, which ones have fail/sucsess and so on.
Is there a way to do this with a SSIS package.
I tried running a SQL job that did 2 SSIS packages and 2 T-SQL statements but, the job just showed the job running and no details about what it was doing or what step it was on and so on.
Thanks in advance for any help you can provide on this topic.
Robin
Have you tried out Logging? Package logging should be able to give you a whole lot of information about the package such as the component run time, success/failure and etc. If you want to view it on the GUI side then I am afraid, you will have to run it in BIDS which is not optimal when you think about production environment.
I realise this would be massively complex under the hood in SSIS.....
But is there a way to run a SSIS package in Visual Studio in a demo mode, that does not alter any data. Only runs a "simulation"?
I can imagine this would be very complex under the hood - running all sorts of transactions and rolling them back reliably.
But other microsoft tools such as MSDeploy are able to simulate (but admittedly it would be much simpler), so I'm hoping (I think I might be asking too much), that SSIS might have a similar feature
thanks for the help
There is no demo feature in SSIS. To do what you want, you'd be best to have a script that you run after the package execution to restore the database to it's original state. Or to restore a stable backup of the database.
No, there's nothing built in to SSIS that will run a package as a simulation.
I'm trying to programmatically add an Execute SQL task from within a script task of my SSIS package. I know that the Microsoft.SqlServer.Dts.Runtime.Package class has an Executables collection which I can add my new task to but how to I get a reference to the package Im inside of?
It is not possible, SSIS does not support self-modifying packages.
To prevent any attempts of doing what you are trying to do, the task code does not have access to the package API, so you can't obtain reference to Package object from a task. But even if you find a way to curcumvent this - the results are not predictable, as package is not allowed to modify itself at runtime.
If you could describe what you really want to achieve (rather than asking for particular way to do it) - someone might find a way to do it. Maybe you can use child package - it is OK to modify child package before its execution, or maybe it is enough to just change some variables that are used by Execute SQL task later in this package?
As Michael says above, it's not possible to do what you are asking. However, you may be able to find a solution by setting variables at runtime, or by enabling or disabling certain packages at runtime using dtexec. For example:
dtexec /f e:\ssis\master.dtsx /set \Package\YourPackageName.Disable;True