Using Apache Airflow to Automate SSRS and/or SSIS - reporting-services

Does it make sense to use Apache Airflow to orchestrate/automate ETLs, and subsequently dispatch reports via SSRS or SSIS? Put another way, I'm wondering if I can wire up an Airflow DAG to SSRS or SSIS. The MsSqlOperator Airflow operator looks like it interfaces with SQL Server, but I can't find any reference to a provider for SSRS or SSIS.
This is meant a very general question, I'm looking for directional guidance, as opposed to code examples (although I'll happily take those). Just wanting to know if I am going to pursue something that isn't going to work or is otherwise a bad idea.

Assuming you're looking to orchestrate the running of a report or running of an SSIS package, yeah sure, that's doable.
SSIS
Assuming you're working with the project deployment model, a package run is a few stored procedure calls strung together
create_execution creates an instance of an execution
set_execution_parameter_value allows you to configure values for an instance of that execution
start_execution begins the actual running of the package
Depending on whether you want the package running in synchronous or asynchronous (default) mode, you might want to set the SYNCHRONIZED bit.
SSRS
I'm not sure what you're looking for here but we trigger the proc add_event to kick off a subscription which then gets emailed out but I have seen plenty of questions from people that want to run an ssis package that pulls a report and exports to csv/pdf/etc
https://businesswintelligence.com/content/26/manually-trigger-ssrs-subscription
Docs/Learn
https://learn.microsoft.com/en-us/sql/integration-services/system-stored-procedures/catalog-create-execution-ssisdb-database?view=sql-server-ver16
https://learn.microsoft.com/en-us/sql/integration-services/system-stored-procedures/catalog-set-execution-parameter-value-ssisdb-database?view=sql-server-ver16
https://learn.microsoft.com/en-us/sql/integration-services/system-stored-procedures/catalog-start-execution-ssisdb-database?view=sql-server-ver16
https://andyleonard.blog/2015/11/the-synchronized-ssis-execution-parameter/

Related

Is it possible to run all the reports in a given folder from report server?

I have a folder with around 15 reports in it, these are Report Server reports. To run each report individually will take a while, so I want them to run together. So, what I want to be able to do is somehow run all the reports in this folder, is this possible?
This is somewhat of an ambiguous question. Let me explain. What are you asking specifically?
Q: Can you run multiple reports at the same time?
A: Yes, and there are several ways to accomplish this.
1. You can use SQL agents
2. Use batch files with task scheduler
3. Use an SSIS package and use an agent to run them at specific times...etc...
Hopefully one of the reports does not depend on another and another thing that you have to take in to consideration is how hard you will be hitting the SSRS or SQL server. Running them all at one time may take longer than one at a time. depending on the bandwidth of the SQL Server and what tables are going to be locked up during each of these processes.
You might want to give a little more detail in your question...
I would recommend an SSIS package, especially as it also one of the options presented by #Michael that can email the Excel workbook too which you mentioned in an earlier comment.
The following resource covers quite well the execution and export of an SSRS report using SSIS, including code you will need as a starting point: Executing an SSRS Report from an SSIS Package.
You could save some time in coding the solution by using the following custom Task that can be integrated into SSIS: SSIS ReportGenerator Task.
There is one problem in your requirements though which is merging reports into one Excel workbook where I assume you want separate sheets for each report within the same workbook?
Reporting Services can use multiple worksheets (to divide a report up into pages a.k.a pagination) but only for a single report; it can't merge reports into one Excel file. This can be accomplished with custom code however. There's a somewhat basic example here: Merging workbooks into a master workbook with separate sheet for each file.
One way to run all the reports at once is to add subscription to all of them and set same subscription start time in all of the reports. what will happen is once the start time arrived all the reports will run simultaneously and will generate excel/pdf (any format specified) file at shared location.

SSIS: printing ssrs report

There is a package with script task which prints SSRS report. Package runs and everything is just perfect. When I deploy it to production server it doesn't work and I know it's because of the printer I'm sending it to (it's invalid, it's not on the list).
The problem is... SQL Server Agent service account has printers configured, not to mention the printer I'm sending this report to is set as default.
The package is run by another account which has printers configured, correct permissions (for testing purposes this account can do pretty much anything), the fuzzy stuff.
The account running SSIS, in the other hand, can do very little (not even fancy kerberos passing :).
Any suggestions? If you could just point me in the right direction, it would be great.
EDIT
Package runs from SSIS, but doesn't run from SQL Server Agent. We are trying to ad-hoc fix the bug in another companys software (it doesn't create few reports, users don't want to use it, people walking backwards, cats living with dogs), waiting for patch. We can't change the code for obvious reasons, the only solution is to find reports which are not created and create them on our own.

SSIS Variables vs Parameters (SSIS Denali)

What is the difference between Variables and Parameters in SSIS Denali?
If there is any difference then What is that which Variables cannot do that Parameters can do ? or vice versa.
When should one go with SSIS Parameters and Variables?
I tried searching on Google, but I failed to get some information.
Thanks In Anticipation!
I think a little bit background will be beneficial to understand the Parameter concept. Here I will explain it in the context of comparing with Variables. To fully grasp the Parameter concept, you might need to look up for the new Project Deployment Model, Environment, Build Configuration as well..
Usage Of Variable
With SSIS prior 2012, if we need to pass any external values to the package before the execution (as we all do all the time), I normally use configuration file (or a couple of other ways). Say we have a file server, which will be used to access a shared file, I will use variable to store the server name, and expose this variable to the configuration file. If the actual file server is changed (dev env to test env etc.), we just need the change the value of that variable in the configuration file and SSIS package remains intact.
Everything looked good, but there are a couple of things that I always ask myself why and could not figure out why:
100% of the time when I am exposing variables to configuration file, I just expose the "Value" properties. Why does SSIS allow to expose all the other variable properties?
Why does SSIS not have "private" variable? By "private" I mean when I chose the variables to configure, the "private" ones just did not get shown on the pick list. The SSIS package could have dozens of variables, for the internal value-holders, what's the point to expose them? Why I have to scroll all the way to find the only one I need to expose?
New Project Deployment Model
SSIS 2012 introduces a new deployment model, Project Deployment Model. For short, this model deploys SSIS project as a single unit to SQL Server SSIS catalog, and package configuration is NOT available in this model (it is available in the old model referenced as Package Deployment Model, with SSIS 2012 you can choose which one to use, 2012 default to the new model).
If we want the pass some values into the SSIS packages, we have to pass them in via Parameters, and use SSIS catalog in SSMS to configure the value for the parameters(only the value, nothing else we can configure). Parameters and connection managers are exposed automatically in SSIS catalog which can be configured, nothing else previously available via configuration files can be configured in Project Deployment Model (The world is much cleaner). Inside SSIS package, parameters can be used in the same way as variables in terms of building up expressions. However, parameters can NOT be modified within the SSIS package, which makes perfect sense. (Why do we need to change a value which is passed in from external? If we have to, pass the value to an variable, and do the changes there..)
Sum Up
Parameter is only available in the Project Deployment Model, and it provides the only mechanism for passing values from external to SSIS packages in this model. If we think SSIS pacakge as an OO class, Parameters could be thought as public properties, which externals can access and assign value to it (the class itself can/will use it, but cannot modify it). Where Variables could be thought as private variables, which are used internally, external world does not need to know anything about it.
For the old Package Deployment model, there is no Parameter, and the world remains the same.
FYI, in short, variable's value can be changed during the runtime, but parameter cannot. Parameter can help you do the project deployment and you can set it up in SSISDB catalog, while variable cannot.
There are many differences between Variables and parameters, few of them mentioned below:
Variables values can change at run time but parameters value can't change .
Variables can be used only with in the package we can't use it for other package with in the solution but we can use Parameters for multiple package (package exist with in the Solution Explorer).
The variables & parameters are similar to that are in java,
we pass/through some values to certain method/task in the form of parameters and we use them in that particular task we cant change those values since they are external things for that method similarly in SSIS the Project Parameters are used to set certain variables or connections dynamically in the package. where as variables are limited internal to the package level.
It works like this:
say you have a project parameter called ServerName:
Lets say you deploy an SSIS package into two integration catalog environments, one which is configured for prod server and another which is configured for test server:
Then your ServerName 'parameter' will be set in prod with prod server address and in test environment to contain test server address.
If any variable in your ssis package needs a runtime value(say the variable is used to set a connection at run time for prod or test servers respectively) then the variable will use the parameter from above to find the right server to connect to.
So parameters are usually needed in environment specific scenarios.
There are two types of parameters based on how you've configured your solution in Visual Studio: Project parameters or Package parameters. Project parameters are accessible to all packages in the project.
Parameters are using send data from outside of the package like usernames, passwords or connectionstrings etc. Variables are using inside of the package. It means you can define a variable in one of your SSIS package and use it in package level.
Parameters in SSIS are like Global Constants in Programming, so if they be applied to the Project can be used anywhere. Their
highest access would be the whole project.
Variables as they are named, can be assigned, from a Query or even a Parameter, etc. Also their highest level access would the
package.

When to use script task in place of sql task and web service task in SSIS

I am new to SSIS. My problem is , I have to do a databse query. Then, I have to do a web service call passing in the result that I got from the db query, to the web service .
Either I can use sql task first for db query and web service task next for web service call. Another way is to do do entire thing in the c# code and put that in script task.
I can use enterprise library for doing db query through the C# code and then call the web service.
What is the better approach to do it ?Which one gives better performance?
Both approaches can work. Both will likely result in approximately the same performance - usually the query execution on SQL Server is the slowest part, and it does not matter how you invoke it.
But if you do everything in C# code and put it in script task - what is the benefit of SSIS? You can as well do everything in C# and put it in standalone console app.
Usually one uses SSIS to avoid writing code, and use declarative program definition - which is more maintainable, easier for others to understand and support, etc. Someone who opens a package and sees SQL task and Web Service task will be able to understand and tweak what's going on without opening Script Task and examining the code. It also requires less developer expertise to maintain such package.
Script Task is then only used when something can't be done directly by existing SSIS task or transform. If I saw a package with all the code in a Script Task - I would ask why the person used SSIS at all? Doing it with standalone console app would be simpler, more reliable (than the same code inside Script Task), avoid unnecessary dependency, etc.
Short: if you DO use SSIS - avoid Script Task when possible.

SSIS 2005/2008 - How do I allow multiple tasks to have a common target task?

In VB pseudocode, I'm trying to accomplish the following using SSIS (either 2008 or 2005)
If FileHasAlreadyBeenDownloaded = False Then
DownloadTheFileFromFTP
End If
ImportTheDownloadedFile
To do this in SSIS I have a script task to check for the file, and if it exists it transfers directly to the DataFlow Task using conditional expressions. If the file doesn't exist, it transfers to the FTP Task, and the FTP Task transfers to the DataFlow Task.
It seems, though, that I can't have two tasks lead into one common shared task, because no matter which path the code takes it won't execute the DataFlow Task. If I make a copy of the DataFlow task and have each path go to its own Task, then all works perfectly.
Is this a documented thing with SSMS that I just haven't found? I looked through 31 pages of questions on SSMS before posting, so hopefully this isn't a stupid question.
I also tried using Expressions on the FTP task to set "Disabled=#FileAlreadyDownloaded=True" but that works only in SSMS 2008 and didn't seem to work in SSMS 2005.
Thanks so much for any pointers on this!
It might be worth trying putting the script task and the FTP task inside a container task, and link the the container to the data flow task on success.