We have many SSIS packages that move, import, export around large amount of data. What is the best way to get alerts or notifications if expected amount of data is not processed? or How to get daily report on how different SSIS packages are functioning. Is there a way to write/use a custom component and simply plug it in to SSIS packages instead of writing custom component for each package?
For your first question, we use user variables in SSIS to log the
number of rows processed by each step along with the package name
and execution id. You can then run reports on the history table,
and if any of the executions have a large variance in the rowcounts
processed, you can trigger an event.
Yes. See here, or in the alternative, google "custom ssis
component tutorial".
Related
I have developed an SSIS package to run 3 reports from Reporting Services that are data driven subscriptions.
When I run the SSIS job it executes all the 3 reports at once, what I need is to run the reports sequentially, in other words, one by one. How can I do this?
This is an expected behavior. When you trigger a data driven subscription job, the SQL Server Agent starts the job and that completes the whole transaction. The SSIS package would then go on to trigger the next data driven subscription job and the next ( assuming you have put the job-triggering in sequence).
Now if you want to create a dependency in the way the jobs should run i.e. Job1 followed by Job2 followed by Job3, you need to manually write additional piece of code. The way to go about it would be to monitor the status code of the subscription.
In the ReportServer database there is a table called dbo.Subscriptions containing a column 'LastStatus'. Currently in my local db, I don't have any subscriptions and also am not able to find any documentation for the table. But I am pretty sure this would be either a boolean or a status flag such as 'Sucess' or 'Failure. Upon triggering the first job, you would need to write a .net Code to monitor this status with a polling interval. Once you get the desired outcome, move on to triggering the next job.
Hope this is clear. I would edit this answer with an working example.
I have a folder with around 15 reports in it, these are Report Server reports. To run each report individually will take a while, so I want them to run together. So, what I want to be able to do is somehow run all the reports in this folder, is this possible?
This is somewhat of an ambiguous question. Let me explain. What are you asking specifically?
Q: Can you run multiple reports at the same time?
A: Yes, and there are several ways to accomplish this.
1. You can use SQL agents
2. Use batch files with task scheduler
3. Use an SSIS package and use an agent to run them at specific times...etc...
Hopefully one of the reports does not depend on another and another thing that you have to take in to consideration is how hard you will be hitting the SSRS or SQL server. Running them all at one time may take longer than one at a time. depending on the bandwidth of the SQL Server and what tables are going to be locked up during each of these processes.
You might want to give a little more detail in your question...
I would recommend an SSIS package, especially as it also one of the options presented by #Michael that can email the Excel workbook too which you mentioned in an earlier comment.
The following resource covers quite well the execution and export of an SSRS report using SSIS, including code you will need as a starting point: Executing an SSRS Report from an SSIS Package.
You could save some time in coding the solution by using the following custom Task that can be integrated into SSIS: SSIS ReportGenerator Task.
There is one problem in your requirements though which is merging reports into one Excel workbook where I assume you want separate sheets for each report within the same workbook?
Reporting Services can use multiple worksheets (to divide a report up into pages a.k.a pagination) but only for a single report; it can't merge reports into one Excel file. This can be accomplished with custom code however. There's a somewhat basic example here: Merging workbooks into a master workbook with separate sheet for each file.
One way to run all the reports at once is to add subscription to all of them and set same subscription start time in all of the reports. what will happen is once the start time arrived all the reports will run simultaneously and will generate excel/pdf (any format specified) file at shared location.
I am using SSIS for ETL and I need to monitor a source folder for the source file to be arrived. When ever a file arrives I need to move that file into another location and rename the file and start executing another SSIS Package. Here we don't have an option to use use any other tool to automate the execution. We have only choice to use SQL Server, SSIS.
I need the mechanism and the logic to implement this logic.
I'm assuming that by "File Watcher" you don't mean FileSystemWatcher class in .NET, as there wouldn't be any point in using this class if you're limited to SQL Server and SSIS (you'd need a job with forever-running SSIS package containing ScriptTask with FileSystemWatcher).
The only solution is to create two-step job. First step would contain SSIS package for reading directory content and comparing it to files history log. Second step would contain your main package and would execute only if first steps succeeds or returns value indicating that there are new files to process.
Your answer is here and here. My personal favorite way of doing it is having an infinite loop package. Yet another way of doing it would be to encapsulate the entire logic in an SSIS package and fire it every X minutes. Vary the value of X depending on the urgency.
We have developed a simple web app which allows a user to upload an excel file. Our system picks the excel file and triggers an SSIS simple task to read an incoming excel file with 6 columns and push the data in the database.
Our concern is what will happen when mutiple users try to upload files simultaneously? Can SSIS serve multiple requests at the same time (which it should)?
What is the upper limit ie how many requests can it perform simultaneously?
What advice and suggestions you can give us in order for us have our server configured so that we can serve maximum requests?
same SSIS package can run more that once at the same time. The concerns you must have are the same concerns as if you were running 2 insert statments at the same time. SSIS is only the way you are using to insert the data. What happens when the data reaches the SQL Server engine is regardless of how it got there.
Just rememebr that SSIS uses bulk inserts
In my SSIS package there is a Script which has message box pops up for each item in foreach loop. This is good for me when debugging. But what happens when I run this package using Job? I know I want see any message boxes but does it stay somewhere in memory? Does it have any influence?
Try using either the event log or a database audit table to record the activity. You will not have a handle over any message box for a deployed package and you do not want a self contained scheduled package to require human intervention to keep it processing.