Does anybody know of an easy way to export test results in Hudson? We have some integration tests that are failing intermittently and our executives would like an excel-style report of the failures, in order to do this we need direct access to the backing data or perhaps the ability to export to csv.
Without knowing specifics of your build languages or integration testing frameworks, I suggest that you write a build task that exports your testing output into some common location, on or off the server.
You could also publish the testing output as an artifact from the job, and have an external tool pull the data from the permalink on the job's page.
Furthermore, you could create a Hudson job that runs automation to transform the testing output into the Excel format that your executives desire. For example, if you are using JUnit, you could write a script to get the test's output XML from the build, parse it, and output into an excel format. This Hudson job might run nightly or could be triggered after the build of the main project.
Related
Does it make sense to use Apache Airflow to orchestrate/automate ETLs, and subsequently dispatch reports via SSRS or SSIS? Put another way, I'm wondering if I can wire up an Airflow DAG to SSRS or SSIS. The MsSqlOperator Airflow operator looks like it interfaces with SQL Server, but I can't find any reference to a provider for SSRS or SSIS.
This is meant a very general question, I'm looking for directional guidance, as opposed to code examples (although I'll happily take those). Just wanting to know if I am going to pursue something that isn't going to work or is otherwise a bad idea.
Assuming you're looking to orchestrate the running of a report or running of an SSIS package, yeah sure, that's doable.
SSIS
Assuming you're working with the project deployment model, a package run is a few stored procedure calls strung together
create_execution creates an instance of an execution
set_execution_parameter_value allows you to configure values for an instance of that execution
start_execution begins the actual running of the package
Depending on whether you want the package running in synchronous or asynchronous (default) mode, you might want to set the SYNCHRONIZED bit.
SSRS
I'm not sure what you're looking for here but we trigger the proc add_event to kick off a subscription which then gets emailed out but I have seen plenty of questions from people that want to run an ssis package that pulls a report and exports to csv/pdf/etc
https://businesswintelligence.com/content/26/manually-trigger-ssrs-subscription
Docs/Learn
https://learn.microsoft.com/en-us/sql/integration-services/system-stored-procedures/catalog-create-execution-ssisdb-database?view=sql-server-ver16
https://learn.microsoft.com/en-us/sql/integration-services/system-stored-procedures/catalog-set-execution-parameter-value-ssisdb-database?view=sql-server-ver16
https://learn.microsoft.com/en-us/sql/integration-services/system-stored-procedures/catalog-start-execution-ssisdb-database?view=sql-server-ver16
https://andyleonard.blog/2015/11/the-synchronized-ssis-execution-parameter/
I am trying to zip excel files using SSIS Process Task. This task asks for Executable. Unfortunately on the machine where I am executing SSIS, does not have any 3rd party provided compression utility. All I have is Windows built in compressor. I want to know how this can be used as executable and zip my files.
Thank you for your time.
You could use 7zip's command line version without installing it.
http://www.7-zip.org/download.html
Also, if your excel files are outputed as '.xlsx' they are already compressed. This is also true for any other office type that ends in an 'x'. Try changing the extension to '.zip' and open them.
Edit: Depending on how much you're doing you might want to consider buying the Cozyroc package which includes a zip task as well as many other useful ones. Not too costly either.
http://cozyroc.com/
I just want to add this note about cozyroc
You can test and develop with the SSIS+ components for as long as needed to determine if the components are suitable for your needs.
When you run the SSIS package from the command line with DTEXEC or SQL Server Agent, the license verification code is activated.
If a license is not found, the package execution will fail, the first time it tries to use SSIS+ library component.
I would like to consolidate all my jUnits onto a single web portal (like fitnesse) and build them every night for a consolidated report. Are there any opensource tools available for it ?
I think you need continuous integration system, such as CruiseControl and Hudson. You could write Ant script to compile and run your test cases periodically.
I am new to SSIS. My problem is , I have to do a databse query. Then, I have to do a web service call passing in the result that I got from the db query, to the web service .
Either I can use sql task first for db query and web service task next for web service call. Another way is to do do entire thing in the c# code and put that in script task.
I can use enterprise library for doing db query through the C# code and then call the web service.
What is the better approach to do it ?Which one gives better performance?
Both approaches can work. Both will likely result in approximately the same performance - usually the query execution on SQL Server is the slowest part, and it does not matter how you invoke it.
But if you do everything in C# code and put it in script task - what is the benefit of SSIS? You can as well do everything in C# and put it in standalone console app.
Usually one uses SSIS to avoid writing code, and use declarative program definition - which is more maintainable, easier for others to understand and support, etc. Someone who opens a package and sees SQL task and Web Service task will be able to understand and tweak what's going on without opening Script Task and examining the code. It also requires less developer expertise to maintain such package.
Script Task is then only used when something can't be done directly by existing SSIS task or transform. If I saw a package with all the code in a Script Task - I would ask why the person used SSIS at all? Doing it with standalone console app would be simpler, more reliable (than the same code inside Script Task), avoid unnecessary dependency, etc.
Short: if you DO use SSIS - avoid Script Task when possible.
I've hit the inevitable state where I have to do a diff on the code within two versions of a SSIS package.
What have you used successfully other than what I'm going to do now by opening up 2 instances of VS and go over it box by box and variable by variable?
Note: The things that are important to compare in my case are:
Variables
Code in Execute SQL Tasks
Order of Tasks
Data Flows
Order of data flow components
Try BIDS Helper. It prepares both versions by normalizing whitespace and splitting long lines before making the comparison. Most changes can be easily recognized.
A Visual Studio extension has been deployed in March 2017 to Visual Studio Marketplace. It is called SSIS Compare and Merge Tool. You can install it using Tools menu than Extensions and updates and search for online extensions or download and install the .VSIX file from Visual Studio Marketplace
https://marketplace.visualstudio.com/items?itemName=TamasTIPost.SSISCompareMergeTool-18170
There are some utilities that will do this:
http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.sqlserver.dts&tid=0619e97f-4dd4-4946-bd41-888e751a5d72&cat=en_US_2b8e81a3-be64-42fa-bd81-c6d41de5a219&lang=en&cr=US&sloc=&p=1
ApexSQLDiff
Randy
I use Notepad++ to compare dtsx (xml) files. Some times even write some code to extract components for example to extract all SQL stored in a large SSIS package with dozens of ExecuteSQL tasks.
UPDATE: Just found this Bi xPress. Here is a video explaining how it works
Updated with the latest locations and consolidated the different answers.
There are a number of tools that try to help managing, comparing and merging SSIS and other otherwise difficult to merge file formats favored by SQL Server Analysis Services, Integration Services and reporting services.
Putting SQL code in stored procedures and managing these using SSDT and Git/TFVC is a useful first step. For the more exotic file formats extend your toolbelt with additional tools like:
BI Developer Extensions (formerly BIDS Helper) free!
Apex SQL Diff Pro
Bi xPress
SSIS Compare Merge Tool
Each work slightly differently and the cost varies, but they all apply normalization and visualization to help you understand and potentially merge the differences between these files.
Aside from that, many changes may be possible with enough understanding of the XML using tools like Notepad++ or Araxis Merge. Many merge tools now have have special XML compare/merge capabilities where you can configure how files should be normalized prior to comparison.