I've hit the inevitable state where I have to do a diff on the code within two versions of a SSIS package.
What have you used successfully other than what I'm going to do now by opening up 2 instances of VS and go over it box by box and variable by variable?
Note: The things that are important to compare in my case are:
Variables
Code in Execute SQL Tasks
Order of Tasks
Data Flows
Order of data flow components
Try BIDS Helper. It prepares both versions by normalizing whitespace and splitting long lines before making the comparison. Most changes can be easily recognized.
A Visual Studio extension has been deployed in March 2017 to Visual Studio Marketplace. It is called SSIS Compare and Merge Tool. You can install it using Tools menu than Extensions and updates and search for online extensions or download and install the .VSIX file from Visual Studio Marketplace
https://marketplace.visualstudio.com/items?itemName=TamasTIPost.SSISCompareMergeTool-18170
There are some utilities that will do this:
http://www.microsoft.com/communities/newsgroups/en-us/default.aspx?dg=microsoft.public.sqlserver.dts&tid=0619e97f-4dd4-4946-bd41-888e751a5d72&cat=en_US_2b8e81a3-be64-42fa-bd81-c6d41de5a219&lang=en&cr=US&sloc=&p=1
ApexSQLDiff
Randy
I use Notepad++ to compare dtsx (xml) files. Some times even write some code to extract components for example to extract all SQL stored in a large SSIS package with dozens of ExecuteSQL tasks.
UPDATE: Just found this Bi xPress. Here is a video explaining how it works
Updated with the latest locations and consolidated the different answers.
There are a number of tools that try to help managing, comparing and merging SSIS and other otherwise difficult to merge file formats favored by SQL Server Analysis Services, Integration Services and reporting services.
Putting SQL code in stored procedures and managing these using SSDT and Git/TFVC is a useful first step. For the more exotic file formats extend your toolbelt with additional tools like:
BI Developer Extensions (formerly BIDS Helper) free!
Apex SQL Diff Pro
Bi xPress
SSIS Compare Merge Tool
Each work slightly differently and the cost varies, but they all apply normalization and visualization to help you understand and potentially merge the differences between these files.
Aside from that, many changes may be possible with enough understanding of the XML using tools like Notepad++ or Araxis Merge. Many merge tools now have have special XML compare/merge capabilities where you can configure how files should be normalized prior to comparison.
Related
I have hundreds of BIML scripts and I have to convert each into SSIS package. The only process I figured out is to manually right click the biml file and convert it click the Generate SSIS Package. (Please follow the link to visualize it). How do I automate this process. In other words, how can I programmatically convert all the biml scripts into their corresponding SSIS packages..
https://www.google.com/url?sa=i&url=http%3A%2F%2Fwww.erikhudzik.com%2Ftag%2Fssis%2F&psig=AOvVaw3vHH8scEdHu5w-JUDrHyLi&ust=1657797254349000&source=images&cd=vfe&ved=0CAkQjRxqFwoTCLDFmZfe9fgCFQAAAAAdAAAAABAR
You should be able to select multiple files, right click and generate them all at the same time.
You can also reference one biml script from another. So you can have your main entry point which contains a <packages> element and then reference other scripts within that which define each package.
Finally, if you have biml studio, this comes with a command line utility which would allow you to do it programmatically.
There are 3 tools for transforming Biml into DTSX packages: BimlExpress, BimlStudio (formerly known as Mist) and the precursor to BimlExpress - BidsHelper. That product no longer exists and the Biml bits in it and has been rebranded.
Under the covers, BimlStudio is going to invoke Bimlc.exe which is the Biml Compiler and that is how scripts become packages. Buy it outright or rent it monthly, depending on your needs. This is your only choice for unattended/automatic/automated builds.
BimlExpress is the free tool that can also transform scripts into packages. It requires mouse clicks to build packages.
The big difference between the two, for the beginner at least, is convenience. I have ScriptedPackageA and ScriptedTablesB which makes Package1. In BimlStudio, I can set the properties so that one is live (tables) and just evaluate/expand the Package script. In BimlExpress, I need to shift/control click the scripts I want to be compiled/referenced.
Also if you have hundreds of Biml scripts... you might not have understood the idea behind Biml. For reference, I have about about 7000 .biml files on my machine but I bet I have less than 30 describing package patterns. The only way I'm at my large number of Biml files is that I have scripted a number of databases with an file per table.
Generally speaking, you want to distill your approach down to distinct patterns and then throw your metadata against it. How many ways can you have a package that loads from file to database?
Using the SSIS Data Tools extension for Visual Studio, it is possible to carry out various operations in a straightforward manner. However VSTA and the Script Task are a major exception—specifically in VS 2019.
After editing an existing Script Task the script text will reset to the default value.
Creating a new Script Task (other than the default) is likewise impossible, as far as I can tell.
I have tried editing the base64 string in which the VSTA task is stored within my .dtsx files.
After that I tried manipulating the files and directory structure under the VSTA temp directory in order to persist changes in the scripting solutions/projects. I have also tried many other tactics, such as including the task implementation as an additional source file, as a binary reference, or via partial classes, a base class etc., within the Script Task.
But none of these approaches (even if they were to work) are necessarily maintainable or simple in the context of a team.
Is there any sustainable long-term alternative that allows Script Task development in VS 2019?
The SQL Server Integration Services Projects extension for Visual Studio 2019 has a fix.
Script Task creation and editing appears to be working as of version 3.2.1 of the extension.
This was apparently released some time in the past few weeks (as of date of writing, 26 November 2019).
I know it's not a new question but maybe somebody find some documentation and/or the sql scripts and SSIS ETL used to create Adventure works DW (2014 at least).
Not sure why Microsoft released a lot about AdventureWorks for Analysis Services ( https://github.com/Microsoft/sql-server-samples/releases/tag/adventureworks-analysis-services ) but nothing for SSIS.
Any help will be much appreciated.
Adventure works DW (tables and procedures) are part of an executable which installs the SQL objects (facts and dimension) tables in the target edition of SQL Server instance.
Never has been the case to create such dimensional model using SSIS since this data is static and one-off. You may want to create SSIS artifacts on top of this DW and there are a lot of samples and workflows available online for reference.
One such repository on Git is - Repo link
I have a data warehousing solution formed of a series of databases, SSIS packages and an SSAS database. The SSIS packages and SSAS database all sit within source control using Team Foundation Server.
What I'd like to be able to do is branch the SSAS and SSIS projects to enable us to work on multiple streams of work and then be able to merge the projects back in prior to release to a production environment.
TFS allows me to branch my projects with little effort, however merging them back together afterwards results in trawling through pages and pages are difficult to consume XML.
How are other people dealing with this situation? Are there any tools available on the market to deal with exactly these situations?
As documented in this blog post by Jamie Thomson, SSIS files are effectively binary files so should be treated as non-mergable.
http://consultingblogs.emc.com/jamiethomson/archive/2007/08/06/SSIS_3A00_-Team-Development-Experiences.aspx
He also recommends making packages as modular as possible if you want to have multiple team members working on the same project - this is something we've adopted.
There is a tool called BIDS Helper which provides a 'smart diff' for SSIS files which can be useful for determining changes between versions.
http://bidshelper.codeplex.com/wikipage?title=Smart%20Diff&referringTitle=Documentation
But, generally, SSIS files should be treated as non-mergable if you want to avoid hours of pain - we've switched on exclusive check out on all .dtsx files in TFS so that people don't tread on each other's toes.
We have a large collection of DTS packages that need to be converted to SSIS as part of SQL upgrade?. How effective is this tool compared to the wizard?. Some of the functionality that is available in DTS
Import/Export
SQL operations
Copying/Renaming/moving files
Activex scripts (Not complex, most of the business functionality is in Stored procedures). Any help in sharing documentation or web links or any insight is well appreciated.
You can find a full comparison here: http://www.pragmaticworks.com/products/business-intelligence/dtsxchange/DTSxChange-vs-MSWizard.htm
At a high-level, the existing Microsoft wizard does not handle some common tasks like Dynamic Properties Task. It also doesn't handle things like ODBC or all the flat file conditions. With the DTS xChange tool, it will migrate pretty much all conditions, re-engineer the logging and auditing framework of the package and turn on some of the new features in SSIS. It also includes BI xPress, which will help migrate ActiveX Scripts post-migration with code snippets.