Modifying my SQL 2008 SSIS Package from C# renders the modification invisible - ssis

I have a small solution that i provide to several people. The solution consists of amongst other things an existing SsisPackage (default.dtsx). I modify this according to some settings in my application to prepare it for transfering data from a datasource to a datamart. I add a sequence container, add a foreachloop and add a dadtaflowtask. Each of these are added (i know i can see them in the package explorer) and the package can execute without problems.
My problem is that when i look at my package i cannot see the elements i added, they are invisible, they exist in the xmla and are visible in the package. Anyone got a possible solution for this - it makes the package almost impossible to maintain for the receiving developer.

EDIT / SOLUTION: For posterity i will document my findings :-)
I have been conparing the underlying xml structure to find this solution so here it comes. The problem is due to the fact that im modifying an already created package. When the package was first made in BIDS layout information was saved within a xml tag called <packagevariable> - all elements have their layout written in encoded xml in the xmla file.
When i go and modify the package through C# and add new elements the layout information is not updated! So when the package is loaded within the BIDS for modification the BIDS recognizes that some layout information is available, so it will use this information to layout the ENTIRE PACKAGE. This means that while we have no layout information for the elements we have added, they are not layed out.
My next step will be to investigate into removing the layout information - then the package will layout the package in a default manner. I will investigate into how to remove it using the API (first priority) else i will have to open the file at XML/Text and make a manual removal (hopefully i will avoid this).
Enjoy!

Related

Opensource JavaDoc editor

I have spent the days looking for an Opensource JavaDoc editor or a way to edit JavaDocs easily or creating package-info files and come back empty.
I am basically after a way to change the way the packages are listed. The automation framework I work on is undergoing some potential future changes. These changes in a nutshell will organise our test scripts in a way where they are split up into business areas and then into smaller packages of business functions within that area.
We use IntelliJ and the default JavaDoc generator creates the JavaDocs in a way where it makes a list of all the packages, what I want to do is have it create a list of the main packages (business areas) and then the sub packages within that contain our test scripts (business functions).
For example:
BusinessAreaOne contain BusinessFunctionOne, BusinessFunctionTwo and BusinessFunctionThree.
BusinessAreaTwo Contains BusinessFunctionFour and BusinessFunctionFive.
The BusinessFunction packages in turn contain scripts that cover just that business function.
I haven't been able to find an editor that does what I would like and I don't think editing the files manually is practical each time there is an update.
If there is no opensource java editor can do this are there any suggestions of a work around?
As I mentioned I have tried creating package-info files but as far as I can tell you can only add a description to that package. Is there any tutorials out there or does anybody know a way I can get it to display the list of packages within the selected package?
Thanks in advance.
Javadoc doesn't treat package hierarchies as nested hierarchies, it just flattens them all into a list, as you probably noticed, so your business functions will just appear as entries in the list between the business areas.
Looking for a Javadoc editor is not going to help you because the Javadocs you end up with is the output from the Javadoc compilation process, and there's no specific tag you can put in your code that will influence the compilation process in the way you're looking for.
You would have to write something to override Javadoc's implementation of its HTML formatter specifically for the package list using:
Javadoc Doclets
You can customize the content and format of the Javadoc tool's output by using doclets. The Javadoc tool has a default "built-in" doclet, called the standard doclet, that generates HTML-formatted API documentation. You can modify or subclass the standard doclet, or write your own doclet to generate HTML, XML, MIF, RTF or whatever output format you'd like. Information about doclets and their use is at the following locations:
Javadoc Doclets
You would use your own implementation of the Standard Doclet, finding the right class to override - try looking first at the PackageDoc class. I'm going through this learning process myself at the moment, but here's the info on Doclet and there's a link at the bottom of that page to the source code which will illustrate how to do it.
Of course there could be an easier way of doing it and if I discover it I'll come back to update this answer.

Wrong HTML report when using -i option in cppcheck

I work for a middleware company. We would like to integrate Cppcheck into our build system to help preventing errors and issues in our code. Our code is big, and it's distributed in several modules (each module in a different folder). These modules have many dependencies between them.
When running cppcheck, we want to run it only once over the whole code to give the whole view to the tool. However, some modules are not related to the core ones, and we want to skip those modules from the analysis. Besides, we have implemented APIs for different languages. So for example, we have some modules for C++ that we would like to analyze separately from the C modules.
We have basically two options: 1) call cppcheck with a list of the modules that we want to analyze, or 2) call cppcheck from the top level folder of the code, and use -i options to ignore all the modules that shouldn't be analyzed.
Both approaches worked fine up to the point of creating the XML report. The problem appears when calling cppcheck-htmlreport. We observed that no index.html or stats.html were generated. Besides, only some of the results appearing in the XML were translated into HTML reports. For many results, the HTML pages were not generated.
Any memory problem can be discarded. We already verified this. Besides, the tool doesn't start creating HTML reports from the XML results consecutively and then it stops. Actually, what happens is that the HTML reports go jumping. I mean, the HTML report for error number 1 in the XML is created, then maybe next one is number 5, and so on.
We called cppcheck-htmlreport with --source-code option pointing to the top level folder of the code. I think the problem may be caused by this. I tried to call cppcheck just from the top level folder, with no -i options, and then the HTML reports were generated without issues. So it looks like the XML created by using -i options cannot be correctly understood by cppcheck-htmlreport.
Is there a way to provide -i options to cppcheck-htmlreport as well? I think this could solve the problem...
I have also noticed that the problem only seems to appear when many modules and code is analyzed. When analyzing only a few modules the HTML report was correct, although we still called cppcheck-htmlreport providing the top level folder as ---source-dir.
Is this a known issue in cppcheck HTML generator? Is there any way to solve this?
Any advice is very much appreciated.
Thanks,
Sonia

How does one make a self-contained package / library of functions

I am writing some support code in the common subset of Matlab/Octave, which comes in the form of a bunch of functions. Let's call it a package.
I want to be able to organize the package, i.e.,
put all the relevant function files in a single place, where users
are not supposed to store their code;
have some internal organization ('subpackages');
prevent namespace pollution;
have some mechanism for user code to 'import' parts of the package;
I don't necessarily want all functions I provide to be
visible from user clients.
On the Matlab side of things, this functionality is pretty much provided by package directories and the 'import' mechanism. This functionality doesn't appear to be available in Octave though (as of 3.6.1).
Given that, I wonder what options remain for organizing my support code package in Octave.
The option of putting everything in a directory and just have the user code do an ADDPATH feels rather unrefined, and doesn't give the level of control I want -- it only addresses point #1 of the list above.
There is plenty documentation here and examples in OctaveForge. Just browse the SVN.
Also there are personal packages all around. For example this one
Happy coding!

SSIS Common rownumber for both outputs on a flatfile source

I have a small problem (I assume...)
I'm loading a flatfile (csv) and I want to add a rownumber to the dataflow. Using the RowNumber transforation works good for both output paths (source and error) individually. But what if you want to use the same rownumber in both paths to be able to track where (in the file) an error occured. I have scratch my head long enough now and I'm just throwing it out here since I'm pretty sure other people has tumbled across this one...
I have tried the script transformation which seems to work for a while but then it hangs the load.
Any suggestion on how to solve this issue is greatly appreciated.
If I understand you correctly, dynamically generating the number with a script component for the dataflow is not a problem for you.
What I would recommend you is to adopt the following philosophy for stable etl processes coming from files:
Never cast anything in the connector, just import the fields as nvarchars of the maximum lenght they will achieve.
Cast and control each column to your specification.
If a row cannot be read, you will not know the index, but you will know that the file is malformed (extremely rare in my experience, for half transferred files), and it should be rejected anyway.
A quick screenshot of a part of a file loading process shows how the rejection (after assigning row_id) can work (link to dataflow image). To this you can add further countless checks (duplicates...) and even have a repository for the loaded files to check upon the rejects and whatever else you might want to control (Link to control flow image).
In some of my processes, I even use a flat file connector and just import each row as a bulk text and then split it in columns with an intermediate script component, allowing for different versions of the columns in the files.
Anyway, sorry not to be more detailed (due to my status I can't add more links or any images), but I hope that you understand the concept.
Regards,
Francisco.

How to display credits

I want to give credit to all open source libraries we use in our (commercial) application. I thought of showing a HTML page in our about dialog. Our build process uses ant and the third party libs are committed in svn.
What do you think is the best way of generating the HTML-Page?
Hard code the HTML-Page?
Switch dependency-management to apache-ivy and write some ant task to generate the html
Use maven-ant-tasks and write some ant task to generate the HTML
Use maven only to handle the dependencies and the HTML once, download them and commit them. The rest is done by the unchanged ant-scripts
Switch to maven2 (Hey boss, I want to switch to maven, in 1 month the build maybe work again...)
...
What elements should the about-dialog show?
Library name
Version
License
Author
Homepage
Changes made with link to source archive
...
Is there some best-practise-advice? Some good examples (applications having a nice about-dialog showing the dependencies)?
There are two different things you need to consider.
First, you may need to identify the licenses of the third-party code. This is often down with a THIRDPARTYLICENSE file. Sun Microsystems does this a lot. Look in the install directory for OpenOffice.org, for example. There are examples of .txt and .html versions of such files around.
Secondly, you may want to identify your dependencies in the About box in a brief way (and also refer to the file of license information). I would make sure the versions appear in the About box. One thing people want to quickly check for is an indication of whether the copy of your code they have needs to be replaced or updated because one of your library dependencies has a recently-disclosed bug or security vulnerability.
So I guess the other thing you want to include in the about box is a way for people to find your support site and any notices of importance to users of the particular version (whether or not you have a provision in your app for checking on-line for updates).
Ant task seems to be the best way. We do a similar thing in one of our projects. All the open source libraries are present in a specified folder. An Ant task reads the manifest of these libraries, versions and so on and generates an HTML, copies into another specified folder from where it is picked up by the web container.
Generating the page with each build would be wasteful if the libraries are not going to change often. Library versions may change, but the actual libraries don't. Easier to just create a HTML page would be the easiest way out, but that's one more maintenance head ache. Generate it once and include it with the package. The script can always be run again in case some changes are being made to the libraries (updating versions, adding new libraries).