I was wondering if anyone could help me with getting the excel (source) filename on my destination database in ssis. The database table has a column called filename (which is equal to the excel filename). Having the filename will help me with my audits. Note: I'm not a vb script person but if vb is the only way out, I'll appreciate sample script as I would have to use that in my logic. Thanks
You can use a "Derived Column" component and add a new column that contains the wanted property. I guess that you want to add a derived column that will store the "FileName" or "ConnectionString" property of your Excel Connection.
I would recommend saving the file as a CSV for multiple reasons. If it is a CSV file, you could create a flat file source for it. Then right click on the item and go to "Show Advanced Editor..." Click on the "Component Properties" tab and go to "Custom Properties" at the bottom. There is a field caled "FileNameColumnName." For the value, put the name of what column you want to store the file name in. Then, the that field will be available for your destination.
http://sqlblog.com/blogs/jamie_thomson/archive/2010/03/31/filenamecolumnname-property-flat-file-source-adapter-ssis-nugget.aspx
Related
I am creating a data flow with excel source and flat file destination.
When I create the flat file connection manager with the "Delimited" format I have no problems creating the txt file.
If I try instead to use the "Fixed width" (like it happens with "Results to Text" in SSMS) I keep getting erros like this one:
when I am in the "Columns" tab.
Also, when I try to run the package it tells me that "It's impossible to write the name of the column "this_column". It could be that the name of the column is bigger than the available column dimensions."
How can I make each column as long as the longest value?
When I create the SSIS package it requires a file to be referenced to pick up the files metadata. For example the column headers will be ColumnA, ColumnB.
I have always assumed that these column names need to be present in the file for it to be loaded. Recently business, for whatever reason, changed one of the column names in the file to something else so the file contains ColumnA, NotColumnB. When the SSIS package runs it ignores this and loads the file. I assumed that it would fail. Is my assumption correct and there is something weird going on or is my assumption incorrect, if so please let me know why.
I have changed the column names in a few other packages that load data from a file and they also dont care what the column names are
Click on the flat file source, and press F4 to show the properties tab. There are a property called ValidateExternalMetadata change it to True.
For more information check the following answer:
Detect new column in source not mapped to destination and fail in SSIS
Update 1
It looks like that flat file connection manager has no validation engine and the metadata defined is used at configuration time to configure the mappings between the data file and the database.
Why Does't SSIS Flat File Data Check If Columns Names or Order Have Changed? What is best way to check?
Flat file destination columns data types validation
I just added a new field to my OLE DB Source in an SSIS package. It then goes to a Flat file Destination. In the Flat file destination object, the field shows up under available inpt columns but doesnt show up anywhere else (Available destination Columns, or the bottom box that lists fields)
How do i get this as an available destination field?
Go into the Flat File Connection manager used by your destination object, go to Advanced, and click "New", then add the column properties.
Delete the destination and recreate it, it usually works.
I'm trying read the data from 2 csv files and export into the new excel file, but I'm not able to export the data in excel destination. While doing the mapping of the columns, there are 4 columns in the input columns but it is showing only 1 column in available output column that is only F1. Please let me know how to resolve this issue.
If I understand the question correctly, you are unable to map columns to a ‘new’ XL file.
If this is the case, the metadata for the mappings is probably the issue.
Try first creating a new xl file with the column headings and column types you want, then map to this.
Alternativly, right click on the excel destination and use the ‘Show Advanced Editor’ option and then adjust the columns in the ‘External Colums’ of ‘Input and Output settings tab.
You may then need to set the ValidateExternalMetadata option to false for the excel destination component in order to allow creation of new files from scratch.
Open an excel sheet and on the 1st row, give the column headings. Column A, B, C and D should have the same names as your source columns. After doing so, save the excel file and close the work book. Go to SSIS and open the Excel destination mappings. You should be able to map them now.
I have an SSIS package using a tab delimited flat file source with a TON of fields. Recently the provider of the tab delimited flat file has decided to change the format of the flat file by sprinkling a couple dozen new fields at random into the file. Needless to say, this hosed the package.
Rather than rebuild another flat file source and redefine all the fields, types, and lengths all over again, is there a way to reorder the fields in the flat file source? Sure would have been nice if Microsoft allowed you to move the fields around in the Advanced Columns pane, but noooooo.
Any help is appreciated.
If you only need to add columns to your file, you can do that in the Flat File connection editor. In the advanced window, you can select the field next to the new one and click the chevron next to the New button. It will give you the choice insert before or insert after.
If you truly have to move things around, you'll need to edit the XML source. If you use the existing file definition as a guide, you can build the new one in Excel or T-SQL relatively easily. Easier than typing everything in all over again at least.
I had a similar issue: I needed to change the order of columns in my flat file destination. The time-saving approach I settled on:
Delete the FF destination and FF connection manager (note down file name/location!),
Clear the check boxes that enable output columns in the source component
Re-enable the columns in the order you want
Add a new FF destination and FF connection right from the FF destination's connection manager drop-down.
Review/sanity check column sizes in FF connection, as usual
Not a direct answer to the question, but I came here looking for advice on "how to rearrange flat file destination columns", perhaps this will help someone.
I haven't seen an solution for that problem. SSIS isn't very strong in changing metadata. You could try to do it in notepad, but that is very tricky and very buggy. I would not recommand that to you.
In the connection managers below of your IDE you can double click your file name and edit everything you want.
This is still a "feature" of SSIS. To work around this I create a flat file connection called "NULL" with a single column named "NULL". Use the "New" button to add the column. I change the default column name from "Column 0" to "NULL". This column name must not match any column name in the list to be re-populated. If you have a real column named "NULL", pick something else for the column name that's not in use. You can keep the "NULL" flat file connection in the project for later use. (I expect to need it a few more times in this project.)
For this example, I use a flat file destination. Change the Flat File Destination to use the NULL connection.
Check the mapping to see there are no columns mapped. Saving this resets the metadata stored for the mapping.
Finally, change the Flat File Destination back to the correct connection to get a new mapping without metadata interference.
My example is a flat file destination. It should work for a flat file source for resetting the metadata. It is similar to the trick of changing a query to "select 1 as [NULL]" and back to purge metadata when using a ODBC source or such.
you could probably try something, but i havent tested.. use expressions to set everything for your flat file source? turn design time validation off