I have a SSIS Package that Imports data from multiple CSV files into a multiple tables within a SQL database, I then do a SQL query to get the data I need and export into a csv file.
My issue is: I need to re-order the columns and I can't do this within the connection manager.
I can add new fields to the csv within the connection manager and can 'add before' or 'add after' to get the positioning correct but I cannot re-order the existing ones.
Is this possible within SSIS ? I have tried looking around but cannot see how to achieve this
i figured this out. I deleted the columns from the connections manager and re added them where I wanted them positioned. I then went into the flat file destination and re mapped the headings with correct field.
Related
I am trying to export a list of fields to a csv file from a database.
It keeps putting all the data onto one column and doesn't separate it. When checking the preview it seems to be okay but on export its not working. Currently trying to following settings. Any help would be appreciated.
SSIS settings
Excel file output issue
Actually it seems to work, Excel is just too dumb to recognize it.
Mark the whole table, then go to Data -> Text in Rows
And configure the wizard (Separeted, Semikolon as Separator):
Now you have seperated the rows and cells:
I have a scenario where my source can be on different versions of our database as a result the in source file I could have different number of columns while my destination have defined number of columns.
now
what we are trying to do is:
load data from source to flat files. move them to central server and
then load that data into central database. but if any column is
missing in flat file i need to add derived column.
what is the best way to do this?? how can i dynamically add derived columns?
You can either do this with BiMLScript as other have suggested in comments, or you can write a script task that reads the file, analyzes the contents, and imports it. Yet another option would be to bulk import the file as is to a staging table (that would have to be dropped and re-created everytime) and write a stored procedure that analyzes the DDL and contents, and imports data to the destination table.
One really annoying thing about Access is that when using the import wizard to import a CSV file, if there is a column in the CSV that isn't a field in the table being imported into, it gives you an error and shuts down the import. It should provide an option to skip columns in the source file if it isn't in the already existing table. Yes, it does give the option to skip if creating a new table from the source file, but that isn't what I am doing. Is there a solution to this or a way to automate it? Does using the VBA Access import function permit this?
The reason I want to skip these columns is because they were calculated columns in Excel file submitted by client, file is converted to CSV for importing in to Access, and I can easily recalculate them with SQL in Access. No reason to have them in the table.
When running the import wizard, you can save the specifications by some name you choose.
Then use:
DoCmd.TransferText acImportDelim, SpecificationName, ... etc.
I am new to ssis. I am actually having a ole db source and a flat file source. I want to look for each value of the specific column of the flat file in to ole db source. If match then get that perticuler row or (2 columns value of that row).
Earlier i was thinking that i can do it with Lookup but i am confused how to send the search key in to Lookup from flat file source.
Or can you suggest me another way around.?
Problem solved.
Actually my concepts were not clear about merge join.
It worked using merge join
Friends, I am using toad for MySQl, and have a huge database ready and validated.
Now i have an excel file which contains data-entries for a particular table. And i am also successfully able to import data into the db using import wizard, mapping the first row header with the column names etc.
But now i have appended a few data entries into it which i wish to insert into the database. However the old values also get selected and hence cause a primary_key_violation exception as the entry already exists! Otherwise a truncate table option is there which i dont wish to use as there may be many files from which i have inserted the data.
I tried my level best but didnt get any solution, atleast in toad for mysql. Please tell me what to do! the solution maybe simple but i need it SOS
An option may be to not append records to that excel file, but create a new excel file with only the new records