I'm trying to move a table into SQL Azure using SSIS. I have an ADO.NET source and destination, but when I run it it always fails with an error relating to IDENTITY INSERT being off. From other posts it mentioned i should remove the check from the box "Use Bulk Insert when possible". I have and still receive the same error. Any ideas on how to move over IDENTITY columns into SQL Azure?
Thanks
by the message, the destination has an identity field, so you have two oprtions:
change the ADO destination to an OLEDB destination and check the
"keep identity" check box on your destination component
manually set identity_insert ON on the table you are trying to insert
records.
Related
In SSIS, using OLE DB command and moved in to another step column
mapping "it is showing only name else blank and i have to map all four column through insert under OLE DB command , even i can share error with this blog.I am using MS BI and SQL-2008 R2 version.Kindly help me out.
I'm looking for the best practice to insert or update rows from a MySQL connection to a SQL Server connection.
First of all, I added a ADO.NET data source to grab MySQL content (a simple table Supplier with two fields id and name). Then, I added a Lookup transformation to split new rows / updated rows. It works well when I need to insert new rows. However, I would like to use a Command OLE DB to update existing rows but It doesn't work due to a incompatibility between my connection manager and the component (ADO.NET vs OLE DB).
Any idea to update modified rows ?! Should I use a cache component ?!
Thanks in advance !
Just get rid of the lookup and conditional split all together.
Outside of your SSIS package, build a staging table that contains the fields you need for inserts/updates.
In your SSIS Package, create a control flow that does the following:
Execute SQL Task to truncate the staging table.
Data Flow task to load the MySQL data from the source system to the staging table. If you can do this based on a "changes-only" type process, such as using a timestamp that you check, it would be faster.
Execute SQL Task to perform an UPDATE statement on your target table using the staging table joined to the target table.
Execute SQL Task to perform an INSERT statement on your target table using a query based on the target table and your staging table (with a WHERE NOT EXISTS or some such on a key fied)
I would change the SQL connection to use OLE DB. As well as allowing the OLE DB Command to work, you may also find the OLE DB Destination is faster.
I am trying to create an SSIS package for integrating between MSSQL and MYSQL. I have no prior experience of working with Bids or SSIS and following the instructions from here.
I added the OLE DB Source, Lookup, Conditional Split, OLE DB Destination and OLE DB Command components to the Data Flow and configured the connection managers and column mappings upto the Conditional Split component.
From here, I am facing two problems -
1) After configuring the OLE DB Destination, it shows error symbol on the component that says could not convert between unicode and non unicode string datatypes. To solve this, I tried to insert a Data Conversion Component between the Conditional Split and the Destination and configured it for the problematic column. But that doesnt seem to help
2) While configuring the OLE DB Command, the right hand side column in Column mappings tab shows zero columns. I have added the Sql command with question marks so i guess it should be showing columns named "Param_0", "Param_1" etc if i am not wrong. I even tried to add them manually from the input and output properties tab but then it shows the warning, external columns for OLE DB command are out of sync with data source
What am I missing here ?
Thanks
The way you describe your first problem, it sounds like it should work. Here are a couple of things to check.
The data conversion component creates a new column for the converted data. Make sure you are referring to it in your following transformations and destination.
Right-click on the Data Conversion component and select Advanced Editor. Select the Input and Output Properties tab in the Advanced Editor. Expand the Data Conversion Output branch of the tree-view and select your new column. Ensure that the Data Type Properties show the data type that you want to convert too. If these values are not right then something is not right with the setup in the component.
For your second problem, the issue can frequently be caused by an error with the SqlCommand value. First, make sure the Connection Manager is correct on the Connection Manager tab. Switch to the Column Mappings tab. Near the bottom of the form, there may be a warning message that indicates that the SQL statement cannot be prepared. In other words, SSIS can't figure out what the statement is supposed to do. Address any problems with the SQL statement and switch back to the Column Mappings tab. The columns will appear once the SQL statement can be parsed.
If you want to avoid the conversion issues then change your destination table column types from char/varchar to nchar/nvarchar. I'm pretty sure you will need to use an ADO connector for mysql source and destinations, you should be able to read data from the mysql source and write to the mssql database w/o using anything other than source and destination components.
When I use ADO.net source in DataFlow to read Blob Column and pass it to Script Component to do further validations - need script compoment to do further validations on each column to generate master / child error records master (for each row) and child (for each error column). This works fine.
As I need to parameterize my source, I can't use ADO.net and instead need to use the OLEDB Source which supports parameters. When I use this OLEDB source, the script component doesnt recognise the BLOB data being passed by OLEDB source. It reports datatype problems i.e., convering nonunicode to unicode.
How can this be done.
Regards
Can you confirm what your source database is (SQL Server, Oracle, etc).
I had the same problem using the 'Oracle OLEDB provider for Oracle' data source. The provider seems to convert every varcahr into an nvarchar. I solved this by adding a 'data conversion' component, and explicitly converting all nvarchar columns to varchar here.
The new columns are incuded in the output of this compnent, so you can link them to the fields on your spreadsheet.
I am trying to retieve data from a MySQL table and insert into a SQL Server table using ADO .NET connections in SQL Server 2008 SSIS. In my data flow task I have an ADO .NET Source which queries the MySQL table (select all invoices) and an ADO .NET Destination which inserts the data in my SQL Server table. Now, I would like to add a parameter in my data source so that I only select the max(invoiceNumber) retrieved from my SQL Server table. I have performed a similar task using "OLE DB Command" but the problem is that I need to query a MySQL database. Any ideas how I can achieve this?
I found that the only way to use parameter with the ADO.NET Data Source is this workaround:
Go to the Flow Control and selenct the Activity Flow containing your ADO.NET source.
In the properties window you can see the ADO.NET source Sql Command
Go to expression and select the properties: [YOU SOURCE NAME].[SqlCommand] and then edit the expression using variables to simulate parameters
Set Data Access Mode in ADO.NET Source to SQL Command and write the query.
You shoudn't have to add a parameter:
select *
from invoices
where invoiceNumber = (select max(invoiceNumber) from invoices)
The above works in SQL Server. I'm assuming that the same query will work in MySQL