I am trying to retieve data from a MySQL table and insert into a SQL Server table using ADO .NET connections in SQL Server 2008 SSIS. In my data flow task I have an ADO .NET Source which queries the MySQL table (select all invoices) and an ADO .NET Destination which inserts the data in my SQL Server table. Now, I would like to add a parameter in my data source so that I only select the max(invoiceNumber) retrieved from my SQL Server table. I have performed a similar task using "OLE DB Command" but the problem is that I need to query a MySQL database. Any ideas how I can achieve this?
I found that the only way to use parameter with the ADO.NET Data Source is this workaround:
Go to the Flow Control and selenct the Activity Flow containing your ADO.NET source.
In the properties window you can see the ADO.NET source Sql Command
Go to expression and select the properties: [YOU SOURCE NAME].[SqlCommand] and then edit the expression using variables to simulate parameters
Set Data Access Mode in ADO.NET Source to SQL Command and write the query.
You shoudn't have to add a parameter:
select *
from invoices
where invoiceNumber = (select max(invoiceNumber) from invoices)
The above works in SQL Server. I'm assuming that the same query will work in MySQL
Related
I need to read data from a DB1 and write them to another DB2.
I use a complex query with CTEs and temp tables and no, i can't put this query in a SProc.
I use an OLE DB source and an OLE DB Destination.
When i put the query as SQL Command in the OLEDBSource I get the usual complaint about not being able to determine metadata because a CTE is using a temp table.
I can't use the "with result sets" workaround because it is not a SProc. So i try with the other workaround, the "SET FMTONLY ON/OFF" .
Now the OLE DB Source accepts my query but it outputs two datasets, the first empty and the second is the data I need. The OLE DB Destination doesn't write a single row because it is reading only the first resultset, the empty one.
How can i solve this?
I cannot change the temp tables in something else and basically i can't change the query. I am looking for a SSIS solution if possible, not a SQL solution.
Thx.
For an SSIS solution, you cannot use an OLE DB Source. That component can only access the first result set.
What you can do is use a Script Transformation as your data source, and access the second result set in the usual way, and send its columns to the output of the script.
I would like to transfer data from a MS SQL Server database to a MySQL database. So, I added a linked server to MS SQL so that I can use Openquery to insert the data in the MySQL database. I want to optimize the performance of the data transfer and I found the guidelines for improving performance of data loading in MySQL.
One optimization consists of disabling AUTOCOMMIT mode, however I was not able to do it using Openquery.
I tried both ways:
SELECT * from openquery(MYSQL,'SET autocommit=0')
exec openquery(MYSQL,'SET autocommit=0')
and I got:
Cannot process the object "SET autocommit=0". The OLE DB provider
"MSDASQL" for linked server "MYSQL" indicates that either the object
has no columns or the current user does not have permissions on that
object.
Is it possible to execute such statements through openquery?
Thanks,
Mickael
OPENDATASOURCE() and OPENROWSET() allow for add-hoc server connections. You do not need to define a linked server ahead of time.
The OPENQUERY() depends upon a static linked server being defined ahead of time.
Here is the MSDN reference.
http://technet.microsoft.com/en-us/library/ms188427.aspx
Most of the examples show a DML (SELECT, UPDATE, DELETE, INSERT) using the OPENQUERY() as the source or destination of the command. What you are trying to do is execute a session command. Therefore it will fail. Also, you might not even know if the session stays open for the next call.
Why not package up the logic on the MYSQL server as a stored procedure. The stored procedure can be executed on a linked server by using a four-part name?
For example:
INSERT INTO #results
EXEC server.database..stored-proc
This assumes MYSQL has the same object structure as ORACLE. Since I am not a MYSQL person, I can not comment. I allow you to research this little item.
But this should work. It will allow you to package any type of logic in the MYSQL database.
If you want to use SSIS to transfer data from SQL Server to MYSQL.
For the ADO.NET Destination to work properly, the MySQL database needs to have the ANSI_QUOTES SQL_MODE option enabled. This option can be enabled globally, or for a particular session. To enable it for a single session:
1 - Create an ADO.NET Connection Manager which uses the ODBC driver
2 - Set the connection manager’s RetainSameConnection property to True
3 - Add an Execute SQL Task before your data flow to set the SQL_MODE – Ex. set sql_mode='STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,ANSI_QUOTES'
4 - Make sure that your Execute SQL Task and your ADO.NET Destination are using the same connection manager.
Matt Mason did this on a reply.. The key is item #2, use the same connection.
http://blogs.msdn.com/b/mattm/archive/2009/01/07/writing-to-a-mysql-database-from-ssis.aspx#comments
Also, CozyRoc has a custom ODBC driver that might be faster / more reliable than the free one from MYSQL.
http://cozyroc.com/ssis/odbc-destination
Monthly, I need to get "new" data from our SQL Server business application into an Access 2010 database that contains all our "Management Reports". The Access database has a "staging table" that is to contain the raw data on which the reports are based.
I have no Access experience, but I suggested that we:
Write a query (stored proc?) on our SQL Server that returns the required raw data (...this bit was easy.)
At the end of each month, call the SQL Server stored proc from within Access 2010 (...click a button?)
Save the results of the stored proc into the staging table within Access.
But I'm finding it harder than I expected. I think I can get something ugly working using ADODB in a code-behind, looping through rows in a recordset one by one, and then setting column values one by one. But there must be a better way :)
How should I go about getting SQL Server data from Access 2010? (ADODB? DAO? QueryDesigner? other?)
Is there a "Insert Recordset Into Table" (or similar) mechanism that I can leverage?
Link the relevant sql server table or view to MS Access. Run a query against the linked table using MS Access syntax and update the staging table.
It is also possible to update an MS Access table using a connection string for SQL Server in-line in your query.
SELECT *
INTO newtable
FROM [odbc;filedsn=Z:\DSN\test.dsn].table1
Working from the SQL Server end, you can use MS Access as a linked server or run a query and update from there.
INSERT INTO
OPENDATASOURCE(
'Microsoft.ACE.OLEDB.12.0', 'Data Source=z:\docs\test.accdb')...[table1]
( atext )
SELECT atext FROM table1 WHERE id=2
I am working with a SQL Server 2008 R2 BIDS 2008 solution.
I need to transfer data from an Oracle(11g) source database to a SQL Server database. I need to copy all data from multiple tables in the Oracle database. I have a ForEach container in which there is one Data Flow Task.
I want to loop round a list of table names, select all columns out of in the Oracle source and copy to dest_ in the SQL Server.
I have added a Property Expression to the Data Flow Task for [ADO NET Source].[SqlCommand] with an expression of "SELECT * FROM " + #[User::ImportPath]
As the table is undefined at this point it isn't clear if I can map columns or setup the ADO NET Destination task correctly.
Is there steps I am missing? Is what I am attempting to do even possible?
You cant loop through tables and dinamically map columns on your source and destinations components. You would need one set of Source -> Ddestination per table.
If that's not feasible, you may want to lokk at the Transfer SQL Server Objects Task
I am writing an SSIS package that has a conditional split from a SQL Server source that splits records to either be updated or inserted into a MYSQL database.
The SQL Server connection has provider .NET Provider for OldDB\SQL Server Native Client 10.0.
The MYSQL connection is a MYSQL ODBC 5.1 ADO.NET connection.
I was thinking about using the OLE DB Command branching off of the conditional split to update records but I connect use this and connect to the MYSQL database.
Does anyone know how to accomplish this task?
I would write to a staging table for updates including the PK and columns to be updated and then execute an UPDATE SQL statement using that table and the table to be updated. The alternative is to use the command for every row and that just doesn't seem to perform that well in my experience - at least compared to a nice fat batch insert and a single update command.
For that matter, I guess you could do without the conditional split altogether, write everything to a staging table and then use an UPDATE and INSERT in SQL back to back.
Probably, the following MSDN blog link might help you. I haven't tried this.
How do I UPDATE and DELETE if I don’t have an OLEDB provider?
The post suggests the following three options.
Script Component
Store the data in a Recordset
Use a custom component (like Merge destination component)
The author also had posted two other articles about MySQL prior to posting the above article.
Connecting to MySQL from SSIS
Writing to a MySQL database from SSIS
Hope that points you in the right direction.