I am writing an SSIS package that has a conditional split from a SQL Server source that splits records to either be updated or inserted into a MYSQL database.
The SQL Server connection has provider .NET Provider for OldDB\SQL Server Native Client 10.0.
The MYSQL connection is a MYSQL ODBC 5.1 ADO.NET connection.
I was thinking about using the OLE DB Command branching off of the conditional split to update records but I connect use this and connect to the MYSQL database.
Does anyone know how to accomplish this task?
I would write to a staging table for updates including the PK and columns to be updated and then execute an UPDATE SQL statement using that table and the table to be updated. The alternative is to use the command for every row and that just doesn't seem to perform that well in my experience - at least compared to a nice fat batch insert and a single update command.
For that matter, I guess you could do without the conditional split altogether, write everything to a staging table and then use an UPDATE and INSERT in SQL back to back.
Probably, the following MSDN blog link might help you. I haven't tried this.
How do I UPDATE and DELETE if I don’t have an OLEDB provider?
The post suggests the following three options.
Script Component
Store the data in a Recordset
Use a custom component (like Merge destination component)
The author also had posted two other articles about MySQL prior to posting the above article.
Connecting to MySQL from SSIS
Writing to a MySQL database from SSIS
Hope that points you in the right direction.
Related
I would like to transfer data from a MS SQL Server database to a MySQL database. So, I added a linked server to MS SQL so that I can use Openquery to insert the data in the MySQL database. I want to optimize the performance of the data transfer and I found the guidelines for improving performance of data loading in MySQL.
One optimization consists of disabling AUTOCOMMIT mode, however I was not able to do it using Openquery.
I tried both ways:
SELECT * from openquery(MYSQL,'SET autocommit=0')
exec openquery(MYSQL,'SET autocommit=0')
and I got:
Cannot process the object "SET autocommit=0". The OLE DB provider
"MSDASQL" for linked server "MYSQL" indicates that either the object
has no columns or the current user does not have permissions on that
object.
Is it possible to execute such statements through openquery?
Thanks,
Mickael
OPENDATASOURCE() and OPENROWSET() allow for add-hoc server connections. You do not need to define a linked server ahead of time.
The OPENQUERY() depends upon a static linked server being defined ahead of time.
Here is the MSDN reference.
http://technet.microsoft.com/en-us/library/ms188427.aspx
Most of the examples show a DML (SELECT, UPDATE, DELETE, INSERT) using the OPENQUERY() as the source or destination of the command. What you are trying to do is execute a session command. Therefore it will fail. Also, you might not even know if the session stays open for the next call.
Why not package up the logic on the MYSQL server as a stored procedure. The stored procedure can be executed on a linked server by using a four-part name?
For example:
INSERT INTO #results
EXEC server.database..stored-proc
This assumes MYSQL has the same object structure as ORACLE. Since I am not a MYSQL person, I can not comment. I allow you to research this little item.
But this should work. It will allow you to package any type of logic in the MYSQL database.
If you want to use SSIS to transfer data from SQL Server to MYSQL.
For the ADO.NET Destination to work properly, the MySQL database needs to have the ANSI_QUOTES SQL_MODE option enabled. This option can be enabled globally, or for a particular session. To enable it for a single session:
1 - Create an ADO.NET Connection Manager which uses the ODBC driver
2 - Set the connection manager’s RetainSameConnection property to True
3 - Add an Execute SQL Task before your data flow to set the SQL_MODE – Ex. set sql_mode='STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,ANSI_QUOTES'
4 - Make sure that your Execute SQL Task and your ADO.NET Destination are using the same connection manager.
Matt Mason did this on a reply.. The key is item #2, use the same connection.
http://blogs.msdn.com/b/mattm/archive/2009/01/07/writing-to-a-mysql-database-from-ssis.aspx#comments
Also, CozyRoc has a custom ODBC driver that might be faster / more reliable than the free one from MYSQL.
http://cozyroc.com/ssis/odbc-destination
I'm getting data from an MSSQL DB ("A") and inserting into a MySQL DB ("B") using the date created in the MSSQL DB. I'm doing it with simple logics, but there's got to be a faster and more efficient way of doing this. Below is the sequence of logics involved:
Create one connection for MSSQL DB and one connection for MySQL DB.
Grab all of data from A that meet the date range criterion provided.
Check to see which of the data obtained are not present in B.
Insert these new data into B.
As you can imagine, step 2 is basically a loop, which can easily max out the time limit on the server, and I feel like there must be a way of doing this must faster and during when the first query is made. Can anyone point me to right direction to achieve this? Can you make "one" connection to both of the DBs and do something like below?
SELECT * FROM A.some_table_in_A.some_column WHERE
"it doesn't exist in" B.some_table_in_B.some_column
A linked server might suit this
A linked server allows for access to distributed, heterogeneous
queries against OLE DB data sources. After a linked server is created,
distributed queries can be run against this server, and queries can
join tables from more than one data source. If the linked server is
defined as an instance of SQL Server, remote stored procedures can be
executed.
Check out this HOWTO as well
If I understand your question right, you're just trying to move things in the MSSQL DB into the MySQL DB. I'm also assuming there is some sort of filter criteria you're using to do the migration. If this is correct, you might try using a stored procedure in MSSQL that can do the querying of the MySQL database with a distributed query. You can then use that stored procedure to do the loops or checks on the database side and the front end server will only need to make one connection.
If the MySQL database has a primary key defined, you can at least skip step 3 ("Check to see which of the data obtained are not present in B"). Use INSERT IGNORE INTO... and it will attempt to insert all the records, silently skipping over ones where a record with the primary key already exists.
Is it possible to update a table in SQL Server using a trigger in MySQL. The scenario is like this.
When a table is updated in MySQL, a corresponding update is to be done in SQL Server for the purpose of syncing certain fields. I understand that the reverse is possible using a linked server in SQL Server.
You can write a UDF function that would update data in the SQL Server. Then call this UDF from the trigger.
Here's some information which may be of help regarding linking SQL Server and MySQL
http://forums.mysql.com/read.php?60,123221,123221
I'm afraid I've no personal experience in the matter as I thankfully have MySQL servers across the board, however had a scan through the links in the above and at least one of them should be able to assist.
I have a mysql database full of data which I need to keep but migrate to SQL Server 2008.
I know end to end where the data should go, table to table but I have no idea how to go about moving the data. I've looked around the web but it seems there are 'solutions' which you have to download and run. I'd rather if possible do something myself in terms of writing scripts or code.
Can anyone recommend the best way to do this please?
You have several options here:
On the sql server side, you can set up a connection to your old mysql db using something called a linked server. This will allow you to write sql code for sql server that returns data from the mysql tables. You can use this to build INSERT or SELECT INTO statements.
You can write queries for mysql to export your data as csv, and then use the BULK INSERT features of sql server to efficiently import the csv data.
You can use Sql Server integration services to set move the data over from mysql.
Regardless of which you choose, non-data artifacts like indexes, foreign keys, triggers, stored procedures, and security will have to be moved manually.
Have you tried tool from MSFT called SQL Server Migration Assistance for MySQL ???
https://www.microsoft.com/download/en/details.aspx?id=1495
Try this tutorial it is very easy to perform migration to SQL Server from Mysql and is straightforward as mentioned
http://www.codeproject.com/Articles/29106/Migrate-MySQL-to-Microsoft-SQL-Server
Thanks
You can use the Import/Export Wizard that comes with SQL Server Standard Edition.
Select your 'data source' from MySQL using the ODBC data source. Note: You will need to first install the from ODBC driver for MySQL (ODBC Connector). Then, select your SQL Server destination. Select all tables, and fire it up. You will need to add your primary and foreign keys, and indexes manually.
A bit more automated means would be by using the SQL Server Migration Assistant for MySQL, also free. It has the benefit of recreating the relationships and indexes automatically for you. Probably your best bet.
I did it once, some time ago. First you could couple your mssql server to the mysql server using the odbc mysql connector
http://dev.mysql.com/downloads/connector/
After the connection is made you can write you database procedure as you would if it were two mssql db's. Probably easiest to write some sql batch scripts including a cursor where you run through every every row of a table an decide on a field basis where you will need the field in the future.
example of a cursor: http://www.mssqltips.com/tip.asp?tip=1599
If you decide to go with the cursor, you can play with the parameter to increase performance. I especially remember the FORWARD_ONLY parameter giving a big boost.
I have 2 Database in my VB.net application. I am using 1st database for daily operations. I would like to send one of the table records to online database. How Can I do that? First database is MSSQL Online database is MYSQL. I have created connections already using MYSQL .net connector.
Any Help will be appreciated.
Regards
Have a look at using a Linked Server instance on SQL Server to write the data to MySQL using the four name notation.
SQL SERVER – Explanation and Example Four Part Name
SQL Server Four-part naming
Ok here is a rough set of steps you need to follow
Query the MSSQL database and retrieve the data you want. Storing it in a DataTable may be the best option starting off.
Loop through the DataTable rows and build an INSERT statement that will be run against the MYSQL database.
Execute the command against the MYSQL db.
This is the basics of what you need to do to get a simple working system. Also take a look at Transactions as a way to manage the rollback of data when something goes wrong.
I'm assuming this is a research project If you are planning on using this code in a production system then i would look into a different alternative such as uploading data files to a service attached to the MYSQL database. This would allow you to batch and retry an import when something goes wrong.