This question already has answers here:
Migrate from Oracle to MySQL
(7 answers)
Closed 8 years ago.
I have 2 databases, the first one is a Mysql db and it is used for a website. The second one is an Oracle db and it has data that I want to show on the website and that data must be fresh, I mean, I need to execute a process to migrate data to Mysql from oracle db every 30 minutes.
Because I am talking about 60000 rows to migrate every 30 minutes, I think that optimal way to do it is something like (thinking about performance):
insert into mysql_db.table.field1, mysql_db.table.field2, mysql_db.table.field3 select oracle_db.table.field1, oracle_db.table.field2, oracle_db.table.field3 from oracle_db.table
The Oracle db is on Windows and Mysql is on Linux (Ubuntu).
Is that possible? how? else, suggest me a different way please.
Having ODBC driver for the MySQL database, you could try Data Export tool (with command line support) in dbForge Studio for Oracle.
You could also use something like GoldenGate to capture changes in oracle and apply them to the mysql database , although if this is the one use case you have, the cost of the product may not be justifiable, in which case probably just a simple perl/php/python/etc script would do the trick.
I've used HSODBC links between mysql 4 and oracle 9i and found the performance less than fantastic, hopefully things have improved, if that is the case, this may be a viable solution, however you would still need to do it as a job ( inside or out of oracle ), as I'm not aware of anyway to make calls from mysql to oracle.
Oracle support the connection between Oracle databases through DBLink. But I don't believe that exists some tool for connect a Mysql and Oracle that let you execute directly your proposed query.
I can suggest that you write an script (by example in Python or Groovy) and schedule it on CRON Linux (if this is your environment). Since the size of data you will need implement Batch Updates (it's specific to the language in which you will implement the script)
Related
I am trying to transfer bulk data on a constant and continuous based from a SQL Server database to a MYSQL database. I wanted to use SQL Server's SSMS's replication but this apparently is only for SQL Server to Oracle or IBM DB2 connection. Currently we are using SSIS to transform data and push it to a temporary location at the MYSQL database where it is copied over. I would like the fastest way to transfer data and am complication several methods.
I have a new way I plan on transforming the data which I am sure will solve most time issues but I want to make sure we do not run into time problems in the future. I have set up a linked server that uses a MYSQL ODBC driver to talk between SQL Server and MYSQL. This seems VERY slow. I have some code that also uses Microsoft's ODBC driver but is used so little that I cannot gauge the performance. Does anyone know of lightening fast ways to communicate between these two databases? I have been researching MYSQL's data providers that seem to communicate with a OleDB layer. Im not too sure what to believe and which way to steer towards, any ideas?
I used the jdbc-odbc bridge in Java to do just this in the past, but performance through ODBC is not great. I would suggest looking at something like http://jtds.sourceforge.net/ which is a pure Java driver that you can drop into a simple Groovy script like the following:
import groovy.sql.Sql
sql = Sql.newInstance( 'jdbc:jtds:sqlserver://serverName/dbName-CLASS;domain=domainName',
'username', 'password', 'net.sourceforge.jtds.jdbc.Driver' )
sql.eachRow( 'select * from tableName' ) {
println "$it.id -- ${it.firstName} --"
// probably write to mysql connection here or write to file, compress, transfer, load
}
The following performance numbers give you a feel for how it might perform:
http://jtds.sourceforge.net/benchTest.html
You may find some performance advantages to dumping data to a mysql dumpfile format and using mysql loaddata instead of writing row by row. MySQL has some significant performance improvements for large data sets if you load infile's and doing things like atomic table swaps.
We use something like this to quickly load large datafiles into mysql from one system to another e.g. This is the fastest mechanism to load data into mysql. But real time row by row might be a simple loop to do in groovy + some table to keep track of what row had been moved.
mysql> select * from table into outfile 'tablename.dat';
shell> myisamchk --keys-used=0 -rq '/data/mysql/schema_name/tablename'
mysql> load data infile 'tablename.dat' into table tablename;
shell> myisamchk -rq /data/mysql/schema_name/tablename
mysql> flush tables;
mysql> exit;
shell> rm tablename.dat
The best way I have found to transfer SQL data (if you have the space) is a SQL dump in one language and then to use a converting software tool (or perl script, both are prevalent) to convert the SQL dump from MSSQL to MySQL. See my answer to this question about what converter you may be interested in :) .
We've used the ado.net driver for mysql in ssis with quite a bit of success. Basically, install the driver on the machine with integration services installed, restart bids, and it should show up in the driver list when you create an ado.net connection manager.
As for replication, what exactly are you trying to accomplish?
If you are monitoring changes, treat it as a type 1 slowly changing dimension (data warehouse terminology, but same principal applies). Insert new records, update changed records.
If you are only interested in new records and have no plans to update previously loaded data, try an incremental load strategy. Insert records where source.id > max(destination.id).
After you've tested the package, schedule a job in the sql server agent to run the package every x minutes.
Cou can also try the following.
http://kofler.info/english/mssql2mysql/
I tried this a longer time before and it worked for me. But I woudn't recommend it to you.
What is the real problem, what you try to do?
Don´t you get a MSSQL DB Connection, for example from Linux?
I have a mysql database full of data which I need to keep but migrate to SQL Server 2008.
I know end to end where the data should go, table to table but I have no idea how to go about moving the data. I've looked around the web but it seems there are 'solutions' which you have to download and run. I'd rather if possible do something myself in terms of writing scripts or code.
Can anyone recommend the best way to do this please?
You have several options here:
On the sql server side, you can set up a connection to your old mysql db using something called a linked server. This will allow you to write sql code for sql server that returns data from the mysql tables. You can use this to build INSERT or SELECT INTO statements.
You can write queries for mysql to export your data as csv, and then use the BULK INSERT features of sql server to efficiently import the csv data.
You can use Sql Server integration services to set move the data over from mysql.
Regardless of which you choose, non-data artifacts like indexes, foreign keys, triggers, stored procedures, and security will have to be moved manually.
Have you tried tool from MSFT called SQL Server Migration Assistance for MySQL ???
https://www.microsoft.com/download/en/details.aspx?id=1495
Try this tutorial it is very easy to perform migration to SQL Server from Mysql and is straightforward as mentioned
http://www.codeproject.com/Articles/29106/Migrate-MySQL-to-Microsoft-SQL-Server
Thanks
You can use the Import/Export Wizard that comes with SQL Server Standard Edition.
Select your 'data source' from MySQL using the ODBC data source. Note: You will need to first install the from ODBC driver for MySQL (ODBC Connector). Then, select your SQL Server destination. Select all tables, and fire it up. You will need to add your primary and foreign keys, and indexes manually.
A bit more automated means would be by using the SQL Server Migration Assistant for MySQL, also free. It has the benefit of recreating the relationships and indexes automatically for you. Probably your best bet.
I did it once, some time ago. First you could couple your mssql server to the mysql server using the odbc mysql connector
http://dev.mysql.com/downloads/connector/
After the connection is made you can write you database procedure as you would if it were two mssql db's. Probably easiest to write some sql batch scripts including a cursor where you run through every every row of a table an decide on a field basis where you will need the field in the future.
example of a cursor: http://www.mssqltips.com/tip.asp?tip=1599
If you decide to go with the cursor, you can play with the parameter to increase performance. I especially remember the FORWARD_ONLY parameter giving a big boost.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Differences Between MySql and MS SQL
What is the key difference between MySQL and SQL Server from the SQL language perspective. The reason why I am asking is that I need to quickly master some basic SQL knowledge and need to find a book to start with.
I find this one through amazon.com Learning SQL, 2nd Edition, however, this book is for MySQL. All other books for MS SQL are big one that cannot be finished in a short time.
So, it is fine I just grab a MySQL book and later I can easily transfer to MS SQL world or I should immediately go into MS SQL Server?
// update the title based on the comments ///
The following links might help you out.
Note that the two are very different.
If you want to make your live easy, use PostgreSQL, the gap between that and SQL-server is much smaller.
Difference between MySQL and MSSQL
From microsoft and mysql respectively:
http://technet.microsoft.com/en-us/library/cc966396.aspx
http://www.mysql.com/why-mysql/white-papers/mysql_wp_mssql2mysql.php (login required sorry)
On stackoverflow: Differences between MySQL and SQL Server
Difference between PostgreSQL and MySQL
http://www.wikivs.com/wiki/MySQL_vs_PostgreSQL
Difference between postgreSQL and MSSQL
On SO: https://stackoverflow.com/questions/907908/postgres-to-sql-server
and: https://stackoverflow.com/questions/4630891/tips-for-sql-server-developer-picking-up-postgresql
Diff between all three :-)
http://www.postgresonline.com/journal/archives/51-Cross-Compare-of-SQL-Server,-MySQL,-and-PostgreSQL.html
or http://troels.arvin.dk/db/rdbms/
This might be a good tutorial to start with SQL - http://philip.greenspun.com/sql/ (rather old, but valid) - though it points more to Oracle, but MySQL is mentioned there.
I have 2 Database in my VB.net application. I am using 1st database for daily operations. I would like to send one of the table records to online database. How Can I do that? First database is MSSQL Online database is MYSQL. I have created connections already using MYSQL .net connector.
Any Help will be appreciated.
Regards
Have a look at using a Linked Server instance on SQL Server to write the data to MySQL using the four name notation.
SQL SERVER – Explanation and Example Four Part Name
SQL Server Four-part naming
Ok here is a rough set of steps you need to follow
Query the MSSQL database and retrieve the data you want. Storing it in a DataTable may be the best option starting off.
Loop through the DataTable rows and build an INSERT statement that will be run against the MYSQL database.
Execute the command against the MYSQL db.
This is the basics of what you need to do to get a simple working system. Also take a look at Transactions as a way to manage the rollback of data when something goes wrong.
I'm assuming this is a research project If you are planning on using this code in a production system then i would look into a different alternative such as uploading data files to a service attached to the MYSQL database. This would allow you to batch and retry an import when something goes wrong.
This question already has answers here:
Migrate database from Postgres to MySQL [closed]
(3 answers)
Closed 8 years ago.
I'm looking to grab a few bits of data from musicbrainz db to use in a mysql based app.
I don't need the entire database, and have been looking at 'migrating' postgreSQL to mysql, which it seems lots of people have difficulty with.
Wouldn't it be simplest to just dump the postgreSQL data into a comma-delimited text file, and then import that into mysql?
I'm just getting started with this, and don't even have postgreSQL installed yet, but trying to look ahead at how I'm going to do it.
You can use COPY (in the psql client) to dump a single table.
Or you can use pg_dump with the -d parameter. This will cause pg_dump to dump INSERT statements, which you can just execute against your MySQL server. You will obviously need to port the schema first - and assuming the datatypes that are used exist in MySQL.
Perhaps you want to dump you Database to a SQL script.