The title pretty much says it all, but to elaborate: If I build a mySQL database on my local dev machine, populate it with data, and subsequently want to migrate the database to a shared host (in this case, Siteground,) how do I do so in a way that keeps structure and data intact?
In this case, I don't have file access to the database server.
use mysqldump (doc) and dump your database (mysqldump [databasename] for a simple configuration) on your development machine to a dump (a file containing sql statements needed to recover both schema and data). Now insert the dump on your shared-host using the provided utilities (normaly you get phpMyAdmin preinstalled from your hoster, which can import dumps)
In addition to the response made by theomega (namely, do a dump of your development database and then insert the dump into your production database), be aware that you may need to enable large SQL insert statements if you have a lot of data. I would recommend you first FTP the file to the host, and then do the insert from a file. Each host has their own way of doing it, but if you can connect to the remote server using SSH, there is likely the ability to run the insert using the command line.
also in addition to theomega: most tools for mysql has dump / execute functions for sql files.
if you're using navicat, for an example, you're just a right-click away:
right-click on the database you want to export, and choose "dump sql file". this will allow you to save the .sql file on your local drive in the folder of your choosing.
then, right click on the destination database and choose "execute batch file". browse to the newly-created .sql file and it will execute all sql commands from that file in the destination database. namely, creating a copy of the exported db.
Related
Disclaimer: I'm a total green horn.
I'm working with PhpStorm on Windows, which offers a convenient way of creating and managing databases during development. Unfortunately, now that I want to push to production, uploading the database to Cloadways (Digital Ocean Server) doesn't seem that simple.
Cloudways' Database Manager has an import function, that requires .gz (gzip) files. gzip files can not be created from directories, but on Windows mysql creates directories for each database and fills them with table files (.ibd).
I've read that you can compress directories into .tar files first and then gzip them (database.tar.gz) and that's what I tried. But when I try to import them with the database manager it only shows this:
Is there any way to do this?
Here's what you need to migrate your data to another MySQL server from your development machine: A text file named whatever.sql containing the definitions and contents of your tables (and views and stored functions and all your other database objects). This kind of text file is often called a mysqldump file. You'll find these files contain a whole mess of SQL statements, mostly CREATE TABLE and INSERT. They also contain some lines like
/*!40103 SET #OLD_TIME_ZONE=##TIME_ZONE */;
These look like comments, but when you load the file they're handled as SQL statements, so leave them alone.
You can use gzip(1) on your file to make it smaller, and Cloudways will handle it correctly. Things will work either with or without gzip.
How to export
With PhpStorm, open up the database panel and right-click on your database name (not the server name itself, but your application's database). You'll see a menu item Export with mysqldump. Click it. You can keep the default checkbox settings.
Then give a filename for your output and run the export.
I use PhpStorm on linux, so I'm not totally sure this export works on Windows. If it doesn't, download Ansgar Becker's free and excellent Windows MySQL client program called HeidiSQL. Right click on the database name then choose Export Database as SQL. Check the Create Tables box and choose Insert from the data pulldown.
How to import to production
Log into the production MySQL server.
Create the database and choose it. Cloudways looks like it does that for you. If not, issue these SQL statements.
CREATE DATABASE myDatabaseName;
USE myDatabaseName;
Use an appropriate tool to run all the SQL in your whatever.sql file. Cloudways looks like it does that for you too. If not, this command line, or something similar, might work.
mysql --user=yourMySQLUserName --password=secret\
--database=myDatabaseName\
--host=cloudways.example.com < whatever.sql
Your migration will be complete.
Extra bonus: If your whatever.sql file contains the initial state of your production database, you can put it into git (or other source control) and use it whenever you deploy a new instance of your software package.
Don't try to copy those .ibd files and other files managed by MySQL to another machine. If you do, you'll be sorry.
I am trying to automate the upload of data to a mysql database. I am using MySql Workbench on a windows pc to remotely access my database on AWS. I have a sql file that I use to load a csv file into the db using LOAD DATA LOCAL INFILE. My csv file is created daily using a scheduled task and I would like to load it into the db using a batch type file and a scheduled task.
Is it possible?
At Windows you may use PHP form Wamp Server, its very straightforward installation. You don't need MySQL Server at your local PC to update remote AWS with the data but only a scripting language.
I would suggest to install MySQL on your local PC to check firstly on that local MySQL if the update does what you expect it to do. Once it meets your expectation you just change the MySQL connection parameters to these from AWS and your set.
In MySQL Workbench you may add additional MySQL Server as local to check the local database and all applied to it changes
Perhaps this example can help link you to do first steps in writing php script to update database
PHP scripts can be executed form command line as well so once you write your script that updates the database you should be able to run it from windows CMD console this way
php -f path-to-your-sript.php
but if so you need to edit php scipt this way that it already knows where the csv file is and reads its content, maybe by this function file_get_contents() or you can also give a try a dedicated to csv files function fgetcsv() that is even more suitable because it reads line by line your CSV file so if you use loop you can even process very big CSV files without running out of the memory.
I'm trying to create a dump of my RDS data and use it locally.
I've used the command:
mysqldump -h myhostname.rds.amazonaws.com -u my_username -p my_dbname > ~/Downloads/dump.sql
When I try to view this data in a tool like DB Browser for sqlite, I get a prompt saying it's encrypted, asking for a password.
I thought maybe it needed to be converted into sqlite first, so I've done this in RazorSQL -> But I still get the same issue. Also, when I try to load the DB into NodeJS's sqlite module I get:
not able to query Table in SQLite DB Error: SQLITE_NOTADB: file is encrypted or is not a database
I've checked my RDS settings, and it says:
Encryption details:
Encryption enabled
No
So I have no idea what's going on here. Any tips? Does the file extension (.sql, .db etc) make a difference here?
When I try to view this data in a tool like DB Browser for sqlite
Sqlite is an entirely different thing than MySQL. There is very little overlap in tools that can work with both.
You're using a tool that can't be used for the purpose to which you're applying it, so you're getting a confusing error:
file is encrypted or is not a database
In other words, the tool is unable to make sense of the file, so one of two things is has happened: the file is encrypted or is not a [sqlite] database [at all].
The problem is the latter.
The file is not encrypted. Even if the RDS instance is encrypted, the generated dump file would still not be encrypted, because encryption in RDS is storage-level encryption of the data, at rest, on the disk volume backing the RDS instance. Encryption in RDS is transparent to the user.
The problem is that what you have here is a dump file -- a series of SQL statements that can be used to reconstruct your database on another MySQL server.
Your file is plain text. You can view it will a text editor. What you can't do is use the file as a database -- that's something Sqlite can do, because Sqlite stores the database inside a single, transportable file. MySQL is a different architecture.
You'll need to have the same version (e.g. 5.7.x) of MySQL Server installed locally, and then load this file onto it.
shell> mysql [options] < my_dump_file.sql
To reload a dump file written by mysqldump that consists of SQL statements, use it as input to the mysql client.
https://dev.mysql.com/doc/refman/5.7/en/reloading-sql-format-dumps.html
You can also use query browser tools for MySQL like Toad or Workbench, but a local MySQL Server is required.
Iam a newbie for DB. I have a running database on godaddy's mysql setup. Now i have a new server whose access is not with me. I have to make a file which can install and create new mysql on that server plus it can copy and store specific tables from my current server database too.
I have to give this file to the person who is having the access so that he can execute it and can have all the content. How can i make such a file?
You can use MySQL's mysqldump to take backup of your databases.
Alternatively, You can always backup your database using MySQL GUI tools like SQLyog.
I guess GoDaddy do not allows direct connection for mysqldump to work. In that case you can use SQLyog's HTTP tunneling capability.
Give the generated backup file (created through SQLyog or through mysqldump) to the person who has access and he will upload the file.
Here is the SQLyog's documentation for taking Backup. Select Databases and Tables that you want to backup through wizard.
Hope it helps....
The title is self explanatory. Is there a way of directly doing such kind of importing?
The .BAK files from SQL server are in Microsoft Tape Format (MTF) ref: http://www.fpns.net/willy/msbackup.htm
The bak file will probably contain the LDF and MDF files that SQL server uses to store the database.
You will need to use SQL server to extract these. SQL Server Express is free and will do the job.
So, install SQL Server Express edition, and open the SQL Server Powershell. There execute sqlcmd -S <COMPUTERNAME>\SQLExpress (whilst logged in as administrator)
then issue the following command.
restore filelistonly from disk='c:\temp\mydbName-2009-09-29-v10.bak';
GO
This will list the contents of the backup - what you need is the first fields that tell you the logical names - one will be the actual database and the other the log file.
RESTORE DATABASE mydbName FROM disk='c:\temp\mydbName-2009-09-29-v10.bak'
WITH
MOVE 'mydbName' TO 'c:\temp\mydbName_data.mdf',
MOVE 'mydbName_log' TO 'c:\temp\mydbName_data.ldf';
GO
At this point you have extracted the database - then install Microsoft's "Sql Web Data Administrator". together with this export tool and you will have an SQL script that contains the database.
MySql have an application to import db from microsoft sql.
Steps:
Open MySql Workbench
Click on "Database Migration" (if it do not appear you have to install it from MySql update)
Follow the Migration Task List using the simple Wizard.
I did not manage to find a way to do it directly.
Instead I imported the bak file into SQL Server 2008 Express, and then used MySQL Migration Toolkit.
Worked like a charm!
In this problem, the answer is not updated in a timely. So it's happy to say that in 2020 Migrating to MsSQL into MySQL is that much easy. An online converter like RebaseData will do your job with one click. You can just upload your .bak file which is from MsSQL and convert it into .sql format which is readable to MySQL.
Additional note: This can not only convert your .bak files but also this site is for all types of Database migrations that you want.
Although my MySQL background is limited, I don't think you have much luck doing that. However, you should be able to migrate over all of your data by restoring the db to a MSSQL server, then creating a SSIS or DTS package to send your tables and data to the MySQL server.
hope this helps
I highly doubt it. You might want to use DTS/SSIS to do this as Levi says. One think that you might want to do is start the process without actually importing the data. Just do enough to get the basic table structures together. Then you are going to want to change around the resulting table structure, because whatever structure tat will likely be created will be shaky at best.
You might also have to take this a step further and create a staging area that takes in all the data first n a string (varchar) form. Then you can create a script that does validation and conversion to get it into the "real" database, because the two databases don't always work well together, especially when dealing with dates.
The method I used included part of Richard Harrison's method:
So, install SQL Server 2008 Express
edition,
This requires the download of the Web Platform Installer "wpilauncher_n.exe"
Once you have this installed click on the database selection ( you are also required to download Frameworks and Runtimes)
After instalation go to the windows command prompt and:
use sqlcmd -S \SQLExpress (whilst
logged in as administrator)
then issue the following command.
restore filelistonly from
disk='c:\temp\mydbName-2009-09-29-v10.bak';
GO This will list the contents of the
backup - what you need is the first
fields that tell you the logical names
- one will be the actual database and the other the log file.
RESTORE DATABASE mydbName FROM
disk='c:\temp\mydbName-2009-09-29-v10.bak' WITH MOVE 'mydbName' TO
'c:\temp\mydbName_data.mdf', MOVE
'mydbName_log' TO
'c:\temp\mydbName_data.ldf'; GO
I fired up Web Platform Installer and from the what's new tab I installed SQL Server Management Studio and browsed the db to make sure the data was there...
At that point i tried the tool included with MSSQL "SQL Import and Export Wizard" but the result of the csv dump only included the column names...
So instead I just exported results of queries like "select * from users" from the SQL Server Management Studio
SQL Server databases are very Microsoft proprietary. Two options I can think of are:
Dump the database in CSV, XML or similar format that you'd then load into MySQL.
Setup ODBC connection to MySQL and then using DTS transport the data. As Charles Graham has suggested, you may need to build the tables before doing this. But that's as easy as a cut and paste from SQL Enterprise Manager windows to the corresponding MySQL window.
For those attempting Richard's solution above, here are some additional information that might help navigate common errors:
1) When running restore filelistonly you may get Operating system error 5(Access is denied). If that's the case, open SQL Server Configuration Manager and change the login for SQLEXPRESS to a user that has local write privileges.
2) #"This will list the contents of the backup - what you need is the first fields that tell you the logical names" - if your file lists more than two headers you will need to also account for what to do with those files in the RESTORE DATABASE command. If you don't indicate what to do with files beyond the database and the log, the system will apparently try to use the attributes listed in the .bak file. Restoring a file from someone else's environment will produce a 'The path has invalid attributes. It needs to be a directory' (as the path in question doesn't exist on your machine).
Simply providing a MOVE statement resolves this problem.
In my case there was a third FTData type file. The MOVE command I added:
MOVE 'mydbName_log' TO 'c:\temp\mydbName_data.ldf',
MOVE 'sysft_...' TO 'c:\temp\other';
in my case I actually had to make a new directory for the third file. Initially I tried to send it to the same folder as the .mdf file but that produced a 'failed to initialize correctly' error on the third FTData file when I executed the restore.
The .bak file from SQL Server is specific to that database dialect, and not compatible with MySQL.
Try using etlalchemy to migrate your SQL Server database into MySQL. It is an open-sourced tool that I created to facilitate easy migrations between different RDBMS's.
Quick installation and examples are provided here on the github page, and a more detailed explanation of the project's origins can be found here.