I'm trying to upload a large database:
$mysql -u USERNAME -p MY_DATABASE < MY_DATABASE.sql
But the process is going extremely slowly, I would like to cancel and try again with different settings. How can I cancel this upload 'safely'? Or what is the best way to clean up my database after forced termination of the import?
You can kill MySQL process involved in your MY_DATABASE.sql
This show you running processes:
show processlist;
Then you should kill the process selecting it by the correct process_id from the user and time shown from previous command to understand which one is the correct one, if others processes are running:
kill process_id
I think that cleaning the DB is the hardest work, because it depends on the MY_DATABASE.sql content: if it populates a table, you can simply truncate it, if it create a DB and upload tables, views, stored procedures and other elements, you should drop the DB.
If you take a look at the MY_DATABASE.sql content and at the server content after killing the process (db created? tables or other elements created?) you will understand what to do.
If, as I can imagine, the script creates entirely a DB, you should only have to drop the DB created from the previous interrupted upload, if any, and restart the upload.
Rather stopping MySQL import with killing the process, I prefer use terminate key which is ctrl+c and service mysqld restart or service mysql restart.
This just ends the import process and it gets sometimes to end. Once it's done you can go with your process. If you end the process with killing it may cause some unexpected errors like a mysql server crash.
Related
I have a MySQL database with multiple tables. When I run a simple SELECT command on one of the tables, it gives the "lost connection" error, but when I run the same command on any of the other tables, it works just fine. Does anyone know what could be the cause of this issue? I'm running the commands from the MySQL Workbench.
You must set the 'Interactive_timeout' and 'wait_timeout' properties in the mysql configuration file to the values you need.
Quite likely there is some sort of a transaction running. Whether you started the transaction via a sql command (i.e. START TRANSACTION) or whether some other query (possibly from a GUI) started a transaction (Like an ALTER TABLE command when adding a column) and the transaction never finished.
Run the command
SHOW FULL PROCESSLIST;
to find the IDs of all processes that are running for your user
You can then kill the processes by running the command
KILL <PROCESS ID> (i.e. KILL 40196;)
When you find and stop the process that contained the transaction that never finished you will likely be able to query the table again. Your table will be queryable again, and you can look further into solving the problem of why the transaction was started but never finished.
Is it possible to cancel a mysql query?
I have the problem that I submit a query which is very time-consuming because of a mistake. And now I can't make a new query because the server is working and working...
Or is there a way to stop all queries in a database or in a table.
For your information I am using phpMyAdmin (MySql).
In phpMyAdmin, go to home page > Status; you'll see a list of your MySQL processes and you have a Kill link for each of them.
Probably your phpMyAdmin will be stuck showing the "Searching" message. What you can do is to open another tab / open a MySQL session with the console and do the following:
Look for all the processes running:
SHOW PROCESSLIST;
And then kill that specific thread with the KILL command:
KILL <thread_id>;
Resources:
SHOW PROCESSLIST page in MySQL
KILL page in MySQL
You could use the KILL statement : http://dev.mysql.com/doc/refman/5.0/en/kill.html
Sometimes, you can't even get to phpmyadmin even on a new tab.
If you have access to the environment via command line,
1) log into mysql
mysql -u yourusername -p
2) SHOW PROCESSLIST;
3) KILL <thread_id>;
This might help you:
Post in Mysql Forum
This will teach you to test your queries before just throwing them into a page. EXPLAIN PLAN is your friend.
Sometimes if the database server is not responding because of a too long query and:
the phpMyAdmin is stuck showing the "Loading..." message...
you can't log into MySQL server with appropriate permissions or if so but theSHOW PROCESSLIST command in CLI is not very useful because of for example too big amount of started processes...
then, the fastest option might be to restart the server, which stops all queries in a database:
service mysql restart
or
sudo /etc/init.d/mysql restart
This is not the safest option, because in some cases the interrupted queries may cause inconsistency of data in the database..., but sometimes it may be the best solution.
What happens to a long lasting query executed from commandline via SSH if the connection to MySQL or SSH is lost?
Context:
We have 2 servers with a very large MySQL database on them. One server acts as the Master, and the other as Slave. During regular maintenance, the replication became corrupt, and we noticed data was missing from the slave, even though it reported Seconds_Behind_Master = 0.
So I am in the process of repairing the replication. I am, as we speak, importing one of two large dumps in to the slave. I am connected to MySQL through SSH, and used the MySQL "\. file.sql" command to import the dump.
Right now I am constantly getting results like so "Query OK, 6798 rows affected".
It has been running for probably 30 minutes now. My question and worry is, what happens if I lose connection through SSH while this is running?
I have another, even larger dump to import after this.
Thanks for the answer!
-Steve
if you lose your connection, all children of your bash process will die, including mysql.
to avoid this problem use the screen command.
I have multiple mysql databases and I want to perform some administration tasks on a particular database. How do I ensure that no one else can connect to that database while the tasks in progress?
Apparently, you can use the FLUSH command for this as such:
> FLUSH TABLES WITH READ LOCK;
and then
> UNLOCK TABLES;
to unlock the database again. Unsure if some setting needs to be set on the tables to allow a readlock. You can test this by trying to do a manual insert after the database is locked and if you get an error message about the table being locked, you know it worked.
More information on FLUSH command
You have a couple of options, as I see it:
Shutdown mysqld, the daemon that connections to be established to databases. This has the downside of preventing access to all the mysql databases on that computer.
Move the file or change the access permissions to it, so that only your user may work with it. (I have no idea if this will work.) The database files are located somewhere like /var/lib/mysql. Just don't forget when you're done to change them back to something that mysqld will be able to work with!
Good luck!
If you can afford stopping the server for the time of maintenance, stop the daemon and reconfigure it to not listen to network connections. Only allow connections through a local UNIX socket.
Then logon locally on the same machine, either physically or via SSH. As long as nobody else has access to the machine, this will ensure you are alone. When done, restore the original configuration file and restart the daemon.
A different approach would be to temporarily disable all user accounts but "root" (or whatever you use to do your maintenance) for the particular database. This has the drawback however of actually messing with account data which is a little more risky than just preventing network connections.
The databases are prohibitively large (> 400MB), so dump > SCP > source is proving to be hours and hours work.
Is there an easier way? Can I connect to the DB directly and import from the new server?
You can simply copy the whole /data folder.
Have a look at High Performance MySQL - transferring large files
Use can use ssh to directly pipe your data over the Internet. First set up SSH keys for password-less login. Next, try something like this:
$ mysqldump -u db_user -p some_database | gzip | ssh someuser#newserver 'gzip -d | mysql -u db_user --password=db_pass some_database'
Notes:
The basic idea is that you are just dumping standard output straight into a command on the other side, which SSH is perfect for.
If you don't need encryption then you can use netcat but it's probably not worth it
The SQL text data goes over the wire compressed!
Obviously, change db_user to user user and some_database to your database. someuser is the (Linux) system user, not the MySQL user.
You will also have to use --password the long way because having mysql prompt you will be a lot of headache.
You could setup a MySQL slave replication and let MySQL copy the data, and then make the slave the new master
400M is really not a large database; transferring it to another machine will only take a few minutes over a 100Mbit network. If you do not have 100M networks between your machines, you are in a big trouble!
If they are running the exact same version of MySQL and have identical (or similar ENOUGH) my.cnf and you just want a copy of the entire data, it is safe to copy the server's entire data directory across (while both instances are stopped, obviously). You'll need to delete the data directory of the target machine first of course, but you probably don't care about that.
Backup/restore is usually slowed down by the restoration having to rebuild the table structure, rather than the file copy. By copying the data files directly, you avoid this (subject to the limitations stated above).
If you are migrating a server:
The dump files can be very large so it is better to compress it before sending or use the -C flag of scp. Our methodology of transfering files is to create a full dump, in which the incremental logs are flushed (use --master-data=2 --flush logs, please check you don't mess any slave hosts if you have them). Then we copy the dump and play it. Afterwards we flush the logs again (mysqladmin flush-logs), take the recent incremental log (which shouldn't be very large) and play only it. Keep doing it until the last incremental log is very small so that you can stop the database on the original machine, copy the last incremental log and then play it - it should take only a few minutes.
If you just want to copy data from one server to another:
mysqldump -C --host=oldhost --user=xxx --database=yyy -p | mysql -C --host=newhost --user=aaa -p
You will need to set the db users correctly and provide access to external hosts.
try importing the dump on the new server using mysql console, not an auxiliar software
I have no experience with doing this with mysql, but to me it seems the bottleneck is transferring the actual data?
4oo MB isnt that much. But if dump -> SCP is slow, i dont think connecting to the db server from the remove box would be any faster?
I'd suggest dumping, compressing, then copying over network or burning to disk and manually transfering the data.
Compressing such a dump will most likely give you quite good compression rate since, most likely , theres a lot of repeptetive data.
If you are only copying all the databases of the server, copy the entire /data directory.
If you are just copying one or more databases and adding them to an existing mysql server:
create the empty database in the new server, set up the permissions for users etc.
copy the folder for the database in /data/databasename to the new server /data/databasename
I like to use BigDump: Staggered Mysql Dump Importer after Exporting my database from the old server.
http://www.ozerov.de/bigdump/
One thing to note though, if you don't set the export options (namely the maximum length of created queries) respective to the load your new server can handle, it'll just fail and you will have to try again with different parameters. Personally, I set mine to about 25,000, but that's just me. Test it out a bit and you'll get the hang of it.