After upgradig mysql to 5.7 form 5.5 few months ago, I forgot to do mysql_upgrade.
And facing some problems.. mysql, sys, performance_schema databases are missing and root privileges are broken. A lot of Access denied for user 'root'... messages pop up, when I try to do some mysql user privileges things.
This stack answer will have to solve my problem. But I need to know it won't affect any of the schemas, data... ect.
Because my database is pretty huge. It amounts to 10 GB and consists of about 50 tables. I'm afraid some bad things could happend. I know the answer will be the mysqldump.
But the full backup will cost a long time, maybe an hour. And the business won't accept that downtime.
So what is the risk of mysql_upgrade without doing mysqldump?
The risk of doing anything administrative to your database without backups is unacceptably high... not because of any limitations in MySQL per se, but because we're talking about something critical to your business. You should be backing it no less often than the time interval of data you are willing to lose.
If you are using InnoDB, then use the --single-transaction option of mysqldump and there should be no locking, because MVCC handles the consistency. If you are not using InnoDB, that is a problem itself, but using --skip-lock-tables should mimimize locking.
Note that it should be quite safe to kill a mysqldump in progress if you find it is causing issues -- find the thread-id of the dump using SHOW PROCESSLIST; and then KILL QUERY #; where # is the ID of the dump connection from the process list.
The potential problem with the answer you cited is that 5.1 > 5.5 is a supported upgrade path, because those two versions are sequential. 5.5 > 5.7 isn't. You should have upgraded to 5.6 and then 5.7, running the appropriate versions of mysql_upgrade both before and after each step (appropriate meaning the version of the utility matching the version of the server running at the moment).
You may be in a more delicate situation than you imagine... or you may not.
Given similar circumstances, I would not want to do anything less than stop the server completely, clone it by copying all the files to a new machine, and test the remediation steps against the clone.
If this system is business-critical, it should have a live, running replica server, that could be promoted to master and permanently replace this machine in the event of a failure. In a circumstance like this one, you would apply your fixes to the replica and promote it.
Access denied for user 'root'... may or may not be related to the schema incompatibilites.
Related
I have a large Drupal 7 site with many concurrent logged-in users running on a server where disk space recently ran out (MySQL InnoDB behind Memcached on Ubuntu 16.04). How/why this happened is a discussion for another day.
I've cleared up the disk space issue, and the site seems to be running fine as far as general interaction indicates, but Drupal log is full of errors like this:
Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 3 Error writing file '/tmp/MYGWmIvU' (Errcode: 28 - No space left on device)' in /var/www/pixelscrapper.com/public_html/includes/database/database.inc:2229
My question now is: will the mysql database that is running Drupal likely be corrupted/flaky at this point? i.e. On a scale of 0-10, how vital is it that I restore the database to a point before the disk space ran out?
(Anything other than 0 means I will likely restore the database--but there are other things that went wrong as well here, which means that we would lose quite a few days of data if I need to restore, which is a huge drag. C'est la vie, etc.).
My assumption is that the data in MySQL may be more or less fine, but that I cannot rely on the integrity of the actual Drupal data (users, nodes, etc.) which are made up of collections of many database rows...
Out of space crashes can cause serious damages to databases. The sole fact that your database is able to startup and to run apparently as usual is already a good indication that it was not totally messed up by the incident.
Next thing you can do is to perform an in-depth scan using the sqlcheck program :
The mysqlcheck client performs table maintenance: It checks, repairs, optimizes, or analyzes tables.
Analyze all tables in all databases :
$ mysqlcheck -A
Analyze and repair all tables in all databases :
$ mysqlcheck -A -r
NB : as explained in the documentation, you would better shutdown your application and make a backup before you run this.
How can I make a more robust database server? I know this is a pretty general question but I'm getting all sort of errors and downtimes using MySQL with a simple service that does some selects and 2 inserts. Nothing complex, no JOINs...
At first I got my host blocked from the DB server because of many connections, so I increased the max_used_connections parameter. And my client is actually closing the connections and handling the pool correctly.
For some days it worked OK. Today I woke up and the MySQL server was down. Tried to restart it with service mysql restart and it failed. Tried to connect and error 1040: too many connections.
So I couldn't even restart the server.
I did killall mysqld and tried to start the server with service mysql start. Service wouldn't start. Run su - mysql -s /bin/sh -c "/usr/bin/mysqld_safe > /dev/null 2>&1 &" and the server finally started.
I also increased max_connections limit because the server would just die after some seconds when starting it today.
Then one table was corrupt ERROR 144 (HY000): Table '<table_name>' is marked as crashed and last (automatic?) repair failed
Tried to REPAIR <table_name> but after some time the server died again ERROR 2006 (HY000): MySQL server has gone away. Tried this same process (killall up to here) about 3 times, everytime the server died.
Now I'm running REPAIR <table_name> QUICK and the server seems not to be dying, but it's taking a really long time to fix a 200k row table.
By the way, I only have two tables, one with 200k rows and another with 11M rows. I peak about 2000 concurrent users, which result in about 100 queries per second max.
Is it normal that MySQL crashes this easy? Should I migrate to other more scalable databases like Cassandra? I actually prefer a relational database for my case (table results has a request_id, FK of a requests table that has a key_id FK of a keys table), but wouldn't mind losing the relational benefits for an actually good uptime.
I'm not sure that your question (Is it normal that MySQL crashes this easy? Should I migrate to other more scalable databases like Cassandra? ) can be answered objectively, so here is my subjective answer:
No, it's not normal, and you probably shouldn't migrate, since you want a relational database and MySQL is well supported and really well known.
More info about your problem (also subjective):
MySQL is really really stable. I have MySQL services that have been running for years with regular use and have not had issues. I'm sure many others have had similar experiences. Many large companies use MySQL with great success.
What concerns me is that killall and service restart didn't work. Especially since you are running on an Ubuntu LTS version that is over 2 years old, and hence pretty stable itself. While I cannot say this with absolute certainty, it looks like you may have hardware issues, or at the very least a corrupt hard drive. You may also have software and/or kernel options or settings that are causing problems.
My suggestion to you would be to do a full hardware check (at a minimum check RAM and HDD), and do a full reinstall.
I understand I might be asking the impossible but, well never hurts to try.
Is it possible to rollback changes to a handful of MySQL databases without having any form of backup. The only thing left are the databases with the changes I wish to undone.
Why this happened you might ask, well simply putting it, Windows XP scheduler decided not to run the backup task as scheduled, therefore no backups were made previous to a few dozen queries queried to those databases.
I doubt it is possible to undo those changes, and if it is possible I don't know how.
Maybe MySQL keeps some sort of record on the changes performed to a database, but I don't know.
Does anybody knows a way to undo changes to a MySql database without any form of backup
(neither dump file or files from the data folder)?
Thank you.
System Details:
Windows XP SP3
Server: localhost via TCP/IP
Software: MySQL
Software version: 5.5.25a - MySQL Community Server (GPL)
Protocol version: 10
Server charset: UTF-8 Unicode (utf8)
Update
Possible that Windows XP system restore kept a previous version of
MySQL DBs on file?
Without having any database backup, you are probably pretty much out of luck. If you had mysql query logging or binary logging on, you could glean some information about what queries had been run since a certain point in time, but if you had destructive queries (UPDATE, DELETE, etc.) you will have no way of knowing what the previous data was.
I have a large (about 10 GB with a 20 GB innodb buffer pool) database, and have noticed that when I start it, for about the first half hour it's running, the database will periodically lock and unlock all tables, making it quite unpleasant for users who attempt to access our site for the first half hour after a database restart.
While I can't be 100% certain of the causal relationship, I notice that the time that the database is locking and unlocking itself coincides with the time that
/usr/bin/mysqlcheck --defaults-file=/etc/mysql/debian.cnf --all-databases --fast --silent
and
perl -e $_=join("", <>); s/^[^\n]+\n(error|note)\s+: The (handler|storage engine) for the table doesn.t support check\n//smg;print;
are running on my database server. My question is whether it's really necessary to run mysqlcheck (which is in the mysql /etc/init.d/mysql script by default) upon starting Mysql? The Google results I've found on mysqlcheck indicate that it "fixes & optimizes" tables, but I don't expect my tables to be broken, and I'm skeptical of the optimization advantages afforded by this utility.
If it matters, I'm running Mysql 5.0.32
No.
This is a 'debian' feature that they modify from the stock MySQL. It is designed to stop people from getting into problems with corrupted MyISAM tables. Since you are using InnoDB, I would recommend you remove it. I believe it is part of the init.d script, so you could edit that to remove it.
What do I have to consider when backing up a database with millions of entries? Are there any tools (maybe bundled with the MySQL server) that I could use?
Depending on your requirements, there's several options that I have been using myself:
if you don't need hot backups, take down the db server and back up on the file system level, i. e. using tar, rsync or similar.
if you do need the database server to keep running, you can start out with the mysqlhotcopy tool (a perl script), which locks the tables that are being backed up and allows you to select single tables and databases.
if you want the backup to be portable, you might want to use mysqldump, which creates SQL scripts to recreate the data, but which is slower than mysqlhotcopy
if you have a copy of the db at a certain point in time, you could also just keep the binlogs (starting at that point in time) somewhere safe. This can be very easy to do and doesn't interfere with the server's operation, but might not be the fastest to restore, and you have to make sure you don't miss part of the logs.
Methods I haven't tried, but that make sense to me:
if you have a filesystem like ZFS or are running on LVM, it might be a good idea to do a snapshot of the database by doing a filesystem snapshot, because they are very, very quick. Just remember to ensure a consistent state of your db during the whole operation, e. g. by doing FLUSH TABLES WITH READ LOCK (and of course, don't forget UNLOCK TABLES afterwards)
Additionally:
you can use a master-slave setup to replicate your production server to either a different machine or a second instance on the same machine and do any of the above to the replicated copy, leaving your production machine alone. Instead of running continously, you can also fire up the slave on regular intervals, let it read the binlog, and switch it off again.
I think, MySQL cluster and the enterprise licensed version have more tools, but I have never tried them.
Mysqlhotcopy is badly described - it only works if you use MyISAM, and it's not hot.
The problem with mysqldump is the time it takes to restore the backup (but it can be made hot if you have all InnoDB tables, see --single-transaction).
I recommend using a hot backup tool, like what is available in XtraBackup:
http://www.percona.com/docs/wiki/percona-xtrabackup:start
Watch out if using mysqldump on large tables using the MyISAM storage engine; it blocks selects while the dump is running on each table and this can take down busy sites for 5-10 minutes in some cases.
Using InnoDB, by comparison, you get non-blocking backups because of its row-level locking, so this is not such an issue.
If you need to use MyISAM, a common strategy is to replicate to a second MySQL instance and do the mysqldump against the replicated copy instead.
Use the export tab in phpMyAdmin. phpMyAdmin is the free easy to use web interface for doing MySQL administration.
I think mysqldump is the proper way of doing it.