We recently did a migration from MySQL 5.5 to MySQL 5.7, on two different servers. For the migration, we used a master-slave replication. The old DB (5.5) was set as master, the new one as slave (5.7).
Everything went well and the migration was successful. The DB is large, about 600G of data.
The problem we have now is that on the new DB, ther is one large .ibd file of 82G. Let's asume the DB name is my_database. This file is located under:
/var/lib/mysql/my_database:
-rw-r----- 1 mysql mysql 82G Apr 20 10:28 articles.ibd
The articles table is big, but why there is such large ibd file on the new MySQL server, and was not there on the old one.
As we have now some storage limits (SSD), we would like to somehow delete this file. How to do it and why this file is created there on MySQL5.7?
Basically just run ALTER TABLE [TABLENAME] ENGINE=InnoDB ALGORITHM=COPY; after disabling innodb_file_per_table.
cf. https://dev.mysql.com/doc/refman/5.7/en/tablespace-copying.html and https://dba.stackexchange.com/questions/136250/migrate-from-innodb-file-per-table-to-off-in-mysql-alter-table-removing-ibd-file, https://dev.mysql.com/doc/refman/5.7/en/innodb-multiple-tablespaces.html
Related
Hi I have an existing mysql master-master replication between 3 servers and all the servers are connected to each other. I'm replicating some tables in a specific database. there is about 80 tables in this database and only 10 of them is replicating. I'm using replicate-do-table option in my mysql config file to tell mysql which tables should be replicated.
In this stage I found that I need another table to be replicated which was not replicated before. When I add new table to mysql config file, Newly added data are coming to all the server but old existing data are not coming. what should I do to bring older data to all the servers.
this is my mysql config file
default_table_encryption=ON
table_encryption_privilege_check=ON
replica_parallel_workers=4
enforce_gtid_consistency=ON
gtid_mode=ON
keyring_file_data=.....
early-plugin-load=keyring_file.so
skip_replica_start=OFF
auto_increment_increment=50
auto_increment_offset=1
server_id=3
replicate_same_server_id=0
replicate-do-db=db
replicate-do-table=db.table1
replicate-do-table=db.table2
replicate-do-table=db.table3
replicate-do-table=db.table4
replicate-do-table=db.table5
replicate-do-table=db.table6
replicate-do-table=db.table7
replicate-do-table=db.table8
replicate-do-table=db.table9
replicate-do-table=db.table10
replicate-do-table=db.table11
replicate-do-table=db.table12
slave-skip-errors=1032
this isn't my exact situation but close enough for description.
On a MySQL 8.0.28 instance on an ubuntu 22.08 server, I have a database 'important' that contains only MyISAM tables. Within MySQL I:
DROP DATABASE important;
then continue to 'do stuff' with other databases etc. Some time later, I decide I want that 'important' database back (without restoring other databases). Luckily I have a physical backup of the MySQL data folder, specifically including the 'important' directory.
If I stop the server, copy the 'important' directory back in place, then restart the server I hoped (and was the case under MySQL 5) that the database would re-appear, but it doesn't.
As far as I can sell, MySQL 5 used to physically scan the directory to look for databases whereas MySQL 8 seems to have a separate idea of what databases exist. To clarify, I am only using MySQL 8.0.28 in this scenario - the backup was made under the same version as I'm restoring to.
My question is, how can I make this database reappear? Perhaps that's "How can I make MySQL 8 rescan its database of databases"?
I think you're in uncharted territory.
It's true that in MySQL 5.x, you could create a schema simply with mkdir, creating a subdirectory under the data directory. Within that directory, each MyISAM table has three files: tablename.MYD, tablename.MYI, and tablename.frm. The frm file has metadata about the table and its columns, so the MyISAM storage engine could discover new tables simply by reading their respective frm file.
This was not possible for InnoDB in 5.x, because InnoDB had an internal data dictionary stored in the ibdata1 tablespace in the data directory. The data dictionary had to maintain duplicate metadata about InnoDB tables.
In MySQL 8.0, they eliminated frm files completely. Tables of all storage engines are managed by the InnoDB data dictionary. There is no interface to the data dictionary besides DDL statements like CREATE/ALTER/DROP. There are several benefits to this change.
This means you can't use external shell commands like mkdir and cp to create schemas and tables. You have to use DDL to communicate to the data dictionary. It's not only stored on disk in the InnoDB tablespace, but it's also cached as objects in the MySQL Server process. Using DDL updates both.
I have not tried this with MyISAM tables (I haven't used MyISAM tables if I can avoid it for over ten years), but in theory you could use CREATE SCHEMA and CREATE TABLE to recreate empty tables that match the definition of the tables you're restoring. Then outside of MySQL, use cp to copy the tables you want to restore over the empty data and index files. I would shut down MySQL Server before doing this, to hopefully avoid any race conditions or conflicting with file caches.
I have no idea if that will work. I would test these steps first on a test instance (not your production instance).
Good day to you.
I used my MySQL server with "innodb_file_per_table" option, and now server is crashed. I want to recover this server using this way:
Uninstall old MySQL
Install new MySQL
Add "innodb_file_per_table" in MySQL configuration
Copy databases folders (only my, not mysql) from old MySQL/data to new MySQL/data
In every folder I have two files, .frm and .ibd, and looks like this files have all data from my databases tables.
But, after copying, tables in this databases didn't work: when I try to open some table, I get error: Table xxx doesn't exist in engine.
I tried REPAIR command, but it isn't helpful.
If you know how to finish my way of repair — please help.
...I know that I need to copy ibdata1 also, but it looks so dead for recovery, so I try way that I try.
REPAIR command won't help with InnoDB.
If you're lucky enough the best you can do is:
1. Start MySQL with innodb_force_recovery=4 (try 5, 6 values if InnoDB fails to start). Make sure innodb_purge_threads=0.
2. Dump the database with mysqldump tool. Yes, it may be slow, but there is no other choice.
3. Create new empty InnoDB table space and reload the dump.
If MySQL fails to start with innodb_force_recovery=6, then recovery from backups is the only option. Well, you can fetch records from *.ibd files, but this is tedious job - google data recovery tool from percona
UPDATE: Data recovery toolkit moved to GitHub
You need to copy everything, not only the data folder
For example, without the ibdata file, mysql don't know where the tables are stored.
https://serverfault.com/questions/487159/what-is-the-ibdata1-file-in-my-var-lib-mysql-directory
Two days ago I have installed RoundCube mail script working on:
PHP
Name : php
Arch : x86_64
Version : 5.3.18
Release : 1.el6.remi
MySQL
Name : mysql
Arch : x86_64
Version : 5.5.28
Release : 1.el6.remi
Anything worked just great till today morning, script refers that Round Cube database has problems. How wonder was I am looking at directory:
/var/lib/mysql/roundcubemail
There is only frm files, no any .myd or .myi
cache.frm
cache_index.frm
cache_messages.frm
cache_thread.frm
contactgroupmembers.frm
contactgroups.frm
contacts.frm
db.opt
dictionary.frm
identities.frm
searches.frm
session.frm
users.frm
My server has 12 databases and now only Round Cube db missing .myd .myi files. Re-install this script is not a problem but I am interesting to figure out what was happened to this database and to be sure that it will not going to happen again.
MySQL error log has only this information:
121117 7:38:06 [ERROR] Cannot find or open table roundcubemail/session from
the internal data dictionary of InnoDB though the .frm file for the
table exists. Maybe you have deleted and recreated InnoDB data
files but have forgotten to delete the corresponding .frm files
of InnoDB tables, or you have moved .frm files to another database?
or, the table contains indexes that this version of the engine
doesn't support.
See http://dev.mysql.com/doc/refman/5.5/en/innodb-troubleshooting.html
how you can resolve the problem.
My question is: Is there any other methods to track this situation and find the reason that database just had disappeared?
.myd and .myi files only exist for MyISAM tables.
If you followed the standard Roudcube installation procedure (verified with v0.8.4), tables were set to use the InnoDB storage engine.
Have you messed with any file under /var/lib/mysql/ ? By default, InnoDB tables are stored in one single file, probably /var/lib/mysql/ibdata1 in your case. If not, look for the innodb_data_file_path directive in your my.cnf configuration file.
We're in the middle of an upgrade from mysql 5.1 to mysql 5.5. Our only remaining holdout is the master, at 5.1, while all the slaves are at 5.5.
However, we have a stats process that runs on an admin slave, which rsyncs the myisam files to the master, who mount it in a stats DB, then imports those stats into the main production db.
All the tables use the default row_format for myisam, which is static. We're only using simple column types like int, varchar, enum, datetime.
We've already run this process, and it didn't fall down, which is encouraging. But I'm concerned with errors that didn't make it into the log.
So the question is: what are the complications of this process? The MyISAM file is created by a 5.5 instance, then moved to a 5.1 instance.
What's going to go wrong?