I'm having an issue with replication for two Mysql servers. The master is running Mysql 5.6 and the slave is running Mysql 8.0 due to another dependency that requires at least version 8. I am only replicating one DB, for this example's sake 'my_db'. However at a certain point in the day the master runs backups and tries to update the DB 'my_db_backups'. I am not replicating this DB and I just want to ignore these SQL commands on the replica. I explicitly ignore this database in my mysql.conf file using:
replicate-ignore-db=my_db_backups
However when I restart the mysql server I continue to see an error:
Last_SQL_Errno: 1049
Last_SQL_Error: Error 'Unknown database 'my_db_backups'' on query. Default database: 'my_db'. Query: 'INSERT INTO my_db.`table_1` SELECT * FROM my_db_backups.`2021_01_04_17_36_22_table_1`'
Everything I've read seems to indicate that setting replicate-do-db should only execute queries related to that DB. Additionally adding an explicit ignore, replicate-ignore-db, I would expect to work as well but I'm having no luck.
My full mysql.conf looks like this:
[mysqld]
server-id=20
lower_case_table_names=1
updatable_views_with_limit=1
relay-log-index=slave-relay-bin.index
relay-log=slave-relay-bin
replicate-ignore-db=my_db_backups
replicate-ignore-table=my_db.mysql_8_incompatible_view_1
replicate-ignore-table=my_db.mysql_8_incompatible_view_2
replicate-ignore-table=my_db.mysql_8_incompatible_view_3
replicate-ignore-table=my_db.mysql_8_incompatible_view_4
replicate-wild-do-table=my_db.%
replicate-do-db=my_db
I have tried every combination of these replicate-do and replicate-ignore-do configurations that I can think of I've tried adding:
replicate-wild-ignore-table=my_db_backups.%
and removing:
replicate-ignore-db=my_db_backups
and I've tried putting them both in the conf file. Nothing seems to work.
I know that I can explicitly skip error 1049 which is what I may have to end up doing. I'm just pulling my hair out trying to figure out why the other configurations aren't having any effect and can't let it go.
Any help would be greatly appreciated!
I noticed that the statement that produces the error isn't actually writing to a table in my_db_backups. It's replicating an INSERT to a table in my_db.
INSERT INTO my_db.`table_1` SELECT * FROM my_db_backups.`2021_01_04_17_36_22_table_1`
The statement is reading from a table in my_db_backups, which it did not find on the replica.
So the statement that inserts into my_db.table_1 will be executed on the replica, because the default database my_db was not ignored.
You could also fix this by using ROW-based binary logs on the master, so it doesn't matter that it's reading from a table that doesn't exist on the replica. The INSERT would result in a row image on the master, where the table does exist.
Related
I'm trying to create an MySQL 5.7.35 read-only replica on GCP from an external origin. The database is enourmous, with aproximately 800G of data.
I have already ajusted the definer on the triggers, views and functions in a way that GCP accepts (root#%) and thefore the full-dump that the Database Migration Service makes worked. Also got the replication working with the schema of this database (no data).
So far made just one attempt with data. On this attempt the full-dump was sucesful (took 2 days and 10 hours) and failed some time after the replication started with the following error:
2021-09-05T06:09:33.293123Z 2 [ERROR] Slave SQL for channel '': Could not execute Write_rows event on table pacsdb.content_item; Duplicate entry '1441957' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000005, end_log_pos 78621021, Error_code: 1062
Selecting this row on the replica returned the same data of the origin (the row was already there).
Since I can't stop slave, skip_counter and start slave or something like that on GCP I have to figure out why this is happening.
My next step would be try to make the dump manualy using the flags that Google recommends.
Someone had a similar problem or have a clue why this is happening?
Any tips are apreciated, thx!
Activating the consistency warnings and GTID-based replication should work. There is information relating replication with Global Transaction Identifiers for MySQL 5.7 here [1].
[1] - https://dev.mysql.com/doc/refman/5.7/en/replication-gtids.html
I run a web application on a shared LiteSpeed server with Mariadb version 10.3.22. I do have access to a terminal.
Cron is set up to perform a daily mysqldump, but I now get the error message:
mysqldump: Error 1412: Table definition has changed, please retry transaction when dumping table `TABLENAME` at row: 0
I do get a sql file, but it is a little smaller than it used to be. The table that is claimed to have changed it's definition is empty, just as it use to be (as far as I know).
I do not know much about mysql, and I do not understand the message "retry transaction when dumping table TABLENAME at row: 0"
Any clues about how to deal with that?
Please check the general log file of mysql service has "TRUNCATE" command on the table or not.
"TRUNCATE" command on the table during the mysqldump process changes table information and changing the command to "DELETE" command solved issue to me.
Default location of general log file is "/var/log/mysql/mysql.log" but it could be changed by "general_log_file" option and "general_log" option on "my.cnf" file.
If you are using mysql and getting error of "table defination has changed, please retry transaction " you have to do only one thing that is reconnect to DBMS, it will definitely solve the problem , tere is a button of "reconnect to DBMS" at home Page.
I'm trying to move from a ISAM mysql database to an InnoDB mysql database (10's of millions of records/rows). So, I'm trying to use mysqldbexport to export just the data from the tables so that we don't need to adjust the sql files once exported. I'm trying to use the following command and it doesn't actually exclude the required tables.
mysqldbexport --server=username:password#localhost db_name --export=data --bulk-insert --exclude=db_name.table_name --output-file=full_db_name_export.sql
Originally started on version:
MySQL Utilities mysqldbexport version 1.5.6
This failed to exclude the table. It also fails to complete the export anyway, presumably due to resource (memory) limitations.
I then updated to:
MySQL Utilities mysqldbexport version 1.6.5
I now get the error:
Source on localhost: ... connected. ERROR: Query failed. 1238 (HY000): Variable 'foreign_key_checks' is a SESSION variable
I have tried using the command SET GLOBAL FOREIGN_KEY_CHECKS=0; on the mysql command line and it just complains saying that it's a SESSION variable.
Can anyone help with either fixing the issue with the first version not excluding or, help me with getting around the issue with the second version? Or better yet, give me a "lmgtfy" link that works :)
I also got this error using mysqldbcopy,you could solve it by 2 ways:
1.upgrade destination MySQL to a higher version that supporting syntax--SET GLOBAL FOREIGN_KEY_CHECKS=0
2.comemnt out this line destination.disable_foreign_key_checks(True) in source file .../site-packages/mysql/utilities/command/dbexport.py
I am trying to migrate a test website on drupal onto a live server on Amazon ec2. I migrated the database using phpmyadmin, and tried to access the site. I got this error:
PDOException: SQLSTATE[42000]: Syntax error or access violation: 1286 Unknown table engine 'InnoDB': SELECT expire, value FROM {semaphore} WHERE name = :name; Array ( [:name] => variable_init ) in lock_may_be_available() (line 167 of /var/www/includes/lock.inc).
I believe the problem here is that MySQL doesn't have InnoDB. I have looked through the my.cnf file and there is no line that says skip-innodb.
I have tried show engines and it showed a bunch of engines but not innodb.
I have tried restarting my server and deleting the logfile, just like has been suggested previously, but that didn't work.
Maybe what needs to be done is to somehow install the Innodb engine. Could you tell me how I may do that?
On Debian 6 this can also happen when you change the innodb_log_file_size parameter... some times mysql does not start.. but other times it just starts up bud disables innodb engine... so solution is to remove ib_logfiles from /var/lib/mysql and restart mysql
Look in your mysql error log. Run
select ##log_error;
To see where exactly that is.
There is probably something in there telling why it failed on startup. (Perhaps trying to allocate more buffer pool than you have memory?)
I have set up replication for MySQL server. I can connect from the slave machine to the master server using the replication user/password. I have got the slave SQL thread running, but the slave I/O thread is not running and the slave I/O status comes as empty when checked using 'show slave status'. What could be the problem?
How do I solve this? Restarting the slave does not help.
This was my bad: Instead of giving a 'replication slave' privilege to *.*, I was only giving it for my_db.*.
Instead of giving a 'replication
slave' privilege to ., I was only
giving it for my_db.*.
Replication slave is only a global privilege (i.e. per-user only), this means that a command such as
GRANT REPLICATION SLAVE on mydb.* TO 'someuser'#'%';
has no effect as you can't grant it per-database/column/table.
The command you need to run is:
GRANT REPLICATION SLAVE on *.* TO 'someuser'#'%';
Then do a START SLAVE. You might also find it useful to look in the mysql error log.
I'd suggest a good read of the replication setup documentation, as it explains all of this in detail.
I have encountered the same problem and I try this steps
First add this code somewhere below [mysqld] in my.cnf or my.ini slave-skip-errors=1046
this will skip all duplicate entry since we will execute the whole binary log file where the replication stop, you may comment this code after successful replication.
1.STOP SLAVE;
2.RESET SLAVE;
3.CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000049';
Note: MASTER_LOG_FILE must be the last file where it stop from replicating
4.CHANGE MASTER TO MASTER_LOG_POS=98;
5.START SLAVE;
check if you are successful
I faced same issue and fixed using following steps. Complete thread link is http://www.percona.com/forums/questions-discussions/percona-xtrabackup/11842-backup-stopped-working-slave-sql-running-no
Steps are same as mentioned by #Luxknight007 except his step 2. However this thread contains more detail which is very helpful. Following is solution which i used and it worked.
"The first issue is that you changed the replication position instead of fixing the error, and used an incorrect binlog file name format (you likely just used the one from that post you linked I'd guess). To get back to where you started, you need to find the binlog file and position that the slave sql_thread stopped at. Based on your slave status output, it looks like the slave is reading from a new binlog file (you can see that the Read_Master_Log_Pos value is smaller than the Exec_Master_Log_Pos value, which means it has to be reading a newer binlog file than where the slave sql_thread stopped at), so you need to find the binlog file that the slave sql_thread actually failed on. So look in the error log for something like the below:
Code:
2013-10-08 12:48:51 37545 [ERROR] Slave SQL: Error 'Table 'testdb.test2' doesn't exist' on query. Default database: 'testdb'. Query: 'insert into test1 select * from test2', Error_code: 1146
2013-10-08 12:48:51 37545 [Warning] Slave: Table 'testdb.test2' doesn't exist Error_code: 1146
2013-10-08 12:48:51 37545 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000001' position 3427
This is a sample I re-created, so yours will be a bit different. Note the ERROR is similar to what you see in your slave status. So find your specific error message in the error log file, and then locate the end part where is gives you the file name and position ("We stopped at log 'mysql-bin.000001' position 3427" in my example). The position should be 315098143 based on your show slave status, as that is when it the slave sql_thread stopped executing events (Exec_Master_Log_Pos ) but the io_thread kept reading in new ones (Read_Master_Log_Pos).
Once you find the correct binlog file name and position, re-run your change master statement on your slave using the information you located in the error log. Note that your file name should be something like "newcrmdb1-bin.XXXXXX", not mysql-bin.XXXXXX (you can see this naming convention your show slave status above).
Code:
mysql> change master to MASTER_LOG_FILE='newcrmdb1-bin.XXXXXX', Master_Log_Pos=315098143;
change master to MASTER_LOG_FILE='mysql-bin.000082' , Master_Log_Pos=47914844;
Once you get pointed back to the original replication location where the slave sql_thread failed, you need to then fix the error that it was complaining about to start with.
The initial replication error appears to be telling you that the table asteriskcdr.bpleadcf does not exist on the slave, so the insert statement is failing when it attempts to select the data from that table. So the problem there is that your slave appears to be already out of sync with your master. If the table in question on the master is static or mostly static, you could likely solve this by exporting the data from just that table on the master using mysqldump and loading it into the slave. If that is not possible, or you do not care about that data, you could always just skip the replication statement with sql_slave_skip_counter, but then the slave would be further out of sync with the master.
And if all else fails, you can always rebuild the slave from the master as a last resort as well. =)"