MySQL slave I/O thread not running - mysql

I have set up replication for MySQL server. I can connect from the slave machine to the master server using the replication user/password. I have got the slave SQL thread running, but the slave I/O thread is not running and the slave I/O status comes as empty when checked using 'show slave status'. What could be the problem?
How do I solve this? Restarting the slave does not help.
This was my bad: Instead of giving a 'replication slave' privilege to *.*, I was only giving it for my_db.*.

Instead of giving a 'replication
slave' privilege to ., I was only
giving it for my_db.*.
Replication slave is only a global privilege (i.e. per-user only), this means that a command such as
GRANT REPLICATION SLAVE on mydb.* TO 'someuser'#'%';
has no effect as you can't grant it per-database/column/table.
The command you need to run is:
GRANT REPLICATION SLAVE on *.* TO 'someuser'#'%';
Then do a START SLAVE. You might also find it useful to look in the mysql error log.
I'd suggest a good read of the replication setup documentation, as it explains all of this in detail.

I have encountered the same problem and I try this steps
First add this code somewhere below [mysqld] in my.cnf or my.ini slave-skip-errors=1046
this will skip all duplicate entry since we will execute the whole binary log file where the replication stop, you may comment this code after successful replication.
1.STOP SLAVE;
2.RESET SLAVE;
3.CHANGE MASTER TO MASTER_LOG_FILE='mysql-bin.000049';
Note: MASTER_LOG_FILE must be the last file where it stop from replicating
4.CHANGE MASTER TO MASTER_LOG_POS=98;
5.START SLAVE;
check if you are successful

I faced same issue and fixed using following steps. Complete thread link is http://www.percona.com/forums/questions-discussions/percona-xtrabackup/11842-backup-stopped-working-slave-sql-running-no
Steps are same as mentioned by #Luxknight007 except his step 2. However this thread contains more detail which is very helpful. Following is solution which i used and it worked.
"The first issue is that you changed the replication position instead of fixing the error, and used an incorrect binlog file name format (you likely just used the one from that post you linked I'd guess). To get back to where you started, you need to find the binlog file and position that the slave sql_thread stopped at. Based on your slave status output, it looks like the slave is reading from a new binlog file (you can see that the Read_Master_Log_Pos value is smaller than the Exec_Master_Log_Pos value, which means it has to be reading a newer binlog file than where the slave sql_thread stopped at), so you need to find the binlog file that the slave sql_thread actually failed on. So look in the error log for something like the below:
Code:
2013-10-08 12:48:51 37545 [ERROR] Slave SQL: Error 'Table 'testdb.test2' doesn't exist' on query. Default database: 'testdb'. Query: 'insert into test1 select * from test2', Error_code: 1146
2013-10-08 12:48:51 37545 [Warning] Slave: Table 'testdb.test2' doesn't exist Error_code: 1146
2013-10-08 12:48:51 37545 [ERROR] Error running query, slave SQL thread aborted. Fix the problem, and restart the slave SQL thread with "SLAVE START". We stopped at log 'mysql-bin.000001' position 3427
This is a sample I re-created, so yours will be a bit different. Note the ERROR is similar to what you see in your slave status. So find your specific error message in the error log file, and then locate the end part where is gives you the file name and position ("We stopped at log 'mysql-bin.000001' position 3427" in my example). The position should be 315098143 based on your show slave status, as that is when it the slave sql_thread stopped executing events (Exec_Master_Log_Pos ) but the io_thread kept reading in new ones (Read_Master_Log_Pos).
Once you find the correct binlog file name and position, re-run your change master statement on your slave using the information you located in the error log. Note that your file name should be something like "newcrmdb1-bin.XXXXXX", not mysql-bin.XXXXXX (you can see this naming convention your show slave status above).
Code:
mysql> change master to MASTER_LOG_FILE='newcrmdb1-bin.XXXXXX', Master_Log_Pos=315098143;
change master to MASTER_LOG_FILE='mysql-bin.000082' , Master_Log_Pos=47914844;
Once you get pointed back to the original replication location where the slave sql_thread failed, you need to then fix the error that it was complaining about to start with.
The initial replication error appears to be telling you that the table asteriskcdr.bpleadcf does not exist on the slave, so the insert statement is failing when it attempts to select the data from that table. So the problem there is that your slave appears to be already out of sync with your master. If the table in question on the master is static or mostly static, you could likely solve this by exporting the data from just that table on the master using mysqldump and loading it into the slave. If that is not possible, or you do not care about that data, you could always just skip the replication statement with sql_slave_skip_counter, but then the slave would be further out of sync with the master.
And if all else fails, you can always rebuild the slave from the master as a last resort as well. =)"

Related

Replication failed with error code 1062 using Google Database Migration Service on big database (~800G)

I'm trying to create an MySQL 5.7.35 read-only replica on GCP from an external origin. The database is enourmous, with aproximately 800G of data.
I have already ajusted the definer on the triggers, views and functions in a way that GCP accepts (root#%) and thefore the full-dump that the Database Migration Service makes worked. Also got the replication working with the schema of this database (no data).
So far made just one attempt with data. On this attempt the full-dump was sucesful (took 2 days and 10 hours) and failed some time after the replication started with the following error:
2021-09-05T06:09:33.293123Z 2 [ERROR] Slave SQL for channel '': Could not execute Write_rows event on table pacsdb.content_item; Duplicate entry '1441957' for key 'PRIMARY', Error_code: 1062; handler error HA_ERR_FOUND_DUPP_KEY; the event's master log mysql-bin.000005, end_log_pos 78621021, Error_code: 1062
Selecting this row on the replica returned the same data of the origin (the row was already there).
Since I can't stop slave, skip_counter and start slave or something like that on GCP I have to figure out why this is happening.
My next step would be try to make the dump manualy using the flags that Google recommends.
Someone had a similar problem or have a clue why this is happening?
Any tips are apreciated, thx!
Activating the consistency warnings and GTID-based replication should work. There is information relating replication with Global Transaction Identifiers for MySQL 5.7 here [1].
[1] - https://dev.mysql.com/doc/refman/5.7/en/replication-gtids.html

MySQL 8.0 Replication 'Unknown Database' Error Causes Replication to Stop

I'm having an issue with replication for two Mysql servers. The master is running Mysql 5.6 and the slave is running Mysql 8.0 due to another dependency that requires at least version 8. I am only replicating one DB, for this example's sake 'my_db'. However at a certain point in the day the master runs backups and tries to update the DB 'my_db_backups'. I am not replicating this DB and I just want to ignore these SQL commands on the replica. I explicitly ignore this database in my mysql.conf file using:
replicate-ignore-db=my_db_backups
However when I restart the mysql server I continue to see an error:
Last_SQL_Errno: 1049
Last_SQL_Error: Error 'Unknown database 'my_db_backups'' on query. Default database: 'my_db'. Query: 'INSERT INTO my_db.`table_1` SELECT * FROM my_db_backups.`2021_01_04_17_36_22_table_1`'
Everything I've read seems to indicate that setting replicate-do-db should only execute queries related to that DB. Additionally adding an explicit ignore, replicate-ignore-db, I would expect to work as well but I'm having no luck.
My full mysql.conf looks like this:
[mysqld]
server-id=20
lower_case_table_names=1
updatable_views_with_limit=1
relay-log-index=slave-relay-bin.index
relay-log=slave-relay-bin
replicate-ignore-db=my_db_backups
replicate-ignore-table=my_db.mysql_8_incompatible_view_1
replicate-ignore-table=my_db.mysql_8_incompatible_view_2
replicate-ignore-table=my_db.mysql_8_incompatible_view_3
replicate-ignore-table=my_db.mysql_8_incompatible_view_4
replicate-wild-do-table=my_db.%
replicate-do-db=my_db
I have tried every combination of these replicate-do and replicate-ignore-do configurations that I can think of I've tried adding:
replicate-wild-ignore-table=my_db_backups.%
and removing:
replicate-ignore-db=my_db_backups
and I've tried putting them both in the conf file. Nothing seems to work.
I know that I can explicitly skip error 1049 which is what I may have to end up doing. I'm just pulling my hair out trying to figure out why the other configurations aren't having any effect and can't let it go.
Any help would be greatly appreciated!
I noticed that the statement that produces the error isn't actually writing to a table in my_db_backups. It's replicating an INSERT to a table in my_db.
INSERT INTO my_db.`table_1` SELECT * FROM my_db_backups.`2021_01_04_17_36_22_table_1`
The statement is reading from a table in my_db_backups, which it did not find on the replica.
So the statement that inserts into my_db.table_1 will be executed on the replica, because the default database my_db was not ignored.
You could also fix this by using ROW-based binary logs on the master, so it doesn't matter that it's reading from a table that doesn't exist on the replica. The INSERT would result in a row image on the master, where the table does exist.

Mariadb master-slave replication stopped without errors

I have setup mariadb master slave replication about a week back.It worked fine for 3-4 days but later the replication has stopped without giving any errors.The value of seconds behind master parameter is increasing.
I have posted the screenshot of
SHOW SLAVE STATUS;
below for reference.
Below is the result of SHOW PROCESSLIST;
Doing mysqlbinlog mysql-relay-bin.000040 > slave.txt this is the error i got
ERROR: Error in Log_event::read_log_event(): 'Found invalid event in binary log', data_len: 39, event_type: -93
ERROR: Could not read entry at offset 535: Error in log format or read error.
How to get the replication to work again?This problem is because of master or slave db?
The problem seems to be because of corrupted mysql binary log.I ended up skipping a counter and replication continued from there.Also shifted to row based replication which seems like much safer option than statement based(this has nothing to do with replication stopping).

mysql: When exactly the statement "Start binlog_dump to slave_server" logged in to the master error log

Recently, I have observed the below statement in the master server's MySQL error log many number of times.
"started bin_log dump to slave server"
I would like to know all the possible cases of when exactly this will be logged in the error log.
How connection failure between master and slave will impact this to be logged into the error log.
I suspect personally when we hit "START SLAVE" on the slave server, But it is not as I tested this.
Thanks in advance.
Exact message is this: Start binlog_dump to slave_server
In the source code this string is saved in the logfile when this function is called: mysql_binlog_send
With a bit of research, I see that just before this function is called, changes the slave state to this: Requesting binlog dump and in the MySQL reference manual says that this state is set just when the slave connects with the master.
So, exactly, this message appears when the slave connects with the master.
So it seems that the slave loses connection with the master and then reconnects.
Is this and error or not? I suspect "started bin_log dump to slave server" is not the entire log line if it is an error. Please add an actual snippet from the log in question of the repeating notice.
If replication looks like its working fine, this isn't an error. Non- errors such as servers start and stopping and slave connection status get logged to the MySQL error log. You can override the default logging levels to lower how much and what gets logged:
http://dev.mysql.com/doc/refman/5.5/en/replication-options-slave.html
Look at the --log-warnings configuration option
I suspect this occurs when you change a slave to a master or when a slave server restarts and try's to reestablish a connection.
I meet the same condition like that
160615 15:40:25 [Note] Start binlog_dump to slave_server(1401), pos(mysql-bin.000020, 219311193)
160622 15:56:58 [Note] Start binlog_dump to slave_server(1401), pos(mysql-bin.000046, 350644600)
when i stop the slave, and start slave again ,the master's mysql error logs would print the note log.
This is nothing to be care. It just means that the slave server (server id is 1401) is begin to connect the master.
Need to be noted that if your masters' mysql error log prints the note log frequently. you will check :
whether the network is stable
if the master have one more slaves ,check the slaves whether have the same server-id.
that's all.

How to find mysql DB is slave?

How to find mysql DB is slave with out using "show slave status" by query?
Here are 3 options you have to detect if Replication is running
OPTION #1 : Check Status Variable 'Slave_running'
Using MySQL 5.1/5.5
select variable_value from information_schema.global_status
where variable_name = 'Slave_running';
Using MySQL 5.0 and back
SHOW VARIABLES LIKE 'Slave_running';
OPTION #2 : Check the Process List
Using MySQL 5.1+/5.5
select COUNT(1) SlaveThreads
from information_schema.processlist
where user = 'system user';
If SlaveThreads = 2, Replication is Running
If SlaveThreads = 1, Replication is Broken
If SlaveThreads = 0, Replication is Stopped or Disabled
Using MySQL 5.0 and back
SHOW PROCESSLIST;
Look for 2 DB Conenctions thaty have 'system user' in the user column.
OPTION #3 : Check for presence of master.info
If replication is setup on a DB Server, look for master.info. By default, master.info is usually in /var/lib/mysql or wherever datadir is defined.
Simply run 'cat master.info' multiple times (For Windows community, type master.info). If the log position is moving, replication is on. If the log position is not moving, it could mean that replication is either broken (SQL Error in SQL Thread), stopped (due to STOP SLAVE;), or disabled (by running CHANGE MASTER TO MASTER_HOST='';).
According to MySQL doc - Checking Replication Status:
Slave_IO_Running: Whether the I/O
thread for reading the master's binary
log is running. Normally, you want
this to be Yes unless you have not yet
started replication or have explicitly
stopped it with STOP SLAVE.
Slave_SQL_Running: Whether the SQL
thread for executing events in the
relay log is running. As with the I/O
thread, this should normally be Yes.
Prior to MySQL 5.7, you can check the 'slave_running' variable by executing the following query:

 SHOW GLOBAL STATUS LIKE 'slave_running';
Since MySQL 5.7, the slave_running has been removed and the above query returns an empty set
You can enable "show_compatibility_56" to get the value but “show_compatibility_56” is deprecated and will be removed soon. The reason for this is because MySQL is moving away from the information_schema GLOBAL_STATUS and SESSION_STATUS tables in preference for performance_schema.
The correct way to get the status of the slave running in MySQL 5.7 outside of SHOW SLAVE STATUS is to use the new replication-based performance_schema tables.
You can execute the following query to get the status of the replication service:

SELECT SERVICE_STATE FROM performance_schema.replication_connection_status;