MySQL replication fails when it replicate from a slave database server - mysql

Question

I have a master MySQL server A with the bin log file bin.000008. And a slave server B with its bin log file bin.000012.
The replication from A to B works well.
Every time I changes the data to server A, the server B will replicate it correctly.
There's a new MySQL server C. I make it to be as a slave to server B.
The problem is:
When I modify the data on server A, server B replicates it correctly. But server C does not be changed. Which means that the replication is invalid from server B to server C.
I tried to restart the slave on server C. And it says Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'
Here are the master and slave status of each server:
Server C (slave of Server B):
db> show slave status\G;
|[
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: luexu.com
Master_User: Aario
Master_Port: 1579
Connect_Retry: 60
Master_Log_File: bin.000012
Read_Master_Log_Pos: 194
Relay_Log_File: cf3613332a61-relay-bin.000004
Relay_Log_Pos: 355
Relay_Master_Log_File: bin.000012
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB: mysql,information_schema,performance_schema,sys
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 194
Relay_Log_Space: 569
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 65535
Master_UUID: dd7aa123-ca18-11e7-9411-0242ac110005
Master_Info_File: /master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: c389aee6-ca16-11e7-81e3-0242ac110004:1-79,
dd7aa123-ca18-11e7-9411-0242ac110005:1-13
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
]|
Server B (slave of Server A, master of Server C):
db> show variables like '%slave%';
+------------------------------+-----------------------+
| Variable_name | Value |
+------------------------------+-----------------------+
| init_slave | |
| log_slave_updates | ON |
| log_slow_slave_statements | OFF |
| pseudo_slave_mode | OFF |
| rpl_stop_slave_timeout | 31536000 |
| slave_allow_batching | OFF |
| slave_checkpoint_group | 512 |
| slave_checkpoint_period | 300 |
| slave_compressed_protocol | OFF |
| slave_exec_mode | STRICT |
| slave_load_tmpdir | /tmp |
| slave_max_allowed_packet | 1073741824 |
| slave_net_timeout | 60 |
| slave_parallel_type | DATABASE |
| slave_parallel_workers | 0 |
| slave_pending_jobs_size_max | 16777216 |
| slave_preserve_commit_order | OFF |
| slave_rows_search_algorithms | TABLE_SCAN,INDEX_SCAN |
| slave_skip_errors | OFF |
| slave_sql_verify_checksum | ON |
| slave_transaction_retries | 10 |
| slave_type_conversions | |
| sql_slave_skip_counter | 0 |
+------------------------------+-----------------------+
db> show master status\G;
|[
*************************** 1. row ***************************
File: bin.000012
Position: 194
Binlog_Do_DB:
Binlog_Ignore_DB: mysql,sys,information_schema,performance_schema
Executed_Gtid_Set: c389aee6-ca16-11e7-81e3-0242ac110004:1-81,
dd7aa123-ca18-11e7-9411-0242ac110005:1-13
1 row in set (0.00 sec)
]|
db> show slave status\G;
|[
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: luexu.com
Master_User: Aario
Master_Port: 5961
Connect_Retry: 60
Master_Log_File: bin.000008
Read_Master_Log_Pos: 27885
Relay_Log_File: 217fcc5843c2-relay-bin.000005
Relay_Log_Pos: 27684
Relay_Master_Log_File: bin.000008
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB: mysql,sys,information_schema,performance_schema
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 27885
Relay_Log_Space: 27938
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: c389aee6-ca16-11e7-81e3-0242ac110004
Master_Info_File: /master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set: c389aee6-ca16-11e7-81e3-0242ac110004:48-81
Executed_Gtid_Set: c389aee6-ca16-11e7-81e3-0242ac110004:1-81,
dd7aa123-ca18-11e7-9411-0242ac110005:1-13
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
1 row in set (0.00 sec)
]|
Server A (master of Server B):
db> show master status\G;
|[
*************************** 1. row ***************************
File: bin.000008
Position: 27885
Binlog_Do_DB:
Binlog_Ignore_DB: mysql,sys,information_schema,performance_schema
Executed_Gtid_Set: c389aee6-ca16-11e7-81e3-0242ac110004:1-81
1 row in set (0.00 sec)
]|
Restart the slave on Server C:
db> stop slave;
db> start slave;
db> show slave status\G;
|[
*************************** 1. row ***************************
Slave_IO_State:
Master_Host: luexu.com
Master_User: Aario
Master_Port: 3306
Connect_Retry: 60
Master_Log_File:
Read_Master_Log_Pos: 4
Relay_Log_File: cf3613332a61-relay-bin.000001
Relay_Log_Pos: 4
Relay_Master_Log_File:
Slave_IO_Running: No
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB: mysql,information_schema,performance_schema,sys
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 0
Relay_Log_Space: 308
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 1236
Last_IO_Error: Got fatal error 1236 from master when reading data from binary log: 'The slave is connecting using CHANGE MASTER TO MASTER_AUTO_POSITION = 1, but the master has purged binary logs containing GTIDs that the slave requires.'
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 65535
Master_UUID: dd7aa123-ca18-11e7-9411-0242ac110005
Master_Info_File: /master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State: Slave has read all relay log; waiting for more updates
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp: 171210 04:01:51
Last_SQL_Error_Timestamp:
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set: 074c1f93-dd08-11e7-b173-0242ac110004:1-3
Auto_Position: 1
Replicate_Rewrite_DB:
Channel_Name:
Master_TLS_Version:
]|

Answer 1

I am not sure about your problem therefore just some thoughts - maybe you will find some inspiration in them:
Do you have some really important reason for setting chain replication A->B->C? If not avoid this complexity and just make C replica of A (you can do it by simple cloning of already working B replica - see here: http://mysql.freeideas.cz/subdom/mysql/2017/08/14/create-new-replica-by-copying-data-and-log-files-on-os-level/
If you really must chain your replication then it generally works OK, we use it for making bigger changes in architecture without downtimes for a long time already and it works even between different MySQL versions.
We used it for the first time when we needed to move from one cloud provider to another - we used (A) old master on old cloud -> (B) new intended master on new cloud -> (C) replicas on new cloud for apps. And currently we set it for new app development with new version of MySQL - (A) old 5.5 Master -> (B) 5.7 replica/master -> (C) 5.7 replicas for app.
You just need to set "log_slave_updates=ON" on "master replica" and create your C replica(s) the same way from B as you created B from A - see general notes here: http://mysql.freeideas.cz/subdom/mysql/2016/11/10/how-to-manually-set-mysql-replication/
Meaning - you have to make dump for C on your B with replication stopped and tables locked against changes and note master status of B and use it to set C. Of course - you have to ensure bin logs on B will live "long enough" - set "expire_logs_days" on B to several days (of course you have to have enough disk space for it). All these changes in setting require restart.