I am creating the MySQL master-slave replication on CentOS 7 below are the configuration files details of both the server:
Master server's my.cnf
[mysqld]
server-id=1
log-bin=mysql-bin
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
Slave Server's my.cnf
[mysqld]
server-id=2
replicate-wild-do-table=db1%.%
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid
Database details given below:
Master Server
create database db1;
create table db1.dbtb1(name varchar(100));
Slave Server
create database db1slave;
create table db1slave.dbtb1(name varchar(100));
AS you can see the Database name on both the sever are different for that I have use the below statement into the slave server for replication.
replicate-wild-do-table=db1%.%
But when I try to insert data into the Master Data base server I got the below error or slave server status:
Error 'Table 'db1.dbtb1' doesn't exist' on query. Default database: ''. Query: 'insert into db1.dbtb1 values ('Punu')'
Slave server Status:
Slave_IO_State: Waiting for master to send event
Master_Host: IP address
Master_User: repl
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000003
Read_Master_Log_Pos: 971
Relay_Log_File: mysqld-relay-bin.000002
Relay_Log_Pos: 283
Relay_Master_Log_File: mysql-bin.000003
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table: db1%.%
Replicate_Wild_Ignore_Table:
Last_Errno: 1146
Last_Error: Error 'Table 'db1.dbtb1' doesn't exist' on query. Default database: ''. Query: 'insert into db1.dbtb1 values ('Punu')'
Skip_Counter: 0
Exec_Master_Log_Pos: 760
Relay_Log_Space: 668
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1146
Last_SQL_Error: Error 'Table 'db1.dbtb1' doesn't exist' on query. Default database: ''. Query: 'insert into db1.dbtb1 values ('Punu')'
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Master_UUID: 2fc2ef76-c87a-11e6-ae22-000d3aa2da57
Master_Info_File: /var/lib/mysql/master.info
SQL_Delay: 0
SQL_Remaining_Delay: NULL
Slave_SQL_Running_State:
Master_Retry_Count: 86400
Master_Bind:
Last_IO_Error_Timestamp:
Last_SQL_Error_Timestamp: 161227 18:40:36
Master_SSL_Crl:
Master_SSL_Crlpath:
Retrieved_Gtid_Set:
Executed_Gtid_Set:
Auto_Position: 0
If I will take the same database name it will work fine. But as per my requirement I can't put the database name same on both the server.
Simply I want that when the table "dbtb1" updated in Master server database then it will be replicated to the slave database table.
I have also try the below options in the slave my.cnf file:
1
replicate-wild-do-table=db1%.dbtb1
2
replicate-rewrite-db="db1->db1slave"
Above option will work fine if I insert record but it will not reflect when I delete or update record.
3
replicate-wild-do-table=%.dbtb1
your option replicate-wild-do-table=db1%.% will replicate all table in all schemas starts with db1% ans also replicate all tables start with %. so it will replicate all tables. thats a wrong config.
use the replicate-rewrite-db option to config this. read this below.
https://mariadb.com/resources/blog/multisource-replication-how-resolve-schema-name-conflicts
Related
We are upgrading webservers and are trying to do the migration without downtime.
We're currently running MySQL 5.1.73 as a master and the new server (slave) is MariaDB 10.1.22
We have temporary tables in MySQL that get used to create statistics and then they are truncated.
The SQL that gets run is an INSERT..SELECT 0, (0 is for the primary column id that has auto increment)..
insert into tempStats select 0,tl.date,tl.sellerId...
I did a mysqldump from MySQL got the position, imported the data to MariaDB, started up the slave and then we keep getting the 'Duplicate entry' SQL errors.
Example:
MariaDB [(none)]> show slave status\G; *************************** 1. row *************************** Slave_IO_State: Waiting for master to send event
Master_Host: SERVERIP_HERE
Master_User: st3slave
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysqld-bin.001097
Read_Master_Log_Pos: 346910538
Relay_Log_File: main-dom0-relay-bin.000002
Relay_Log_Pos: 69379592
Relay_Master_Log_File: mysqld-bin.001093
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB: database_name_here
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1062
Last_Error: Error 'Duplicate entry '1923' for key 'PRIMARY'' on query. Default database: 'XYZ'. Query: 'insert into tempStats select 0,tl.date,tl.sellerId from tempLeadStatsSellersLeadsData tl left join tempLeadStatsSellers t on (t.date = tl.date AND t.sellerId = tl.sellerId AND t.type = tl.type AND t.subId = tl.subId) where isnull(t.date)'
Skip_Counter: 0
Exec_Master_Log_Pos: 410583345
Relay_Log_Space: 4300695570
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: Yes
Master_SSL_CA_File: /etc/mysql/sslcerts/ca-cert.pem
Master_SSL_CA_Path:
Master_SSL_Cert: /etc/mysql/sslcerts/new-live-server-cert.pem
Master_SSL_Cipher:
Master_SSL_Key: /etc/mysql/sslcerts/new-live-server-key.pem
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1062
Last_SQL_Error: Error 'Duplicate entry '1923' for key 'PRIMARY'' on query. Default database: 'XYZ'. Query: 'insert into tempStats select 0,tl.date,tl.sellerId from tempLeadStatsSellersLeadsData tl left join tempLeadStatsSellers t on (t.date = tl.date AND t.sellerId = tl.sellerId AND t.type = tl.type AND t.subId = tl.subId) where isnull(t.date)'
Replicate_Ignore_Server_Ids:
Master_Server_Id: 4
Master_SSL_Crl: /etc/mysql/sslcerts/ca-cert.pem
Master_SSL_Crlpath:
Using_Gtid: No
Gtid_IO_Pos:
Replicate_Do_Domain_Ids:
Replicate_Ignore_Domain_Ids:
Parallel_Mode: none
1 row in set (0.00 sec)
It's easy for us to do:
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1; stop slave; start slave;
show slave status\G;
But the problem is these reports get rebuilt every 5 minutes. We thought it was the parallel mode and have set these options in Maria db cnf:
slave-parallel-threads=0
slave_parallel_threads=0
slave_parallel_mode=none
slave-parallel-mode=none
The SQL that it's failing on is a table that has the first column as the primary key that is on auto_increment and the query inserts "0" as the first column, so it seems like MariaDB is still doing some kind of parallel insertion here?
here's a copy of the server.cnf from the slave that's running MariaDB:
[mysqld]
slow_query_log = 1
slow_query_log_file = /var/log/mysql-slow.log
long_query_time = 2 # log_queries_not_using_indexes
slave which db to fetch from master
replicate-do-db=DB_NAME
end slave config
server_id=55
ssl
enable mysql replication over ssl
ssl
ssl-ca=/etc/mysql/sslcerts/ca-cert.pem
ssl-cert=/etc/mysql/sslcerts/new-live-server-cert.pem
ssl-key=/etc/mysql/sslcerts/new-live-server-key.pem
#master
#which db to dump to binlog
log-bin=mysql-bin
#lets log the updates we get from the master to binlog too
#so that the dev box can read these too
log-slave-updates
binlog-do-db=DB_NAME_HERE
#end master config
Anyone have any insight into why we keep getting these duplicate inserts errors?
Spent two days so far looking though stack overflow answers and google and I just cannot get it working..
Im trying to setup a master / slave replication in mysql/mariadb.. yet when I start the replication. Its just errors on the slave saying tables dont exist.
Do I need to create the database and tables first?
If so, what happens if a new table is made on the master? will this break
replication?
This is the current state of the slave:
MariaDB [(none)]> show slave status \G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: somedomain.com
Master_User: someuser
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000002
Read_Master_Log_Pos: 63687969
Relay_Log_File: mariadb-relay-bin.000002
Relay_Log_Pos: 382
Relay_Master_Log_File: mysql-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: No
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 1146
Last_Error: Error 'Table 'xxxxx.xxxx' doesn't exist' on query. Default database: 'xxxxxxxxxxxx'. Query: 'UPDATE xxxx SET lastused = NOW(), lingertime = 7 WHERE siteid = 'xxxxxxxxxxx''
Skip_Counter: 0
Exec_Master_Log_Pos: 98
Relay_Log_Space: 63763807
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: NULL
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 1146
Last_SQL_Error: Error 'Table 'xxxxxxxx.xxxxx' doesn't exist' on query. Default database: 'xxxxxxx'. Query: 'UPDATE xxxxx SET lastused = NOW(), lingertime = 7 WHERE siteid = 'xxxxxxxxxx''
Replicate_Ignore_Server_Ids:
Master_Server_Id: 1
Do I need to create the database and tables first?
Yes, you have to do that first. You must have the same schemas and tables as you have on master.
If so, what happens if a new table is made on the master? will this break replication?
Every command you execute on master will be replicated on slave as well. This includes CREATE statements for tables and schemas. So, you won't have any problem.
I am new to MySQL and tried to create replication
Here are my.cnf on Master
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
log-bin = /var/lib/mysql/logs/mysql-bin.log
binlog-do-db=fedorareptest1
server-id=1
----
My.cnf on Slave
[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
server-id=2
master-host=130.14.165.25
master-connect-retry=60
master-user=reptest
master-password=nopass!123
replicate-do-db=fedorareptest1
relay-log = /var/lib/mysql/logs/slave-relay.log
relay-log-index = /var/lib/mysql/logs/slave-relay-log.index
--------
Note: The database mentioned in "replicate-do-db=fedorareptest1" is the database on the master. This database name on the slave is fedoratest2. Should I use this above or it should be the one on master
When I do Slave status:
mysql> show slave status\G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 130.14.165.25
Master_User: reptest
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000001
Read_Master_Log_Pos: 107
Relay_Log_File: slave-relay.000005
Relay_Log_Pos: 252
Relay_Master_Log_File: mysql-bin.000001
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB: fedorareptest1
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 107
Relay_Log_Space: 549
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
1 row in set (0.00 sec)
--------------------------------------
When I do update on the table the row does not show up on the slave!
PLEASE HELP AND I AM NEW TO MYSQL.
You should avoid the *do-db options, as they can cause problems
Check that the queries are coming through to your slave:
mysqlbinlog --short-form /path/to/relaylogfile | less
.. or ..
tcpdump -vv -A -i lo0 tcp port 3306 | tee dump.log
UPDATE having reread your post I noticed the issue: you have change the DB name and not used replicate-rewrite-db
I have a slave that was started after the master had been running. I started it at a position in the master bin log that was when I started to import databases on the master. This slave has been running, but now is stuck and not progressing.
It has been stuck at relay master log 000055 and position for hours.
Results of show slave status:
mysql> show slave status \G
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: db10.domain.com
Master_User: replicator
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: DB10-bin.000102
Read_Master_Log_Pos: 917727958
Relay_Log_File: dbbk9-relay-bin.000152
Relay_Log_Pos: 863694346
Relay_Master_Log_File: DB10-bin.000055
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB:
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table: %.rmallaccountinfo,%.rmaccountbalance,%.rmoldestcharge,%.rmautotemp,%.rmitemtemp,%.rmcust
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 863694199
Relay_Log_Space: 56169138270
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 842902
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 100
1 row in set (0.00 sec)
I look up what is at that position in the master binlog and its just a drop temp table:
# at 863694053
#120827 15:32:50 server id 100 end_log_pos 863694199 Query thread_id=6068 exec_time=0 error_code=0
SET TIMESTAMP=1346095970/*!*/;
DROP TEMPORARY TABLE IF EXISTS `rmtemptableinclude` /* generated by server */
/*!*/;
# at 863694199
#120827 15:32:51 server id 100 end_log_pos 863694282 Query thread_id=7152 exec_time=1 error_code=0
SET TIMESTAMP=1346095971/*!*/;
BEGIN
/*!*/;
Whenever I see replication hungup on something, there's an error displayed in the STATUS output...
You can try skipping one statement in the replication:
STOP SLAVE;
SET GLOBAL SQL_SLAVE_SKIP_COUNTER = 1;
START SLAVE;
I just experienced the same thing, slave not reporting any error, just stuck on an old "Read_Master_Log_Pos"
I executed a "STOP SLAVE;" followed by a "START SLAVE;" and now it's catching up with the master! This is a little disconcerting b/c the slave wasn't even reporting that it was stuck.
Here are the steps i did.
I was Replicating the mysql 5.0.95 as master and 5.5 as a slave. I'm using linux OS. My DB engine is InnoDBenter code here
I have set the master with
/etc/my.cnf
**#mysql Server setup**
server-id=1
bind-address = 192.168.1.41
innodb_flush_log_at_trx_commit=1
sync_binlog=1
log-bin = /var/log/mysql/mysql-bin.log
binlog-do-db=my_db
I have set the slave with
/etc/my.cnf
**#mysql replication client setup**
server-id=5
master-host=192.168.1.41
master-user=web_master
master-password=webmaster
master-connect-retry=60
replicate-do-db=dev_my_db
log-slave-updates
I have granted permission on master as
GRANT SUPER,REPLICATION CLIENT,REPLICATION SLAVE,RELOAD ON *.* TO 'web_master'#'192.168.1.41' identified by 'webmaster';
I have set change master and bin position on the slave
CHANGE MASTER TO MASTER_HOST='192.168.1.41', MASTER_USER='web_master', MASTER_PASSWORD='webmaster', MASTER_LOG_FILE='mysql-bin.00002', MASTER_LOG_POS=107;
I did 2 insert and a update and log my queries
and checked My Master status as
mysql> SHOW master STATUS\G
File: mysql-bin.000004
Position: 13790
Binlog_Do_DB: my_db
Binlog_Ignore_DB:
1 row in set (0.00 sec)
I chcekd my slave and slave status as
mysql> SHOW SLAVE STATUS \G;
*************************** 1. row ***************************
Slave_IO_State: Waiting for master to send event
Master_Host: 192.168.1.41
Master_User: web_master
Master_Port: 3306
Connect_Retry: 60
Master_Log_File: mysql-bin.000004
Read_Master_Log_Pos: 13790
Relay_Log_File: mysqld-relay-bin.000007
Relay_Log_Pos: 244
Relay_Master_Log_File: mysql-bin.000004
Slave_IO_Running: Yes
Slave_SQL_Running: Yes
Replicate_Do_DB: dev_CHGV2_dbo
Replicate_Ignore_DB:
Replicate_Do_Table:
Replicate_Ignore_Table:
Replicate_Wild_Do_Table:
Replicate_Wild_Ignore_Table:
Last_Errno: 0
Last_Error:
Skip_Counter: 0
Exec_Master_Log_Pos: 13790
Relay_Log_Space: 401
Until_Condition: None
Until_Log_File:
Until_Log_Pos: 0
Master_SSL_Allowed: No
Master_SSL_CA_File:
Master_SSL_CA_Path:
Master_SSL_Cert:
Master_SSL_Cipher:
Master_SSL_Key:
Seconds_Behind_Master: 0
Master_SSL_Verify_Server_Cert: No
Last_IO_Errno: 0
Last_IO_Error:
Last_SQL_Errno: 0
Last_SQL_Error:
Replicate_Ignore_Server_Ids:
Master_Server_Id: 4
1 row in set (0.00 sec)
ERROR:
No query specified
I could see the bin log are updated on slave, i couldn't figure out what was causing the issue. As my updates on the master is not replicated on slave.
can any one suggest did i have missed out something?