I have 2 servers set up using MySQL. It's using a standard replication setup, with one slave, no circular replication.
Is there a way to programmatically tell how far behind the slave is in reading the data from the binary log?
If I run the statement:
SHOW MASTER STATUS;
On the master, and run
SHOW SLAVE STATUS;
on the slave, I can compare the Position column from master status, and the Read_Master_Log_Pos column from slave status to determine how far behind the slave is.
However, this only works if the slave is reading from the same file the master is writing to. So if the slave is still reading a previous log file, because it is running behind, I can't figure out how to determine how much data is left until it catches up to the current position that the master is at. A solution using only SQL would be optimal, but I'm open to other solutions. Hopefully not one that requires reading the directory containing the log files.
I like to use the 'Seconds_behind_master' field from SHOW SLAVE STATUS in order to determine if the MASTER-SLAVE servers are caught up. As a secondary guard I also to a COUNT(*) query on a specific table (ie one that gets updated frequently) on both servers and then compare the record counts.
Related
I have one master and two slaves.
Is it possible to restrict a particular read-only user to query only against the second slave (disallowing him from running any queries on the master and the first slave)?
I see that one can do the following to make un-replicated changes to the master, but what I think I need is to make changes to one slave and not the other.
SET sql_log_bin = {OFF|ON}
And the GRANT syntax allows one to limit what host users come from, but -- as far as I understand -- not which DB server(s) the person can use.
I didn't find much in a web search -- perhaps that's a hint that there's a better way to solve this problem. Basically I'm asking if this can be enforced by the database since the restriction I want applies to just this one user.
For context: a slave is basically just a server that copies every action that happened on the master. Depending on your configuration, the slaves will either just run the same queries that have been executed on the master, or apply a list of changes to individual rows to the slave.
To add a user just for a specific slave, you can do this directly on the slave. Anything you do here will only affect this slave. If your user currently exists on the master (and slaves), you would first have to drop him/remove his permissions, wait until this change has been replicated to all the slaves (which might also depend on your configuration), then add/modify this user directly on the slave.
You may need to temporarily disable a read_only or super_read_only setting (on the slave), which exists to prevent accidently executing something on the slave - but that is what you want to do.
Since your slave now deviates (slightly) from the master, if you would now run a query that alters that user on your master (e.g. drop it again), it might have a different effect on the master and the slave. This will depend on your configuration, but keep it in mind.
Is there a way to see when the last byte of data was copied over from the master to the slave? Currently to check how 'current' the data is I'm doing some pretty crude such as:
select max(last_updated) from one-my-my-tables
But it doesn't work too well. Is there a more formal way to do this?
From a privileged MySQL account (e.g. root) use:
mysql> show slave status;
The field:
Seconds_Behind_Master: 0
tells you how far out of date the slave is.
Unfortunately, there's no way to get the information from SHOW SLAVE STATUS without using that statement. I searched for this recently and learned that there are replication info tables in the PERFORMANCE_SCHEMA, however none of them contain the Seconds_Behind_Master.
Strictly speaking, the Seconds_Behind_Master doesn't tell you what you asked, anyway. You asked "when the last byte of data was copied over from the master to the replica?" Seconds_Behind_Master tells you the difference between the system time on the replica vs. the timestamp of the last executed event from the relay log.
Or if the replica has executed all downloaded events, it reports 0.
But suppose the replica has lost contact with its master, and there are more logs sitting around on the master waiting to be downloaded? The replica doesn't report this, because it has no idea there are more logs.
A more accurate way of measuring replication lag is to use the pt-heartbeat script, which is included in the free Percona Toolkit.
You execute the script on the master, and it inserts a timestamp to a table once per second, like a heartbeat.
Then on the replica you can query the timestamp and compare it to the system time.
If the replica is caught up, the difference will be zero.
If the replica has downloaded all the logs, but executing events is lagging, the timestamp difference will show this.
If the replica has lost contact with its master and hasn't downloaded all the logs, but we know the heartbeat timestamp should update once per second, then you can still get an accurate measure of the replication lag.
We have a one master and two VIP slave database servers. We changed data type of column from VARCHAR(255) to TEXT on master.
The application is currently configured to use master only for writing operations and configured slaves for reading operation.
After changing the data type on master server using ALTER TABLE command the slave server becomes unresponsive.
We are using Mariadb 10.0
[PROCESSES INFORMATION]
Id User Host Db Command Time(sec) State Info
-----------------------------------------------------------------------
203739 repl slave1 Binlog Dump 75,143,121 Master has sent all binlog to slave; waiting for binlog to be updated
203740 repl slave2 Binlog Dump 75,143,121 Master has sent all binlog to slave; waiting for binlog to be updated
The slave instance becomes very slow due to slow queries.
number of sessions: 1590
thread_pool_max_thread=500
Current value =648
After performing ALTER TABLE on Master server, it was replicating to slave server and in the same time number of sessions were get increased rapidly on slave server.
I think slaves becomes unresponsive because of slow queries.
But I don't know why this queries became so slow and slaves got unresponsive.
The DBA's saying that after executing ANALYZE TABLE command, the issue has been solved.
But I don't understand why this happened because ANALYZE TABLE only update the statistic information.
It would be helpful if anyone comment on this why it happened?
How to avoid such issues in future.
There is one minor case where TEXT is slower than VARCHAR. When a SELECT needs to build a temporary table (often for sorting due to GROUP BY or ORDER BY), it first tries to build a MEMORY table. But, TEXT and BLOB prevent it from using such, so it uses MyISAM instead. This is slower (but gets the job done).
I say this is a "minor case" because users rarely identify it with phrases like "very slow" and "becomes unresponsive". I would guess that a SELECT might run twice as slow.
Also, the ANALYZE TABLE discussion does not hold water. Again it may be coincidence, not causation.
So, the change to TEXT may be a 'red herring'. Instead, let's discover what is being slow by using the slowlog. See this for what I like to work from.
mysql ignore-db works according to server my.cnf AFAIK,
i.e.
binlog-ignore-db = mysql
replicate-ignore-db = mysql
I am not sure, if this works from client side too, can anyone explain the mechanism, how can i be able to send from master but not accept in client side.
Why i want to do this? I have multiple slave "2 slave" must replicate MySQL table where as in other 2 should not be overwriten. Where as every other table will be replicated.
Reading this: http://dev.mysql.com/doc/refman/5.6/en/replication-rules-db-options.html didnt make me clear enough.
binlog-ignore-db is a master-side setting, it tells the Master not to log changes taking place on the listed DB.
replicate-ignore-db is a slave-side setting, it tells the Slave to ignore incoming log information related to the listed DB
The typical use case is when you want to replicate different databases from one single Master to different Slaves. The Master must log all changes occurring in all databases (minus those possibly excluded by binlog-ignore-db, i.e. database that will not be replicated anywhere).
Each Slave will receive the full binary log, but will only replicate changes related to the selected databases (i.e. databases not excluded by replicate-ignore-db -- this list would be different on each Slave).
(mysql database being a system database, it should be ignored from both ends, unless you really, really really know what you are doing).
I have a number mysql servers running version 5.1.63 and whilst running some queries against the slave earlier this week, I noticed some data on the slave that should have been removed using an update statement on the master.
My initial thoughts were:
someone on the team was updating the slave, which I have since disproved
that the column being updated had changed
So, I investigated by running a mysql show status "table" query. This was run against a test database on each of the servers to see what the data length was, in a lot of cases it was showing me the data length differed between servers, but on an eyeball look at the data I could see the data was the same, so I couldn't use this method to see if there were any differences as it appears to be prone to error.
Next I ran a simple (across all dbs) row count for each table to confirm the row count was the same - it was.
I then started looking in the bin logs for replication. I could see the update statements that should have run clearly visible in the logs, but the update never ran.
What I need to know is:
is replication broken? I'm assuming it is
if I create new slave servers, will I encounter the same issue?
how do I find out the extent of the issue on my servers?
Any help is appreciated.
If you are using statement based replication then it is easily possible to end up with different results on master and slave due to badly constructed INSERT statements.
INSERT SELECT without ORDER BY, or where the ORDER BY can leave non deterministic results will cause the slaves to diverge from master.
From the MySQL site http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
The order in which rows are returned by a SELECT statement with no
ORDER BY clause is not determined. This means that, when using
replication, there is no guarantee that such a SELECT returns rows in
the same order on the master and the slave; this can lead to
inconsistencies between them. To prevent this from occurring, you
should always write INSERT ... SELECT statements that are to be
replicated as INSERT ... SELECT ... ORDER BY column. The choice of
column does not matter as long as the same order for returning the
rows is enforced on both the master and the slave. See also Section
16.4.1.15, “Replication and LIMIT”.
If this has happened then your replicas have diverged and the only safe way to bring them back in line is to rebuild them from a recent backup of the master DB. The worst part of this is the error may never cause the replication to fail, yet the results are inconsistent. Normally replication fails when an UPDATE or DELETE statement affects a different number of rows than on master, this is confusing as it was not the UPDATE that actually caused the error and the only way I know to fix the issue is to inspect every INSERT query in the code base!
Status details are from information_schema which collates data from databases statistics for Mysql instance and it never remained the same at every execution. It can be considered as just a rough estimation of data sizes in bytes but never an exact value as for index and data length. It can be used for estimations but not for cross check. For replication you may check the slave io and sql against the master is running or not. And relay-info you might see the corresponding log details from master and that of slave.
Of,course (1) way of doing is count(*) of tables EOD ensures the data in tables on master and slave are consistent or not. But to be accurate either (2) take random value fields and cross check with master and slave. Also if you aren't satisfied with it, (3) you may take them into outfile and take diff or checksum. I prefer (1) and (2). If (1) is not possible (2) still convinces me. ;)
There is a tool to verify replication named pt-table-checksum