We currently use a trigger on our MySQL database that sets a "last-modified" timestamp to CURRENT_TIMESTAMP. It is called on update.
We also need to use statement based reproduction for the cluster.
Is there a way to modify the trigger so that the propagated CURRENT_TIMESTAMP is identical on every cluster instance?
Currently the statement based reproduction calls the statement for every cluster instance, resulting in slightly different timestamps.
You must switch to mixed binlog format to save inside the binlog not only the statement but also the data for non deterministic writes.
You can do that without service disruption with:
SET GLOBAL binlog_format = 'MIXED';
On the master server generating the binlog.
Related
The for row based replication MySql documentation states that:
"For statements such as CREATE TABLE ... SELECT, a CREATE statement is generated from the table definition and replicated using statement-based format, while the row insertions are replicated using row-based format."
how does the row based replication handle alter and delete statements? Couldn't find any documentation on that, do i need to re-run those commands onto the replicas?
From the documentation at http://dev.mysql.com/doc/refman/5.7/en/binary-log-setting.html:
With the binary log format set to ROW, many changes are written to the binary log using the row-based format. Some changes, however,
still use the statement-based format. Examples include all DDL (data
definition language) statements such as CREATE TABLE, ALTER TABLE, or
DROP TABLE.
DDL statements are handled with statement based replication, and DML, including deletes are handling with row based replication.
Under normal operation, you should not have to re-run any statements.
On mysql, I have two data bases "parque_test" and "tabelas_temporais", and binary logs are activated.
Every action that modifies an InnoDB table belonging to "parque_test" is recorded on the binary log. However, "parque_test" has stored procedures that use temporary tables to retrieve a result (they are not used to perform update, delete or insert).
To avoid recording the activity of the temporary tables on the bin log, I have set the
"/etc/mysql/my.cnf" file so that mysql register all the activities on "parque_test" with the exception of "tabelas_temporais".
cat /etc/mysql/my.cnf"
...
#log_bin = /var/log/mysql/mysql-bin.log
log_bin=/mysql-log/bin-log
binlog_do_db=parque_test
binlog_do_db=parque_prod
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
binlog_ignore_db=tabelas_temporais
...
All the temporary tables are created on the "tabelas_temporais" schema; however, the binary log still records the activities on "tabelas_temporais" when for example a stored procedure from "parque_test" is executed a containing a command such as
DROP TEMPORARY TABLE IF EXISTS tabelas_temporais.temp_mod_user;
Any help would be much appreciated!
mysql Ver 14.14 Distrib 5.5.40, for debian-linux-gnu (x86_64) using readline 6.2
Database filtering in the MySQL binary log can be somewhat unexpected if you don't know exactly how it works. from the manual
When using statement-based logging, the following example does not work as you might expect. Suppose that the server is started with --binlog-ignore-db=sales and you issue the following statements:
USE prices;UPDATE sales.january SET amount=amount+1000;
The UPDATE statement is logged in such a case because --binlog-ignore-db applies only to the default database (determined by the USE statement). Because the sales database was specified explicitly in the statement, the statement has not been filtered. However, when using row-based logging, the UPDATE statement's effects are not written to the binary log, which means that no changes to the sales.january table are logged; in this instance, --binlog-ignore-db=sales causes all changes made to tables in the master's copy of the sales database to be ignored for purposes of binary logging.
In short: it seems you might want to look into ROW based logging instead of STATEMENT or MIXED. However:
You should keep in mind that the format used to log a given statement may not necessarily be the same as that indicated by the value of binlog_format. For example, DDL statements such as CREATE TABLE and ALTER TABLE are always logged as statements, without regard to the logging format in effect, so the following statement-based rules for --binlog-ignore-db always apply in determining whether or not the statement is logged.
DROP is also a DDL which gets logged. So, does that mean there's no way? On the contrary:
.... temporary tables are logged only when using statement-based replication, whereas with row-based replication they are not logged. With mixed replication, temporary tables are usually logged; exceptions happen with user-defined functions (UDFs) and with the UUID() function....
So, in short, for 'normal' tables this becomes next to impossible while working in a schema that is logged, however, TEMPORARY tables are discarded in ROW based replication by default. This means: switch to ROW based replication, and you don't need to use a different schema for true temporary tables.
However, if you need to switch from STATEMENT / MIXED to ROW based replication, do check performance of this, and if you often do a bulk update (a lot of rows affected), your binlogs will quite a bit larger, as it will log every row changed rather then the single 'simple' UPDATE statement which caused it.
can a MySQL slave instance have different row values for the same ID when binlog_format is set to STATEMENT and we insert something like:
insert into foo values(CURRENT_TIMESTAMP)
As I understand it, the slave read the SQL statement and execute it thus, if the replication is lagging, could lead to differences for the same row. Right or wrong ?
How can I avoid this situation ?
Thank you.
Your approach is perfectly safe in statement level replication. The TIMESTAMP is written to the binary log, so the value for CURRENT_TIMESTAMP will be consistent across the master and the slave even if the slave is behind. You can also use the NOW() function safely for the same reason.
The function to avoid is SYSDATE(), which will not use the TIMESTAMP from the binary log, and therefore the slave's value will represent when the statement ran on the slave, rather than when the statement ran on the master.
I am running a couple of databases on MySQL 5.0.45 and am trying to get my legacy database to sync with a revised schema, so I can run both side by side. I am doing this by adding triggers to the new database but I am running into problems with replication. My set up is as follows.
Server "master"
Database "legacydb", replicates to server "slave".
Database "newdb", has triggers which update "legacydb" and no replication.
Server "slave"
Database "legacydb"
My updates to "newdb" run fine, and set off my triggers. They update "legacydb" on "master" server. However, the changes are not replicated down to the slaves. The MySQL docs say that for simplicity replication looks at the current database context (e.g. "SELECT DATABASE();" ) when deciding which queries to replicate rather than looking at the product of the query. My trigger is run from the context of database "newdb", so replication ignores the updates.
I have tried moving the update statement to a stored procedure in "legacydb". This works fine (i.e. data replicates to slave) when I connect to "master" and manually run "USE newdb; CALL legacydb.do_update('Foobar', 1, 2, 3, 4);". However, when this procedure is called from a trigger it does not replicate.
So far my thinking on how to fix this has been one of the following.
Force the trigger to set a new current database. This would be easiest, but I don't think this is possible. This is what I hoped to achieve with the stored procedure.
Replicate both databases, and have triggers in both master and slave. This would be possible, but a pain to set up.
Force the replication to pick up all changes to "legacydb", regardless of the current database context.
If replication runs at too high a level, it will never even see any updates run by my trigger, in which case no amount of hacking is going to achieve what I want.
Any help on how to achieve this would be greatly appreciated.
This may have something to do with it:
A stored function acquires table locks before executing, to avoid inconsistency in the binary log due to mismatch of the order in which statements execute and when they appear in the log. Statements that invoke a function are recorded rather than the statements executed within the function. Consequently, stored functions that update the same underlying tables do not execute in parallel.
In contrast, stored procedures do not acquire table-level locks. All statements executed within stored procedures are written to the binary log.
Additionally, there are a whole list of issues with Triggers:
http://dev.mysql.com/doc/refman/5.0/en/routine-restrictions.html
Ok,I'm running a setup with a single master and a number of slaves. All writes go through the master and are replicated down to the slaves which are used strictly for reads.
Now I have a stored procedure (not function) which is called by a trigger on an insert. According to the MySQL docs, for replication triggers log the call to the trigger while stored procedures actually log the result of the stored procedure.
So my question is, when my trigger gets fired, will it replicate both the trigger and the results of the procedure that the trigger calls (resulting in the procedure effectively being run twice)? Or will it simply replicate the trigger have the slaves re-run the stored procedure on their own?
Thanks
In MySQL 5.0 (and MySQL 5.1 with statement based binary logging), only the calling query is logged, so in your case, the INSERT would be logged.
On the slave, the INSERT will be executed and then the trigger will be re-run on the slave. So the trigger needs to exist on the slave, and assuming it does, then it will be executed in exactly the same way as the master.
In MySQL 5.1, there is row-based binary logging, which will log only the rows being changed, so the trigger would not be re-fired on the slave, but all rows that changed would still be propagated.
In addition to Harrison's excellent answer:
Assuming the databases are in sync (schema, data, same version) to start with, it should just work
If it doesn't, then it may be that you're using something non deterministic in your queries or trigger. Fix that.
Regardless of how you use replication, you need to have monitoring to check that the slaves are always in sync. Without any monitoring, they will become out of sync (subtly) and you won't notice. MySQL has no automatic built-in feature for checking this or fixing it.