On mysql, I have two data bases "parque_test" and "tabelas_temporais", and binary logs are activated.
Every action that modifies an InnoDB table belonging to "parque_test" is recorded on the binary log. However, "parque_test" has stored procedures that use temporary tables to retrieve a result (they are not used to perform update, delete or insert).
To avoid recording the activity of the temporary tables on the bin log, I have set the
"/etc/mysql/my.cnf" file so that mysql register all the activities on "parque_test" with the exception of "tabelas_temporais".
cat /etc/mysql/my.cnf"
...
#log_bin = /var/log/mysql/mysql-bin.log
log_bin=/mysql-log/bin-log
binlog_do_db=parque_test
binlog_do_db=parque_prod
expire_logs_days = 10
max_binlog_size = 100M
#binlog_do_db = include_database_name
#binlog_ignore_db = include_database_name
binlog_ignore_db=tabelas_temporais
...
All the temporary tables are created on the "tabelas_temporais" schema; however, the binary log still records the activities on "tabelas_temporais" when for example a stored procedure from "parque_test" is executed a containing a command such as
DROP TEMPORARY TABLE IF EXISTS tabelas_temporais.temp_mod_user;
Any help would be much appreciated!
mysql Ver 14.14 Distrib 5.5.40, for debian-linux-gnu (x86_64) using readline 6.2
Database filtering in the MySQL binary log can be somewhat unexpected if you don't know exactly how it works. from the manual
When using statement-based logging, the following example does not work as you might expect. Suppose that the server is started with --binlog-ignore-db=sales and you issue the following statements:
USE prices;UPDATE sales.january SET amount=amount+1000;
The UPDATE statement is logged in such a case because --binlog-ignore-db applies only to the default database (determined by the USE statement). Because the sales database was specified explicitly in the statement, the statement has not been filtered. However, when using row-based logging, the UPDATE statement's effects are not written to the binary log, which means that no changes to the sales.january table are logged; in this instance, --binlog-ignore-db=sales causes all changes made to tables in the master's copy of the sales database to be ignored for purposes of binary logging.
In short: it seems you might want to look into ROW based logging instead of STATEMENT or MIXED. However:
You should keep in mind that the format used to log a given statement may not necessarily be the same as that indicated by the value of binlog_format. For example, DDL statements such as CREATE TABLE and ALTER TABLE are always logged as statements, without regard to the logging format in effect, so the following statement-based rules for --binlog-ignore-db always apply in determining whether or not the statement is logged.
DROP is also a DDL which gets logged. So, does that mean there's no way? On the contrary:
.... temporary tables are logged only when using statement-based replication, whereas with row-based replication they are not logged. With mixed replication, temporary tables are usually logged; exceptions happen with user-defined functions (UDFs) and with the UUID() function....
So, in short, for 'normal' tables this becomes next to impossible while working in a schema that is logged, however, TEMPORARY tables are discarded in ROW based replication by default. This means: switch to ROW based replication, and you don't need to use a different schema for true temporary tables.
However, if you need to switch from STATEMENT / MIXED to ROW based replication, do check performance of this, and if you often do a bulk update (a lot of rows affected), your binlogs will quite a bit larger, as it will log every row changed rather then the single 'simple' UPDATE statement which caused it.
Related
I wanted to create a temporary table in MySQL database which is a member of Master-Master replication. I run the query and created the temporary table, however, the table was created in one of the servers and has not been created in the other member of replication.
My question is, should not all of the queries be run in both of the replication members?
No, not at all.
Quoting the manual:
A TEMPORARY table is visible only within the current session, and is dropped automatically when the session is closed.
This means, even when you open another connection on the same server, you won't see the table.
Therefore it wouldn't make sense to create the temporary table on multiple servers, at least when you don't use it in a DML statement.
If you use this table to do an UPDATE, DELETE or INSERT statement, it gets replicated, but it depends on the way you replicate.
If your binlog format is STATEMENT, your temporary table is created on disk on the replicating server in the directory you specified in your tmp_data variable.
If it's MIXED or ROW, it gets transmitted in ROW format, if I remember correctly.
The for row based replication MySql documentation states that:
"For statements such as CREATE TABLE ... SELECT, a CREATE statement is generated from the table definition and replicated using statement-based format, while the row insertions are replicated using row-based format."
how does the row based replication handle alter and delete statements? Couldn't find any documentation on that, do i need to re-run those commands onto the replicas?
From the documentation at http://dev.mysql.com/doc/refman/5.7/en/binary-log-setting.html:
With the binary log format set to ROW, many changes are written to the binary log using the row-based format. Some changes, however,
still use the statement-based format. Examples include all DDL (data
definition language) statements such as CREATE TABLE, ALTER TABLE, or
DROP TABLE.
DDL statements are handled with statement based replication, and DML, including deletes are handling with row based replication.
Under normal operation, you should not have to re-run any statements.
Can anyone tell me the exact difference in replicate_wild_do_table and replicate_do_table while creating Slave.
Thanks
replicate-wild-do-table allows you to restrict replication statements to databases and tables using % and _ wildcard characters, that is perform pattern matching.
Where replicate-do-table uses concrete databases and tables names.
Here's a manual reference: link
UPDATE:
Seems like you are having trouble with USE statement or fully qualified table names (those with database name specified) in your setup. First of all, there are 2 types of binary logging:
Statement-based
Row-based
In short, statement-based replication logs every statement that could modify data and row-based logs modified rows.
Here's a manual reference: link
Here's a comparison from the manual: link
Those two types imply different behaviour for replication options. The key one here is that for statement-based replication MySQL replicates statements that are performed on default (selected via USE) database only.
So the question is: do you need cross-database updates? If so, you can't use replicate-do-db, because it restricts replication on database level. In terms of cross-database updates, there is no difference between replicate-wild-do-table and replicate-do-table.
I have a Mysql master-slave(s) replication with MyISAM tables. All updates are done on the master and selects are done on either the master or slaves.
It appears that we might need to manually lock a few tables when we do certain updates. While this write lock is on the tables, no selects can happen on the locked table. But what about on the slaves? Does the lock propagate out?
Say I have table_A and table_B. I initiate a lock on table_A and table_B on the master and start performing the update. At this time no other connection can read table_A and table_B off the master? But what if at this time another connection tries to read the tables off of a slave, can they do so?
Everything that MySQL replicates can be found in the binary logs.
You can run the following command to see the details.
show global variables like 'log_bin%';
log_bin_basename will tell you the path to your binary logs with base file name.
and run
show binary logs
to find the binary files that are currently present on your server.
You can check the actual commands that are written to the file by using mysqlbinlog command together with the file name or by running show binlog events ... from the MySQL CLI.
Also, check what binlog_format are you using.
Basically - the lock of the tables is not directly propagated to slaves, but at the time, whey will execute the performed updates they will perform a lock of the updated table if needed.
As far as I know write locks do not propagate into the binlog, You can verify that by doing quick test and looking at the binlog. If you want to avoid issues on the master aswell and for some reason can not migrate to InnoDB consider integrating something like GET_LOCK() into your application instead of completely locking a table. MyISAM is quite iffy when it comes to concurrency.
I want to get mysqldump after certain checkpoint e.g. if i take the mysqldump now then next time when i will take the dump it should give me only the commands which executed between this time interval. is there anyway to get this using mysqldump.
One more thing how to show the commands delete, update in the mysqldump files.
Thanks
I dont think this is possible from a MySQLdump, however that feature exists as part of MySQL core - its called Binlogging or binary logging.
The binary log contains “events” that describe database changes such as table creation operations or changes to table data. It also contains events for statements that potentially could have made changes (for example, a DELETE which matched no rows). The binary log also contains information about how long each statement took that updated data
Check this out http://dev.mysql.com/doc/refman/5.0/en/binary-log.html
Word of warning, binlogs can slow down the performance of your server.