I have one master and 2 slaves and have replication set at statement level.
I want to purge records older than specific date on master and one of the slave. I created a procedure to do that.
Is there any way I can skip procedure call on 2nd slave and execute that on 1st slave?
Please note that I want to execute all other statements and I want to schedule purge procedure call as MySQL event.
Thanks and best regards,
Santosh!
Declare the procedure to some sort of No-Op on the 2nd slave. The statement will still get sent and processed but won't do anything.
Related
I am a beginner in MySQL, looking for expert advise ...
I need to create 4 logging stored procedures:
OPEN_LOG(code, comment)
CLOSE_LOG(result)
BEGIN_ACTION(action, comment)
END_ACTION(result)
to use them in my stored procedures like this:
START TRANSACTION;
CALL open_log('ETL_ABC', 'Running ETL process ABC');
CALL begin_action('Refreshing Dimensions', 'CUSTOMERS, PRODUCTS, STORES');
MERGE INTO customers ...;
MERGE INTO products ...;
MERGE INTO stores' ...;
CALL end_action('Done');
CALL begin_action('Loading new SALES data', 'For 01-APR-2018');
INSERT INTO sales ...;
CALL end_action('... rows added');
CALL close_log('Successfully completed');
COMMIT;
The OPEN_LOG procedure should insert 1 row into the PROCESS_LOGS table.
The CLOSE_LOG procedure should update that row with RESULT and END_DATE_TIME values. The BEGIN_ACTION procedure should insert 1 row into the LOG_DATA table. The END_ACTION procedure should insert yet another row into the same LOG_DATA table. All these inserts and updates should be independent from the main transaction.
There may be many ETL processes running simultaneously in the system and using this logging mechanism. At the same time, support staff can manually query these logging tables to see what is running and at what stage the processes are.
In Oracle, I was able to achieve this by declaring my logging procedures with PRAGMA AUTONOMOUS_TRANSACTION.
In MySQL/MariaDB, I believe my only option is using MyISAM storage engine for the logging tables. My concern is concurrency. I estimate that I may need to allow up to 100 log writes per second and up to 10 queries per minute. The queries should be quick because both log tables will properly indexed.
Please, give me your opinion/alternatives.
Is there a way to test the mysql queries, without modifying my database?
For example, can I run a delete from my_table where id=101 without actual delete anything? What about insert and update?
TY
You could start a transaction before running your queries and then rollback after running them. Note that to do this you'll require InnoDB or XtraDb tables (won't work on MyISAM).
To start a transaction send to MySQL the following statement:
START TRANSACTION;
And at the end of your queries run the following statement:
ROLLBACK;
Your database will never be modified from the point of view of other connections. Your current connection will see the changes until ROLLBACK, and after that the original state will be restored.
If you just want to see how many rows would be deleted why not substitute 'delete' with 'select count(*)'?
Copy your database to a test-database or auto-generate a import-script. Than you can test whatever you want on the test-database and restore it again using the script or import.
can a MySQL slave instance have different row values for the same ID when binlog_format is set to STATEMENT and we insert something like:
insert into foo values(CURRENT_TIMESTAMP)
As I understand it, the slave read the SQL statement and execute it thus, if the replication is lagging, could lead to differences for the same row. Right or wrong ?
How can I avoid this situation ?
Thank you.
Your approach is perfectly safe in statement level replication. The TIMESTAMP is written to the binary log, so the value for CURRENT_TIMESTAMP will be consistent across the master and the slave even if the slave is behind. You can also use the NOW() function safely for the same reason.
The function to avoid is SYSDATE(), which will not use the TIMESTAMP from the binary log, and therefore the slave's value will represent when the statement ran on the slave, rather than when the statement ran on the master.
I am running a couple of databases on MySQL 5.0.45 and am trying to get my legacy database to sync with a revised schema, so I can run both side by side. I am doing this by adding triggers to the new database but I am running into problems with replication. My set up is as follows.
Server "master"
Database "legacydb", replicates to server "slave".
Database "newdb", has triggers which update "legacydb" and no replication.
Server "slave"
Database "legacydb"
My updates to "newdb" run fine, and set off my triggers. They update "legacydb" on "master" server. However, the changes are not replicated down to the slaves. The MySQL docs say that for simplicity replication looks at the current database context (e.g. "SELECT DATABASE();" ) when deciding which queries to replicate rather than looking at the product of the query. My trigger is run from the context of database "newdb", so replication ignores the updates.
I have tried moving the update statement to a stored procedure in "legacydb". This works fine (i.e. data replicates to slave) when I connect to "master" and manually run "USE newdb; CALL legacydb.do_update('Foobar', 1, 2, 3, 4);". However, when this procedure is called from a trigger it does not replicate.
So far my thinking on how to fix this has been one of the following.
Force the trigger to set a new current database. This would be easiest, but I don't think this is possible. This is what I hoped to achieve with the stored procedure.
Replicate both databases, and have triggers in both master and slave. This would be possible, but a pain to set up.
Force the replication to pick up all changes to "legacydb", regardless of the current database context.
If replication runs at too high a level, it will never even see any updates run by my trigger, in which case no amount of hacking is going to achieve what I want.
Any help on how to achieve this would be greatly appreciated.
This may have something to do with it:
A stored function acquires table locks before executing, to avoid inconsistency in the binary log due to mismatch of the order in which statements execute and when they appear in the log. Statements that invoke a function are recorded rather than the statements executed within the function. Consequently, stored functions that update the same underlying tables do not execute in parallel.
In contrast, stored procedures do not acquire table-level locks. All statements executed within stored procedures are written to the binary log.
Additionally, there are a whole list of issues with Triggers:
http://dev.mysql.com/doc/refman/5.0/en/routine-restrictions.html
Ok,I'm running a setup with a single master and a number of slaves. All writes go through the master and are replicated down to the slaves which are used strictly for reads.
Now I have a stored procedure (not function) which is called by a trigger on an insert. According to the MySQL docs, for replication triggers log the call to the trigger while stored procedures actually log the result of the stored procedure.
So my question is, when my trigger gets fired, will it replicate both the trigger and the results of the procedure that the trigger calls (resulting in the procedure effectively being run twice)? Or will it simply replicate the trigger have the slaves re-run the stored procedure on their own?
Thanks
In MySQL 5.0 (and MySQL 5.1 with statement based binary logging), only the calling query is logged, so in your case, the INSERT would be logged.
On the slave, the INSERT will be executed and then the trigger will be re-run on the slave. So the trigger needs to exist on the slave, and assuming it does, then it will be executed in exactly the same way as the master.
In MySQL 5.1, there is row-based binary logging, which will log only the rows being changed, so the trigger would not be re-fired on the slave, but all rows that changed would still be propagated.
In addition to Harrison's excellent answer:
Assuming the databases are in sync (schema, data, same version) to start with, it should just work
If it doesn't, then it may be that you're using something non deterministic in your queries or trigger. Fix that.
Regardless of how you use replication, you need to have monitoring to check that the slaves are always in sync. Without any monitoring, they will become out of sync (subtly) and you won't notice. MySQL has no automatic built-in feature for checking this or fixing it.