That's a strange issue.
There are source MySQL DB (MASTER) and its replic (SLAVE). It's a (as you understand its a MASTER-SLAVE) statement-bases replication, because I need triggers to run on the SLAVE side.
The original triggers were replaced by new ones.
Every table has 3 trigers: on INSERT, on UPDATE and on DELETE.
Trigers were generated by a single pattern and differ only by params.
Every trigger does a single INSERT query to a table (CHANGES) on the SLAVE.
This table is not replicated and exists only on the SLAVE.
There is an autoincremented column (ID - bigint) in this table.
None of the triggers set or modify values of the column ID. The DB sets a default values for it.
It's about 20 inserts executed on CHANGES per minute.
I see errors with duplicated values.
How it's possible?
Let's again:
An INSERT / UPDATE / DELETE query is executed on MASTER.
This change is replicated to SLAVE.
A trigger is called and inserts a row to the CHANGES.
Duplicated values error is generated.
And as I said before, none of the triggers set or modify value of the autoincremented field (ID). And only triggers work with table CHANGES.
I understand that two or more triggers can be called together, and try to do INSERT together, but I think DB should easy solve this. Or that's a very bad DB.
UPD:
CREATE TABLE `CHANGES` (
`id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT,
`field1` ENUM(...) NOT NULL,
`field2` BOOL NOT NULL DEFAULT FALSE,
`field3` VARCHAR(64) NOT NULL,
`field4` VARCHAR(255) NOT NULL,
`field5` TEXT,
PRIMARY KEY (`id`)
) ENGINE=INNODB DEFAULT CHARSET=utf8;
CREATE TRIGGER `tr_TABLE_insert` AFTER INSERT ON `TABLE`
FOR EACH ROW BEGIN
INSERT INTO `CHANGES` (`field1`, `field3`, `field4`, `field5`)
VALUES ("value1", "value3", "value4", "value5");
END
UPD 2:
I found a temporary and more dirty method - added BEFORE INSERT trigger for CHANGES to set ID manually.
It works. But I still can't figure out why the native AUTO_INCREMENT mechanism is generating duplicated ids.
Related
I've created a log system based on trigger.
Every time a row is inserted or updated the trigger store a new row in another table.
The trigger works fine but after some time I found this message in logs:
[Warning] Unsafe statement written to the binary log using statement format since BINLOG_FORMAT = STATEMENT. Statement is unsafe because it invokes a trigger or a stored function that inserts into an AUTO_INCREMENT column. Inserted values cannot be logged correctly. Statement: update `gl_item` set `is_shown` = '0', `updated_at` = '2016-03-21 16:56:28' where `list_id` = '1' and `is_shown` = '1'
I've already red some post related to this issue i like:
- MySQL Replication & Triggers
But I don't understand the nature of the problem.
What this warning mean?
I don't have to insert into auto increment column with triggers?
Which is the best way to create a log system in order to avoid this warning?
Update
Output of SHOW CREATE TABLE, this is the table where the trigger will enter THE rows.
gl_item_log | CREATE TABLE `gl_item_log` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Item log unique id',
`item_id` bigint(20) unsigned DEFAULT NULL ,
`updated_by` bigint(20) unsigned DEFAULT NULL ,
`switch_shown` tinyint(4) DEFAULT NULL ,
`switch_checked` tinyint(4) DEFAULT NULL ,
`logged_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`logged_at_microtime` decimal(6,6) unsigned NOT NULL ,
`logged_at_microtime_int` mediumint(8) unsigned NOT NULL DEFAULT '0' ,
PRIMARY KEY (`id`),
KEY `gl_item_log_updated_by_foreign` (`updated_by`),
KEY `gl_item_log_item_id_updated_by_switch_shown_switch_checked_index`
(`item_id`,`updated_by`,`switch_shown`,`switch_checked`),
CONSTRAINT `gl_item_log_item_id_foreign`
FOREIGN KEY (`item_id`) REFERENCES `gl_item` (`id`),
CONSTRAINT `gl_item_log_updated_by_foreign`
FOREIGN KEY (`updated_by`) REFERENCES `gl_general_user` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=32 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
Could be a good idea drop the id column with the auto increment field in order to have logs entries without a unique identifier?
Thanks
What this warning mean?
According to the MySQL documentation:
A statement invoking a trigger (or function) that causes an update to an AUTO_INCREMENT column is not replicated correctly using statement-based replication. MySQL 5.7 marks such statements as unsafe. (Bug #45677)
With statement-based replication, the exact SQL which is run on your master database is also run on your slave(s). When your trigger is fired, if it exists on every one of your databases, it is run on each database and inserts into your log. This can be a tricky situation for your databases to remain in sync.
I don't have to insert into auto increment column with triggers?
Correct. It's never necessary to insert your own values into an auto-increment column.
Which is the best way to create a log system in order to avoid this warning?
First, either keep the trigger on every database and turn off replication for your log table, or have the trigger only on your master database and let replication copy the log table inserts to the other databases.
To work around this specific warning, configure your log table to have no auto-increment column. Your trigger can then insert into it and it shouldn't cause any replication warnings.
Another option is to switch to row-based replication. Then the trigger will only be fired automatically on the master and the auto-increment values will always replicate without issue.
I have a task that must be executed by MySql (v5.1.72) stored procedure.
The task consists of several steps:
select rowset from one table by some condition
delete all rows, containing in rowset from step 1 (actually, delete rows by condition from step 1)
return from procedure rows, retrieved on step 1 (that were deleted on step 2)
And this procedure have some additional constraints:
it's expected to be invoked rather frequently, so there's a possible problem with creating temporary tables in request.
all data assigned to procedure call must be removed after procedure returned result and finished working. So, if result data is stored in table, this table must be automatically removed after procedure call.
Is it possible to solve this problem without additional tables?
And if not, how this can be done the best way?
I would create a table that has a extra column name Guid, of type CHAR(38).
Something like this.
CREATE TABLE `test_table` (
`TestId` int(11) NOT NULL,
`Value` varchar(45) DEFAULT NULL,
`Value1` varchar(45) DEFAULT NULL,
`Guid` char(38) NOT NULL,
PRIMARY KEY (`TestId`),
KEY `Guid` (`Guid`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The index KEY Guid (Guid) will help in the SP to faster process the data.
If you in your procedure define a golbal variable
DECLARE v_Guid CHAR(38);
Then you can set the value like this
SET v_Guid = SELECT UUID();
This would return a string like this 18d2e4ea-5d8c-11e3-b923-00ff90aef4a9.
Now you can use this value v_Guid for every action that you make. SELECT, INSERT or DELETE.
When you later want to delete the rows. You just execute
DELETE FROM test_table WHERE Guid=v_Guid;
In this manner, the SP can use the same table even if the procedure runs and overlaps eachother. You never DROP the table, just delete the rows with the related GUID.
Have you considered someting like this?
We are in the process of migrating between 2 systems and need to have 2 fields for one of our database tables that always stay in sync. Here is the table structure:
CREATE TABLE `example` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`object_id` int(11) NOT NULL DEFAULT '0',
`value` varchar(200) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `object_id` (`object_id`)
);
Every time one of the systems inserts a new row we need to have object_id set to id. We can't use 'before insert' since the id column is an auto_increment column so it's value is NULL before insert and due to the limitations of the MySQL 'after insert' on triggers I can't do the following:
CREATE TRIGGER insert_example
AFTER INSERT ON example
FOR EACH ROW
SET NEW.object_id = NEW.id;
I can't update the code for either system so I need a way to accomplish this on the database side. Both systems are going to be inserting new rows. How can I accomplish this?
Using a trigger which fires before the insert should do the job
CREATE TRIGGER insert_example
BEFORE INSERT ON example
FOR EACH ROW
SET NEW.object_id = NEW.id;
EDIT:
As the OP pointed out NEW.id won't work with auto-increment; one could use the following trigger (use at own risk):
CREATE TRIGGER insert_example
BEFORE INSERT ON example
FOR EACH ROW
SET NEW.object_id = (
SELECT AUTO_INCREMENT
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'example'
);
But I'd rather re-think this somewhat strange requirement - why do you need the pk value twice in the table?
Is there any reason you cant use a BEFORE INSERT trigger?
I've always seen AFTER INSERT triggers as a method to manipulate other tables rather than the table for which the trigger was executed on.
Rule of thumb, manipulate table the trigger is running on = BEFORE INSERT, manipulate other tables AFTER INSERT :)
I think your trigger will never create in the first place because you can't refer NEW.column_name in an AFTER INSERT trigger.
Try doing this in a BEFORE INSERT trigger (PLEASE IGNORE THIS FIX AS IT WILL NOT WORK):
CREATE TRIGGER `insert_example` BEFORE INSERT ON `t`
FOR EACH ROW
SET NEW.`object_id` = NEW.`id`;
Please change the table and column names as per your schema.
Hope this helps.
Say you are running two mysql servers: one a master, the other the slave. The master has triggers set that update columns with the COUNT of the number of rows in other tables. For instance, you have a news table and a comments table. News contains an INT column called "total_comments" which is incremented via trigger every time a new row is put into "comments." Does the slave need this trigger as well (to keep "news.total_comments" up to date) or will it get be told to update the appropriate "news.total_comments" directly?
From the docs http://dev.mysql.com/doc/refman/5.0/en/faqs-triggers.html:
22.5.4: How are actions carried out through triggers on a master
replicated to a slave? First, the
triggers that exist on a master must
be re-created on the slave server.
Once this is done, the replication
flow works as any other standard DML
statement that participates in
replication. For example, consider a
table EMP that has an AFTER insert
trigger, which exists on a master
MySQL server. The same EMP table and
AFTER insert trigger exist on the
slave server as well. The replication
flow would be: An INSERT statement is
made to EMP. The AFTER trigger on EMP
activates. The INSERT statement is
written to the binary log. The
replication slave picks up the INSERT
statement to EMP and executes it. The
AFTER trigger on EMP that exists on
the slave activates.
And
22.5.4 Actions carried out through triggers on a master are not
replicated to a slave server.
Thus, you DO need the triggers on the slave.
It depends on the replication you're using. If you use statement based replication, then you must use matching triggers in the master and the slave. If you use row-based replication, then you must not include the triggers on the slave.
You can have the action of requests made by triggers in the binary log with federated tables (MySQL5) by adding the same table with a local connection.
---------------------------------------------------------------------------------------
-- EXEMPLE :
-- We want install a replication of the table test_table that will be managed by Trg_Update triggers.
---------------------------------------------------------------------------------------
Create database TEST;
USE TEST;
CREATE TABLE test_trigger (
id INT(20) NOT NULL AUTO_INCREMENT,
name VARCHAR(32) NOT NULL DEFAULT '',
PRIMARY KEY (id),
INDEX name (name),
) ;
DELIMITER |
CREATE TRIGGER Trg_Update AFTER INSERT ON test_trigger
FOR EACH ROW BEGIN
INSERT INTO federated_table (name, other) values (NEW.name, ‘test trigger on federated table -> OK’)
END|
DELIMITER ;
CREATE TABLE test_table (
id INT(20) NOT NULL AUTO_INCREMENT,
name VARCHAR(32) NOT NULL DEFAULT '',
other VARCHAR(32) NOT NULL DEFAULT '',
PRIMARY KEY (id),
INDEX name (name),
INDEX other_key (other)
) ;
CREATE TABLE federated_table (
id INT(20) NOT NULL AUTO_INCREMENT,
name VARCHAR(32) NOT NULL DEFAULT '',
other VARCHAR(32) NOT NULL DEFAULT '',
PRIMARY KEY (id),
INDEX name (name),
INDEX other_key (other)
)
ENGINE=FEDERATED
CONNECTION='mysql://root#localhost/TEST/test_table';
---------------------------------------------------------------------------------------
I am a user of a some host company which serves my MySql database. Due to their replication problem, the autoincrement values increses by 10, which seems to be a common problem.
My question is how can I simulate (safely) autoincrement feature so that the column have an consecutive ID?
My idea was to implement some sequence mechanism to solve my problem, but I do not know if it is a best option. I had found such a code snipset over the web:
DELIMITER ;;
DROP TABLE IF EXISTS `sequence`;;
CREATE TABLE `sequence` (
`name` CHAR(16) NOT NULL,
`value` BIGINT UNSIGNED NOT NULL,
PRIMARY KEY (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;;
DROP FUNCTION IF EXISTS `nextval`;
CREATE FUNCTION `nextval`(thename CHAR(16) CHARSET latin1)
RETURNS BIGINT UNSIGNED
MODIFIES SQL DATA
SQL SECURITY DEFINER
BEGIN
INSERT INTO `sequence`
SET `name`=thename,
`value`=(#val:=##auto_increment_offset)+##auto_increment_increment
ON DUPLICATE KEY
UPDATE `value`=(#val:=`value`)+##auto_increment_increment;
RETURN #val;
END ;;
DELIMITER ;
which seems quite all correct. My second question is if this solution is concurrent-safe? Of course INSERT statement is, but what about ON DUPLICATE KEY update?
Thanks!
Why do you need to have it in the first place?
Even with auto_increment_increment == 1 you are not guaranteed, that the autoincrement field in the table will have consecutive values (what if the rows are deleted, hmm?).
With autoincrement you are simply guaranteed by the db engine, that the field will be unique, nothing else, really.
EDIT: I want to reiterate: In my opinion, it is not a good idea to assume things like concurrent values of an autoincrement column, because it is going to bite you later.
EDIT2: Anyway, this can be "solved" by an "on insert" trigger
create trigger "sequence_b_ins" before insert on `sequence`
for each row
begin
NEW.id = select max(id)+1 from `sequence`;
end
Or something along these lines (sorry, not tested)
Another option would be to use a stored proc to do the insert and have it either select max id from your table or keep another table with the current id being used and update as id's are used.