I am a user of a some host company which serves my MySql database. Due to their replication problem, the autoincrement values increses by 10, which seems to be a common problem.
My question is how can I simulate (safely) autoincrement feature so that the column have an consecutive ID?
My idea was to implement some sequence mechanism to solve my problem, but I do not know if it is a best option. I had found such a code snipset over the web:
DELIMITER ;;
DROP TABLE IF EXISTS `sequence`;;
CREATE TABLE `sequence` (
`name` CHAR(16) NOT NULL,
`value` BIGINT UNSIGNED NOT NULL,
PRIMARY KEY (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;;
DROP FUNCTION IF EXISTS `nextval`;
CREATE FUNCTION `nextval`(thename CHAR(16) CHARSET latin1)
RETURNS BIGINT UNSIGNED
MODIFIES SQL DATA
SQL SECURITY DEFINER
BEGIN
INSERT INTO `sequence`
SET `name`=thename,
`value`=(#val:=##auto_increment_offset)+##auto_increment_increment
ON DUPLICATE KEY
UPDATE `value`=(#val:=`value`)+##auto_increment_increment;
RETURN #val;
END ;;
DELIMITER ;
which seems quite all correct. My second question is if this solution is concurrent-safe? Of course INSERT statement is, but what about ON DUPLICATE KEY update?
Thanks!
Why do you need to have it in the first place?
Even with auto_increment_increment == 1 you are not guaranteed, that the autoincrement field in the table will have consecutive values (what if the rows are deleted, hmm?).
With autoincrement you are simply guaranteed by the db engine, that the field will be unique, nothing else, really.
EDIT: I want to reiterate: In my opinion, it is not a good idea to assume things like concurrent values of an autoincrement column, because it is going to bite you later.
EDIT2: Anyway, this can be "solved" by an "on insert" trigger
create trigger "sequence_b_ins" before insert on `sequence`
for each row
begin
NEW.id = select max(id)+1 from `sequence`;
end
Or something along these lines (sorry, not tested)
Another option would be to use a stored proc to do the insert and have it either select max id from your table or keep another table with the current id being used and update as id's are used.
Related
I recently encountered an error in my application with concurrent transactions. Previously, auto-incrementing for compound key was implemented using the application itself using PHP. However, as I mentioned, the id got duplicated, and all sorts of issues happened which I painstakingly fixed manually afterward.
Now I have read about related issues and found suggestions to use trigger.
So I am planning on implementing a trigger somewhat like this.
DELIMITER $$
CREATE TRIGGER auto_increment_my_table
BEFORE INSERT ON my_table FOR EACH ROW
BEGIN
SET NEW.id = SELECT MAX(id) + 1 FROM my_table WHERE type = NEW.type;
END $$
DELIMITER ;
But my doubt regarding concurrency still remains. Like what if this trigger was executed concurrently and both got the same MAX(id) when querying?
Is this the correct way to handle my issue or is there any better way?
An example - how to solve autoincrementing in compound index.
CREATE TABLE test ( id INT,
type VARCHAR(192),
value INT,
PRIMARY KEY (id, type) );
-- create additional service table which will help
CREATE TABLE test_sevice ( type VARCHAR(192),
id INT AUTO_INCREMENT,
PRIMARY KEY (type, id) ) ENGINE = MyISAM;
-- create trigger which wil generate id value for new row
CREATE TRIGGER tr_bi_test_autoincrement
BEFORE INSERT
ON test
FOR EACH ROW
BEGIN
INSERT INTO test_sevice (type) VALUES (NEW.type);
SET NEW.id = LAST_INSERT_ID();
END
db<>fiddle here
creating a service table just to auto increment a value seems less than ideal for me. – Mohamed Mufeed
This table is extremely tiny - you may delete all records except one per group with largest autoincremented value in this group anytime. – Akina
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=61f0dc36db25dd5f0cf4647d8970cdee
You may schedule excess rows removing (for example, daily) in service event procedure.
I have managed to solve this issue.
The answer was somewhat in the direction of Akina's Answer. But not quite exactly.
The way I solved it did indeed involved an additional table but not like the way He suggested.
I created an additional table to store meta data about transactions.
Eg: I had table_key like this
CREATE TABLE `journals` (
`id` bigint NOT NULL AUTO_INCREMENT,
`type` smallint NOT NULL DEFAULT '0',
`trans_no` bigint NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
KEY `transaction` (`type`,`trans_no`)
)
So I created a meta_journals table like this
CREATE TABLE `meta_journals` (
`type` smallint NOT NULL,
`next_trans_no` bigint NOT NULL,
PRIMARY KEY (`type`),
)
and seeded it with all the different types of journals and the next sequence number.
And whenever I insert a new transaction to the journals I made sure to increment the next_trans_no of the corresponding type in the meta_transactions table. This increment operation is issued inside the same database TRANSACTION, i.e. inside the BEGIN AND COMMIT
This allowed me to use the exclusive lock acquired by the UPDATE statement on the row of meta_journals table. So when two insert statement is issued for the journal concurrently, One had to wait until the lock acquired by the other transaction is released by COMMITing.
I have the same question as described here and also I think the answer https://stackoverflow.com/a/22343265/297487 is a good solution but I have another question about this answer.
Is the following trigger (copied from answer) thread-safe? I mean if two concurrent record inserted to table, does "priority" column (as describe in question) have consistent value (The same value as id)?
delimiter //
drop trigger if exists bi_table_name //
create trigger bi_table_name before insert on table_name
for each row begin
set #auto_id := ( SELECT AUTO_INCREMENT
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME='table_name'
AND TABLE_SCHEMA=DATABASE() );
set new.priority= #auto_id;
end;
//
delimiter ;
How do we interpret "for each row" clause?
Suppose that MySQL wants insert two concurrent rows. How does this trigger works?
One interpretation is as follows:
MySQL locks table and before insert one of the rows (one of concurrent record) MySQL trigger starts and gets current AUTO_INCREMENT value and sets it to "priority" column and then inserts a record. After that MySQL starts inserting another record and then the same situation applies for new record.
Another interpretation might be as follows:
When two concurrent records are inserted to MySQL, MySQL locks the table and then before inserting two concurrent records a trigger starts and "for each row" clause iterate between two record and set "priority" column value to the same value and then insert two concurrent record in database. In this situation the trigger does not work as expected.
Which one of the above interpretation is correct?
Update :
I have the following table :
CREATE TABLE `t_file` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`name` varchar(255) COLLATE utf8_persian_ci NOT NULL,
`p_name` bigint(20) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `p_name_2` (`p_name`)
) ENGINE=InnoDB AUTO_INCREMENT=206284 DEFAULT CHARSET=utf8 COLLATE=utf8_persian_ci;
I want to insert value of p_name the same as id when row inserted.
The trigger that sets value of p_name is as follows(copied from your code)
delimiter $$
drop trigger if exists file_p_name $$
create trigger file_p_name before insert on t_file
for each row begin
set #id := ( SELECT AUTO_INCREMENT
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_NAME='t_file'
AND TABLE_SCHEMA=DATABASE() );
set new.p_name= #id;
end;
$$
delimiter ;
In our application, i surround the code that inserts int_file table with try catch,in almost always everything is OK,but sometimes(in concurrent insert to t_file table), i see the following exception in our application's log:
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry \'387456\' for key \'p_name_2\''
It seems the trigger does not work as expected or maybe i am wrong!!!
The answer referred in your question was posted by me some time back.
While in "For Each Row", The NEW is the corresponding new row, being inserted, in context. And the set new.xxx is only applicable for that row and but not 'for all rows' in batch. So there won't be any collision and the question of failure should not arise.
I have also answered a similar question
How does “for each row” work in triggers in mysql?. Please go through the examples given in the answer.
Refer to Documentation:
MySQL: Triggers
Can you help me with this problem? I have two tables in a MySQL database:
ServiceProvider(SPID, Name, ... etc.)
hasTag(SPID, TagID)
Each service provider is supposed to have at least one tag, and a maximum of five tags. The max-constraint is not a problem, but the min-constraint refuses to work properly. I first tried to implement this via assertions, but then I found out, that MySQL does not support assertions. Thus, I wrote the following trigger:
delimiter |
CREATE TRIGGER MinTags BEFORE INSERT
ON ServiceProvider
FOR EACH ROW BEGIN
IF EXISTS (SELECT SPID FROM ServiceProvider
WHERE NOT EXISTS (SELECT DISTINCT SPID FROM hasTag))
THEN INSERT INTO stop_action VALUES(1, 'Assert Failure');
END IF;
END;
|
delimiter ;
The insert of 'Assert Failure' into the stop_action table is only to create a constraint violation, so that the DB would abort the action.
Now, normally, when I insert any value into the ServiceProvider table, without inserting anything in to the hasTag table, I should get an error, right? But, somehow it doesn't work ... I can insert anything I want into the ServiceProvider table, without receiving any kind of error. Do you know, what is wrong with my code?
How about denormalising a tad:
ALTER TABLE ServiceProvider
ADD COLUMN TagID1 BIGINT UNSIGNED NOT NULL,
ADD COLUMN TagID2 BIGINT UNSIGNED NULL,
ADD COLUMN TagID3 BIGINT UNSIGNED NULL,
ADD COLUMN TagID4 BIGINT UNSIGNED NULL,
ADD COLUMN TagID5 BIGINT UNSIGNED NULL;
Include foreign key constraints, if appropriate.
As written, this trigger does not even use the values from the row to be inserted.
The syntax to get the value of the SPID column is:
NEW.SPID
Also, consider using the SIGNAL statement to raise an error.
If you want to use ASSERT in SQL, this post may help:
SQL Scripts - Does the equivalent of a #define exist?
In my MySQL table I've created an ID column which I'm hoping to auto-increment in order for it to be the primary key.
I've created my table:
CREATE TABLE `test` (
`id` INT( 11 ) NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`name` VARCHAR( 50 ) NOT NULL ,
`date_modified` DATETIME NOT NULL ,
UNIQUE (
`name`
)
) TYPE = INNODB;
then Inserted my records:
INSERT INTO `test` ( `id` , `name` , `date_modified` )
VALUES (
NULL , 'TIM', '2011-11-16 12:36:30'
), (
NULL , 'FRED', '2011-11-16 12:36:30'
);
I'm expecting that my ID's for the above are 1 and 2 (respectively). And so far this is true.
However when I do something like this:
insert into test (name) values ('FRED')
on duplicate key update date_modified=now();
then insert a new record, I'm expecting it to be 3, however now I'm shown an ID of 4; skipping the place spot for 3.
Normally this wouldn't be an issue but I'm using millions of records which have thousands of updates every day.. and I don't really want to even have to think about running out of ID's simply because I'm skipping a ton of numbers..
Anyclue to why this is happening?
MySQL version: 5.1.44
Thank you
My guess is that the INSERT itself kicks off the code that generates the next ID number. When the duplicate key is detected, and ON DUPLICATE KEY UPDATE is executed, the ID number is abandoned. (No SQL dbms guarantees that automatic sequences will be without gaps, AFAIK.)
MySQL docs say
In general, you should try to avoid using an ON DUPLICATE KEY UPDATE
clause on tables with multiple unique indexes.
That page also says
If a table contains an AUTO_INCREMENT column and INSERT ... ON
DUPLICATE KEY UPDATE inserts or updates a row, the LAST_INSERT_ID()
function returns the AUTO_INCREMENT value.
which stops far short of describing the internal behavior I guessed at above.
Can't test here; will try later.
Is it possible to change your key to unsigned bigint - 18,446,744,073,709,551,615 is a lot of records - thus delaying the running out of ID's
Found this in mysql manual http://dev.mysql.com/doc/refman/5.1/en/example-auto-increment.html
Use a large enough integer data type for the AUTO_INCREMENT column to hold the
maximum sequence value you will need. When the column reaches the upper limit of
the data type, the next attempt to generate a sequence number fails. For example,
if you use TINYINT, the maximum permissible sequence number is 127.
For TINYINT UNSIGNED, the maximum is 255.
More reading here http://dev.mysql.com/doc/refman/5.6/en/information-functions.html#function_last-insert-id it could be inferred that the insert to a transactional table is a rollback so the manual says "LAST_INSERT_ID() is not restored to that before the transaction"
What about for a possible solution to use a table to generate the ID's and then insert into your main table as the PK using LAST_INSERT_ID();
From the manual:
Create a table to hold the sequence counter and initialize it:
mysql> CREATE TABLE sequence (id INT NOT NULL);
mysql> INSERT INTO sequence VALUES (0);
Use the table to generate sequence numbers like this:
mysql> UPDATE sequence SET id=LAST_INSERT_ID(id+1);
mysql> SELECT LAST_INSERT_ID();
The UPDATE statement increments the sequence counter and causes the next call to
LAST_INSERT_ID() to return the updated value. The SELECT statement retrieves that
value. The mysql_insert_id() C API function can also be used to get the value.
See Section 20.9.3.37, “mysql_insert_id()”.
It's really a bug how you can see here: http://bugs.mysql.com/bug.php?id=26316
But, apparently, they fixed it on 5.1.47 and it was declared as INNODB plugin problem.
A duplicate, but same problem, you can see here too: http://bugs.mysql.com/bug.php?id=53791 referenced to the first page mentioned here in this answer.
We are in the process of migrating between 2 systems and need to have 2 fields for one of our database tables that always stay in sync. Here is the table structure:
CREATE TABLE `example` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`object_id` int(11) NOT NULL DEFAULT '0',
`value` varchar(200) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `object_id` (`object_id`)
);
Every time one of the systems inserts a new row we need to have object_id set to id. We can't use 'before insert' since the id column is an auto_increment column so it's value is NULL before insert and due to the limitations of the MySQL 'after insert' on triggers I can't do the following:
CREATE TRIGGER insert_example
AFTER INSERT ON example
FOR EACH ROW
SET NEW.object_id = NEW.id;
I can't update the code for either system so I need a way to accomplish this on the database side. Both systems are going to be inserting new rows. How can I accomplish this?
Using a trigger which fires before the insert should do the job
CREATE TRIGGER insert_example
BEFORE INSERT ON example
FOR EACH ROW
SET NEW.object_id = NEW.id;
EDIT:
As the OP pointed out NEW.id won't work with auto-increment; one could use the following trigger (use at own risk):
CREATE TRIGGER insert_example
BEFORE INSERT ON example
FOR EACH ROW
SET NEW.object_id = (
SELECT AUTO_INCREMENT
FROM information_schema.TABLES
WHERE TABLE_SCHEMA = DATABASE()
AND TABLE_NAME = 'example'
);
But I'd rather re-think this somewhat strange requirement - why do you need the pk value twice in the table?
Is there any reason you cant use a BEFORE INSERT trigger?
I've always seen AFTER INSERT triggers as a method to manipulate other tables rather than the table for which the trigger was executed on.
Rule of thumb, manipulate table the trigger is running on = BEFORE INSERT, manipulate other tables AFTER INSERT :)
I think your trigger will never create in the first place because you can't refer NEW.column_name in an AFTER INSERT trigger.
Try doing this in a BEFORE INSERT trigger (PLEASE IGNORE THIS FIX AS IT WILL NOT WORK):
CREATE TRIGGER `insert_example` BEFORE INSERT ON `t`
FOR EACH ROW
SET NEW.`object_id` = NEW.`id`;
Please change the table and column names as per your schema.
Hope this helps.