We have an table bellow:
CREATE TABLE usable (
id bigint(20) NOT NULL AUTO_INCREMENT primary key ,
device VARCHAR(64) NOT NULL,
key1 VARCHAR(128) NOT NULL,
key2 VARCHAR(128) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ;
create unique index uidx_usable on usable (key1);
CREATE TABLE used
(
usedid bigint(20) PRIMARY KEY NOT NULL AUTO_INCREMENT primary key,
devid VARCHAR(64) NOT NULL,
key1 VARCHAR(128) NOT NULL ,
key2 VARCHAR(64) NOT NULL '
created_time timestamp DEFAULT CURRENT_TIMESTAMP NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ;
create unique index uidx_used on used (key1);
create unique index uidx_used_deviceid on used (devid);
select 1 record from usable table wih muti-thread.
insert into record to used table.
delete this record from usable table.
my work goes here:
START TRANSACTION
SELECT * FROM usable WHERE device = 555 order by rand() limit 1 FOR UPDATE;
insert into used (key1, key2) values (key1, key2 from step 1 select result);
delete from usable where id = (id from step 1 select result);
COMMIT
The tables are in InnoDB engine. AutoCommit is set to ON in the global variables.
There is no problem for just one thread, about 100ms with these three sql cmds. Is there solution to improve it?
but, thread begin to hang up when 100 QPS.
How to congfirm transaction worked?
How to improve QPS
(0.7~5ms) SELECT ; (47ms~250ms) insert ; (48ms~250ms) delete
why insert and delete need large time? how to tune insert and delete sql?
Related
I like to set internal AUTO_INCREMENT counter for the table, which uses MyISAM and composite primary key - but just for specific PK combination.
If I use
ALTER TABLE tablename AUTO_INCREMENT = 1;
It will set internal counter for all composite PK combinations, which I don't want.
I need something like
ALTER TABLE tablename AUTO_INCREMENT = 1 WHERE prefix = 5 AND suffix = X;
It does not work this way. Is there any possibility to change only counter for specific PK combination in MyISAM table?
Table:
CREATE TABLE `ENG__faktury_counter`
(
`year` int(10) NOT NULL,
`prefix` varchar(10) NOT NULL,
`DIC` varchar(50) NOT NULL,
`id_counter` int(15) NOT NULL AUTO_INCREMENT ,
`created` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`year`,`prefix`,`DIC`,`id_counter`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
There is virtually zero use for ALTER TABLE ... AUTO_INCREMENT=....
If you are using MyISAM and have
foo ...
id ... AUTO_INCREMENT
PRIMARY KEY(foo, id) -- where the _2nd_ column is auto-inc
Then there is nothing to do to get
foo id
--- --
cat 1
cat 2
dog 1
bird 1
cat 3
bird 2
bird 3
dog 2
regardless of the order in which you insert the rows.
If this does not address your question, please enhance it with an example and SHOW CREATE TABLE.
I have a large table named 'roomlogs' which has nearly 1 million entries.
The structure of the table:
id --> PK
roomId --> varchar FK to rooms table
userId --> varchar FK to users table
enterTime --> Date and Time
exitTime --> Date and Time
status --> bool
I have the previous indexing on roomID, I recently added an index on the userId column.
So, When I run a stored procedure with following code it is taking more time like on average 50 seconds. WHich it should not take.
DELIMITER ;;
CREATE DEFINER=`root`#`%` PROCEDURE `enter_room`(IN pRoomId varchar(200), IN puserId varchar(50), IN ptime datetime, IN phidden int, pcheckid int, pexit datetime)
begin
update roomlogs set
roomlogs.exitTime = ptime,
roomlogs.`status` = 1
where
roomlogs.userId = puserId
and roomlogs.`status` = 0
and DATEDIFF(ptime,roomlogs.enterTime) = 0;
INSERT into roomlogs
( roomlogs.roomId,
roomlogs.userId,
roomlogs.enterTime,
roomlogs.exitTime,
roomlogs.hidden,
roomlogs.checkinId )
value
( pRoomId,
userId,
ptime,
pexit,
phidden,
pcheckid);
select *
from
roomlogs
where
roomlogs.id= LAST_INSERT_ID();
end ;;
DELIMITER ;
What Can be the reason for it to take this much time:
I added an index recently so previous rows are not indexed.
There is no selection on storage type for any indexes right now. Should I change it to B-tree?
On my website, I get 20-30 simultaneous call on other procedures also while this procedure has 10-20 simultaneous calls, does the update query in the procedure make a lock? But in MySQL.slow_logs table for each query the lock _time shows 0.
Is there any other reason for this behaviour?
Edit: Here is the SHOW TABLE:
CREATE TABLE `roomlogs` (
`roomId` varchar(200) CHARACTER SET latin1 DEFAULT NULL,
`userID` varchar(50) CHARACTER SET latin1 DEFAULT NULL,
`enterTime` datetime DEFAULT NULL,
`exitTime` datetime DEFAULT NULL,
`id` int(11) NOT NULL AUTO_INCREMENT,
`status` int(11) DEFAULT '0',
`hidden` int(11) DEFAULT '0',
`checkinId` int(11) DEFAULT '-1',
PRIMARY KEY (`id`),
KEY `RoomLogIndex` (`roomId`),
KEY `RoomLogIDIndex` (`id`),
KEY `USERID` (`userID`)
) ENGINE=InnoDB AUTO_INCREMENT=1064216 DEFAULT CHARSET=utf8
I can also see that this query is running more number of times like 100000 times per day (nearly continuously).
SELECT count(*) from roomlogs where roomId=proomId and status='0';
Because of this query reads from the same table, does InnoDB block or create a lock on update query because I can see that when the above-stored procedure is running more number of times then this query is taking more time.
Here is the link for MySQL variables: https://docs.google.com/document/d/17_MVaU4yvpQfVDT83yhSjkLHsgYd-z2mg6X7GwvYZGE/edit?usp=sharing
roomlogs needs this 'composite' index:
INDEX(userId, `status`, enterTime)
I added an index recently so previous rows are not indexed.
Not true. Adding an INDEX indexes the entire table.
The default index type is BTree; no need to explicitly specify it.
does the update query in the procedure make a lock?
It does some form of locking. What is the value of autocommit? Do you explicitly use BEGIN and COMMIT? Is the table ENGINE=InnoDB? Please provide SHOW CREATE TABLE.
MySQL.slow_logs table for each query the lock _time shows 0.
The INSERT you show seems to be inserting the same row as the UPDATE. Maybe you need INSERT ... ON DUPLICATE KEY UPDATE ...?
Don't "hide an index column in a function"; instead of DATEDIFF(roomlogs.enterTime,NOW()) = 0, do
AND enterTime >= CURDATE()
AND enterTime < CURDATE() + INTERVAL 1 DAY
This allows the index to be used more fully.
KEY `RoomLogIndex` (`roomId`), Change to (roomId, status)
KEY `RoomLogIDIndex` (`id`), Remove, redundant with the PK
Buffer pool in only 97,517,568 -- make it more like 9G.
I am in a problematic situation and found dozens of questions on same topic, but may b i am not able to understand those solutions as per my issue.
I have a system built in Codeigniter, and it does the following
codeigniter->start_transaction()
UPDATE T SET A = 1, MODIFIED = NOW()
WHERE PK IN
( SELECT PK FROM
(SELECT PK, LAST_INSERT_ID(PK) FROM T
where FK = 31 AND A=0 AND R=1 AND R_FK = 21
AND DEAD = 0 LIMIT 0,1) AS TBL1
) and A=0 AND R = 1 AND R_FK = 21 AND DEAD = 0
-- what this query does is , it takes a row dynamically which is not dead yet,
--and not assigned and it's linked to 21 id (R_FK) from R table,
-- when finds the row, update it to be marked as assigned (A=1).
-- PK = LAST_INSERT_ID(PK) ensures that last_insert_id is updated with this row id, so i can retrieve it from PHP
GOTO MODULE B
MODULE B {
INSERT INTO T(A,B,C,D,E,F,R,FK,R_FK,DEAD,MODIFIED) VALUES(ALL VALUES)
-- this line gives me lock wait timeout exceeded.
}
MySQL version is 5.1.63-community-log
Table T is an INNODB table and has only one normal type index on FK field, and no foreign key constraints are there. PrimaryKey (PK) field is an auto_increment field.
I get lock wait timeout in the above case , and that is due to first transactional update holding lock on table, how can i avoid lock on table with that update query ,while using transactions, I cannot commit the transaction until i receive response from MODULE B .
I don't have much detailed knowledge about DB and structural things, so please bear with me if i said something not making sense.
--UPDATE--
-- TABLE T Structure
CREATE TABLE `T` (
`PK` int(11) NOT NULL AUTO_INCREMENT,
`FK` int(11) DEFAULT NULL,
`P` varchar(1024) DEFAULT NULL,
`DEAD` tinyint(1) NOT NULL DEFAULT '0',
`A` tinyint(1) NOT NULL DEFAULT '0',
`MODIFIED` datetime DEFAULT NULL,
`R` tinyint(4) NOT NULL DEFAULT '0',
`R_FK` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`PK`),
KEY `FK_REFERENCE_54` (`FK`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
-- Indexes Information
SHOW INDEX FROM T;
1- Field FK, Cardinality 65 , NULL => Yes , Index_Type => BTRee
2- Field PK, Cardinality 11153, Index_Type => BTRee
Scenario:
I have a table which references two foreign keys, and for each unique combination of these foreign keys, has its own auto_increment column. I need to implement a Composite Key that will help identify the row as unique using combination of these three (one foreign keys and one auto_increment column, and one other column with non-unique values)
Table:
CREATE TABLE `issue_log` (
`sr_no` INT NOT NULL AUTO_INCREMENT ,
`app_id` INT NOT NULL ,
`test_id` INT NOT NULL ,
`issue_name` VARCHAR(255) NOT NULL ,
primary key (app_id, test_id,sr_no)
);
Of course, there has to be something wrong with my query, because of which the error thrown is:
ERROR 1075: Incorrect table definition; there can be only one auto
column and it must be defined as a key
What I am trying to achieve:
I have an Application Table (with app_id as its primary key), each Application has a set of Issues to be resolved, and each Application has multiple number of tests (so the test_id col)
The sr_no col should increment for unique app_id and test_id.
i.e. The data in table should look like:
The database engine is InnoDB.
I want to achieve this with as much simplicity as possible (i.e. avoid triggers/procedures if possible - which was suggested for similar cases on other Questions).
You can't have MySQL do this for you automatically for InnoDB tables - you would need to use a trigger or procedure, or user another DB engine such as MyISAM. Auto incrementing can only be done for a single primary key.
Something like the following should work
DELIMITER $$
CREATE TRIGGER xxx BEFORE INSERT ON issue_log
FOR EACH ROW BEGIN
SET NEW.sr_no = (
SELECT IFNULL(MAX(sr_no), 0) + 1
FROM issue_log
WHERE app_id = NEW.app_id
AND test_id = NEW.test_id
);
END $$
DELIMITER ;
You can do this with myISAM and BDB engines. InnoDB does not support this. Quote from MySQL 5.0 Reference Manual.
For MyISAM and BDB tables you can specify AUTO_INCREMENT on a secondary column in a multiple-column index. In this case, the generated value for the AUTO_INCREMENT column is calculated as MAX(auto_increment_column) + 1 WHERE prefix=given-prefix.
http://dev.mysql.com/doc/refman/5.0/en/example-auto-increment.html
I don't fully understand your increment requirement on the test_id column, but if you want an ~autoincrement sequence that restarts on every unique combination of (app_id, test_id), you can do an INSERT ... SELECT FROM the same table, like so:
mysql> INSERT INTO `issue_log` (`sr_no`, `app_id`, `test_id`, `issue_name`) SELECT
IFNULL(MAX(`sr_no`), 0) + 1 /* next sequence number */,
3 /* desired app_id */,
1 /* desired test_id */,
'Name of new row'
FROM `issue_log` /* specify the table name as well */
WHERE `app_id` = 3 AND `test_id` = 1 /* same values as in inserted columns */
This assumes a table definition with no declared AUTO_INCREMENT column. You're essentially emulating autoincrement behavior with the IFNULL(MAX()) + 1 clause, but the manual emulation works on arbitrary columns, unlike the built-in autoincrement.
Note that the INSERT ... SELECT being a single query ensures atomicity of the operation. InnoDB will gap-lock the appropriate index, and many concurrent processes can execute this kind of query while still producing non-conflicting sequences.
You can use a unique composite key for sr_no,app_id & test_id. You cannot use incremental in sr_no as this is not unique.
CREATE TABLE IF NOT EXISTS `issue_log` (
`sr_no` int(11) NOT NULL,
`app_id` int(11) NOT NULL,
`test_id` int(11) NOT NULL,
`issue_name` varchar(255) NOT NULL,
UNIQUE KEY `app_id` (`app_id`,`test_id`,`sr_no`)
) ENGINE=InnoDB ;
I have commented out unique constraint violation in sql fiddle to demonstrate (remove # in line 22 of schema and rebuild schema )
This is what I wanted
id tenant
1 1
2 1
3 1
1 2
2 2
3 2
1 3
2 3
3 3
My current table definition is
CREATE TABLE `test_trigger` (
`id` BIGINT NOT NULL,
`tenant` varchar(255) NOT NULL,
PRIMARY KEY (`id`,`tenant`)
);
I created one table for storing the current id for each tenant.
CREATE TABLE `get_val` (
`tenant` varchar(255) NOT NULL,
`next_val` int NOT NULL,
PRIMARY KEY (`tenant`,`next_val`)
) ENGINE=InnoDB ;
Then I created this trigger which solve my problem
DELIMITER $$
CREATE TRIGGER trigger_name
BEFORE INSERT
ON test_trigger
FOR EACH ROW
BEGIN
UPDATE get_val SET next_val = next_val + 1 WHERE tenant = new.tenant;
set new.id = (select next_val from get_val where tenant=new.tenant);
END$$
DELIMITER ;
This approach will be thread safe also because any insertion for the same tenant will happen sequentially because of the update query in the trigger and for different tenants insertions will happen parallelly.
Just add key(sr_no) on auto-increment column:
CREATE TABLE `issue_log` (
`sr_no` INT NOT NULL AUTO_INCREMENT ,
`app_id` INT NOT NULL ,
`test_id` INT NOT NULL ,
`issue_name` VARCHAR(255) NOT NULL ,
primary key (app_id, test_id,sr_no),
key (`sr_no`)
);
Why don't you try to change the position of declare fields as primary key, since when you use "auto_increment" it has to be referenced as the first. Like in the following example
CREATE TABLE `issue_log` (
`sr_no` INT NOT NULL AUTO_INCREMENT ,
`app_id` INT NOT NULL ,
`test_id` INT NOT NULL ,
`issue_name` VARCHAR(255) NOT NULL ,
primary key (sr_no,app_id, test_id)
);
If I create a table with the following syntax,
CREATE TABLE IF NOT EXISTS `hashes` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`hash` binary(20) NOT NULL,
PRIMARY KEY (`id`,`hash`),
UNIQUE KEY (`hash`)
) ENGINE=InnoDB ROW_FORMAT=COMPRESSED KEY_BLOCK_SIZE = 4 AUTO_INCREMENT=1
PARTITION BY KEY(`hash`)
PARTITIONS 10;
And insert queries with the following syntax
INSERT INTO hashes (hash) VALUES ($value) ON DUPLICATE KEY UPDATE hash = hash
Then the auto increment column works as expected both if the row is inserted or updated.
Although creating the table without the partition like below and inserting with the query above the auto increment value will increase by 1 on every update or insert causing the A_I column to be all over place as the query could do 10 updates and then 1 insert causing the column value to jump 10 places.
CREATE TABLE IF NOT EXISTS `hashes` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`hash` binary(20) NOT NULL,
PRIMARY KEY (`id`,`hash`),
UNIQUE KEY (`hash`)
) ENGINE=InnoDB AUTO_INCREMENT=1;
I understand why the value increases on an update with INNO_DB but I do not understand why it doesn't when the table is partitioned?
you cannot change that, but you can try something like this:
mysql> set #a:= (select max(id) + 2 from hashes);
mysql> insert into hashes (id) values ($value) on duplicate key update id=#a;
NOTE: the partitions change a little bit after mysql 5.6, which version do you have?