Deadlocks happened when massive inserting executed simultaneously in mysql 5.0 - mysql

I am using mysql 5.0 and I met some mysql deadlock problems when there were lots of inserts from different session at the same time (We estimated there would be a maximum about 900 insert statements executed in one second).
Here is the error I got:
1213, Deadlock found when trying to get lock; try restarting transaction
Here is one of my failure insert statement:
INSERT `cj_202203qmoh_prize_log` (`user_id`, `lottery_id`, `create_ip`, `flags`, `created_at`, `create_mac`)VALUES('388','58','???.???.???.???','0','2022-04-01 20:00:33','444937f4bc5d5aa8f4af3d96d31dbf61');
My table definition:
CREATE TABLE `cj_202203qmoh_prize_log` (
`id` int(10) unsigned NOT NULL auto_increment,
`user_id` int(10) unsigned NOT NULL,
`lottery_id` int(10) unsigned default NULL,
`code` int(11) default NULL,
`flags` int(10) unsigned default '0',
`create_ip` varchar(64) NOT NULL,
`create_mac` varchar(255) character set ascii NOT NULL,
`created_at` timestamp NOT NULL default '0000-00-00 00:00:00',
`updated_at` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
PRIMARY KEY USING BTREE (`id`),
KEY `user_id` USING BTREE (`user_id`,`created_at`),
KEY `user_id_2` USING BTREE (`user_id`,`lottery_id`),
KEY `create_ip` USING BTREE (`create_ip`),
KEY `create_mac` USING BTREE (`create_mac`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ROW_FORMAT=COMPACT;
I didn't use transactions at all in my business. Except for the insert statement, the only possible (in fact it's extremely improbable to be executed simultaneously) sql that requires a x-lock executes at the same time is:
UPDATE `cj_202203qmoh_prize_log` SET `code` = ? WHERE `id` = ?;
There are several select statements using index 'user_id' or 'user_id_2' could be executed simultaneously, but they don't need a s-lock.
And same user_id could only be inserted in the same session.
According to my company's policy, I have no privileges to run SHOW ENGINE INNODB STATUS, so I am afraid I could not provide further information.
After I set the transaction level to READ COMMITTED, execute the statement in a transaction and drop both create_ip and create_mac indexes, it seemed this problem have not happened again. But I still couldn't figure out what caused the deadlock.

Related

MySQL keeps ignoring MAX_EXECUTION_TIME

I am running a node server with mysql8.
This query keeps popping up and freezing.
SELECT /*+ MAX_EXECUTION_TIME(2000) */ COUNT(*) FROM my_table
Even though this query have MAX_EXECUTION_TIME mentioned in it, It keeps executing well past this limit (basically it never stops). The longest I have seen it running was around 20-30 days and even after that it didn't stopped I had restarted the server.
Main issue is I can't even kill this query. Even after killing it, It keeps executing and never stops.
I can't even restart mysql. After trying to restart the mysql. It just shuts down and never starts back.
I had to reboot the server (which is totally unacceptable).
SHOW OPEN TABLES;
Shows that this table is in use. It doesn't effect Reading, updating or inserting of data in the table. But as soon as I try to alter this table or any other table which has any reference from or to this table. It freezes whole node server as deadlock of queries occur and every query keeps waiting for the first query to end, which as already stated, never does.
This query in question isn't there in any of my node js code. But is being executed by the user (mysql user) which my node server uses, I also use this user with adminer.
This query always shows up with this specific table. Below is the sample output of show create table my_table; (table name and column name is changed)
CREATE TABLE `my_table` (
`id` int NOT NULL AUTO_INCREMENT,
`col1` varchar(150) COLLATE utf8_unicode_ci DEFAULT NULL,
`col2` int NOT NULL,
`col3` longtext COLLATE utf8_unicode_ci,
`col4` longtext COLLATE utf8_unicode_ci,
`col5` mediumtext COLLATE utf8_unicode_ci,
`col5` int DEFAULT NULL COMMENT 'in seconds',
`col6` tinyint(1) NOT NULL DEFAULT '0',
`col7` int DEFAULT NULL,
`col8` int DEFAULT NULL,
`col9` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`col9` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `test_level` (`col7`),
KEY `col9` (`col9`),
KEY `col6` (`col6`),
KEY `col5` (`col5`),
KEY `col2` (`col2`),
CONSTRAINT `my_table_ibfk_1` FOREIGN KEY (`col7`) REFERENCES `table_1` (`id`),
CONSTRAINT `my_table_ibfk_2` FOREIGN KEY (`col2`) REFERENCES `table_2` (`id`) ON DELETE RESTRICT ON UPDATE RESTRICT
) ENGINE=InnoDB AUTO_INCREMENT=4078 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
No of rows in this table is around 4000.
exact mysql-version 8.0.19

INSERT INTO SELECT takes long time on cluster

My mysql cluster: Ver 5.6.30-76.3-56 for debian-linux-gnu on x86_64 (Percona XtraDB Cluster (GPL), Release rel76.3, Revision aa929cb, WSREP version 25.16, wsrep_25.16)
I've a complicated sql query which inserts for about 36k rows into a table with this syntax:
INSERT INTO `sometable` (SELECT ...);
The select is a bit complicated but not slow (0.0023s) but the insert takes about 40-50s. The table is not in use when I'm inserting the rows.
My questions are:
Can I speed it up somehow?
The slow insert causes locking problems on the other tables (because of select)
This workflow is good or bad practice? Is there any better?
Thanks
UPDATE:
The table schema:
CREATE TABLE `sometable` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`user_id` int(11) unsigned DEFAULT NULL,
`a` varchar(255) DEFAULT NULL,
`b` smallint(6) unsigned DEFAULT NULL,
`c` smallint(6) unsigned DEFAULT NULL,
`d` smallint(6) unsigned DEFAULT NULL,
`e` smallint(6) unsigned DEFAULT NULL,
`f` varchar(255) DEFAULT '',
`country_id` int(10) unsigned DEFAULT NULL,
`city_id` int(10) unsigned DEFAULT NULL,
`g` smallint(6) unsigned DEFAULT NULL,
`h` smallint(6) unsigned DEFAULT NULL,
`i` smallint(6) unsigned DEFAULT NULL,
`j` smallint(6) unsigned DEFAULT NULL,
`k` smallint(6) unsigned DEFAULT NULL,
`l` varchar(3) DEFAULT NULL,
`m` varchar(3) DEFAULT NULL,
`n` text,
`o` varchar(255) DEFAULT NULL,
`p` varchar(32) DEFAULT NULL,
`q` varchar(32) DEFAULT NULL,
`r` varchar(32) DEFAULT NULL,
`s` time DEFAULT NULL,
`t` time DEFAULT NULL,
`u` text,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `country_id` (`country_id`),
KEY `city_id` (`city_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
UPDATE2:
When I try to run the query I get an error in some cases:
ERROR 1213 (40001): Deadlock found when trying to get lock; try restarting transaction
MY SOLUTION:
Here is my final solution if somebody interested in:
gist
The main problem was that while I fill mytable the other queries are stuck and the cluster had serious performance problems. In this solution I create a temporary table and fill it with data in "dirty read" mode, then I copy these data to mytable in chunks so it takes a bit more time but there is no performance problem and not stuck the queries.
A SELECT operation that returns a row of the length you describe every 64 nanoseconds is very fast. That's what 36 kilorows in 2.3 milliseconds works out to. It seems likely that your SELECT query timing doesn't account for the transport of the result set to the MySQL client. At any rate, using that performance as a comparison to an INSERT operation sets your expectations unreasonably high.
You might try issuing this command before starting your operation. It will allow your SELECT operation to proceed with fewer contentions with your application's traffic on the source tables for the SELECT. See here https://dev.mysql.com/doc/refman/5.7/en/set-transaction.html
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
You might try a two step process, involving a temporary table. This will have the advantage of not having to update all the indexes in some_table at the same time as the SELECT operation. That operation will look like this.
SET TRANSACTION ISOLATION LEVEL READ UNCOMMITTED;
CREATE TEMPORARY TABLE insert_batch AS SELECT ... ;
INSERT INTO some_table SELECT * FROM insert_batch;
DROP TEMPORARY TABLE insert_batch;
You should understand that InnoDB posts your batch of insertions to your table as a single transaction. If you can do this in a way that handles about 500 rows at a time rather than 36K, you'll have more transactions, but they will be smaller. That's generally a way to get higher throughput.
If all else fails, this may be a viable solution. First, see http://mysql.rjweb.org/doc.php/deletebig#deleting_in_chunks
Load your corrections into a temp table (or non-replicated MyISAM table).
Loop through the temp table (using code similar to that link). Pick 100 rows at a time.
Do the INSERT ... SELECT ... of 100 rows in a separate transaction.
This technique may (or may not) take longer than 40-50s, but at least is much less likely to timeout or deadlock.
In general, avoid running any transaction that lasts longer than a few seconds. This link is somewhat generic on how to "chunk" lengthy (and repetitive) operations to avoid long transactions.

Handling rapid microtransactions in MySQL

I'm running across an issue where innoDB locks rows while doing updates. When I run SHOW FULL PROCESSLIST I end up getting tons of sleeping executions that look like this:
UPDATE accounts SET account_balance = account_balance + 0.000004514 WHERE account_id = 1234 LIMIT 1
The row gets lock and the server ends up with a traffic jam of update connections.
How do I scale my mySQL application, so that I can handle rapid micro-transactions?
EDIT: Table structure:
CREATE TABLE `accounts` (
`account_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`account_balance` float NOT NULL DEFAULT '0',
`account_created` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`account_updated` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`account_id`)
) ENGINE=InnoDB AUTO_INCREMENT=2756 DEFAULT CHARSET=utf8mb4;

Slow Updates for Single Records by Primary Key

I am using MySQL 5.5.
I have an InnoDB table definition as follows:
CREATE TABLE `table1` (
`col1` int(11) NOT NULL AUTO_INCREMENT,
`col2` int(11) DEFAULT NULL,
`col3` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`col4` int(11) DEFAULT NULL,
`col5` datetime DEFAULT NULL,
`col6` tinyint(1) NOT NULL DEFAULT '0',
`col7` datetime NOT NULL,
`col8` datetime NOT NULL,
`col9` int(11) DEFAULT NULL,
`col10` tinyint(1) NOT NULL DEFAULT '0',
`col11` tinyint(1) DEFAULT '0',
PRIMARY KEY (`col1`),
UNIQUE KEY `index_table1_on_ci_ai_tn_sti` (`col2`,`col4`,`col3`,`col9`),
KEY `index_shipments_on_applicant_id` (`col4`),
KEY `index_shipments_on_shipment_type_id` (`col9`),
KEY `index_shipments_on_created_at` (`col7`),
KEY `idx_tracking_number` (`col3`)
) ENGINE=InnoDB AUTO_INCREMENT=7634960 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
The issue is UPDATES. There are about 2M rows in this table.
A typical UPDATE query would be :
UPDATE table1 SET col6 = 1 WHERE col1 = 7634912;
We have about 5-10k QPS on this production server. These queries are often in "Updating" state when looked at through the process list. The InnoDB locks show that there are many rec but not gap locks on index_table1_on_ci_ai_tn_sti. No transaction is waiting for lock.
My feeling is that the Unique Index is causing the lag but I'm not sure why. This is the only table we have that is defined this way using the Unique Index.
I don't think the UNIQUE key has any impact (in this case).
Are you really setting a DATETIME to "1"? (Please check for other typos -- they could make a big difference.)
Are you trying to do 10K UPDATEs per second?
Is innodb_buffer_pool_size bigger than the table, but no bigger than 70% of available RAM?
What is the value of innodb_flush_log_at_trx_commit? 1 is default and secure, but slower than 2.
Can you put a bunch of updates into a single transaction? That would cut down the transaction overhead.

Non-null auto-incrementing column — but 'is null' query returns just-inserted row

I am using MySQL 5.1.60
And i have a table called "folder"
CREATE TABLE `folder` (
`folder_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(50) DEFAULT NULL,
`parent_id` int(10) unsigned DEFAULT NULL,
`update_date` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`folder_id`)
) ENGINE=InnoDB AUTO_INCREMENT=0 DEFAULT CHARSET=utf8
And i insert a value to it.
INSERT INTO `folder` (`name`) VALUES ('test_name');
just after that i execute a select query
query is
SELECT * FROM folder f WHERE f.folder_id IS NULL
But the surprising thing is that it actually return the last inserted row (only at the first run).
why this behavior happening in MySQL.
actually it is a bug in mysql 5.1.60 which you are using, and it is not a problem in early version of mysql.
however you can fix it in your current version of mysql by running this command:
mysql> SET sql_auto_is_null=0;
Check your "innodb_flush_log_at_trx_commit" system variable
and set it to 1 and then try again