I'm getting "Lock wait timeout exceeded" error when inserting a record in MySQL table (amazon aurora to be precise). We're inserting hundreds of records in one transaction with savepoint for every few inserts.
We're using the InnoDB engine. We have innodb_lock_wait_timeout set to 50 sec. We're getting this error only in production so I am limited in debugging ability here.
I've tried to pull info about locked transactions when insert statements runs and here is what I got:
MySQL [(none)]> SELECT w.* FROM information_schema.innodb_lock_waits w INNER JOIN information_schema.innodb_trx b ON b.trx_id = w.blocking_trx_id INNER JOIN information_schema.processlist p on b.trx_mysql_thread_id = p.ID LIMIT 10;
+-------------------+--------------------------+-----------------+--------------------------+
| requesting_trx_id | requested_lock_id | blocking_trx_id | blocking_lock_id |
+-------------------+--------------------------+-----------------+--------------------------+
| 1177444193 | 1177444193:5437:2161:286 | 1177444168 | 1177444168:5437:2161:286 |
+-------------------+--------------------------+-----------------+--------------------------+
1 row in set (0.01 sec)
MySQL [(none)]> select * from information_schema.innodb_trx where trx_id=1177444168 limit 10;
+------------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+------------------+----------------------------+
| trx_id | trx_state | trx_started | trx_requested_lock_id | trx_wait_started | trx_weight | trx_mysql_thread_id | trx_query | trx_operation_state | trx_tables_in_use | trx_tables_locked | trx_lock_structs | trx_lock_memory_bytes | trx_rows_locked | trx_rows_modified | trx_concurrency_tickets | trx_isolation_level | trx_unique_checks | trx_foreign_key_checks | trx_last_foreign_key_error | trx_adaptive_hash_latched | trx_adaptive_hash_timeout | trx_is_read_only | trx_autocommit_non_locking |
+------------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+------------------+----------------------------+
| 1177444168 | RUNNING | 2022-09-26 12:08:49 | NULL | NULL | 1844 | 64308215 | NULL | NULL | 0 | 8 | 908 | 1136 | 2579 | 936 | 0 | READ COMMITTED | 1 | 1 | NULL | 0 | 0 | 0 | 0 |
+------------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+------------------+----------------------------+
1 row in set (0.01 sec)
As I see blocking tx isn't a query but rather some transaction with a lot of locked records because I see nothing in trx_query.
So my question is how can I get more info about this blocking transaction to see what exactly is blocking my insert for 50 sec until timeout?
p.s. we haven't had such problems on standalone MySQL but started to see lock wait errors when migrated to aurora.
+----+---------+--------------------+----------+------+---------------------+---------+-------------+------------+----------+
| id | pname | pmail | pgender | page | pdetails | pnumber | ddepartment | date | time |
+----+---------+--------------------+----------+------+---------------------+---------+-------------+------------+----------+
| 1 | karuna | karuna#gmail.com | female | 21 | hfsdh gfhdf shdh | 2332 | Surgeon | 2018-05-15 | 16:00:00 |
+----+---------+--------------------+----------+------+---------------------+---------+-------------+------------+----------+
This is my database table appointment.
Whenever I execute first two queries, they run perfectly
select * from appointment;(execute )
select * from appointment where pname = 'karuna';(execute)
but when I try to execute these two queries, the result I get is empty set
select *from appointment where pmail='karuna#gmail.com';(empty set (0.00 sec))
select *from appointment where ddepartment='Surgeon';(empty set (0.00 sec))
ModelName.all(:having=>"count(receipt_no)>1",:select=>"school_id,group_concat(id SEPARATOR ',') as f_ids,receipt_no,count(distinct id) as id_count,count(receipt_no) as rec_count",:conditions=>"receipt_no is not null",:group=>"receipt_no")
Output is
+------------+-----------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| receipt_no | school_id | id_count | f_ids | rec_count |
+------------+-----------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| 1261 | 1783 | 2 | 557660,557661 | 2 |
| 14/15- | 1783 | 1209 | 68352,77056,113664,56320,68353,77057,113665,56321,68354,56322,68355,81923,173571,113667,56323,68356,94980,56324,68357,56325,68358,80390,56326,68359,80391,110599,56327,80392,885... | 1209 |
| 15- | 1783 | 112 | 344067,344068,344069,344070,344075,326923,373261,373262,345882,360218,344091,361755,347685,341542,347689,360233,351530,358705,352829,324674,341576,324684,360018,368469,371541,3... | 112 |
Here group_concat does not show all the values but the count of items as same as the count receipt no. Suppose the items in the f_ids column is more than 200 character then its not showing all the values . In other case it will show correct value
I got the solution
SET SESSION group_concat_max_len = 1000000;
Run this code in MySQL console, then this code will change default group_concat character limit to 1000000 characters.
If you want to use in rails console,you can use in this following way
sql = "SET SESSION group_concat_max_len = 1000000"
ActiveRecord::Base.connection.execute(sql)
Please note:
This configuration will work only in that session
I'm working on a application for which I need to read m rows at a time out of total 'N' rows where m < N. Every time I read 'm' rows I have to set their status as read in the same table.
For example consider following table
+------+-------------------------+--------------------------+----------+-------+---------+------+
| ID | from_email_address | to_email_address | subject | body | inqueue | sent |
+------+-------------------------+--------------------------+----------+-------+---------+------+
| 1 | 0120sushil#gmail.com | kumar.sushil#outlook.com | Subject1 | Body1 | 0 | 0 |
| 2 | 0120ksushil#gmail.com | kumar.sushil#outlook.com | Subject1 | Body1 | 0 | 0 |
| 3 | shivaseth1#gmail.com | kumar.sushil#outlook.com | Subject1 | Body1 | 0 | 0 |
| 4 | shivaseth1#gmail.com | amanrajg#outlook.com | Subject1 | Body1 | 0 | 0 |
| 5 | shivamprakash#gmail.com | amanrajg#outlook.com | Subject1 | Body1 | 0 | 0 |
| 6 | shivamprakash#gmail.com | poorvanagpal#outlook.com | Subject1 | Body1 | 0 | 0 |
| 7 | shivankgupta#gmail.com | poorvanagpal#outlook.com | Subject1 | Body1 | 0 | 0 |
+------+-------------------------+--------------------------+----------+-------+---------+------+
I want to read lets say 3 rows at a time and once I have read the rows I want to set inqueue status of those rows as 1.
I can use following query in stored procedure to select the rows
select * from EmailQueue where inqueue=1 LIMIT 3
After this how to update the same rows and set their inqueue to 1.
EDIT
Here is the stored procedure I created which is giving some error.
DELIMITER $$
DROP PROCEDURE IF EXISTS GetUnsentMails;
CREATE PROCEDURE GetUnsentMails()
BEGIN
START TRANSACTION;
CREATE TEMPORARY TABLE temp_EmailQueue AS SELECT * FROM EmailQueue WHERE inqueue = 0 LIMIT 5 FOR UPDATE;
UPDATE EmailQueue SET inqueue=1 where id in (SELECT id from temp_EmailQueue) AND inqueue = 0;
COMMIT;
END
It gives following error on calling
ERROR 1746 (HY000): Can't update table 'emailqueue' while 'temp_EmailQueue' is being created.
Suggesting to use Transaction and "SELECT FOR UPDATE" to fulfill your requirements.
Please refer following links for the examples:
MySQL 'select for update' behaviour
http://dev.mysql.com/doc/refman/5.0/en/innodb-locking-reads.html
http://www.sqlines.com/mysql/how-to/select-update-single-statement-race-condition
UPDATE - Added the QUERY EXAMPLE:
Example:
.......
#Before starting procedure
.......
START TRANSACTION;
CREATE TEMPORARY TABLE zzz_EmailQueue AS SELECT * FROM EmailQueue WHERE inqueue=1 LIMIT 3 FOR UPDATE;
.....
.....
#Section for other activities....
.....
.....
UPDATE EmailQueue SET inqueue=<<New_Value>> WHERE id IN (SELECT id FROM zzz_EmailQueue) AND inqueue=1;
COMMIT;
.......
#Remaining lines of Prodecure
.......
Update **
**Try with following method:
DECLARE v_EmailQueue_ID DOUBLE;
SELECT ID INTO v_EmailQueue_ID FROM EmailQueue WHERE inqueue = 0 LIMIT 1 FOR UPDATE;
UPDATE EmailQueue SET inqueue=1 WHERE id=v_EmailQueue_ID AND inqueue = 0;
I got a table like this:
mysql> select * from users;
+--------+----------+------------+-----------+
| userid | username | password | privilege |
+--------+----------+------------+-----------+
| 1 | user1 | password | 1 |
| 2 | david | goodboy | 1 |
| 3 | admin | mastermold | 5 |
| 4 | user4 | password4 | 1 |
| 5 | user5 | password5 | 2 |
| 6 | user6 | password6 | 1 |
| 7 | user7 | password7 | 1 |
+--------+----------+------------+-----------+
7 rows in set
Now, how to extract password of username who is called "david" by the only select query without "password" stored in it and result is in one field. (Don't accept "select * from users")?
Try
select password from users where username='david';
without using mentioning "password"?
well you could grab the column name into a variable and use that like this sqlFiddle
SET #columnName = (SELECT COLUMN_NAME FROM
INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'users'
AND ORDINAL_POSITION = 3);
SET #sql = CONCAT("SELECT ",#columnName," FROM users WHERE username = 'david'");
prepare statement FROM #sql;
execute statement;
Well, you mean that your query result shouldn't have the string "password", if I have got it correctly. Even though I am posting the answer, You should really consider #MarcB comment and try learning SQL first.
Your query should go like this
select `password` from users where username='david'
and lower(`password`) not like '%password%';