Why does SELECT FOR UPDATE works only within a transaction? - mysql

I think I am confused with the SELECT FOR UPDATE construct.
Example:
mysql> select * from employees2;
+-------+----------+--------+-----------+
| EmpId | EmpName | DeptId | EmpSalary |
+-------+----------+--------+-----------+
| 1 | John | 1 | 5000.00 |
| 2 | Albert | 1 | 4500.00 |
| 3 | Crain | 2 | 6000.00 |
| 4 | Micheal | 2 | 5000.00 |
| 5 | David | NULL | 34.00 |
| 6 | Kelly | NULL | 457.00 |
| 7 | Rudy | 1 | 879.00 |
| 8 | Smith | 2 | 7878.00 |
| 9 | Karsen | 5 | 878.00 |
| 10 | Stringer | 5 | 345.00 |
| 11 | Cheryl | NULL | NULL |
+-------+----------+--------+-----------+
11 rows in set (0.00 sec)
I do the following in a script:
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect('dbi:mysql:testdb','root','1234', {'RaiseError' => 1, 'AutoCommit' => 0}) or die "Connection Error: $DBI::errstr\n";
my $sql = "select * from employees2 where EmpId IN (2,10) for update";
my $sth = $dbh->prepare($sql);
$sth->execute or die "SQL Error: $DBI::errstr\n";
while (my #row = $sth->fetchrow_array) {
print "#row\n";
}
sleep(9000);
$dbh->commit;
I also in parallel a console and connect to the database.
So I run the script first and then in another session I do:
mysql> select * from employees2 where EmpId IN (10) for update;
The second select blocks as it refers to the same row.
This blocks either I do:
mysql> set autocommit = 0;
mysql> begin;
mysql> select * from employees2 where EmpId IN (10) for update;
mysql> commit;
or just
mysql> select * from employees2 where EmpId IN (10) for update;
So it blocks irrelevant if it is in a transaction or not.
Now if I change the script as:
my $dbh = DBI->connect('dbi:mysql:practice','root','') or die "Connection Error: $DBI::errstr\n";
I.e the script does not run within a transaction the second session does not block!
Why does it block only if the script runs within a transaction?

According to the documentation:
Locking of rows for update using SELECT FOR UPDATE only applies when autocommit is disabled (either by beginning transaction with START TRANSACTION or by setting autocommit to 0. If autocommit is enabled, the rows matching the specification are not locked.
In other words, if you don't execute your first SELECT FOR UPDATE inside a transaction, no rows are locked.

Related

How to debug "Lock wait timeout exceeded" in mysql (aurora)

I'm getting "Lock wait timeout exceeded" error when inserting a record in MySQL table (amazon aurora to be precise). We're inserting hundreds of records in one transaction with savepoint for every few inserts.
We're using the InnoDB engine. We have innodb_lock_wait_timeout set to 50 sec. We're getting this error only in production so I am limited in debugging ability here.
I've tried to pull info about locked transactions when insert statements runs and here is what I got:
MySQL [(none)]> SELECT w.* FROM information_schema.innodb_lock_waits w INNER JOIN information_schema.innodb_trx b ON b.trx_id = w.blocking_trx_id INNER JOIN information_schema.processlist p on b.trx_mysql_thread_id = p.ID LIMIT 10;
+-------------------+--------------------------+-----------------+--------------------------+
| requesting_trx_id | requested_lock_id | blocking_trx_id | blocking_lock_id |
+-------------------+--------------------------+-----------------+--------------------------+
| 1177444193 | 1177444193:5437:2161:286 | 1177444168 | 1177444168:5437:2161:286 |
+-------------------+--------------------------+-----------------+--------------------------+
1 row in set (0.01 sec)
MySQL [(none)]> select * from information_schema.innodb_trx where trx_id=1177444168 limit 10;
+------------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+------------------+----------------------------+
| trx_id | trx_state | trx_started | trx_requested_lock_id | trx_wait_started | trx_weight | trx_mysql_thread_id | trx_query | trx_operation_state | trx_tables_in_use | trx_tables_locked | trx_lock_structs | trx_lock_memory_bytes | trx_rows_locked | trx_rows_modified | trx_concurrency_tickets | trx_isolation_level | trx_unique_checks | trx_foreign_key_checks | trx_last_foreign_key_error | trx_adaptive_hash_latched | trx_adaptive_hash_timeout | trx_is_read_only | trx_autocommit_non_locking |
+------------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+------------------+----------------------------+
| 1177444168 | RUNNING | 2022-09-26 12:08:49 | NULL | NULL | 1844 | 64308215 | NULL | NULL | 0 | 8 | 908 | 1136 | 2579 | 936 | 0 | READ COMMITTED | 1 | 1 | NULL | 0 | 0 | 0 | 0 |
+------------+-----------+---------------------+-----------------------+------------------+------------+---------------------+-----------+---------------------+-------------------+-------------------+------------------+-----------------------+-----------------+-------------------+-------------------------+---------------------+-------------------+------------------------+----------------------------+---------------------------+---------------------------+------------------+----------------------------+
1 row in set (0.01 sec)
As I see blocking tx isn't a query but rather some transaction with a lot of locked records because I see nothing in trx_query.
So my question is how can I get more info about this blocking transaction to see what exactly is blocking my insert for 50 sec until timeout?
p.s. we haven't had such problems on standalone MySQL but started to see lock wait errors when migrated to aurora.

mysql query returns empty set when data is there

+----+---------+--------------------+----------+------+---------------------+---------+-------------+------------+----------+
| id | pname | pmail | pgender | page | pdetails | pnumber | ddepartment | date | time |
+----+---------+--------------------+----------+------+---------------------+---------+-------------+------------+----------+
| 1 | karuna | karuna#gmail.com | female | 21 | hfsdh gfhdf shdh | 2332 | Surgeon | 2018-05-15 | 16:00:00 |
+----+---------+--------------------+----------+------+---------------------+---------+-------------+------------+----------+
This is my database table appointment.
Whenever I execute first two queries, they run perfectly
select * from appointment;(execute )
select * from appointment where pname = 'karuna';(execute)
but when I try to execute these two queries, the result I get is empty set
select *from appointment where pmail='karuna#gmail.com';(empty set (0.00 sec))
select *from appointment where ddepartment='Surgeon';(empty set (0.00 sec))

group_concat does not show all the values mysql

ModelName.all(:having=>"count(receipt_no)>1",:select=>"school_id,group_concat(id SEPARATOR ',') as f_ids,receipt_no,count(distinct id) as id_count,count(receipt_no) as rec_count",:conditions=>"receipt_no is not null",:group=>"receipt_no")
Output is
+------------+-----------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| receipt_no | school_id | id_count | f_ids | rec_count |
+------------+-----------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| 1261 | 1783 | 2 | 557660,557661 | 2 |
| 14/15- | 1783 | 1209 | 68352,77056,113664,56320,68353,77057,113665,56321,68354,56322,68355,81923,173571,113667,56323,68356,94980,56324,68357,56325,68358,80390,56326,68359,80391,110599,56327,80392,885... | 1209 |
| 15- | 1783 | 112 | 344067,344068,344069,344070,344075,326923,373261,373262,345882,360218,344091,361755,347685,341542,347689,360233,351530,358705,352829,324674,341576,324684,360018,368469,371541,3... | 112 |
Here group_concat does not show all the values but the count of items as same as the count receipt no. Suppose the items in the f_ids column is more than 200 character then its not showing all the values . In other case it will show correct value
I got the solution
SET SESSION group_concat_max_len = 1000000;
Run this code in MySQL console, then this code will change default group_concat character limit to 1000000 characters.
If you want to use in rails console,you can use in this following way
sql = "SET SESSION group_concat_max_len = 1000000"
ActiveRecord::Base.connection.execute(sql)
Please note:
This configuration will work only in that session

SELECT and UPDATE same rows in MYSQL stored procedure

I'm working on a application for which I need to read m rows at a time out of total 'N' rows where m < N. Every time I read 'm' rows I have to set their status as read in the same table.
For example consider following table
+------+-------------------------+--------------------------+----------+-------+---------+------+
| ID | from_email_address | to_email_address | subject | body | inqueue | sent |
+------+-------------------------+--------------------------+----------+-------+---------+------+
| 1 | 0120sushil#gmail.com | kumar.sushil#outlook.com | Subject1 | Body1 | 0 | 0 |
| 2 | 0120ksushil#gmail.com | kumar.sushil#outlook.com | Subject1 | Body1 | 0 | 0 |
| 3 | shivaseth1#gmail.com | kumar.sushil#outlook.com | Subject1 | Body1 | 0 | 0 |
| 4 | shivaseth1#gmail.com | amanrajg#outlook.com | Subject1 | Body1 | 0 | 0 |
| 5 | shivamprakash#gmail.com | amanrajg#outlook.com | Subject1 | Body1 | 0 | 0 |
| 6 | shivamprakash#gmail.com | poorvanagpal#outlook.com | Subject1 | Body1 | 0 | 0 |
| 7 | shivankgupta#gmail.com | poorvanagpal#outlook.com | Subject1 | Body1 | 0 | 0 |
+------+-------------------------+--------------------------+----------+-------+---------+------+
I want to read lets say 3 rows at a time and once I have read the rows I want to set inqueue status of those rows as 1.
I can use following query in stored procedure to select the rows
select * from EmailQueue where inqueue=1 LIMIT 3
After this how to update the same rows and set their inqueue to 1.
EDIT
Here is the stored procedure I created which is giving some error.
DELIMITER $$
DROP PROCEDURE IF EXISTS GetUnsentMails;
CREATE PROCEDURE GetUnsentMails()
BEGIN
START TRANSACTION;
CREATE TEMPORARY TABLE temp_EmailQueue AS SELECT * FROM EmailQueue WHERE inqueue = 0 LIMIT 5 FOR UPDATE;
UPDATE EmailQueue SET inqueue=1 where id in (SELECT id from temp_EmailQueue) AND inqueue = 0;
COMMIT;
END
It gives following error on calling
ERROR 1746 (HY000): Can't update table 'emailqueue' while 'temp_EmailQueue' is being created.
Suggesting to use Transaction and "SELECT FOR UPDATE" to fulfill your requirements.
Please refer following links for the examples:
MySQL 'select for update' behaviour
http://dev.mysql.com/doc/refman/5.0/en/innodb-locking-reads.html
http://www.sqlines.com/mysql/how-to/select-update-single-statement-race-condition
UPDATE - Added the QUERY EXAMPLE:
Example:
.......
#Before starting procedure
.......
START TRANSACTION;
CREATE TEMPORARY TABLE zzz_EmailQueue AS SELECT * FROM EmailQueue WHERE inqueue=1 LIMIT 3 FOR UPDATE;
.....
.....
#Section for other activities....
.....
.....
UPDATE EmailQueue SET inqueue=<<New_Value>> WHERE id IN (SELECT id FROM zzz_EmailQueue) AND inqueue=1;
COMMIT;
.......
#Remaining lines of Prodecure
.......
Update **
**Try with following method:
DECLARE v_EmailQueue_ID DOUBLE;
SELECT ID INTO v_EmailQueue_ID FROM EmailQueue WHERE inqueue = 0 LIMIT 1 FOR UPDATE;
UPDATE EmailQueue SET inqueue=1 WHERE id=v_EmailQueue_ID AND inqueue = 0;

MYSQL. Extract data with column filtered

I got a table like this:
mysql> select * from users;
+--------+----------+------------+-----------+
| userid | username | password | privilege |
+--------+----------+------------+-----------+
| 1 | user1 | password | 1 |
| 2 | david | goodboy | 1 |
| 3 | admin | mastermold | 5 |
| 4 | user4 | password4 | 1 |
| 5 | user5 | password5 | 2 |
| 6 | user6 | password6 | 1 |
| 7 | user7 | password7 | 1 |
+--------+----------+------------+-----------+
7 rows in set
Now, how to extract password of username who is called "david" by the only select query without "password" stored in it and result is in one field. (Don't accept "select * from users")?
Try
select password from users where username='david';
without using mentioning "password"?
well you could grab the column name into a variable and use that like this sqlFiddle
SET #columnName = (SELECT COLUMN_NAME FROM
INFORMATION_SCHEMA.COLUMNS
WHERE TABLE_NAME = 'users'
AND ORDINAL_POSITION = 3);
SET #sql = CONCAT("SELECT ",#columnName," FROM users WHERE username = 'david'");
prepare statement FROM #sql;
execute statement;
Well, you mean that your query result shouldn't have the string "password", if I have got it correctly. Even though I am posting the answer, You should really consider #MarcB comment and try learning SQL first.
Your query should go like this
select `password` from users where username='david'
and lower(`password`) not like '%password%';