MySql Innodb table strange SELECT lock / timeout - mysql

I'm having a strange problem with a SELECT in an Innodb table, it never returns, I waited more than two hours to see if I get the results but no, still waiting.
CREATE TABLE `example` (
`id` int(11) NOT NULL,
`done` tinyint(2) NOT NULL DEFAULT '0',
`agent` tinyint(4) NOT NULL DEFAULT '0',
`text` varchar(256) NOT NULL,
PRIMARY KEY (`id`),
KEY `da_idx` (`done`,`agent`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
The query that I can't obtain the results is:
SELECT id, text FROM example WHERE done = 0 AND agent = 0 LIMIT 120;
First I thought in some index optimization or lock problem, I was some time researching that but then I found this:
SELECT id FROM example WHERE done = 0 AND agent = 0 LIMIT 120;
...
...
...
120 rows in set (0.27 sec)
SELECT text FROM example WHERE done = 0 AND agent = 0 LIMIT 120;
...
...
...
120 rows in set (0.83 sec)
Now I'm lost, how obtaining id or text column separately with the exactly same query (same WHERE and LIMIT) works perfect and then obtaining both of them not??
Executing the "SELECT id, text..." again after that two queries have the same effect, never returns.
Any help is appreciated, an Innodb guru could help ;)
Added information:
Doesn't look like a transaction lock problem, look at the exponential increase of the response times for the next queries:
SELECT id, text FROM example WHERE done = 0 AND agent = 0 limit 109;
...
109 rows in set (0.31 sec)
SELECT id, text FROM example WHERE done = 0 AND agent = 0 limit 110;
...
110 rows in set (3.98 sec)
SELECT id, text FROM example WHERE done = 0 AND agent = 0 limit 111;
...
111 rows in set (4 min 5.00 sec)

I found the solution to my own question and I want to share here because it was quite strange, at least for me.
I always thought that the time delay that the mysql client returns after each query (for example "120 rows in set (0.27 sec)") was the time that takes to the mysql server to generate that result, but not.
The problem was a network problem! With no relation at all with the mysql server!
So, I found, opposed to what I always thought, that the time after each query showed by the mysql client includes the network delay. The same requests at the same time from a server in the same datacenter as the mysql server and another from another country returns a different time(considerabily different).

Related

Process getting too many Lock wait timeout exceeded errors

So I'm working on a project that have the following schema on a MySQL 5.7 database:
CREATE TABLE `voucher` (
`id` varchar(36) NOT NULL,
`voucher_status_id` int(11) NOT NULL,
`situation_code` varchar(30) DEFAULT NULL,
`organization_id` int(11) DEFAULT NULL,
`code` varchar(15) NOT NULL,
`authorization_code` varchar(7) NOT NULL
.. other columns omitted for brevity
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
This table have ~ 2M rows and stores a pool of Vouchers.
The system get vouchers with voucher_status_id = 1 to sell.
The sale process is basically:
The user inputs the quantity X of Vouchers he wants to buy.
The application query the database with this select: SELECT v.* FROM voucher v where v.voucher_status_id = ? and v.organization_id = ? limit X for update
Then do the rest (update the voucher status, etc), and commit the transaction.
Things to notice: there's no SKIP LOCKED feature available, because this application is using MySQL 5.7; this pool of vouchers exist because the process of generating a voucher code is complex and made by a scheduled process on another system that load this table; the id column is a UUID v4
As you noticed, the select statement from above does lock some rows and it causes a lot of problems to the other sales concurrently happening and often (really) inccurs in Lock wait timeout exceeded errors.
By reading MySQL documentation, I found that it locks all the rows it has to scan while doing select ... for update statements.
A SELECT ... FOR UPDATE reads the latest available data, setting
exclusive locks on each row it reads. Thus, it sets the same locks a
searched SQL UPDATE would set on the rows.
The application is getting tons of Lock wait timeout exceeded errors. How can this be avoided in this case?
Another dev proposed randomly select rows by using ORDER BY RAND(), but it would end up with the same problem as a test I made clearly showed us:
Transaction A:
begin;
select * from voucher v where v.voucher_status_id = 1
and organization_id = 5
order by rand()
limit 5
for update;
Transaction B:
begin;
select * from voucher v where v.voucher_status_id = 1
and organization_id = 5
order by rand()
limit 10
for update;
Transaction B gets locked until Transaction A is commited.
How would you solve such situation?

Mysql Strange auto increment increase.

Yesterday I found one problem on my project. For some reason for usual insert mysql did increase auto increment from 8 symbols to 10. In binlogs I found this
SET INSERT_ID=2147483646/*!*/;
# at 2638426
#140514 18:49:36 server id 31245 end_log_pos 2638810 Query thread_id=178500933 exec_time=0 error_code=0
SET TIMESTAMP=1400093376/*!*/;
INSERT INTO deals SET NAME = '###', PRICE = 125
But it must be around 26513863
ID field is: `ID` int(10) NOT NULL AUTO_INCREMENT
Table type: InnoDB
Mysql version: 5.5.31
Maybe someone know how can it be, or have any ideas?
A failed insert can still cause the auto-increment column to increase. If your program went into an infinite loop of failures it could cause the limit to be reached.
It's also possible to set the auto-increment programmatically to a specific value.
ALTER TABLE yourtable AUTO_INCREMENT = 12345;
from "Mark Byers" in this question stackoverflow

SQL syntax working on local server but not on my live server?

I'm trying to run a SQL query to update a table if an ID does not exists in another table.
Heres the query:
UPDATE product
SET active = 0
WHERE id NOT IN (SELECT product_id FROM image);
This query works as intended on my local server(running apache/mysql).
But on my dedicated server(also running apache/mysql), it just returns this:
Query OK, 0 rows affected (0.19 sec)
Rows matched: 0 Changed: 0 Warnings: 0
But I know it should effect some rows.
I've double checked everything and yet can't see why the query wont go through correctly.
Is there some kind of setting I'm missing out perhaps?
Kind regards,
Daniel
Do you have some rows where product_id is null? You must filter these out when using NOT IN:
UPDATE product
SET active = 0
WHERE id NOT IN (SELECT product_id FROM image WHERE product_id IS NOT NULL);
The reason is that x NOT IN y can only be false or NULL but never true if y contains a NULL value.
You could also use one of the other approaches to find values that occur in one table but not in another. For example, using EXISTS:
UPDATE product
SET active = 0
WHERE NOT EXISTS (SELECT * FROM image WHERE product_id = product.id);
This approach does not have the same problem with NULL values.

Why does MySQL allow to update a NOT NULL column to NULL?

I'm running MySql in ubuntu 10.10. I created a table called 'employee' having 3 field names empno, name and salary. Inserted few entities. In the middle of the process i want to change salary attribute as 'NOT NULL'. I Alter the table as
ALTER TABLE employee MODIFY salary int(10) NOT NULL;
Query executed. I wanted to test by using command,
UPDATE employee SET salary=NULL;
Query OK, 15 rows affected, 15 warnings (0.06 sec)
Rows matched: 15 Changed: 15 Warnings: 15
also gave warnings " (Code 1048): Column 'salary' cannot be null "(Repeated for every row)
But when i saw my table , All salaries were Zeros('0').
Same queries result in error instead of warning in WINDOWS XP's MySql
I checked in both INNODB and MYISAM engines but same Result.
Please help me to know what happened beside processing.
You must not have SQL_MODE set to strict on you ubuntu installation.
Issue
SET SQL_MODE='STRICT_ALL_TABLES'
or add
SQL_MODE='STRICT_ALL_TABLES'
under [mysqld] to your my.cnf on Ubuntu.
I don't see the problem, you set the column to NOT NULL, (which doesn't allow NULL values) and now it won't let you set it to NULL, which would be the expected behaviour.
The reason you have 0s in your DB is because 0 would be the result of casting NULL to an int.

How do I log just the raw queries in MySQL?

I have my log file working, but I get extraneous info on each line like "29 Query", and I can't tell, but it looks like the queries logged are the interpretation of how MySQL treats each query internally. Is there a way to automatically log each query as they were executed by the application without any additional information added to the log by MySQL? Thanks!
EDIT:
As a part of offering the bounty, let me explain my situation. We're using Magento Commerce, which has an EAV database architecture. Tracking anything down, and where it is stored is an absolute nightmare. My thought was to insert a product into the database in the application, and then log every query that was executed during that process. This worked well, but the logs have a ton of other cruft around the queries. I really do just want something like this:
1.) SELECT * FROM <TABLE>;
2.) UPDATE <TABLE> SET <VALUE> = <VALUE>;
3.) ...
4.) ...
Something simple that tells me what was executed so that I don't have to go sifting through controllers and models to try and get all this. I don't need dates, times, line numbers or anything extra.
To enable full Log Query add the following to your my.cnf:
log=/var/log/mysqldquery.log
The above will log all queries to the log file.
Don't forgot to restart mysql service after making changes in my.cnf file.
Example output from actions via SequelPro (mac client):
090721 11:06:45 51 Query ALTER TABLE `test` ADD `name` varchar(10) DEFAULT NULL
51 Query SHOW COLUMNS FROM `test`
51 Query SHOW INDEX FROM `test`
090721 11:06:57 51 Query SHOW COLUMNS FROM `test`
51 Query UPDATE `test` SET `id`='1', `name`='test' WHERE `id` = '1' AND `name` IS NULL LIMIT 1
51 Query SELECT * FROM `test` LIMIT 0,100
51 Query SELECT COUNT(1) FROM `test`
090721 11:07:00 51 Query UPDATE `test` SET `id`='2', `name`='test' WHERE `id` = '2' AND `name` IS NULL LIMIT 1
51 Query SELECT * FROM `test` LIMIT 0,100
51 Query SELECT COUNT(1) FROM `test`
On *NIX based systems you can use grep to start
grep 'SELECT\|INSERT\|UPDATE' querylog.log
Or get more tricky and start doing things like:
grep 'SELECT\|INSERT\|UPDATE' querylog.log | awk '{$1="";$2="";print}'
This would give you something like this, not perfect but closer:
51 Query UPDATE `test` SET `id`='2', `name`='test' WHERE `id` = '2' AND `name` IS NULL LIMIT 1
SELECT * FROM `test` LIMIT 0,100
SELECT COUNT(1) FROM `test`
51 Query INSERT INTO `test` (`id`,`name`) VALUES ('3','testing')
SELECT * FROM `test` LIMIT 0,100
SELECT COUNT(1) FROM `test`
You could use the mysql query log file. Add this parameter when you start mysql:
--log=/var/log/mysqld.log
If you are referring to the binary log, you need to use mysqlbinlog to pass it through to get meaningful output.
cat log100.log | mysqlbinlog
This any help to you?
http://www.bigdbahead.com/?p=99
There's 2 solutions there - one is easier but requires mysql 5.1+.