I have simple function consist of one sql query
CREATE FUNCTION `GetProductIDFunc`( in_title char (14) )
RETURNS bigint(20)
BEGIN
declare out_id bigint;
select id into out_id from products where title = in_title limit 1;
RETURN out_id;
END
Execution time of this function takes 5 seconds
select Benchmark(500 ,GetProductIdFunc('sample_product'));
Execution time of plain query takes 0.001 seconds
select Benchmark(500,(select id from products where title = 'sample_product' limit 1));
"Title" field is indexed. Why function execution takes so much time and how can I optimize it?
edit:
Execution plan
mysql> EXPLAIN EXTENDED select id from products where title = 'sample_product' limit 1;
+----+-------------+----------+-------+---------------+------------+---------+-------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+----------+-------+---------------+------------+---------+-------+------+----------+-------------+
| 1 | SIMPLE | products | const | Index_title | Index_title | 14 | const | 1 | 100.00 | Using index |
+----+-------------+----------+-------+---------------+------------+---------+-------+------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
mysql> EXPLAIN select GetProductIdFunc('sample_product');
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | NULL | NULL | NULL | NULL | NULL | NULL | NULL | No tables used |
+----+-------------+-------+------+---------------+------+---------+------+------+----------------+
1 row in set (0.00 sec)
This could be a character set issue. If the function is using a different character set than the table column, it would lead to very slow performance despite the index.
Run show create table products\G to determine the character set for the column.
Run show variables like 'character_set%'; to see what the relevant default character sets are for your DB.
Try this:
CREATE FUNCTION `GetProductIDFunc`( in_title char (14) )
RETURNS bigint(20)
BEGIN
declare out_id bigint;
set out_id = (select id from products where title = in_title limit 1);
RETURN out_id;
END
Related
Recently I have considered how Index condition Pushdown work, and whether make a difference of locking in Repeatable read isolation level.
However, I find ICP does not take effect in MySQL 8.0, but work in 5.7. The repro steps as below shown ( based on guide in Index Condition Pushdown Optimization
Prepare table and data
CREATE TABLE `fruit` (
`id` bigint NOT NULL AUTO_INCREMENT,
`name` varchar(32) NOT NULL,
`age` int DEFAULT NULL,
`data` varchar(16) DEFAULT '',
PRIMARY KEY (`id`),
KEY `i_age_name` (`age`,`name`)
) ENGINE=InnoDB;
-- Prepare data
delimiter ;;
CREATE PROCEDURE `load_data`()
BEGIN
DECLARE i int DEFAULT 1;
WHILE i < 300 DO
INSERT INTO fruit (`name`, `age`, `data`) VALUES
(substring(MD5(RAND()),1,20), i / 10 + 1, 'test data'),
(substring(MD5(RAND()),1,20), i / 10 + 1, 'hello world');
SET i = i + 1;
END WHILE;
END ;;
delimiter ;
call load_data();
execute a query
explain select * from fruit where age = 10 and name like '%a';
We hope it Extra column could show Using index condition, but not found, I do not know why?
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-------------+
| 1 | SIMPLE | fruit | NULL | ref | i_age_name | i_age_name | 5 | const | 20 | 11.11 | Using where |
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-------------+
In Mysql 5.7, the query execution plan shows ICP is used
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-----------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-----------------------+
| 1 | SIMPLE | fruit | NULL | ref | i_age_name | i_age_name | 5 | const | 20 | 11.11 | Using index condition |
+----+-------------+-------+------------+------+---------------+------------+---------+-------+------+----------+-----------------------+
My problem is: simple select query takes a long time (3 minutes).
Structure:
mysql> show create table seventhcont_exceptionreport;
seventhcont_exceptionreport | CREATE TABLE `seventhcont_exceptionreport` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`body_html` longtext NOT NULL,
`datetime_created` datetime NOT NULL,
`subject` varchar(256) NOT NULL,
`host` varchar(128) NOT NULL,
`exc_value` varchar(512) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=74607 DEFAULT CHARSET=utf8 |
Rows count:
mysql> select count(*) from seventhcont_exceptionreport;
+----------+
| count(*) |
+----------+
| 7064 |
+----------+
1 row in set (0.00 sec)
Query 1 (normal):
mysql> select id, datetime_created from seventhcont_exceptionreport order by id LIMIT 100 OFFSET 6000;
...
100 rows in set (0.30 sec)
Query 2 (very slow):
mysql> select id, datetime_created from seventhcont_exceptionreport order by id LIMIT 100 OFFSET 7000;
...
63 rows in set (3 min 40.56 sec)
!!! 3 minutes and 40 sec.
Why?
UPDATE
Explain for query 1:
mysql> EXPLAIN select id, datetime_created from seventhcont_exceptionreport order by id LIMIT 100 OFFSET 6000;
+----+-------------+-----------------------------+-------+---------------+---------+---------+------+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------------------+-------+---------------+---------+---------+------+------+-------+
| 1 | SIMPLE | seventhcont_exceptionreport | index | NULL | PRIMARY | 4 | NULL | 6100 | |
+----+-------------+-----------------------------+-------+---------------+---------+---------+------+------+-------+
1 row in set (0.00 sec)
Explain for query 2:
mysql> EXPLAIN select id, datetime_created from seventhcont_exceptionreport order by id LIMIT 100 OFFSET 7000;
+----+-------------+-----------------------------+------+---------------+------+---------+------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------------------+------+---------------+------+---------+------+------+----------------+
| 1 | SIMPLE | seventhcont_exceptionreport | ALL | NULL | NULL | NULL | NULL | 7067 | Using filesort |
+----+-------------+-----------------------------+------+---------------+------+---------+------+------+----------------+
1 row in set (0.00 sec)
UPDATE
Analyze table:
mysql> ANALYZE TABLE seventhcont_exceptionreport;
+--------------------------------+---------+----------+----------+
| Table | Op | Msg_type | Msg_text |
+--------------------------------+---------+----------+----------+
| 7k.seventhcont_exceptionreport | analyze | status | OK |
+--------------------------------+---------+----------+----------+
1 row in set (2.51 sec)
I am no MySQL specialist, but I might be able to point you in the right direction.
In the first query, we can see in the Explain Plan, that an index access was used. In contrast, for the second query, we can see that a non-index access is performed (type index vs ALL). Also, we can see that MySQL is using Using filesort.
This means MySQL cannot perform the sort operation on the index and is therefore performing it on the data itself. This could be because the sort buffer is too small (also see https://www.percona.com/blog/2009/03/05/what-does-using-filesort-mean-in-mysql/).
Therefore, try to increase the size of your sort buffer (soft_buffer_size).
I'm having a curious issue with my MySQL database. The following query runs in 0.003 seconds:
SELECT * FROM `post` where `thread_id` > 12117484 and `index` > -1 limit 1;
If I change the second > to an =, the query doesn't complete (it runs for over a minute):
SELECT * FROM `post` where `thread_id` > 12117484 and `index` = 0 limit 1;
It's worth noting that the result from the first query has index = 0. I know it's bad form to name a column index...but it's the database I've been given. Here's the MySQL Explain for the second query:
+----+-------------+-------+-------+---------------------+---------------------+---------+------+------+------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------------+---------------------+---------+------+------+------------------------------------+
| 1 | SIMPLE | post | range | post_thread_id_idx1 | post_thread_id_idx1 | 5 | NULL | 1 | Using index condition; Using where |
+----+-------------+-------+-------+---------------------+---------------------+---------+------+------+------------------------------------+
This is MariaDB + TokuDB 7.1 community downloaded from Tokutek. Please accept my ignorance if this is normal behavior but I have a question about sorting results. I'm experiencing huge time difference in sorting between the two sort directions - ascending and descending:
SELECT sql_no_cache id, createts, deleted
FROM sort_test
WHERE createts > '2000098'
ORDER BY createts asc
+---------+----------+---------+
| id | createts | deleted |
+---------+----------+---------+
| 1999999 | 2000099 | NULL |
| 2000000 | 2000100 | NULL |
+---------+----------+---------+
2 rows in set (0.00 sec)
SELECT sql_no_cache id, createts, deleted
FROM sort_test
WHERE createts > '2000098'
ORDER BY createts desc
+---------+----------+---------+
| id | createts | deleted |
+---------+----------+---------+
| 2000000 | 2000100 | NULL |
| 1999999 | 2000099 | NULL |
+---------+----------+---------+
2 rows in set (0.55 sec)
Below I present my simplified test case. Here is the table:
CREATE TABLE `sort_test` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`createts` int(11) DEFAULT NULL,
`deleted` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `idx_createts` (`createts`)
) ENGINE=TokuDB
Here I populate the table with 2 million rows using this procedure:
delimiter ;;
drop procedure if exists sort_test_populate;;
create procedure sort_test_populate()
begin
DECLARE int_val INT DEFAULT 1;
myloop : LOOP
if (int_val > 2000000) THEN
LEAVE myloop;
end if;
insert into sort_test (id, createts) values (int_val, int_val+100);
set int_val = int_val +1;
end loop;
end;;
call sort_test_populate();;
Query OK, 1 row affected (28 min 2.80 sec)
Here are my test queries again:
SELECT sql_no_cache id, createts, deleted
FROM sort_test
WHERE createts > '2000098'
ORDER BY createts asc
2 rows in set (0.00 sec)
SELECT sql_no_cache id, createts, deleted
FROM sort_test
WHERE createts > '2000098'
ORDER BY createts desc
2 rows in set (0.55 sec)
And here is the "explain extended" result, it's identical for both queries:
+------+-------------+-----------+-------+---------------+--------------+---------+------+------+----------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+------+-------------+-----------+-------+---------------+--------------+---------+------+------+----------+-------------+
| 1 | SIMPLE | sort_test | range | idx_createts | idx_createts | 5 | NULL | 2 | 100.00 | Using where |
+------+-------------+-----------+-------+---------------+--------------+---------+------+------+----------+-------------+
Please note that this is not my exact data I'm working with, that would be too much to include here. I just wanted to create some test data to demonstrate the problem. My question is - why it's behaving like this and how to make the descending order query faster?
This is a known bug with Index Condition Pushdown (ICP). The workaround is to disable ICP by setting the optimizer_switch either globally or within the session executing this query.
mysql> SET optimizer_switch='index_condition_pushdown=off';
(full disclosure, I'm an employee at Tokutek, makers of TokuDB)
What I'm trying to do is index the first name of a person and the date they were born.
The table is laid out like this:
CREATE TABLE test
(
id INT NOT NULL AUTO_INCREMENT,
fname VARCHAR(10) NOT NULL,
sname VARCHAR(10) NOT NULL,
age INT NOT NULL,
born DATETIME NOT NULL,
PRIMARY KEY(id),
INDEX name_age(fname, sname, age),
INDEX name_date(fname, born)
)
However the index isn't recognised in a where statement like so:
mysql> EXPLAIN SELECT *
-> FROM `test`
-> WHERE fname = "coby"
-> AND born
-> BETWEEN "1900-05-02 06:23:00"
-> AND "2100-05-02 06:23:00";
+----+-------------+-------+------+--------------------+----------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+--------------------+----------+---------+-------+------+-------------+
| 1 | SIMPLE | test | ref | name_age,name_date | name_age | 12 | const | 45 | Using where |
+----+-------------+-------+------+--------------------+----------+---------+-------+------+-------------+
1 row in set (0.00 sec)
However it is recognised in an order by statement:
mysql> EXPLAIN SELECT *
-> FROM `test`
-> WHERE fname = "coby"
-> ORDER BY born;
+----+-------------+-------+------+--------------------+-----------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+--------------------+-----------+---------+-------+------+-------------+
| 1 | SIMPLE | test | ref | name_age,name_date | name_date | 12 | const | 45 | Using where |
+----+-------------+-------+------+--------------------+-----------+---------+-------+------+-------------+
1 row in set (0.00 sec)
How do I make it so that the index is recognised in a where statement?
Any help would be appreciated.
The index is recognized, as seen in the column possible_keys of the result of the first EXPLAIN statement. It just happens, that for 45 rows the other index produces an equal or better query plan: The selectivity of your date range is close to zero.
The ORDER BY is another pair of shoes: As you use the index not only for selection, but also for ordering, it now becomes useful.