Abstruse difference in query speed - mysql

Question

I do not understand the difference (line 2) of those two EXPLAINs. Maybe someone has a hint for me why mysql acts so different on those, which heavily affects query speed.
The slow query lasts 12 seconds (which equals querying all rows with that query) and uses a join on integer columns while the joined table has just 3 records:
SELECT `inv_assets`.`id` AS `id`, `site`.`description` AS `sitename`,
(SELECT COALESCE(DATE_FORMAT(CONVERT_TZ(MIN(inspdate),'UTC','Europe/Vienna'),'%Y-%m-%d'),'')
FROM `mobuto_inv_inspections` AS `nextinsp`
WHERE ((`nextinsp`.`objectlink` = `inv_assets`.`id`
AND `nextinsp`.`inspdate` >= NOW()))
) AS `nextinsp`
FROM `mobuto_inv_assets` AS `inv_assets`
LEFT JOIN `mobuto_inv_sites` AS `site`
ON (`site`.`siteid` = `inv_assets`.`site`
AND `site`.`_state` IN (2,0))
ORDER BY `inv_assets`.`type` ASC LIMIT 0, 20;
+----+--------------------+------------+--------+----------------+---------+---------+------------------------------+-------+----------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+--------+----------------+---------+---------+------------------------------+-------+----------------------------------------------------+
| 1 | PRIMARY | inv_assets | ALL | NULL | NULL | NULL | NULL | 24857 | Using temporary; Using filesort |
| 1 | PRIMARY | site | ALL | PRIMARY,_state | NULL | NULL | NULL | 3 | Using where; Using join buffer (Block Nested Loop) |
| 2 | DEPENDENT SUBQUERY | nextinsp | ALL | inspdate | NULL | NULL | NULL | 915 | Using where |
+----+--------------------+------------+--------+----------------+---------+---------+------------------------------+-------+----------------------------------------------------+
The fast query consumes just a few fractions of a second, uses a join on varchar(32) columns and the joined table has 1352 records:
SELECT `inv_assets`.`id` AS `id`, `guarantor`.`lastname` AS `guarantoruname`,
(SELECT COALESCE(DATE_FORMAT(CONVERT_TZ(MIN(inspdate),'UTC','Europe/Vienna'),'%Y-%m-%d'),'')
FROM `mobuto_inv_inspections` AS `nextinsp`
LEFT JOIN `users` AS `saveuser`
ON (`saveuser`.`uid` = `nextinsp`.`saveuser`
AND `saveuser`.`_state` = '0')
WHERE ((`nextinsp`.`objectlink` = `inv_assets`.`id`
AND `nextinsp`.`inspdate` >= NOW()))
) AS `nextinsp`
FROM `mobuto_inv_assets` AS `inv_assets`
LEFT JOIN `users` AS `guarantor`
ON (`guarantor`.`uid` = `inv_assets`.`guarantor`
AND `guarantor`.`_state` = '0')
ORDER BY `inv_assets`.`type` ASC LIMIT 0, 20;
+----+--------------------+------------+--------+----------------+---------+---------+---------------------------------+-------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+--------+----------------+---------+---------+---------------------------------+-------+----------------+
| 1 | PRIMARY | inv_assets | ALL | NULL | NULL | NULL | NULL | 24857 | Using filesort |
| 1 | PRIMARY | guarantor | eq_ref | PRIMARY,_state | PRIMARY | 98 | mobuto_dev.inv_assets.guarantor | 1 | Using where |
| 2 | DEPENDENT SUBQUERY | nextinsp | ALL | inspdate | NULL | NULL | NULL | 915 | Using where |
| 2 | DEPENDENT SUBQUERY | saveuser | eq_ref | PRIMARY,_state | PRIMARY | 98 | mobuto_dev.nextinsp.saveuser | 1 | Using where |
+----+--------------------+------------+--------+----------------+---------+---------+---------------------------------+-------+----------------+
The strange thing to me is, when I remove the column (description) of the joined table in the 'column-select-part' (while the join still persists and IMHO mysql does not optimize it away when not used), the speed is back (because mysql does not use a temporary table any longer and the explain looks same as the fast one, having type=eq_ref).
But why does this work for the first sample only when no column selected, whereas I can select one in the second one!?
CREATE TABLE `mobuto_inv_assets` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`invnum` varchar(10) NOT NULL,
`oebglcat` varchar(4) NOT NULL,
`mark` varchar(100) NOT NULL,
`type` varchar(100) NOT NULL,
`serialnum` varchar(100) NOT NULL,
`desc` varchar(100) NOT NULL,
`site` int(11) NOT NULL DEFAULT '0',
`licnum` varchar(20) NOT NULL DEFAULT '',
`inquirer` varchar(100) NOT NULL DEFAULT '',
`inqdate` date NOT NULL DEFAULT '0000-00-00',
`supplier` varchar(100) NOT NULL DEFAULT '',
`suppldate` date NOT NULL DEFAULT '0000-00-00',
`supplnumber` varchar(30) NOT NULL DEFAULT '',
`invoicedate` date NOT NULL DEFAULT '0000-00-00',
`invoicenumber` varchar(30) NOT NULL DEFAULT '',
`purchaseprice` decimal(11,2) NOT NULL DEFAULT '0.00',
`leased` varchar(1) NOT NULL DEFAULT 'N',
`leasingcompany` varchar(100) NOT NULL DEFAULT '',
`leasingnumber` varchar(30) NOT NULL DEFAULT '',
`notes` text NOT NULL,
`inspnotes` text NOT NULL,
`inactive` varchar(1) NOT NULL DEFAULT 'N',
`maintain` varchar(1) NOT NULL DEFAULT 'Y',
`asset` varchar(1) NOT NULL DEFAULT 'Y',
`inspection` varchar(1) NOT NULL DEFAULT '',
`inspperson` varchar(100) NOT NULL DEFAULT '',
`guarantor` varchar(32) NOT NULL DEFAULT '',
`saveuser` varchar(32) NOT NULL,
`savetime` int(11) NOT NULL,
`recordid` varchar(32) NOT NULL,
`_state` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `invnum` (`invnum`),
KEY `_state` (`_state`),
KEY `site` (`site`)
) ENGINE=InnoDB AUTO_INCREMENT=30707 DEFAULT CHARSET=utf8;
CREATE TABLE `mobuto_inv_sites` (
`siteid` int(11) NOT NULL AUTO_INCREMENT,
`description` varchar(100) NOT NULL,
`saveuser` varchar(32) NOT NULL,
`savetime` int(11) NOT NULL,
`recordid` varchar(32) NOT NULL,
`_state` int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (`siteid`),
KEY `_state` (`_state`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8;
mysql> SHOW INDEX FROM mobuto_inv_assets;
+-------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| mobuto_inv_assets | 0 | PRIMARY | 1 | id | A | 24857 | NULL | NULL | | BTREE | | |
| mobuto_inv_assets | 0 | invnum | 1 | invnum | A | 24857 | NULL | NULL | | BTREE | | |
| mobuto_inv_assets | 1 | _state | 1 | _state | A | 4 | NULL | NULL | | BTREE | | |
+-------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Changes as requested by #Wilson Hauck:
Added index to column site in mobuto_inv_assets (reduced execution speed by almost half a second)
Seems that column nextinsp was missing in first query. Maybe lost while formatting the query. Of course there should be the same as in the fast one
Removed the saveuser join as it is not used there (saved another 2 seconds) and updated its EXPLAIN (last line removed)
SHOW INDEX FROM mobuto_inv_sites added
+------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| mobuto_inv_sites | 0 | PRIMARY | 1 | siteid | A | 3 | NULL | NULL | | BTREE | | |
| mobuto_inv_sites | 1 | _state | 1 | _state | A | 3 | NULL | NULL | | BTREE | | |
+------------------+------------+----------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+

Answer 1

Your first query is making less use of keys than the second. The possible_keys column in the explain plan shows where keys are available to be used, however, the key column shows where they are actually being used.
I would advise, short of seeing the structure of your DB, to make more use of these keys in your JOIN and WHERE clauses to speed it up.
I'd make sure that the query isn't being cached when you say you're modifying the select columns and the speed is varying.

Answer 2

12 seconds first query likely caused by ROWS column clues of simply 24857*3*915*1 = 68,232,465 total rows considered. Less than 1 second for second query ROWS column clues of simply 24857*1*915*1 = 22,744,155 total rows considered. The first query's use of Block Nested Loop processing is a major contributor to delaying the response.
Please post results of SHOW CREATE TABLE mobuto_inv_assets and mobuto_inv_sites. Please also post results of SHOW INDEX FROM mobuto_inv_assets and mobuto_inv_sites. With this additional information someone may be able to suggest improvements to SELECT .... queries that will avoid Block Nested Loop processing that is very time CPU intense with RBAR (Row By Agonizing Row processing). Additional indexing may be required.

Answer 3

Thanks for posting your two SHOW CREATE TABLE's, immensely helpful.
Please consider adding index with
ALTER TABLE mobuto_inv_sites ADD INDEX site --
if space permits on your system.
Also, the EXPLAIN showing for query1 is mismatched to the query.
The query does not refer to nextinsp or saveused that I can see in EXPLAIN.
Please replace the EXPLAIN for query1 after creating the index when you have an opportunity to test again and indicate any reduction in execution time required.
It would also be nice if you could post results of
SHOW INDEX FROM mobuto_inv_sites so we can see the scope of your data and cardinality.

Answer 4

If the inv_assets rows are populated with ACCURATE _state data
consider changing query1 to something like the following:
SELECT inv_assets.id AS id, site.description AS sitename,
(SELECT COALESCE(DATE_FORMAT(CONVERT_TZ(MIN(inspdate),'UTC','Europe/Vienna'),'%Y-%m-%d'),'')
FROM mobuto_inv_inspections AS nextinsp
WHERE ((nextinsp.objectlink = inv_assets.id
AND nextinsp.inspdate >= NOW()))
) AS nextinsp
FROM mobuto_inv_assets AS inv_assets
WHERE inv_assets._state = 2 OR inv_assets._state = 0
LEFT JOIN mobuto_inv_sites AS site
ON (site.siteid = inv_assets.site
AND site._state IN (2,0))
ORDER BY inv_assets.type ASC LIMIT 0, 20;
EXPLAIN should avoid table scan and subsequent Block Nested Loop processing.
If _state data in inv_assets is not ACCURATE on every row, this will not work.
2017-08-10 update 09:42 CT please post QUERY, EXPLAIN result, SHOW CREATE TABLE tblname for tables involved and SHOW INDEX FROM tblname for tables involved.