Can I make this mySQL query faster? - mysql

Question

I have the following entry in mysql-slow.log:
# Time: 180506 21:57:03
# User#Host: mysqlserver[mysqlserver] # localhost []
# Query_time: 88.963476 Lock_time: 0.000088 Rows_sent: 50 Rows_examined: 114197
SET timestamp=1525633023;
SELECT n1.full_name AS sender_full_name, s1.email AS sender_email, e.subject, e.body,
e.attach, e.date, e.id, r.status, n2.full_name AS receiver_full_name,
s2.email AS receiver_email, r.basket
FROM people_emails p
JOIN email_routing r ON r.receiver_email_id = 3223 AND r.status = 2
JOIN email e ON e.id = r.message_id
JOIN people_emails s1 ON s1.id = r.sender_email_id
JOIN people n1 ON n1.id = s1.people_id
JOIN people_emails s2 ON s2.id = r.receiver_email_id
JOIN people n2 ON n2.id = s2.people_id
WHERE p.internal_user_id = 314
ORDER BY e.date desc
LIMIT 0, 50;
The result of that query is similar to this:
----------------------------------------------------------------------------------------------------
|sender_full_name|sender_email|subject|body| attach | date | id |status|receiver_full_name|basket|
----------------------------------------------------------------------------------------------------
|John Blow |jb#corp.lan |Aloha |Text| |180506|856050|2 |Mary Johns |1 |
----------------------------------------------------------------------------------------------------
Here is all the data about the query and the used tables:
EXPLAIN SELECT n1.full_name AS sender_full_name, s1.email AS sender_email,
e.subject, e.body, e.attach, e.date, e.id, r.status, n2.full_name AS receiver_full_name,
s2.email AS receiver_email, r.basket, 'user777' FROM people_emails p
JOIN email_routing r ON r.receiver_email_id = 3233 AND r.status = 2
JOIN email e ON e.id = r.message_id
JOIN people_emails s1 ON s1.id = r.sender_email_id
JOIN people n1 ON n1.id = s1.people_id
JOIN people_emails s2 ON s2.id = r.receiver_email_id
JOIN people n2 ON n2.id = s2.people_id
WHERE p.internal_user_id = 314 ORDER BY e.date desc LIMIT 0, 50;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE s2 const PRIMARY PRIMARY 4 const 1 Using temporary; Using filesort
1 SIMPLE n2 const PRIMARY PRIMARY 4 const 1
1 SIMPLE p ALL NULL NULL NULL NULL 18631 Using where
1 SIMPLE r ALL NULL NULL NULL NULL 899567 Using where; Using join buffer
1 SIMPLE e eq_ref PRIMARY PRIMARY 4 server.r.message_id 1
1 SIMPLE s1 eq_ref PRIMARY PRIMARY 4 server.r.sender_email_id1
1 SIMPLE n1 eq_ref PRIMARY PRIMARY 4 server.s1.people_id 1
SHOW CREATE TABLE people_emails;
CREATE TABLE `people_emails` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`nick` varchar(255) NOT NULL,
`email` varchar(255) NOT NULL,
`key_name` varchar(255) NOT NULL,
`people_id` int(11) NOT NULL,
`status` int(11) NOT NULL DEFAULT '0',
`activity` int(11) NOT NULL,
`internal_user_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
FULLTEXT KEY `email` (`email`)
) ENGINE=MyISAM AUTO_INCREMENT=22114 DEFAULT CHARSET=utf8
SHOW CREATE TABLE email_routing;
CREATE TABLE `email_routing` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`message_id` int(11) NOT NULL,
`sender_email_id` int(11) NOT NULL,
`receiver_email_id` int(11) NOT NULL,
`basket` int(11) NOT NULL,
`status` int(11) NOT NULL,
`popup` int(11) NOT NULL,
`tm` int(11) NOT NULL,
KEY `id` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=987389 DEFAULT CHARSET=utf8
SHOW CREATE TABLE email;
CREATE TABLE `email` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`subject` text NOT NULL,
`body` text NOT NULL,
`date` datetime NOT NULL,
`attach` text NOT NULL,
`attach_ondisk` text NOT NULL,
`attach_dir` varchar(255) CHARACTER SET cp1251 DEFAULT NULL,
`attach_subject` varchar(255) DEFAULT NULL,
`attach_content` longtext,
PRIMARY KEY (`id`),
KEY `Index_2` (`attach_dir`),
FULLTEXT KEY `path` (`attach_dir`)
) ENGINE=MyISAM AUTO_INCREMENT=856151 DEFAULT CHARSET=utf8
SHOW CREATE TABLE people;
CREATE TABLE `people` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`fname` varchar(255) CHARACTER SET cp1251 NOT NULL,
`lname` varchar(255) CHARACTER SET cp1251 NOT NULL,
`patronymic` varchar(255) CHARACTER SET cp1251 NOT NULL,
`gender` tinyint(1) NOT NULL,
`full_name` varchar(255) NOT NULL DEFAULT ' ',
`category` int(11) NOT NULL,
`people_type_id` int(255) DEFAULT NULL,
`tags` varchar(255) CHARACTER SET cp1251 NOT NULL,
`job` varchar(255) CHARACTER SET cp1251 NOT NULL,
`post` varchar(255) CHARACTER SET cp1251 NOT NULL,
`profession` varchar(255) CHARACTER SET cp1251 DEFAULT NULL,
`zip` varchar(16) CHARACTER SET cp1251 NOT NULL,
`country` int(11) DEFAULT NULL,
`region` varchar(10) NOT NULL,
`city` varchar(255) CHARACTER SET cp1251 NOT NULL,
`address` varchar(255) CHARACTER SET cp1251 NOT NULL,
`address_date` date DEFAULT NULL,
`inner` tinyint(4) NOT NULL,
`contact_through` varchar(255) DEFAULT '',
`next_call` date NOT NULL,
`additional` text CHARACTER SET cp1251 NOT NULL,
`user_id` int(11) NOT NULL,
`changed` datetime NOT NULL,
`status` int(11) DEFAULT NULL,
`nick` varchar(255) DEFAULT NULL,
`birthday` date DEFAULT NULL,
`last_update_ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`area` text NOT NULL,
`reviewed_` tinyint(4) NOT NULL,
`phones_old` text NOT NULL,
`post_sticker` text NOT NULL,
`permissions` int(120) NOT NULL DEFAULT '0',
`internal_user_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `most_used` (`category`,`status`,`city`,`lname`,`next_call`),
KEY `registrars` (`category`,`status`,`contact_through`,`next_call`),
FULLTEXT KEY `lname` (`lname`),
FULLTEXT KEY `fname` (`fname`),
FULLTEXT KEY `mname` (`patronymic`),
FULLTEXT KEY `Full Name` (`full_name`)
) ENGINE=MyISAM AUTO_INCREMENT=415009 DEFAULT CHARSET=utf8
While getting the above output as per request from the comment I have also noticed that my tables are all in different format - MyISAM and InnoDB. Can that be a part of the problem too?
Did I made the tables structure too complicated? I would like to understand what part of the query makes this so slow so I can re-arrange my tables.

Answer 1

In general, you want to eliminate the entries from your EXPLAIN report where type=ALL. This means it's doing a table-scan, and that's bad for performance if it happens on a large table.
In your case, you have two tables that are doing table-scans. Check the numbers in the row column of the explain, 18631 and 899567. Multiply them together = 16,759,832,777. That's how many row combinations the query will potentially examine!
Part of the problem is that your query is doing a Cartesian product. You have no conditions relating your table p to the other tables. So for every row examined in p, it combines this with the rows examined in other tables. This has a very high cost.
It's not clear why you even have p in your query, since it's not related to the other tables, and you don't fetch any columns from it in the select-list. I can produce the result set you described even when I take p out of the query:
SELECT n1.full_name AS sender_full_name, s1.email AS sender_email,
e.subject, e.body, e.attach, e.date, e.id, r.status, n2.full_name AS receiver_full_name,
s2.email AS receiver_email, r.basket, 'user777'
FROM email_routing r
JOIN email e ON e.id = r.message_id
JOIN people_emails s1 ON s1.id = r.sender_email_id
JOIN people n1 ON n1.id = s1.people_id
JOIN people_emails s2 ON s2.id = r.receiver_email_id
JOIN people n2 ON n2.id = s2.people_id
WHERE r.receiver_email_id = 3233 AND r.status = 2
ORDER BY e.date desc LIMIT 0, 50;
I also suggest adding this index:
ALTER TABLE email_routing ADD KEY bk1 (receiver_email_id, status,
sender_email_id, message_id, basket);
This helps the search for r.receiver_email_id = 3233 AND r.status = 2.
The additional columns are in the index to make it a covering index. This means the query doesn't have to read the email_routing table at all, if it gets all the columns it needs from the index.
EXPLAIN for this query looks better, now that none of the tables are doing type=ALL, and one of them shows "Using index" which is the indicator of the covering index.
+----+-------------+-------+--------+---------------+---------+---------+------------------------+------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+------------------------+------+---------------------------------+
| 1 | SIMPLE | s2 | const | PRIMARY | PRIMARY | 4 | const | 1 | Using temporary; Using filesort |
| 1 | SIMPLE | n2 | const | PRIMARY | PRIMARY | 4 | const | 1 | NULL |
| 1 | SIMPLE | r | ref | bk1 | bk1 | 8 | const,const | 1 | Using index |
| 1 | SIMPLE | s1 | eq_ref | PRIMARY | PRIMARY | 4 | test.r.sender_email_id | 1 | NULL |
| 1 | SIMPLE | n1 | eq_ref | PRIMARY | PRIMARY | 4 | test.s1.people_id | 1 | NULL |
| 1 | SIMPLE | e | eq_ref | PRIMARY | PRIMARY | 4 | test.r.message_id | 1 | NULL |
+----+-------------+-------+--------+---------------+---------+---------+------------------------+------+---------------------------------+
P.S.: MyISAM vs. InnoDB makes little difference for this query optimization. The index will help a lot for both storage engines. But I always recommend to convert to InnoDB (see my answer to MyISAM versus InnoDB).

Answer 2

This looks wrong:
FROM people_emails p
JOIN email_routing r ON r.receiver_email_id = 3223
AND r.status = 2
p is not used in any ON clauses. Perhaps you are missing so way to tie p and r together? Without it, you have a "cross join". If there are 1K rows in each, you end up with 1M rows in the join.
Also, please use ON for showing how tables relate; use WHERE for filtering (3222 & 2).