MySQL Extremely Slow Query - mysql

Question

I am trying to learn about optimizations to MySQL, table engines and when to use them, etc.
I have a query that is running up against the time-out limit of 10 minutes and which needs to complete in seconds because its function is a user-generated report.
The Query:
SELECT em.employeeId, tsk.taskId
FROM employee em INNER JOIN
task tsk
ON tsk.employeeId = em.employeeId
WHERE em.employeeId <> 'Not Done'
AND tsk.employeeId (
SELECT employeeId FROM task
WHERE templateId
IN ( '5', '6', '7', '8' )
AND tsk.status = 'Done'
)
AND tsk.employeeId IN
(
SELECT employeeId FROM task
WHERE templateId IN
( '55', '56', '57', '58' )
AND status = 'Not Done'
)
Explain:
# id, select_type, table, type, possible_keys, key, key_len, ref, rows, Extra
1, PRIMARY, tsk, ALL, , , , , 61326, Using where
1, PRIMARY, em, eq_ref, PRIMARY, PRIMARY, 4, newhire.tsk.employeeId, 1, Using index
3, DEPENDENT SUBQUERY, task, ALL, , , , , 61326, Using where
2, DEPENDENT SUBQUERY, task, ALL, , , , , 61326, Using where
The DB server uses MyISAM as default, so most schemas including this one are MyISAM.
I also realize that the text searches (status=Done or status LIKE 'Done') are adding a lot to the query.
EDIT1:
# Table, Create Table
employee, CREATE TABLE `employee` (
`employeeId` int(10) unsigned NOT NULL AUTO_INCREMENT,
`lastName` varchar(255) NOT NULL,
`firstName` varchar(255) NOT NULL,
`applicantId` varchar(255) NOT NULL,
`fEmployeeId` varchar(255) DEFAULT NULL,
`opId` varchar(255) DEFAULT NULL,
`rehire` tinyint(3) unsigned NOT NULL DEFAULT '0',
`sDate` date DEFAULT NULL,
`oDate` date DEFAULT NULL,
`cDate` date DEFAULT NULL,
`additionalDate` date DEFAULT NULL,
`additionalType` varchar(255) DEFAULT NULL,
`processingDate` date DEFAULT NULL,
`created` datetime NOT NULL,
`recruiterId` int(10) unsigned NOT NULL,
`processorId` int(10) unsigned DEFAULT NULL,
`position` tinyint(3) unsigned NOT NULL DEFAULT '1',
`status` varchar(255) NOT NULL,
`campus` varchar(255) DEFAULT NULL,
`phone` varchar(255) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`requisition` varchar(255) DEFAULT NULL,
`Position` varchar(255) DEFAULT NULL,
`department` varchar(255) DEFAULT NULL,
`jobClass` varchar(255) DEFAULT NULL,
`hiringManager` varchar(255) DEFAULT NULL,
`badge` varchar(255) DEFAULT NULL,
`currentAddress` varchar(255) DEFAULT NULL,
`holding` tinyint(3) unsigned DEFAULT '0',
PRIMARY KEY (`employeeId`)
) ENGINE=MyISAM AUTO_INCREMENT=3959 DEFAULT CHARSET=latin1
EDIT 2:
# Table, Create Table
task, CREATE TABLE `task` (
`taskId` int(10) unsigned NOT NULL AUTO_INCREMENT,
`templateId` int(10) unsigned NOT NULL,
`employeeId` int(10) unsigned NOT NULL,
`name` varchar(255) NOT NULL,
`description` text,
`naAvailable` tinyint(3) unsigned DEFAULT '0',
`fileRequired` tinyint(3) unsigned DEFAULT '0',
`fileHrCatalog` int(10) unsigned DEFAULT NULL,
`quickFileName` varchar(255) DEFAULT NULL,
`fileUploaded` tinyint(3) unsigned DEFAULT '0',
`fileExt` varchar(255) DEFAULT NULL,
`level` tinyint(3) unsigned NOT NULL,
`status` varchar(255) NOT NULL,
`due` date DEFAULT NULL,
`daysDue` int(10) unsigned DEFAULT NULL,
`routeIncentives` tinyint(3) unsigned DEFAULT '0',
`requiresAudit` tinyint(3) unsigned DEFAULT '0',
`auditStatus` varchar(255) DEFAULT NULL,
`auditUser` int(10) unsigned DEFAULT NULL,
`auditDate` datetime DEFAULT NULL,
`stampOption` tinyint(3) unsigned DEFAULT '0',
`done` tinyint(3) unsigned DEFAULT '0',
`doneBy` int(10) unsigned DEFAULT NULL,
`doneWhen` datetime DEFAULT NULL,
`sortOrder` tinyint(3) unsigned NOT NULL DEFAULT '255',
PRIMARY KEY (`taskId`),
KEY `status` (`status`,`templateId`)
) ENGINE=MyISAM AUTO_INCREMENT=176802 DEFAULT CHARSET=latin1

Answer 1

I would write the query as below, but to help the optimization, have a covering indexes on your tables.
Employee table -- index on ( status, employeeID )
Task table -- index on ( employeeid, templateid, status )
By the first join, you are prequalifying to get the first task as a "Done" status.
The second join is looking for the OTHER task you are interested in that is NOT Done.
Doing subqueries (especially correlated sub queries) can be harder on performance. By doing a JOIN, it's either there or its not...
SELECT
em.employeeId,
tsk.taskId
FROM
employee em
INNER JOIN task tsk1
ON em.employeeId = tsk1.employeeId
AND tsk1.templateID in ( '5', '6', '7', '8' )
AND tsk1.status = 'Done'
INNER JOIN task tsk2
ON em.employeeId = tsk2.employeeId
AND tsk2.templateID in ( '55', '56', '57', '58' )
AND tsk2.status = 'Not Done'
WHERE
em.status <> 'Not Done'

Answer 2

Your first change should be to create an index on task that covers both the status and templateId columns:
ALTER TABLE task ADD INDEX (status, templateId);
That'll prevent the full-table scans of 61326 rows each time that table is accessed in your query.
Also, it looks like you might have made a typo here:
SELECT employeeId FROM task
WHERE templateId
IN ( '5', '6', '7', '8' )
AND tsk.status = 'Done'
That tsk.status should be just status like the 2nd subquery.