MYSQL query is taking forever - mysql

Question

I have a report that needs running to satisfy our reporting requirements for a government body. The report is supposed to return the study load for each student in each module for a given period of time.
For example the report needs to return the students enrolled in a given module for a given intake in a given year and semester, with a census date (a government specified date that after which the student is liable for the cost of the unit even if they withdraw)
So I've written this mysql query
SELECT
e.enrolstudent AS '313',
(SELECT c.ntiscode FROM course c WHERE c.courseid=ec.courseid) AS '307',
e.startdate as '534',
'AOU' as '333',
m.mod_eftsl as '339',
e.enrolmod as '354',
e.census_date as '489',
m.diciplinecode as '464',
(CASE
WHEN m.mode = 'Face to Face' THEN 1
WHEN m.mode = 'Online' THEN 2
WHEN m.mode = 'RPL' THEN 5
ELSE 3
END) AS '329',
'A6090' as '477',
up.citizen AS '358',
vf.maxcontribute as '392',
vf.studentstatus as '490',
vf.total_amount_charged as '384',
vf.amount_paid as '381',
vf.loan_fee as '529',
u.chessn as '488',
m.workexp as '337',
'0' as '390',
m.sumwinschool as '551',
vf.help_debt as '558'
FROM
enrolment e
INNER JOIN enrolcourse AS ec ON ec.studentid=e.enrolstudent
INNER JOIN vetfee AS vf ON vf.userid=e.enrolstudent
INNER JOIN users AS u ON u.userid = e.enrolstudent
INNER JOIN users_personal AS up ON up.userid = e.enrolstudent
INNER JOIN module AS m ON m.modshortname = e.enrolmod
WHERE
e.online_intake in (select oi.intakecode from online_intake oi where STR_TO_DATE(oi.censusdate,'%d-%m-%Y') > '2015-07-01' and STR_TO_DATE(oi.censusdate,'%d-%m-%Y') < '2015-09-31') AND
e.enrolstudent NOT LIKE '%onlinetutor%' AND
e.enrolstudent NOT LIKE '%tes%' AND
e.enrolstudent NOT like '%student%' AND
e.enrolrole = 'student'
ORDER BY e.enrolstudent;"
It seems to hang, I've left it running for an hour with no result. There's only 10189 records in th enrolment table, 1538 in enrolcourse,650 in module. I don't think its the number of records, I'm guessing I've just constructed my query wrong, first time using joins (other than natural). Any ideas or tips in improving this would be greatly appreciated.
select count(*) from enrolment;
+----------+
| count(*) |
+----------+
| 10189 |
+----------+
select count(*) from enrolcourse;
+----------+
| count(*) |
+----------+
| 1538 |
+----------+
select count(*) from vetfee;
+----------+
| count(*) |
+----------+
| 1538 |
+----------+
select count(*) from users;
+----------+
| count(*) |
+----------+
| 1249 |
+----------+
select count(*) from users_personal;
+----------+
| count(*) |
+----------+
| 941 |
+----------+
select count(*) from module;
+----------+
| count(*) |
+----------+
| 650 |
Here's the results of the EXPLAIN
+----+--------------------+-------+------+---------------+------+---------+------+-------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+-------+------+---------------+------+---------+------+-------+---------------------------------+
| 1 | PRIMARY | m | ALL | NULL | NULL | NULL | NULL | 691 | Using temporary; Using filesort |
| 1 | PRIMARY | up | ALL | NULL | NULL | NULL | NULL | 987 | Using join buffer |
| 1 | PRIMARY | u | ALL | NULL | NULL | NULL | NULL | 1180 | Using where; Using join buffer |
| 1 | PRIMARY | ec | ALL | NULL | NULL | NULL | NULL | 1607 | Using where; Using join buffer |
| 1 | PRIMARY | e | ALL | NULL | NULL | NULL | NULL | 10629 | Using where; Using join buffer |
| 1 | PRIMARY | vf | ALL | NULL | NULL | NULL | NULL | 10959 | Using where; Using join buffer |
| 3 | DEPENDENT SUBQUERY | oi | ALL | NULL | NULL | NULL | NULL | 42 | Using where |
| 2 | DEPENDENT SUBQUERY | c | ALL | NULL | NULL | NULL | NULL | 23 | Using where |
+----+--------------------+-------+------+---------------+------+---------+------+-------+---------------------------------+

Answer 1

Get rid of those correlated subqueries. Use a join instead.
Also, use BETWEEN to reduce one STR_TO_DATE call
Finally, you should look at a way of eliminating all those LIKE calls.
SELECT
e.enrolstudent AS '313',
c.ntiscode AS '307',
e.startdate as '534',
'AOU' as '333',
m.mod_eftsl as '339',
e.enrolmod as '354',
e.census_date as '489',
m.diciplinecode as '464',
(CASE
WHEN m.mode = 'Face to Face' THEN 1
WHEN m.mode = 'Online' THEN 2
WHEN m.mode = 'RPL' THEN 5
ELSE 3
END) AS '329',
'A6090' as '477',
up.citizen AS '358',
vf.maxcontribute as '392',
vf.studentstatus as '490',
vf.total_amount_charged as '384',
vf.amount_paid as '381',
vf.loan_fee as '529',
u.chessn as '488',
m.workexp as '337',
'0' as '390',
m.sumwinschool as '551',
vf.help_debt as '558'
FROM
enrolment e
INNER JOIN enrolcourse AS ec ON ec.studentid=e.enrolstudent
INNER JOIN course AS c ON c.courseid = ec.courseid
INNER JOIN vetfee AS vf ON vf.userid=e.enrolstudent
INNER JOIN users AS u ON u.userid = e.enrolstudent
INNER JOIN users_personal AS up ON up.userid = e.enrolstudent
INNER JOIN module AS m ON m.modshortname = e.enrolmod
INNER JOIN online_intake oi ON oi.intakecode = e.online_intake
AND STR_TO_DATE(oi.censusdate, '%d-%m-%Y') BETWEEN '2015-07-01' AND '2015-09-31'
WHERE e.enrolstudent NOT LIKE '%onlinetutor%'
AND e.enrolstudent NOT LIKE '%tes%'
AND e.enrolstudent NOT like '%student%'
AND e.enrolrole = 'student'
ORDER BY e.enrolstudent;
Given your posted EXPLAIN output, you'll also want to add the following indexes:
ALTER TABLE enrolment
ADD INDEX (enrolstudent),
ADD INDEX (enrolmod),
ADD INDEX (online_intake);
ALTER TABLE enrolcourse
ADD INDEX (studentid),
ADD INDEX (courseid);
ALTER TABLE course
ADD INDEX (courseid);
ALTER TABLE vetfee
ADD INDEX (userid);
ALTER TABLE users
ADD INDEX (userid);
ALTER TABLE users_personal
ADD INDEX (userid);
ALTER TABLE module
ADD INDEX (modshortname);
ALTER TABLE online_intake
ADD INDEX (intakecode);