Group Concat Or Something Else? Mysql Query Taking So Long - mysql

Question

This is my Tables structures:
DROP TABLE IF EXISTS `alljobs`;
CREATE TABLE IF NOT EXISTS `alljobs`
(
`ID` int(11) NOT NULL AUTO_INCREMENT,
`Department` varchar(50) NULL,
`SourceSite` varchar(50) NULL,
`Title` varchar(255) NULL,
`Description` text NULL,
`JobType` varchar(50) NULL,
`MainCategory` varchar(100) NULL,
`SubCategory` varchar(100) NULL,
`Url` varchar(255) NULL,
`TimeOfCreation` varchar(50) NULL,
`UnixTime` varchar(25) NULL,
PRIMARY KEY (`ID`),
KEY `Department` (`Department`),
KEY `SourceSite` (`SourceSite`),
KEY `Title` (`Title`),
KEY `MainCategory` (`MainCategory`),
KEY `SubCategory` (`SubCategory`),
KEY `UnixTime` (`UnixTime`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='To Hold All Job Information';
DROP TABLE IF EXISTS `joblocations`;
CREATE TABLE IF NOT EXISTS `joblocations`
(
`ID` int(11) NOT NULL AUTO_INCREMENT,
`jobId` varchar(25) NULL,
`Location` varchar(100) NULL,
PRIMARY KEY (`ID`),
KEY `jobId` (`jobId`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='To Hold Job Locations';
this is the query i am trying:
(it is basically the query if user doesn't apply any filter on the search page)
Select j.Department, j.SourceSite, j.Title, j.Url,
Group_Concat(l.Location Separator '||') As Locations
From alljobs as j
Left Outer Join joblocations as l On l.jobId = j.ID
Group By j.ID
Order By j.Title
Limit 25 Offset 0;
But it's taking 4-5 minutes to execute the query on phpMyadmin and on php it's just timing out..
there is only 30-40k data in both tables.
but if the user apply any search filter and execute it, it's takes less than a second (as per phpMyAdmin results saying):
Select j.Department, j.SourceSite, j.Title, j.Url,
Group_Concat(l.Location Separator '||') As Locations
From alljobs as j
Left Outer Join joblocations as l On l.jobId = j.ID
Where Department In ('Healthcare', 'Food', 'Technology', 'Lifestyle')
And MainCategory Like '%Marketing%'And Location Like '%California%'
Group By j.ID
Order By j.Title
Limit 25 Offset 0 ;
so, i don't understand what i am doing wrong here? i just wanted to have first 25 rows of data and concat the locations on 2nd table (matched by job id) in to single column.
any suggestion will be highly appreciated.
best regards

Answer 1

I don't see a real reason why your query should be so slow. However, since you group by j.ID and then order by j.Title, followed by a limit, the query has to process all data in the two tables before it can output the result. You only need 25 of the 30-40k rows from the alljobs table.
So let's rethink this. First write down the query for one table:
SELECT
j.Department,
j.SourceSite,
j.Title,
j.Url,
<... Locations... >
FROM alljobs AS j
ORDER BY j.Title
LIMIT 25 OFFSET 0;
This should be very quick, since j.Title is an index. The only problem left is, of course, that Locations doesn't work. We could write a subquery for this:
SELECT
j.Department,
j.SourceSite,
j.Title,
j.Url,
(SELECT
GROUP_CONCAT(l.Location SEPARATOR '||')
FROM joblocations AS l
WHERE l.jobId = j.ID) AS Locations
FROM alljobs AS j
ORDER BY j.Title
LIMIT 25 OFFSET 0;
The query now has to execute this subquery 25 times. That's quite a lot but still a lot less than going through the whole table. What should make it reasonable quick is that l.jobId is an index.
The reason your query with the search filters is a lot quicker is because you already filter out a lot of rows in advance with those search filters. A smaller data set is just quicker. However, the data set might not be small for some search filters, making the query take a lot longer.
If you want to use a query with a subquery, as shown above, it should look something like:
SELECT
j.Department,
j.SourceSite,
j.Title,
j.Url,
(SELECT
GROUP_CONCAT(l.Location SEPARATOR '||')
FROM joblocations AS l
WHERE l.jobId = j.ID) AS Locations
FROM alljobs AS j
WHERE j.Department IN ('Healthcare', 'Food', 'Technology', 'Lifestyle')
AND j.MainCategory LIKE '%Marketing%'
AND EXISTS (SELECT ID
FROM joblocations AS m
WHERE m.jobId = j.ID
AND m.Location LIKE '%California%')
ORDER BY j.Title
LIMIT 25 OFFSET 0;
I hope this works.