Optimize MySQL UPDATE query that contains WHERE and ORDER BY? - mysql

Question

How can I optimize this query? If I run it without the ORDER BY clause, it executes in <100ms. With the ORDER BY clause it takes many seconds, and crushes the server when more than one system is trying to make this query at once.
UPDATE companies
SET
crawling = 1
WHERE
crawling = 0
AND url_host IS NOT NULL
ORDER BY
last_crawled ASC
LIMIT 1;
If I run this query as a SELECT, it's also fast ( <100ms ).
SELECT id
FROM companies
WHERE
crawling = 0
AND url_host IS NOT NULL
ORDER BY
last_crawled ASC
LIMIT 1;
Here is my table schema:
CREATE TABLE `companies` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`url` varchar(255) DEFAULT NULL,
`url_scheme` varchar(10) DEFAULT NULL,
`url_host` varchar(255) DEFAULT NULL,
`name` varchar(255) DEFAULT NULL,
`crawl` tinyint(1) unsigned NOT NULL DEFAULT '1',
`crawling` tinyint(1) unsigned NOT NULL DEFAULT '0',
`last_crawled` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `name` (`name`),
KEY `url_host` (`url_host`),
KEY `crawl` (`crawl`),
KEY `crawling` (`crawling`),
KEY `last_crawled` (`last_crawled`),
KEY `url_scheme` (`url_scheme`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
UPDATE ONE
This query gives me the following error: You can't specify target table 'companies' for update in FROM clause
UPDATE companies
SET crawling = 1
WHERE id = (
SELECT id
FROM companies
WHERE
crawling = 0
AND url_host IS NOT NULL
ORDER BY
last_crawled ASC
LIMIT 1
);
This query gives me the following error: This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery'
UPDATE companies
SET crawling = 1
WHERE id in (
SELECT id
FROM companies
WHERE
crawling = 0
AND url_host IS NOT NULL
ORDER BY
last_crawled ASC
LIMIT 1
);

Answer 1

try not to use ORDER-BY and LIMIT for such small number of updates.
UPDATE companies t1
join
(
SELECT c.id,#RowNum:=#RowNum+1 AS RowID
FROM companies c, (SELECT #RowNum := 0)r
WHERE c.crawling = 0 AND c.url_host IS NOT NULL
ORDER BY c.last_crawled ASC
)t2
ON t2.RowID=1 AND t1.id=t2.id
SET t1.crawling = 1
EDIT:1
make sure you have the index on (last_crawled ASC , id ASC)
UPDATE companies t1
join
(
Select ID,RowID
From
(
SELECT c.id,#RowNum:=#RowNum+1 AS RowID
FROM companies c, (SELECT #RowNum := 0)r
WHERE c.crawling = 0 AND c.url_host IS NOT NULL
ORDER BY c.last_crawled ASC
)t2
WHERE ROWID=1
)t3
ON t1.id=t3.id
SET t1.crawling = 1