Nodejs Mysql optimizing query - mysql

Question

I am using mysql2 module in nodejs v8.9.4.
This is my function to get a message from message queue which meets this conditions :
status==0
if count of botId with status==1 is less than 10
if retry_after in wait table for botId+chatId and just botId is less than NOW(timestamp)
if there is no same chatId with status==1
static async Find(activeMessageIds, maxActiveMsgPerBot) {
let params = [maxActiveMsgPerBot];
let filterActiveMessageIds = ' ';
let time = Util.GetTimeStamp();
if (activeMessageIds && activeMessageIds.length) {
filterActiveMessageIds = 'q.id NOT IN (?) AND ';
params.push(activeMessageIds);
}
let q =
`select q.*
from bot_message_queue q
left join bot_message_queue_wait w on q.botId=w.botId AND q.chatId=w.chatId
left join bot_message_queue_wait w2 on q.botId=w2.botId AND w2.chatId=0
where
q.status=0 AND
q.botId NOT IN (select q2.botId from bot_message_queue q2 where q2.status=1 group by q2.botId HAVING COUNT(q2.botId)>?) AND
${filterActiveMessageIds}
q.chatId NOT IN (select q3.chatId from bot_message_queue q3 where q3.status=1 group by q3.chatId) AND
(w.retry_after IS NULL OR w.retry_after <= ?) AND
(w2.retry_after IS NULL OR w2.retry_after <= ?)
order by q.priority DESC,q.id ASC
limit 1;`;
params.push(time);
params.push(time);
let con = await DB.connection();
let result = await DB.query(q, params, con);
if (result && result.length) {
result = result[0];
let updateQ = `update bot_message_queue set status=1 where id=?;`;
await DB.query(updateQ, [result.id], con);
} else
result = null;
con.release();
return result;
}
This query runs fine on my local dev system. It also runs fine in servers phpmyadmin in couple of milliseconds.
BUT when it runs throw nodejs+mysql2 The cpu usage goes up to 100%
There is only 2K rows in this table.
CREATE TABLE IF NOT EXISTS `bot_message_queue` (
`id` int(10) UNSIGNED NOT NULL AUTO_INCREMENT,
`botId` int(10) UNSIGNED NOT NULL,
`chatId` varchar(50) CHARACTER SET utf8 NOT NULL,
`type` varchar(50) DEFAULT NULL,
`message` longtext NOT NULL,
`add_date` int(10) UNSIGNED NOT NULL,
`status` tinyint(2) UNSIGNED NOT NULL DEFAULT '0' COMMENT '0=waiting,1=sendig,2=sent,3=error',
`priority` tinyint(1) UNSIGNED NOT NULL DEFAULT '5' COMMENT '5=normal messages,<5 = bulk messages',
`delay_after` int(10) UNSIGNED NOT NULL DEFAULT '1000',
`send_date` int(10) UNSIGNED DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `botId` (`botId`,`status`),
KEY `botId_2` (`botId`,`chatId`,`status`,`priority`),
KEY `chatId` (`chatId`,`status`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE TABLE IF NOT EXISTS `bot_message_queue_wait` (
`botId` int(10) UNSIGNED NOT NULL,
`chatId` varchar(50) CHARACTER SET utf8 NOT NULL,
`retry_after` int(10) UNSIGNED NOT NULL,
PRIMARY KEY (`botId`,`chatId`),
KEY `retry_after` (`retry_after`),
KEY `botId` (`botId`,`chatId`,`retry_after`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
UPDATE: Real table data here
UPDATE 2:
FetchMessageTime :
- Min : 1788 ms
- Max : 44285 ms
- Average : 20185.4 ms
The max was like 20ms until yesterday :( now its 40 seconds!!!
UPDATE 3: I merged these 2 joins and wheres:
left join bot_message_queue_wait w on q.botId=w.botId AND q.chatId=w.chatId
left join bot_message_queue_wait w2 on q.botId=w2.botId AND w2.chatId=0
(w.retry_after IS NULL OR w.retry_after <= ?) AND
(w2.retry_after IS NULL OR w2.retry_after <= ?)
into a single one, I hope this will work as intended!
left join bot_message_queue_wait w on q.botId=w.botId AND ( q.chatId=w.chatId OR w.chatId=0 )
and for the time being I removed the 2 wheres and the query time went back to normal.
q.botId NOT IN (select ...)
q.chatId NOT IN (select ...)
So these 2 where queries are the chock points and needs to be fixed.

Answer 1

NOT IN ( SELECT ... ) is difficult to optimize.
OR cannot be optimized.
In ORDER BY, mixing DESC and ASC eliminates use of an index (until 8.0). Consider changing ASC to DESC. After that, INDEX(priority, id) might help.
What is ${filterActiveMessageIds}?
The GROUP BY is not needed in
NOT IN ( SELECT q3.chatId
from bot_message_queue q3
where q3.status=1
group by q3.chatId )
INDEX(status, chatid) in this order would benefit that subquery.
INDEX(status, botid) in this order
More on index creation: http://mysql.rjweb.org/doc.php/index_cookbook_mysql

Answer 2

I would replace the NOT IN subquery with a NOT EXISTS in this case, as it can perform better.
Switch the ORDER BY to either all DESC or all ASC
So to optimize the query, first, add these indexes:
ALTER TABLE `bot_message_queue` ADD INDEX `bot_message_queue_idx_status_botid_chatid_priori_id` (`status`,`botId`,`chatId`,`priority`,`id`);
ALTER TABLE `bot_message_queue` ADD INDEX `bot_message_queue_idx_priority_id` (`priority`,`id`);
ALTER TABLE `bot_message_queue` ADD INDEX `bot_message_queue_idx_botid_status` (`botId`,`status`);
ALTER TABLE `bot_message_queue` ADD INDEX `bot_message_queue_idx_chatid_status` (`chatId`,`status`);
ALTER TABLE `bot_message_queue_wait` ADD INDEX `bot_message_queue_wa_idx_chatid_botid` (`chatId`,`botId`);
Now, you can try to run this query (please note I changed the order by to all DESC, so you can change it to ASC if that's a requirement):
SELECT
bot_message_queue.*
FROM
bot_message_queue q
LEFT JOIN
bot_message_queue_wait w
ON q.botId = w.botId
AND q.chatId = w.chatId
LEFT JOIN
bot_message_queue_wait w2
ON q.botId = w2.botId
AND w2.chatId = 0
WHERE
q.status = 0
AND NOT EXISTS (
SELECT
1
FROM
bot_message_queue AS q21
WHERE
q21.status = 1
AND q.botId = q21.botId
GROUP BY
q21.botId
HAVING
COUNT(q21.botId) > ?
ORDER BY
NULL
)
AND NOT EXISTS (
SELECT
1
FROM
bot_message_queue AS q32
WHERE
q32.status = 1
AND q.chatId = q32.chatId
GROUP BY
q32.chatId
ORDER BY
NULL
)
AND (
w.retry_after IS NULL
OR w.retry_after <= ?
)
AND (
w2.retry_after IS NULL
OR w2.retry_after <= ?
)
ORDER BY
q.priority DESC,
q.id DESC LIMIT 1