I have this query in mysql with very poor performance.
select `notifiables`.`notification_id`
from `notifiables`
where `notifiables`.`notification_type` in (2, 3, 4)
and ( ( `notifiables`.`notifiable_type` = 16
and `notifiables`.`notifiable_id` = 53642)
or ( `notifiables`.`notifiable_type` = 17
and `notifiables`.`notifiable_id` = 26358)
or ( `notifiables`.`notifiable_type` = 18
and `notifiables`.`notifiable_id` = 2654))
order by `notifiables`.`id` desc limit 20
Is this query can be optimized in any way. Please help
This table has 2M rows. and taking upto 1-4 seconds in searching
Updated indexes and Explain select
Possible solutions:
Turning OR into UNION (see #hongnhat)
Row constructors (see #Akina)
Adding
AND notifiable_type IN (16, 17, 18)
Index hint. I dislike this because it often does more harm than good. However, the Optimizer is erroneously picking the PRIMARY KEY(id) (because of the ORDER BY instead of some filter which, according to the Cardinality should be very good.
INDEX(notification_type, notifiable_type, notifiable_id, id, notification_id) -- This is "covering", which can help because the index is probably 'smaller' than the dataset. When adding this index, DROP your current INDEX(notification_type) since it distracts the Optimizer.
VIEW is very unlikely to help.
More
Give this a try: Add this to the beginning of the WHERE
WHERE notifiable_id IN ( 53642, 26358, 2654 )
AND ... (all of what you have now)
And be sure to have an INDEX starting with notifiable_id. (I don't see one currently.)
Use the next syntax:
SELECT notification_id
FROM notifiables
WHERE notification_type IN (2, 3, 4)
AND (notifiable_type, notifiable_id) IN ( (16, 53642), (17, 26358), (18, 2654) )
ORDER BY id DESC LIMIT 20
Create index by (notification_type, notifiable_type, notifiable_id) or (notifiable_type, notifiable_id, notification_type) (depends on separate conditions selectivity).
Or create covering index ((notification_type, notifiable_type, notifiable_id, notification_id) or (notifiable_type, notifiable_id, notification_type, notification_id)).
You can make different kinds of "VIEW" from the data you want and then join them.
Related
The sql throws timeout exception in the PRD environment.
SELECT
COUNT(*) totalCount,
SUM(IF(t.RESULT_FLAG = 'success', 1, 0)) successCount,
SUM(IF(b.ERROR_CODE = 'Y140', 1, 0)) unrecognizedCount,
SUM(IF(b.ERROR_CODE LIKE 'Y%' OR b.ERROR_CODE = 'E008', 1, 0)) connectCall,
SUM(IF(b.ERROR_CODE = 'N004', 1, 0)) hangupUnconnect,
SUM(IF(b.ERROR_CODE = 'Y001', 1, 0)) hangupConnect
FROM
lbl_his b LEFT JOIN lbl_error_code t ON b.TASK_ID = t.TASK_ID AND t.CODE = b.ERROR_CODE
WHERE
b.TASK_ID = "5f460e4ffa99f51697ad4ae3"
AND b.CREATE_TIME BETWEEN "2020-07-01 00:00:00" AND "2020-10-28 00:00:00"
The size of table lbl_his is super large. About 20,000,000 rows data which occupied 20GB disk.
The size of table lbl_error_code is small. Only 305 rows.
The indexes of table lbl_his:
TASK_ID
UPDATE_TIME
CREATE_TIME
RECORD_ID
The union indexes of table lbl_his:
TASK_ID, ERROR_CODE, UPDATE_TIME
TASK_ID, CREATE_TIME
There are no index created for table lbl_error_code.
I ran EXPLAIN SELECT and found the sql hit the index of lbl_his.TASK_ID and lbl_error_code.primary.
How to avoid to execute timeout?
For an index solution on lbl_his, try putting a non-clustered index on
firstly the things you filter on by exact match
then the things you filter on as ranges (or inexact matches)
e.g., the initial part of the index should be TASK_ID then CREATE_TIME. Putting these first is very important as it means the engine can do one seek to get the data.
Then include any other fields in use (either as part of index, or includes - doesn't matter) - in this case, ERROR_CODE. This makes your index a covering index.
Therefore your final new non-clustered index on lbl_his should be (TASK_ID, CREATE_TIME, ERROR_CODE)
I'd like to know if it's possible to pass rows of data directly into a select subquery, rather than setting up a temporary table and joining on that.
My actual use case is trying to prevent thousands of individual queries, and for architectural reasons adding a temporary table would be a pain (but not impossible, so it's where I may have to go.)
An simplified example of my issue is :
I have a table giving the number plate of the cars in a car park, with each row containing section (a letter), space (a number), and reg_plate (a tinytext).
My boss gives me a list of 3 places and wants the reg number of the car in each (or null if empty).
Now I can do this by creating a temporary table containing the section/space sequence I'm interested in, and then join my carpark table against that to give me each of the rows.
I'm wondering is there a syntax where I could do this in a select, perhaps with a subselect something like this - obviously invalid, but hopefully it shows what I'm getting at:
SELECT targets.section, targets.space, cp.reg_plate
FROM carpark cp
LEFT JOIN (
SELECT field1 AS section, field2 AS space
FROM (
('a', 7), ('c', 14), ('c', 23)
)
) targets ON (cp.section = targets.section AND cp.space = targets.space)
Any ideas gratefully received!
You can use UNION:
SELECT targets.section, targets.space, cp.reg_plate
FROM carpark cp
LEFT JOIN (
SELECT 'a' as section, 7 AS space
UNION ALL
SELECT 'c', 14
UNION ALL
SELECT 'c', 23
) targets ON cp.section = targets.section AND cp.space = targets.space
A followup;
I realised I was hoping to find something analagous to the VALUES used in an insert statement.
It turns out this is available in MySQL 8+, and VALUES is available as a table constructor:
https://dev.mysql.com/doc/refman/8.0/en/values.html
SELECT targets.section, targets.space, cp.reg_plate
FROM carpark cp
LEFT JOIN (
VALUES
ROW ('a', 7), ROW('c', 14), ROW('c', 23)
)
) targets ON (cp.section = targets.column_0 AND cp.space = targets.column_1)
Not available in earlier MySQL (so the UNION method is OK) but great in 8+.
My mysql query is like below, I need to create index to boost up result fetching.
SELECT * FROM tbl_name
WHERE seasonid = 1 AND status = 'N'
AND month = 10 AND (team_a = 'India' OR team_b = 'India');
Thanks in advance
(a = 1 OR a = 3) is turned into a IN (1,3), which is sometimes optimizable. However, you don't have that case. Therefore, the expressions on either side of OR are useless for indexing.
INDEX(seasonid,status,month)
with the 3 fields in any order is the 'best' index for that query.
See also my index cookbook.
I am using MySQL 5.1 on a Windows Server 2008 (with 4GB RAM) and have the following configuration:
I have 2 MyISAM tables. One is in 1 database (DB1) and has 14 columns, which are mostly varchar. This table has about 5,000,000 rows and is the DB1.games table below. It has a primary key on GameNumber (int(10)).
The other table is the DB2.gameposition and consists of the columns GameNumber (links to
DB1.games) and PositionCode (int(10)). This table has about 400,000,000 rows and there is an index IX_PositionCode on PositionCode.
These 2 databases are on the same server.
I want to run a query on DB2.gameposition to find a particular PositionCode, and have the results sorted by the linking DB1.games.Yr field (smallint(6) - this represents a Year). This sorting of results I only introduced recently. There is an index on this Yr field in DB1.games.
In my stored procedure, I perform the following:
CREATE TEMPORARY TABLE tblGameNumbers(GameNumber INT UNSIGNED NOT NULL PRIMARY KEY);
INSERT INTO tblGameNumbers(GameNumber)
SELECT DISTINCT gp.GameNumber
FROM DB2.gameposition gp
WHERE PositionCode = var_PositionCode LIMIT 1000;
I just get 1000 to make it quicker
And then join it to the DB1.games table.
In order to generate an EXPLAIN from that, I took out the temporary table (I use in the stored procedure) and refactored it as seen in the inner subquery below:
EXPLAIN
SELECT *
FROM DB1.games g
INNER JOIN (SELECT DISTINCT gp.GameNumber
FROM DB2.gameposition gp
WHERE PositionCode = 669312116 LIMIT 1000
) B ON g.GameNumber = B.GameNumber
ORDER BY g.Yr DESC
LIMIT 0,28
Running the EXPLAIN above, I see the following:
1, 'PRIMARY', '', 'ALL', '', '', '', '', 1000, 'Using temporary; Using filesort'
1, 'PRIMARY', 'g', 'eq_ref', 'PRIMARY', 'PRIMARY', '4', 'B.GameNumber', 1, ''
2, 'DERIVED', 'gp', 'ref', 'IX_PositionCode', 'IX_PositionCode', '4', '', 1889846, 'Using temporary'
The query used to be almost instant before I introduced the ORDER BY clause. Now, sometimes it is quick (depending on different PositionCode), but other times it can take up to 10 seconds to return the rows. Before I introduced the sorting, it was always virtually instantaneous. Unfortunately, I am not too proficient in interpreting the EXPLAIN output. Or how to make the query faster.
Any help would be greatly appreciated!
Thanks in advance,
Tim
Without the order by, your limit means the first 28 results are returned and then the query stops. With order by, all results need to be retrieved so they can be sorted and the first 28 returned.
The explain shows what MySql is doing:
sort 5000000 games records by yr
for each games record from sorted list
get the games record by primary key (to get all the columns)
read gamepositions by position code
if it does not match gamenumber, discard it
when 1000 matches found, stop reading
end read
end for
Try this instead:
select distinct ... from gameposition gp
inner join games g on g.gamenumber = gp.gamenumber
where gp.positioncode = ...
order by g.yr limit ...
i have simple query:
SELECT data FROM table WHERE id IN (5, 2, 8, 1, 10)
Question is, how can i select my data and order it like in my IN.
Order must be 5, 2, 8, 1, 10.
Problem is that i have no key for order. IN data is from other query (1), but i need to safe order.
Any solutions?
(1)
SELECT login
FROM posts
LEFT JOIN users ON posts.post_id=users.id
WHERE posts.post_n IN (
2280219,2372244, 2345146, 2374106, 2375952, 2375320, 2371611, 2360673, 2339976, 2331440, 2279494, 2329266, 2271919, 1672114, 2301856
)
Thanx for helping, solutions works but very slow, maybe find something better later, thanx anyway
The only way I can think to order by an arbitrary list would be to ORDER BY comparisons to each item in that list. It's ugly, but it will work. You may be better off sorting in whatever code you are doing the selection.
SELECT data FROM t1 WHERE id IN (5, 2, 8, 1, 10)
ORDER BY id = 10, id = 1, id = 8, id = 2, id = 5
The order is reversed because otherwise you would have to add DESC to each condition.
You can use a CASE statement
SELECT data
FROM table WHERE id IN (5, 2, 8, 1, 10)
ORDER BY CASE WHEN id = 5 THEN 1 WHEN id = 2 THEN 2 WHEN id = 8 THEN 3 WHEN id = 1 THEN 4 WHEN id = 10 THEN 5 END
SELECT data FROM table
WHERE id IN (5, 2, 8, 1, 10)
ORDER BY FIELD (id, 5, 2, 8, 1, 10)
http://dev.mysql.com/doc/refman/5.5/en/string-functions.html#function_field
Might be easier to auto-generate (because it basically just needs inserting the wanted IDs comma-separated in the same order a second time) than the other solutions suggested using CASE or a number of ID=x, ID=y ...
http://sqlfiddle.com/#!2/40b299/6
I think that's what you're looking for :D Adapt it to your own situation.
To do this dynamically, and within MySql, I would suggest to do the following:
Create a temp table or table variable (not sure if MySql has these), with two columns:
OrderID mediumint not null auto_increment
InValue mediumint -(or whatever type it is)
Insert the values of the IN clause in order, which will generate ID's in order of insertion
Add a JOIN to your query on this temp table
Change your Order By to be
order by TempTable.OrderID
Drop temp table (again, in SQL inside a stored proc, this is automatic, not sure about MySql so mentioning here for full disclosure)
This effectively circumvents the issue of you not having a key to order by in your table ... you create one. Should work.