I am using a simple MySQL query, but the performance is realy bad because of using ORDER BY. I can't figure out why MySQL is using filesort and temporary.
My query is:
EXPLAIN
SELECT * FROM Events
INNER JOIN EventLogFiles ON ServerID = 42
AND Events.LogFileID = EventLogFiles.LogFileID
ORDER BY ReportID DESC , TimeWritten DESC
LIMIT 100
This is the output of EXPLAIN:
Table Events structure
Table Events indexes
Table EventLogFiles structure
Table EventLogFiles indexes
UPDATE:
I tried to create two new indexes, but both still force MySQL to use filesort and temporary.
ALTER TABLE Events ADD INDEX `ReportID_TimeWritten_ServerID_LogFileID` ( ReportID DESC, TimeWritten DESC, ServerID, LogFileID)
ALTER TABLE Events ADD INDEX `ServerID_LogFileID_ReportID_TimeWritten` ( ServerID, LogFileID, ReportID DESC, TimeWritten DESC)
In order to utilize the index for both selection and sorting, you need to have all of the following columns in a multi-column index on Events: (ServerID, LogFileID, ReportID, TimeWritten).
Currently, MySQL cannot utilize the existing multi-column index because it doesn't include LogFileID, which is in your ON clause.
If you ever have problems where MySQL is selecting from EventLogFiles first, you can change the INNER JOIN to a STRAIGHT JOIN to ensure that MySQL always elects from Events first, using your index.
In order to get that working without temp table and file sort, you must to create an index (ServerID, LogFileID, ReportID) and TimeWritten must to be sequential like the auto_increment that you have been using in ReportID, so making the ORDER BY ReportID(PK - Order Physical) you must remove the filesort.
Related
I'm a beginner in php and I want to ask you if the query and table schema I have set up is the right way for performance. Note: If you want me to follow a different way, please provide sample for me, thanks
$digerilanlar = DB::get('
SELECT Count(siparisler.hid) AS siparissayisi,
siparisler.hid, ilanlar.id, ilanlar.seflink, ilanlar.kategori, ilanlar.baslik,
ilanlar.yayin, ilanlar.tutar, ilanlar.sure, ilanlar.onecikan, ilanlar.guncellemetarihi,
uyeler.nick, uyeler.foto, uyeler.online, uyeler.ban FROM ilanlar
inner join uyeler ON uyeler.id=ilanlar.ilansahibi
LEFT JOIN siparisler ON ilanlar.id = siparisler.hid
WHERE ilanlar.kategori= '.$kat->id.' and ilanlar.yayin=1 and uyeler.ban=0
GROUP BY ilanlar.id
ORDER BY guncellemetarihi DESC
LIMIT 0,12');
DATABASE DESİGN
Table engine MyISAM MYSQL versiyon 5.7.14
TABLE:İLANLAR
ilansahibi (int)= index
kategori (int)= index
yayin (int)= index
TABLE:UYELER
ban (int)= index
TABLE:SİPARİSLER
hid (int)= index
This will probably require two temp tables and two sorts:
GROUP BY ilanlar.id
ORDER BY guncellemetarihi DESC
Assuming that guncellemetarihi is update_date, this is not identical, but probably gives you what you want, but with only one temp table and sort:
GROUP BY guncellemetarihi, id
ORDER BY guncellemetarihi DESC, id DESC
COUNT(x) checks x for being NOT NULL. If that is not necessary, simply do COUNT(*).
SELECT COUNT(hid), hid
does not make sense. The COUNT implies that there may be multiple "hids", but hid implies that there is only one. (Since I don't understand to objective, I cannot advise which direction to change things.)
This composite INDEX may help:
ilanlar: INDEX(kategori, yayin, ilansahibi, id)
You should switch from ENGINE=MyISAM to ENGINE=InnoDB.
More on making indexes: Index Cookbook
To discuss further, please provide SHOW CREATE TABLE and EXPLAIN SELECT ...
I am running a basic select on a table with 189,000 records. The table structure is:
items
id - primary key
ad_count - int, indexed
company_id - varchar, indexed
timestamps
the select query is:
select *
from `items`
where `company_id` is not null
and `ad_count` <= 100
order by `ad_count` desc, `items`.`id` asc
limit 50
On my production servers, just the MySQL portion of the execution takes 300 - 400ms
If I run an explain, I get:
select type: SIMPLE
table: items
type: range
possible_keys: items_company_id_index,items_ad_count_index
key: items_company_id_index
key_len: 403
ref: NULL
rows: 94735
Extra: Using index condition; Using where; Using filesort
When fetching this data in our application, we paginate it groups of 50, but the above query is "the first page"
I'm not too familiar with dissecting explain queries. Is there something I'm missing here?
An ORDER BY clause with different sorting order can cause the creation of temporary tables and filesort. MySQL below (and including) v5.7 doesn't handle such scenarios well at all, and there is actually no point in indexing the fields in the ORDER BY clause, as MySQL's optimizer will never use them.
Therefore, if the application's requirements allow, it's best to use the same order for all columns in the ORDER BY clause.
So in this case:
order by `ad_count` desc, `items`.`id` asc
Will become:
order by `ad_count` desc, `items`.`id` desc
P.S, as a small tip to read more about - it seems that MySQL 8.0 is going to change things and these use cases might perform significantly better when it's released.
Try replacing items_company_id_index with a multi-column index on (company_id, ad_count).
DROP INDEX items_company_id_index ON items;
CREATE INDEX items_company_id_ad_count_index ON items (company_id, ad_count);
This will allow it to use the index to test both conditions in the WHERE clause. Currently, it's using the index just to find non-null company_id, and then doing a full scan of those records to test ad_count. If most records have non-null company_id, it's scanning most of the table.
You don't need to retain the old index on just the company_id column, because a multi-column index is also an index on any prefix columns, because of the way B-trees work.
I could be wrong here (depending on your sql version this could be faster) but try a Inner Join with your company table.
Like:
Select *
From items
INNER JOIN companies ON companies.id = items.company_id
and items.ad_count <= 100
LIMIT 50;
because of your high indexcount building the btrees will slow down the database each time a new entry is inserted. Maybe remove the index of ad_count?! (this depends on how often you use that entry for queries)
Does creating index on a column which is there in sort order improves performance? I need to know this for Mysql, Postgresql and Oracle database.
E.g query:
select * from article_comments where article_id = 245 order by date_created desc limit 30;
In article_comments table, article_id is an indexed field, where as date_created is not.
If we create an index on date_created, will it improve performance.
Table size is around 5-7 million rows.
Does creating index on a column which is there in sort order improves
performance?
A general answer is: it depends on many factors, esspecialy on conditions used in WHERE clause.
In case of your query an index on date_created column doesn't eliminate a sort operation due to where article_id = 245 clause.
In this case you need to create a multicolum index in order to skip sorting:
CREATE INDEX somename ON article_comments(article_id, date_created DESC)
I can tell only for Oracle. Yes an Index can make your sorting faster.
In this example Function-Based Index for Language-Dependent Sorting Oracle shows an index which intention is only to improve performance of an ORDER BY operation.
Another example regarding sorting is shown here: Full Index Scan
i couldn't find a way to optimize the following Query:
SELECT *
FROM tbl
WHERE type='51' AND `start`<='2012-01-19'
ORDER BY end DESC
LIMIT 5
I've tried by indexing each column in a separate index (type,start,end), and all of them in the same index, but MySQL keeps telling me that needs to do a filesort
Is this query just impossible to optimize?
Yes, as long you have range comparison in WHERE and sort by another field - mysql cannot use index for sorting.
It could if you had WHERE type='51' ANDstart='2012-01-19' ORDER BY end DESC or WHERE type='51' ANDstart<= '2012-01-19' ORDER BY start DESC
http://dev.mysql.com/doc/refman/5.5/en/order-by-optimization.html -- and here is a chapter relevant to your problem
It depends greatly on the column types and what's all in the table, but all you should really need are indexes on type, start, and end columns.
For an extra boost, you can make an index across type and start.
My first thought is (*) is expanding into rows that are not indexed.
If not, you will most certainly benefit from a Multiple-Column Index.
I would experiment and learn to see how to create the order of columns in the index definition with the lowest cardinality (approx unique combinations) of the columns.
Thanks for all your answers, it made me learn a lot about the issue.
And finally i got it! Looks like this is actually possible to Optimize, or at least to remove that using filesort in the EXPLAIN sentence.
Here's the indexes i used:
KEY `start` (`start`),
KEY `typeend` (`type`,`end`)
Now executing:
EXPLAIN SELECT *
FROM tbl
WHERE type='51' AND `start`<='2012-01-19'
ORDER BY end
DESC LIMIT 5
Leads to:
SIMPLE tbl ref start,type,typeend typeend 5 const 19 Using where
I would recommend the following:
add index (type, end, start)
rewrite the query:
SELECT *
FROM (
SELECT id -- `id` is the primary key
FROM tbl
WHERE type='51' AND `start`<='2012-01-19'
ORDER BY end DESC
LIMIT 5) as ids
JOIN tbl
USING (id); -- `id` is the primary key
I have an index problem with my order by statement. I have this query witch is runing fast:
SELECT name from members where kat = 2 order by date DESC;
I have a composite index on members table:
kat_index(kat,date);
Now my aplication has changed and i have added a query like this:
SELECT name from members where kat = 2 order by logged_in DESC, status DESC, date DESC;
Now the query i slow because it's using filesort. So the question is... What type of index should i add to satisfy both queries?
You should have two indexes:
(kat, date)
(kat, logged_in, status, date)
Each query will use the index that is best for that query.
You cannot cover both queries with one index. You must form a leftmost prefix on the composite index along with the ORDER BY clause as well.