Here is a screenshot of mysql explain command on a common query:
http://cl.ly/3r34251M320A1P2s3e1Y
I have 3 different tables I have to join together to extract the data I want. This is the main CI model code using activerecord
$this->db->select('articles.id, articles.title, articles.link, articles.updated_time, articles.created_time, shows.network,shows.show_id, shows.name');
$this->db->from('articles')->order_by('updated_time','desc')->offset($offset)->limit($limit);
$this->db->join('labels', 'articles.remote_id = labels.articleid');
$this->db->join('shows', 'shows.show_id = labels.showid');
Can anyone suggest any ways to improve the schema or query performance?
Thanks
What makes your query slower is mysql use of temporary tables and filesort which means it can't efficiently use the index and creates a temporary table (many times on disk!).
It usually happen when you join a table using one index and sort it by another index or use a condition on another index.
First thing you can do is read about this issue and see if you can, at least, avoid the use of temporary disk tables.
How mysql uses temp tables
Related
I have a many-to-many relationship database with 3 tables. It's very slow to load the data into the tables, especially the join table. Several hours for 3 millions rows.
I was suggested to create the tables first without creating index. I am using Hibernate. If I don't annotate index in classes, then what's the best time and way to add index? Should I do it directly on MySql database using SQL statement? Or the index should be added somewhere in Hibernate, without affecting loading performance?
You should add indexes directly to MySQL database using CREATE INDEX statement.
If you have very big table you can use pt-online-schema-change to prevent blocking your application
How can I optimize the performance of a query that contains the following code?
FROM members m
JOIN settings s ON MD5(m.ID) = s.ID...
In MySQL, the only way you can optimize the performance is to put an index on settings(ID). MySQL should then do a full table scan of the members table, with an index lookup in the settings table.
You could change the data structure to include the md5(id) in the members table and then build an index on that value.
Other databases have other capabilities for resolving these queries (notably hash-based join algorithms and functional indexes). MySQL does not support these capabilities.
I have the following query:
SELECT t.*, a.hits AS ahits
FROM t, a
WHERE (t.TRACK LIKE 'xxx')
AND a.A_ID = t.A_ID
ORDER BY t.hits DESC, a.hits DESC
which runs very frequently. Table t has around 15M+ rows and a has around 3M+ rows.
When I did an EXPLAIN on the above query, I received a note saying that it always created a temp table. I noticed that creating a temp table based on the above query took quite a while. And, this is done plenty of time.
Thus, I am wondering if I create a view using the above say:
CREATE VIEW v_t_a
SELECT t.*, a.hits AS ahits
FROM t, a
WHERE a.A_ID = t.A_ID
And change my code to:
SELECT * FROM v_t_a WHERE TRACK LIKE 'xxx' ORDER BY hits DESC, ahits DESC
Will it improve the performance? Will it remove the create temp table time?
Thank you so much for your suggestions!
It is very dangerous if you assume MySQL would optimize your VIEWs same way as more advanced database systems would. Same as with subqueries and derived tables MySQL 5.0 will fail and perform very inefficiently in many counts.
MySQL has two ways of handling the VIEWS – query merge, in which case VIEW is simply expanded as a macro or Temporary Table in which case VIEW is materialized to temporary tables (without indexes !) which is later used further in query execution.
There does not seems to be any optimizations applied to the query used for temporary table creation from the outer query and plus if you use more then one Temporary Tables views which you join together you may have serious issues because such tables do not get any indexes.
So be very careful implementing MySQL VIEWs in your application, especially ones which require temporary table execution method. VIEWs can be used with very small performance overhead but only in case they are used with caution.
MySQL has long way to go getting queries with VIEWs properly optimized.
VIEW internally JOINS the TWO tables everytime you QUERY a VIEW...!!
To prevent this, create MATERIALIZED VIEW...
It is a view that is more of a TABLE ...You can query it directly as other table..
But you have to write some TRIGGERS to update it automatically, if any underlying TABLE data changes...
See this : http://tech.jonathangardner.net/wiki/PostgreSQL/Materialized_Views
It's rare that doing exactly the same operations in a view will be more efficient than doing it as a query.
The views are more to manage complexity of queries rather than performance, they simply perform the same actions at the back end as the query would have.
One exception to this is materialised query tables which actually create a separate long-lived table for the query so that subsequent queries are more efficient. I have no idea whether MySQL has such a thing, I'm a DB2 man myself :-)
But, you could possibly implement such a scheme yourself if performance of the query is an issue.
It depends greatly on the rate of change of the table. If the data is changing so often that a materialised query would have to be regenerated every time anyway, it won't be worth it.
I often break my complicated queries into temporary tables so I can comment them and help me comprehend the steps.
Does this differ greatly from how mysql handle's nested joins internally?
e.g. select * from t1,t2,t3,t4 where t1.id=t2.id and t2.id2 = t3.id2, t4.id3 = t3.id3
Does an index(s) defined on t1 get "carried over" to whatever internal table mysql creates to hold intermediate results?
Is there any major difference performance wise between explicitly defining temporary tables or using one single query full of nested joins?
Indexes are not carried over to temporary tables you explicitly create. They're just like regular tables, except that they'll disappear when you no longer have the session open. The database has no knowledge that the data in the table originally came from a query on some other tables that had indexes; for one thing you might have inserted/deleted/updated rows in the temp table since the query.
I would expect mysql to make use of any indexes it thought would be beneficial when you run joins, but the index has to be on a table in the join.
i was wondering, if i add one index for each field in every table of my DB, will that make my queries run faster?
or do i have to analyze my queries and create indexes only when required?
Adding an index on each column will probably make most of your queries faster, but it's not necessarily the best approach. It is better to tune your indexes to your specific queries, using EXPLAIN and performance measurements to guide you in adding the correct indexes.
In particular you need to understand when you shouldn't index a column, and when you need multi-column indexes.
I would advise reading the MySQL manual for optimization of SELECT statements which explains under what conditions indexes can be used.
The more indexes you have, the heavier inserting/updating gets. So it's a tradeoff. The select queries that cannot use an index now will get quicker ofcourse, but if you check what fields you're joining on (or using in a where) you will not trade off that much
(and, ofcourse, there is the disk-space, but most of the time I don't really care bout that: ) )
Another point is that MySql can only use a single index for a query, so if your query is
SELECT * FROM table WHERE status = 1 AND col1='foob' AND col2 = 'bar'
MySql will use 1 of the indexes, and filter out the rest reading the data from the table.
If you have queries like this, its better to create a composite index on (status, col1, col2)
Adding index on every field in every table is not smart.
You should add indexes ONLY on columns that you use in the WHERE clause in select OR on which you sort.
Often, the best results are achieved by using multi-column indexes that are specific to your SQL selects.
There are also a partial indexes with limit on the length of field which can also be used to optimize performance and reduce the index site.
Every unnecessary index will slow down the database during the insert because on every insert, every index has to be updated.
Also the more indexes you have, the more chances you have of data corruption. And lastly, indexes take extra storage space on disk, sometimes a lot of space.
Also MySQL tries to keep indexes in memory. If you have unnecessary indexes, there is a good change MySQL will end up using up the available memory with unnecessary indexes in which case your performance will degrade considerable.
Creating the right kind of indexes is probably the single most important optimization technique. That's why when someone asks something like this I thought it was a joke.
This question can only be asked by someone who have not read a single book on MySQL. Just get a good book and read it, then you will not have to ask questions like this.