SELECT vt.vtid, vt.tag, vt.typeid, vt.id, vt.count, tt.type, u.username, vt.date_added, tc.context, tc.contextid
FROM ( vt, tt, u )
LEFT JOIN tc ON ( vt.vtid = tc.vtid AND tc.userid = vt.userid )
WHERE vt.typeid = tt.typeid
AND vt.verified =0
AND vt.userid = u.userid
ORDER BY vt.date_added DESC
LIMIT 1
takes .0007s to complete
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE vt ref typeid,userid,verified verified 1 const 9 Using where; Using filesort
1 SIMPLE tt eq_ref PRIMARY PRIMARY 4 vt.typeid 1
1 SIMPLE tc ref vtid vtid 4 vt.vtid 3
1 SIMPLE u eq_ref PRIMARY PRIMARY 4 vt.userid 1 Using where
How can I change this to not show up in the slow query log?
Just a guess. It's possible that you set log-queries-not-using-indexes flag. According to documentation, it may cause queries to be logged in slow log even if indexes are used.
I'm pretty sure that a1ex07 is correct.
However if you want to speed this query up slightly you can change your index on tc from being an index on vtid to being an index on (vtid, userid). Compound keys like that are much faster if you're joining on both keys, and are almost exactly as fast if you're just joining on the first field.
Related
I have this short snippet of code
SELECT candidate.ID
FROM users u
JOIN users candidate ON candidate.a = u.a AND candidate.b < 1
JOIN user_meta meta ON candidate.id = meta.user_id
WHERE u.id = 1
AND candidate.count > 0
ORDER BY meta.updated_at DESC
LIMIT 100
And it finishes in around 8s which I think is far to slow so I started to investigate a bit. I tried experiment with the join conditions
SELECT candidate.ID
FROM users u
JOIN users candidate ON candidate.a = u.a AND candidate.b < 2
JOIN user_meta meta ON candidate.id = meta.user_id
WHERE u.id = 1
AND candidate.count > 0
ORDER BY meta.updated_at DESC
LIMIT 100
and interesting enough this finishes in ~80ms. The only thing changed is the less than 1 to a less than 2.
Running EXPLAIN on the query yields the following for both queries
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE u const PRIMARY,index_a PRIMARY 4 const 1 NULL
1 SIMPLE meta index PRIMARY index_meta_on_updated_at 5 NULL 100 Using index
1 SIMPLE candidate eq_ref PRIMARY,index_a PRIMARY 4 db.meta.user_id 1 Using where
Probably something I have missed but what can cause this behavior?
Could you provide more information about:
- The table ( you can use describe )
- How many records has every table
- Does the tables has index ?
For troubleshooting you can use in MySQL the explain extended like Mjh told you. Provide us the explain for every query this will help us give you a better advice or even help you to make your query better.
SELECT `Nen Straatnaam` as street, `Nen Woonplaats` as city, Gemeente,
Postcode, acn_distinct.zipcodes, acn_distinct.lat, acn_distinct.lng
FROM `acn_distinct` INNER JOIN crimes as c
ON `Nen Woonplaats` = c.place AND c.street_check = 0
ORDER BY street ASC
EXPLAIN gives me this information:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE c ref idx_place,idx_street_check,fulltext_place idx_street_check 1 const 67556 Using temporary; Using filesort
1 SIMPLE acn_distinct ref ID_nen_woonplaats ID_nen_woonplaats 768 crimes.c.place 42 Using index condition
So why is it not using the suggested indexes?
It is using an index, idx_street_check, however, the performance will be terrible, since it's also creating a temporary table and using filesort which are both notorious culprits.
The question is why it's not using an index on crimes.place. I would try creating a multi-column index on (place, street_check).
But more importantly, I think your table schema is very, very bad. Your JOIN is poorly formed:
FROM `acn_distinct` INNER JOIN crimes as c
ON `Nen Woonplaats` = c.place AND c.street_check = 0
When you do a JOIN between tables A and B, you should join as such: A.x = B.y. But in this case you're not even referring to table A. You're multiplying the # of rows from A by a particular subset of B.
MySQL query taking 1.6 seconds for 40000 records in table
SELECT aggsm.topicdm_id AS topid,citydm.city_name
FROM AGG_MENTION AS aggsm
JOIN LOCATIONDM AS locdm ON aggsm.locationdm_id = locdm.locationdm_id
JOIN CITY AS citydm ON locdm.city_id = citydm.city_id
JOIN STATE AS statedm ON citydm.state_id = statedm.state_id
JOIN COUNTRY AS cntrydm ON statedm.country_id = cntrydm.country_id
WHERE cntrydm.country_id IN (1,2,3,4)
GROUP BY aggsm.topicdm_id,aggsm.locationdm_id
LIMIT 0,200000
I have 40000 to 50000 records in AGG_MENTION,LOCATIONDM,CITYDM tables....500records in STATEDM abd 4 records in COUNTRY table.
When i run above query it is taking 1.6 sec..Is there a way to optimize the query or index on which columns will improve the performance....
Following is the EXPLAIN output:
1 SIMPLE aggsm index agg_sm_locdm_fk_idx agg_sm_datedm_fk_idx 4 36313 Using index; Using temporary; Using filesort
1 SIMPLE locdm eq_ref PRIMARY,city_id_UNIQUE,locationdm_id_UNIQUE,loc_city_fk_idx PRIMARY 8 opinionleaders.aggsm.locationdm_id 1
1 SIMPLE citydm eq_ref PRIMARY,city_id_UNIQUE,city_state_fk_idx PRIMARY 8 opinionleaders.locdm.city_id 1
1 SIMPLE statedm eq_ref PRIMARY,state_id_UNIQUE,state_country_fk_idx PRIMARY 8 opinionleaders.citydm.state_id 1 Using where
1 SIMPLE cntrydm eq_ref PRIMARY,country_id_UNIQUE PRIMARY 8 opinionleaders.statedm.country_id 1 Using index
I would reverse the query and start with the STATE first as that is what your criteria is based upon. Since you are not actually doing anything with the country table (except the country ID)... This column also exists in the State table, so you can the State.Country_ID and remove the country table from the join.
Additionally, I would have the following indexes
Table Index
State (Country_ID) as that will be basis of your WHERE criteria.
City (State_ID, City_Name).
Location (City_ID)
Agg_Mention (LocationDM_ID, TopicDM_id).
By having the "City_Name" as part of the index, the query doesn't have to go to the actual page data for it. Since part of the index, it can use it directly.
Many times, the keyword "STRAIGHT_JOIN" included here helps optimizer to run query in the order stated so it doesn't try to take one of the other tables as its primary basis of querying the data. If that doesn't perform well, you can try it again without it.
SELECT STRAIGHT_JOIN
aggsm.topicdm_id AS topid,
citydm.city_name
FROM
STATE AS statedm
JOIN CITY AS citydm
ON statedm.state_id = citydm.state_id
JOIN LOCATIONDM AS locdm
ON citydm.city_id = locdm.city_id
join AGG_MENTION AS aggsm
ON locdm.locationdm_id = aggsm.locationdm_id
WHERE
statedm.country_id IN (1,2,3,4)
GROUP BY
aggsm.topicdm_id,
aggsm.locationdm_id
LIMIT 0,200000
UPDATE: Added the query that runs second most:
(maybe needed when taking an index in consideration???)
SELECT m.time, m.message, m.receiver_uid AS receiver, m.sender_uid AS sender
FROM messages AS m, users AS u
WHERE u.uid = '$coID'
AND ( (m.receiver_uid = '$meID' AND m.sender_uid = '$coID') OR
(m.receiver_uid = '$coID' AND m.sender_uid = '$meID') )
ORDER BY m.time DESC
$meID is the iD of the user who runs the wuery,
$coID is the ID of the contact.
I've got a somewhat big query and it runs everytime an user visits my page.
SELECT m2.message, m2.time, m2.sender_uid AS sender, m2.receiver_uid AS receiver,
m.contact, u.ufirstname
FROM ( SELECT CASE
WHEN sender_uid = '$me' THEN receiver_uid
ELSE sender_uid
END AS contact,
MAX(time) AS maxtime
FROM messages
WHERE sender_uid = '$me' OR receiver_uid = '$me'
GROUP BY CASE
WHEN sender_uid = '$me' THEN receiver_uid
ELSE sender_uid
END ) AS m
INNER JOIN messages m2 ON m.maxtime = m2.time
AND ((m2.sender_uid = '$me' AND m2.receiver_uid = m.Contact)
OR (m2.receiver_uid = '$me' AND m2.sender_uid = m.Contact))
INNER JOIN users AS u ON m.contact = u.uid
ORDER BY time DESC
$me is the ID of the user who runs the query
This query will (successfully) retrieve:
LAST MESSAGE from EVERY 'CONVERSATION' ordered by TIME.
So it will get the last message (whether the message is send or received) in every PM session
And than sort those by time, and retrieves the contacts information.
Please tell me if I didn't explain it correctly.
My MySQL table looks like this:
receiver_id | sender_id | message | time
From what index(es) would this query benefit?
(The user table already has an primary key on the ID so the part where the join retrieves the contacts name should be efficient)
EXPLAIN OUTPUTs:
The BIG query:
id select_type table type possible_keys key key_len ref rows Extra
1 PRIMARY <derived2> ALL NULL NULL NULL NULL 4 Using temporary; Using filesort
1 PRIMARY m2 ALL NULL NULL NULL NULL 42 Using where
1 PRIMARY u eq_ref PRIMARY PRIMARY 4 m.contact 1 Using where
2 DERIVED messages ALL NULL NULL NULL NULL 42 Using where; Using temporary; Using filesort
The query in the update part:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE u const PRIMARY PRIMARY 4 const 1 Using index; Using filesort
1 SIMPLE m ALL NULL NULL NULL NULL 42 Using where
As your messages table grows, this query will start becoming slower and slower. Depending on the number of conversations that the user is a part of, you will start seeing exponentially decaying performance. While an individual index on messages.time, messages.sender_uid and messages.receiver_uid will help for now, no index will help you in your long run, unless you trim your messages table. Especially when you have more than a few hundred thousand messages.
I would suggest maintaining an association type type that links a user to a conversation and their last message id. Something looking like:
user_id | conversation_id | message_id
You then look up this table, instead of performing a complicated and expensive query. This greatly reduces the number of scans that you need to do on your messages table. While, it does slightly increase the complexity, the performance will not degrade as much as your query above.
I found by trail and error that indexing the time decreases the load time.
So that will probably be the answer to my question.
I have some querys using views, and these run a lot slower than I would expect them to given all relevant tables are indexed (and not that large anyway).
I hope I can explain this:
My main Query looks like this (grossly simplified)
select [stuff] from orders as ord
left join calc_order_status as ors on (ors.order_id = ord.id)
calc_order_status is a view, defined thusly:
create view calc_order_status as
select ord.id AS order_id,
(sum(itm.items * itm.item_price) + ord.delivery_cost) AS total_total
from orders ord
left join order_items itm on itm.order_id = ord.id
group by ord.id
Orders (ord) contain orders, order_items contain the individual items associated with each order and their prices.
All tables are properly indexed, BUT the thing runs slowly and when I do a EXPLAIN I get
# id select_type table type possible_keys key key_len ref rows Extra
1 1 PRIMARY ord ALL customer_id NULL NULL NULL 1002 Using temporary; Using filesort
2 1 PRIMARY <derived2> ALL NULL NULL NULL NULL 1002
3 1 PRIMARY cus eq_ref PRIMARY PRIMARY 4 db135147_2.ord.customer_id 1 Using where
4 2 DERIVED ord ALL NULL NULL NULL NULL 1002 Using temporary; Using filesort
5 2 DERIVED itm ref order_id order_id 4 db135147_2.ord.id 2
My guess is, "derived2" refers to the view. The individual items (itm) seem to work fine, indexed by order _ id. The problem seems to be Line # 4, which indicates that the system doesn't use a key for the orders table (ord). But in the MAIN query, the order id is already defined:
left join calc_order_status as ors on (ors.order _ id = ord.id)
and ord.id (both in the main query and within the view) refer to the primary key.
I have read somewhere than MySQL simpliy does not optimize views that well and might not utilize keys under some conditions even when available. This seems to be one of those cases.
I would appreciate any suggestions. Is there a way to force MySQL to realize "it's all simpler than you think, just use the primary key and you'll be fine"? Or are views the wrong way to go about this at all?
If it is at all possible to remove those joins remove them. Replacing them with subquerys will speed it up a lot.
you could also try running something like this to see if it has any speed difference at all.
select [stuff] from orders as ord
left join (
create view calc_order_status as
select ord.id AS order_id,
(sum(itm.items * itm.item_price) + ord.delivery_cost) AS total_total
from orders ord
left join order_items itm on itm.order_id = ord.id
group by ord.id
) as ors on (ors.order_id = ord.id)
An index is useful for finding a few rows in a big table, but when you query every row, an index just slows things down. So here MySQL probably expects to be using the whole [order] table, so it better not use an index.
You can try if it would be faster by forcing MySQL to use an index:
from orders as ord force index for join (yourindex)