I am having issue with mysql query.
SELECT Count(*) AS aggregate
FROM (SELECT Group_concat(gateways.public_name) AS client_gateways,
`clients`.`id`,
`clients`.`name`,
`clients`.`status`,
`clients`.`api_key`,
`clients`.`user_name`,
`clients`.`psp_id`,
`clients`.`suspend`,
`clients`.`secret_key`,
`clients`.`created_at`,
`companies`.`name` AS `company_name`,
`mid_groups_mid`.`mid_id`,
`mid_groups_mid`.`mid_group_id`,
`mid_groups`.`id` AS `group_id`,
`mid_groups`.`user_id`,
`mids`.`mid_group_id` AS `id_of_mid`
FROM `clients`
LEFT JOIN `client_site_gateways`
ON `clients`.`id` = `client_site_gateways`.`client_id`
LEFT JOIN `gateways`
ON `client_site_gateways`.`gateway_id` = `gateways`.`id`
LEFT JOIN `client_broker`
ON `client_broker`.`client_id` = `clients`.`id`
LEFT JOIN `mid_groups`
ON `mid_groups`.`user_id` = `clients`.`psp_id`
LEFT JOIN `mid_groups_mid`
ON `mid_groups_mid`.`mid_group_id` = `mid_groups`.`id`
LEFT JOIN `mids`
ON `mids`.`mid_group_id` = `mid_groups_mid`.`mid_group_id`
INNER JOIN `companies`
ON `companies`.`id` = `clients`.`company_id`
WHERE `is_corp` = 0
AND `clients`.`suspend` = '0'
AND ( `clients`.`company_id` = 1 )
AND `clients`.`deleted_at` IS NULL
GROUP BY `clients`.`id`,
`clients`.`name`,
`clients`.`status`,
`clients`.`api_key`,
`clients`.`suspend`,
`clients`.`secret_key`,
`clients`.`created_at`,
`companies`.`name`,
`clients`.`user_name`,
`clients`.`psp_id`,
`mid_groups_mid`.`mid_id`,
`mid_groups_mid`.`mid_group_id`,
`mid_groups`.`id`,
`mid_groups`.`user_id`,
`mids`.`mid_group_id`) count_row_table
all table have few hundreds records. here is explain query result
+------+-------------+----------------------+--------+-------------------------------------+-------------------------------------+---------+----------------------------------------------+------------+-------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+----------------------+--------+-------------------------------------+-------------------------------------+---------+----------------------------------------------+------------+-------------------------------------------------+
| 1 | PRIMARY | <derived2> | ALL | NULL | NULL | NULL | NULL | 2849642280 | |
| 2 | DERIVED | companies | const | PRIMARY | PRIMARY | 4 | const | 1 | Using temporary; Using filesort |
| 2 | DERIVED | clients | ref | clients_company_id_foreign | clients_company_id_foreign | 4 | const | 543 | Using where |
| 2 | DERIVED | client_site_gateways | ref | client_id | client_id | 4 | knox_staging.clients.id | 5 | |
| 2 | DERIVED | gateways | eq_ref | PRIMARY | PRIMARY | 4 | knox_staging.client_site_gateways.gateway_id | 1 | Using where |
| 2 | DERIVED | client_broker | ALL | NULL | NULL | NULL | NULL | 6 | Using where; Using join buffer (flat, BNL join) |
| 2 | DERIVED | mid_groups | ref | mid_groups_user_id_foreign | mid_groups_user_id_foreign | 4 | knox_staging.clients.psp_id | 1 | Using where; Using index |
| 2 | DERIVED | mid_groups_mid | ref | mid_groups_mid_mid_group_id_foreign | mid_groups_mid_mid_group_id_foreign | 8 | knox_staging.mid_groups.id | 433 | Using where |
| 2 | DERIVED | mids | ref | mids_mid_group_id_foreign | mids_mid_group_id_foreign | 9 | knox_staging.mid_groups_mid.mid_group_id | 404 | Using where; Using index |
+------+-------------+----------------------+--------+-------------------------------------+-------------------------------------+---------+----------------------------------------------+------------+-------------------------------------------------+
in explain results what is causing to have 2849642280 row. while tables have only few hundreds records. all tables have proper indexing.
what i am thinking causing storage full is tmp table with above records. i tried to scale storage upto 60GB database size is few MBs. all storage filled up as soon as i run above query. i am not sure what causing left join to filter 2849642280 rows
The problem is probably the "aggregate." If the only thing you need is the count of records, you should write a new query which gets that count.
I am trying to optimize a stored procedure. I have identified some issues with it, but I don't know enough to actually correct the problems. One of the subqueries looks like this
select d.districtCode,
b.year Year,
if(bli.releaseAdjustment = 3 and bli.id not in (select billLineItem_id from AppliedDiscount),
bli.amount ,0) *-1 Refund
from Bill b
join BillLineItem bli on bli.bill_id = b.id
left join Bill_District d on d.bill_id = b.id
and bli.type = d.type
left join AppliedDiscount a on a.billLineItem_id = bli.id
where bli.releaseAdjustment in (1,2,3)
and bli.type in (4,6)
group by d.districtCode, b.year
The EXPLAIN outputs this
+----+------------------------+---------------+--------------+--------------------------------+-----------------+---------+---------------------+---------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+------------------------+---------------+--------------+--------------------------------+-----------------+---------+---------------------+---------+----------------------------------------------+
| 1 | PRIMARY | bli | ALL | FKF4236A,type,releaseAdjustment| NULL | NULL | NULL | 2787322 | Using where; Using temporary; Using filesort |
| 1 | PRIMARY | b | eq_ref | PRIMARY | PRIMARY | 8 | tax.bli.bill_id | 1 | |
| 1 | PRIMARY | d | ref | bill_id,type | bill_id | 8 | tax.bli.bill_id | 1 | |
| 1 | PRIMARY | a | ref | billLineItem_idx | billLineItem_idx| 8 | tax.bli.id | 1 | Using index |
| 1 | DEPENDENT SUBQUERY |AppliedDiscount|index_subquery| billLineItem_idx | billLineItem_idx| 8 | func | 1 | Using index |
+----+------------------------+---------------+--------------+--------------------------------+-----------------+---------+---------------------+---------+----------------------------------------------+
How would you suggest I fix this? This problem, or one very similar, is found throughout this stored procedure numerous times. AppliedDiscount only consists of 3 columns, all of which are indexed already.
Edit: Removing the group by changes the first row of the explain to
| 1 | PRIMARY | bli | ALL | FKF4236A,type,releaseAdjustment,bill_id| NULL | NULL | NULL | 2613847 | Using where |
That's better and technically answers my question, but that just means that I was asking the wrong question.
The 'type' is still ALL. What can I do to improve that?
I have the following query that runs really slow on mysql (83 seconds) but really fast on mariadb (.4 seconds).
I verified the data database has the same indexes and data. Maria Db server has less cpu (1VCPU), memory (2gb)
Mysql servers have 8 - 32GB ram and full quad core processors (tried 5.6,5.7, and 8.0 with similar results).
The phppos_inventory table has ~170000 rows and the phppos_items table has ~3000 rows
Here is the query and the tables and explains
SELECT /*+ SEMIJOIN(#subq MATERIALIZATION) */ SQL_CALC_FOUND_ROWS
1 AS _h,
`phppos_location_items`.`location_id` AS `location_id`,
`phppos_items`.`item_id`,
`phppos_items`.`name`,
`phppos_categories`.`id` AS `category_id`,
`phppos_categories`.`name` AS `category`,
`location`,
`company_name`,
`phppos_items`.`item_number`,
`size`,
`product_id`,
Coalesce(phppos_location_item_variations.cost_price,
phppos_item_variations.cost_price, phppos_location_items.cost_price,
phppos_items.cost_price, 0) AS cost_price,
Coalesce(phppos_location_item_variations.unit_price,
phppos_item_variations.unit_price, phppos_location_items.unit_price,
phppos_items.unit_price, 0) AS unit_price,
Sum(Coalesce(inv.trans_current_quantity, 0)) AS quantity,
Coalesce(phppos_location_item_variations.reorder_level,
phppos_item_variations.reorder_level, phppos_location_items.reorder_level,
phppos_items.reorder_level) AS reorder_level,
Coalesce(phppos_location_item_variations.replenish_level,
phppos_item_variations.replenish_level, phppos_location_items.replenish_level,
phppos_items.replenish_level) AS replenish_level,
description
FROM `phppos_inventory` `inv`
LEFT JOIN `phppos_items`
ON `phppos_items`.`item_id` = `inv`.`trans_items`
LEFT JOIN `phppos_location_items`
ON `phppos_location_items`.`item_id` = `phppos_items`.`item_id`
AND `phppos_location_items`.`location_id` = `inv`.`location_id`
LEFT JOIN `phppos_item_variations`
ON `phppos_items`.`item_id` = `phppos_item_variations`.`item_id`
AND `phppos_item_variations`.`id` = `inv`.`item_variation_id`
AND `phppos_item_variations`.`deleted` = 0
LEFT JOIN `phppos_location_item_variations`
ON `phppos_location_item_variations`.`item_variation_id` =
`phppos_item_variations`.`id`
AND `phppos_location_item_variations`.`location_id` =
`inv`.`location_id`
LEFT OUTER JOIN `phppos_suppliers`
ON `phppos_items`.`supplier_id` =
`phppos_suppliers`.`person_id`
LEFT OUTER JOIN `phppos_categories`
ON `phppos_items`.`category_id` = `phppos_categories`.`id`
WHERE inv.trans_id = (SELECT Max(inv1.trans_id)
FROM phppos_inventory inv1
WHERE inv1.trans_items = inv.trans_items
AND ( inv1.item_variation_id =
phppos_item_variations.id
OR phppos_item_variations.id IS NULL )
AND inv1.location_id = inv.location_id
AND inv1.trans_date < '2019-12-31 23:59:59')
AND inv.location_id IN( 1 )
AND `phppos_items`.`system_item` = 0
AND `phppos_items`.`deleted` = 0
AND `is_service` != 1
GROUP BY `phppos_items`.`item_id`
LIMIT 20
Explain mysql (slighly different than maria db but I tried use index to match the execution plan and still was slow)
+------------------------------------------+-------+----------+------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+---------------------------------+------------+--------+------------------------------+-------+----------+------------------------------------+
| 1 | PRIMARY | phppos_items | NULL | ref | PRIMARY,item_number,product_id,phppos_items_ibfk_1,deleted,phppos_items_ibfk_3,phppos_items_ibfk_4,phppos_items_ibfk_5,description,size,reorder_level,cost_price,unit_price,promo_price,last_modified,name,phppos_items_ibfk_6,deleted_system_item,custom_field_1_value,custom_field_2_value,custom_field_3_value,custom_field_4_value,custom_field_5_value,custom_field_6_value,custom_field_7_value,custom_field_8_value,custom_field_9_value,custom_field_10_value,verify_age,phppos_items_ibfk_7,item_inactive_index,tags,full_search,name_search,item_number_search,product_id_search,description_search,size_search,custom_field_1_value_search,custom_field_2_value_search,custom_field_3_value_search,custom_field_4_value_search,custom_field_5_value_search,custom_field_6_value_search,custom_field_7_value_search,custom_field_8_value_search,custom_field_9_value_search,custom_field_10_value_search | deleted | 4 | const | 21188 | 9.00 | Using index condition; Using where |
| 1 | PRIMARY | inv | NULL | ref | phppos_inventory_ibfk_1,location_id,phppos_inventory_custom | phppos_inventory_custom | 8 | pos.phppos_items.item_id,const | 3 | 100.00 | NULL |
| 1 | PRIMARY | phppos_location_items | NULL | eq_ref | PRIMARY,phppos_location_items_ibfk_2 | PRIMARY | 8 | const,pos.phppos_items.item_id | 1 | 100.00 | NULL |
| 1 | PRIMARY | phppos_item_variations | NULL | eq_ref | PRIMARY,phppos_item_variations_ibfk_1 | PRIMARY | 4 | pos.inv.item_variation_id | 1 | 100.00 | Using where |
| 1 | PRIMARY | phppos_location_item_variations | NULL | eq_ref | PRIMARY,phppos_item_attribute_location_values_ibfk_2 | PRIMARY | 8 | pos.phppos_item_variations.id,const | 1 | 100.00 | NULL |
| 1 | PRIMARY | phppos_suppliers | NULL | ref | person_id | person_id | 4 | pos.phppos_items.supplier_id | 1 | 100.00 | NULL |
| 1 | PRIMARY | phppos_categories | NULL | eq_ref | PRIMARY | PRIMARY | 4 | pos.phppos_items.category_id | 1 | 100.00 | NULL |
| 2 | DEPENDENT SUBQUERY | inv1 | NULL | ref | phppos_inventory_ibfk_1,location_id,trans_date,phppos_inventory_ibfk_4,phppos_inventory_custom | phppos_inventory_custom | 8 | pos.inv.trans_items,pos.inv.location_id | 3 | 50.00 | Using where; Using index |
+----+--------------------+---------------------------------+------------+--------+---------------------------------------------------------------------------------------------------------
Explain maria db:
+------+---------------------------------------------+-------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+--------------------+---------------------------------+--------+------------------------------+
| 1 | PRIMARY | phppos_items | ref | PRIMARY,deleted,deleted_system_item | deleted | 4 | const | 23955 | Using where |
| 1 | PRIMARY | inv | ref | phppos_inventory_ibfk_1,location_id,phppos_inventory_custom | phppos_inventory_ibfk_1 | 4 | freelance_pos5.phppos_items.item_id | 2 | Using where |
| 1 | PRIMARY | phppos_location_items | eq_ref | PRIMARY,phppos_location_items_ibfk_2 | PRIMARY | 8 | const,freelance_pos5.phppos_items.item_id | 1 | |
| 1 | PRIMARY | phppos_item_variations | eq_ref | PRIMARY,phppos_item_variations_ibfk_1 | PRIMARY | 4 | freelance_pos5.inv.item_variation_id | 1 | Using where |
| 1 | PRIMARY | phppos_location_item_variations | eq_ref | PRIMARY,phppos_item_attribute_location_values_ibfk_2 | PRIMARY | 8 | freelance_pos5.phppos_item_variations.id,const | 1 | Using where |
| 1 | PRIMARY | phppos_suppliers | ref | person_id | person_id | 4 | freelance_pos5.phppos_items.supplier_id | 1 | Using where |
| 1 | PRIMARY | phppos_categories | eq_ref | PRIMARY | PRIMARY | 4 | freelance_pos5.phppos_items.category_id | 1 | Using where |
| 2 | DEPENDENT SUBQUERY | inv1 | ref | phppos_inventory_ibfk_1,location_id,trans_date,phppos_inventory_ibfk_4,phppos_inventory_custom | phppos_inventory_custom | 8 | freelance_pos5.inv.trans_items,freelance_pos5.inv.location_id | 2 | Using where; Using index |
+------+--------------------+---------------------------------+--------+------------------------------------------------------------------------------------------------+-------------------------+---------+---------------------------------------------------------------+-------+--------------------------+
Tables described (Reached StackOverflow char limit)
https://pastebin.com/nhngSHb8
Create tables:
https://pastebin.com/aWMeriqt
MYSQL (DEV BOX)
mysql> SHOW GLOBAL STATUS LIKE '%thread%';
+------------------------------------------+-------+
| Variable_name | Value |
+------------------------------------------+-------+
| Delayed_insert_threads | 0 |
| Performance_schema_thread_classes_lost | 0 |
| Performance_schema_thread_instances_lost | 0 |
| Slow_launch_threads | 0 |
| Threads_cached | 4 |
| Threads_connected | 1 |
| Threads_created | 5 |
| Threads_running | 1 |
+------------------------------------------+-------+
8 rows in set (0.06 sec)
MARIA DB
MariaDB [freelance_pos5]> SHOW GLOBAL STATUS LIKE '%thread%';
+------------------------------------------+-------+
| Variable_name | Value |
+------------------------------------------+-------+
| Delayed_insert_threads | 0 |
| Performance_schema_thread_classes_lost | 0 |
| Performance_schema_thread_instances_lost | 0 |
| Slow_launch_threads | 0 |
| Threadpool_idle_threads | 0 |
| Threadpool_threads | 0 |
| Threads_cached | 3 |
| Threads_connected | 2 |
| Threads_created | 5 |
| Threads_running | 1 |
| wsrep_applier_thread_count | 0 |
| wsrep_rollbacker_thread_count | 0 |
| wsrep_thread_count | 0 |
+------------------------------------------+-------+
13 rows in set (0.00 sec)
Moving the
WHERE inv.trans_id = (SELECT Max(inv1.trans_id)
into the INNER JOIN is the game changer.
INNER JOIN (
SELECT inv1.trans_items, inv1.item_variation_id, inv1.location_id, MAX(inv1.trans_id) as trans_id
FROM phppos_inventory inv1
WHERE inv1.trans_date < '2019-12-31 23:59:59'
GROUP BY inv1.trans_items, inv1.item_variation_id, inv1.location_id
ORDER BY inv1.trans_items, inv1.item_variation_id, inv1.location_id
) inv1 on inv1.trans_id = inv.trans_id
AND inv1.trans_items = inv.trans_items
AND (inv1.item_variation_id = phppos_item_variations.id OR phppos_item_variations.id IS NULL)
AND inv1.location_id = inv.location_id
The execution is reduced from 80+s down to ~ <0.4s, on MySQL 8.0.
MariaDB's and MySQL's Optimizers started diverging significantly at 5.6. Certain queries will run queries faster in one than the other.
I think I see a way to speed up the query, perhaps on both versions.
Don't use LEFT JOIN when it is the same as JOIN, which seems to be the case for at least phppos_items, which has items in the WHERE that override LEFT.
Please provide SHOW CREATE TABLE; meanwhile, I will guess that what indexes you have/don't have, and that each table has PRIMARY KEY(id)
Use composite indexes where appropriate. (More below.)
Get the 20 rows before JOINing to the rest of the tables:
SELECT ...
FROM ( SELECT inv.id, pi.id
FROM `phppos_inventory` AS inv `inv`
JOIN `phppos_items` AS pi
ON pi.`item_id` = `inv`.`trans_items`
AND inv.location_id IN( 1 )
AND pi.`system_item` = 0
AND pi.`deleted` = 0
AND `is_service` != 1 -- Which table is this in???
GROUP BY pi.`item_id`
LIMIT 20 )
LEFT JOIN .... (( all the other tables ))
-- no GROUP BY or LIMIT needed (I think)
phppos_items: INDEX(item_id, deleted, system_item, is_service)
phppos_items: INDEX(deleted, system_item, is_service)
phppos_inventory: INDEX(trans_items, location_id, location_id, item_variation_id, trans_date, trans_id)
phppos_inventory: INDEX(location_id)
Aside with the fact that the query is misleading since the outer join is discarded, the main difference is that the second engine operation in MariabDB is an index range scan (ref) using the phppos_inventory_custom index. MySQL also chose an index range scan but over phppos_inventory_ibfk_1.
However, without the definition of these two indexes it's difficult to asses why the engines may have chosen a different path.
Please add to your question the definition of these indexes, and alse their selectivity (percent of estimated rows selected / total table rows) to elaborate more.
Ive got a search query that uses some joins to search in different correlating tables. But recently I've added about 3000 contacts to the contactpersonen table. And it got really slow.
Tables are these:
debiteuren : 1445 entries
contactpersonen: 3711 entries
debiteuren_toegang: 3008 entries
SELECT
contactpersonen.id,
contactpersonen.voornaam,
contactpersonen.achternaam,
debiteuren.bedrijfsnaam,
debiteuren.id as debid
FROM
debiteuren
LEFT JOIN
contactpersonen ON contactpersonen.bedrijf = debiteuren.id
LEFT JOIN
debiteuren_toegang ON debiteuren_toegang.bedrijf = debiteuren.id
WHERE
(contactpersonen.voornaam LIKE '%henk%'
OR contactpersonen.achternaam LIKE '%henk%'
OR debiteuren.id LIKE '%henk%'
OR debiteuren.bedrijfsnaam LIKE '%henk%'
OR contactpersonen.id LIKE '%henk%')
AND debiteuren_toegang.website = 'web1'
LIMIT 10
When I remove the part that searches trough contactpersonen.voornaam LIKE '%henk%' OR contactpersonen.achternaam LIKE '%henk%' The query is really fast again.
Ive added an index in phpmyadmin on voornaam and achternaam, but that didnt help anything.
Any ideas on how to make this quicker? I don't think this is a lot of rows right? Queries last for even 5 seconds at times.
Thanks!
FULL QUERY EXPLAIN:
+----+-------------+--------------------+--------+---------------+---------+---------+---------------------------------------+------+-------------+--+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | |
+----+-------------+--------------------+--------+---------------+---------+---------+---------------------------------------+------+-------------+--+
| 1 | SIMPLE | debiteuren_toegang | ALL | NULL | NULL | NULL | NULL | 3008 | Using where | |
| 1 | SIMPLE | debiteuren | eq_ref | PRIMARY | PRIMARY | 4 | deb12311_1.debiteuren_toegang.bedrijf | 1 | | |
| 1 | SIMPLE | contactpersonen | ALL | NULL | NULL | NULL | NULL | 4169 | Using where | |
+----+-------------+--------------------+--------+---------------+---------+---------+---------------------------------------+------+-------------+--+
PARTIAL QUERY EXPLAIN:
+----+-------------+--------------------+--------+---------------+---------+---------+---------------------------------------+------+-------------+--+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra | |
+----+-------------+--------------------+--------+---------------+---------+---------+---------------------------------------+------+-------------+--+
| 1 | SIMPLE | debiteuren_toegang | ALL | NULL | NULL | NULL | NULL | 3008 | Using where | |
| 1 | SIMPLE | debiteuren | eq_ref | PRIMARY | PRIMARY | 4 | deb12311_1.debiteuren_toegang.bedrijf | 1 | Using where | |
| 1 | SIMPLE | contactpersonen | ALL | NULL | NULL | NULL | NULL | 4098 | | |
+----+-------------+--------------------+--------+---------------+---------+---------+---------------------------------------+------+-------------+--+
Try this,
` SELECT
contactpersonen.id,
contactpersonen.voornaam,
contactpersonen.achternaam,
debiteuren.bedrijfsnaam,
debiteuren.id as debid
FROM
debiteuren
LEFT JOIN
contactpersonen ON contactpersonen.bedrijf = debiteuren.id
LEFT JOIN
debiteuren_toegang ON debiteuren_toegang.bedrijf = debiteuren.id
WHERE
debiteuren_toegang.website = 'web1'
AND
instr ( concat(contactpersonen.voornaam, contactpersonen.achternaam, debiteuren.id, debiteuren.bedrijfsnaam,contactpersonen.id) , 'henk'
)>0
LIMIT 10`
I've recently noticed that a query I have is running quite slowly, at almost 1 second per query.
The query looks like this
SELECT eventdate.id,
eventdate.eid,
eventdate.date,
eventdate.time,
eventdate.title,
eventdate.address,
eventdate.rank,
eventdate.city,
eventdate.state,
eventdate.name,
source.link,
type,
eventdate.img
FROM source
RIGHT OUTER JOIN
(
SELECT event.id,
event.date,
users.name,
users.rank,
users.eid,
event.address,
event.city,
event.state,
event.lat,
event.`long`,
GROUP_CONCAT(types.type SEPARATOR ' | ') AS type
FROM event FORCE INDEX (latlong_idx)
JOIN users ON event.uid = users.id
JOIN types ON users.tid=types.id
WHERE `long` BETWEEN -74.36829174058 AND -73.64365405942
AND lat BETWEEN 40.35195025942 AND 41.07658794058
AND event.date >= '2009-10-15'
GROUP BY event.id, event.date
ORDER BY event.date, users.rank DESC
LIMIT 0, 20
)eventdate
ON eventdate.uid = source.uid
AND eventdate.date = source.date;
and the explain is
+----+-------------+------------+--------+---------------+-------------+---------+------------------------------+-------+---------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+--------+---------------+-------------+---------+------------------------------+-------+---------------------------------+
| 1 | PRIMARY | | ALL | NULL | NULL | NULL | NULL | 20 | |
| 1 | PRIMARY | source | ref | iddate_idx | iddate_idx | 7 | eventdate.id,eventdate.date | 156 | |
| 2 | DERIVED | event | ALL | latlong_idx | NULL | NULL | NULL | 19500 | Using temporary; Using filesort |
| 2 | DERIVED | types | ref | eid_idx | eid_idx | 4 | active.event.id | 10674 | Using index |
| 2 | DERIVED | users | eq_ref | id_idx | id_idx | 4 | active.types.id | 1 | Using where |
+----+-------------+------------+--------+---------------+-------------+---------+------------------------------+-------+---------------------------------+
I've tried using 'force index' on latlong, but that doesn't seem to speed things up at all.
Is it the derived table that is causing the slow responses? If so, is there a way to improve the performance of this?
--------EDIT-------------
I've attempted to improve the formatting to make it more readable, as well
I run the same query changing only the 'WHERE statement as
WHERE users.id = (
SELECT users.id
FROM users
WHERE uidname = 'frankt1'
ORDER BY users.approved DESC , users.rank DESC
LIMIT 1 )
AND date & gt ; = '2009-10-15'
GROUP BY date
ORDER BY date)
That query runs in 0.006 seconds
the explain looks like
+----+-------------+------------+-------+---------------+---------------+---------+------------------------------+------+----------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------+-------+---------------+---------------+---------+------------------------------+------+----------------+
| 1 | PRIMARY | | ALL | NULL | NULL | NULL | NULL | 42 | |
| 1 | PRIMARY | source | ref | iddate_idx | iddate_idx | 7 | eventdate.id,eventdate.date | 156 | |
| 2 | DERIVED | users | const | id_idx | id_idx | 4 | | 1 | |
| 2 | DERIVED | event | range | eiddate_idx | eiddate_idx | 7 | NULL | 24 | Using where |
| 2 | DERIVED | types | ref | eid_idx | eid_idx | 4 | active.event.bid | 3 | Using index |
| 3 | SUBQUERY | users | ALL | idname_idx | idname_idx | 767 | | 5 | Using filesort |
+----+-------------+------------+-------+---------------+---------------+---------+------------------------------+------+----------------+
The only way to clean up that mammoth SQL statement is to go back to the drawing board and carefully work though your database design and requirements. As soon as you start joining 6 tables and using an inner select you should expect incredible execution times.
As a start, ensure that all your id fields are indexed, but better to ensure that your design is valid. I don't know where to START looking at your SQL - even after I reformatted it for you.
Note that 'using indexes' means you need to issue the correct instructions when you CREATE or ALTER the tables you are using. See for instance MySql 5.0 create indexes