mysql query taking long time to excute - mysql

This is my query taking 3 second to execute :
SELECT I.itemname,
I.overdue,
D.value,
I.itemid,
icd.ecwstatus AS status,
C.inactiveflag AS inactive,
icd.validfrom,
icd.validto
FROM items I
JOIN itemdetail D
ON I.itemtype = 'I'
AND I.itemid = D.itemid
AND D.propid = 13
LEFT OUTER JOIN icd
ON icd.code = D.value
LEFT OUTER JOIN edi_icdcodes C
ON I.itemid = C.itemid
WHERE I.deleteflag = 0
AND ( icd.validfrom <= '2012-12-06'
OR icd.validfrom IS NULL )
AND ( icd.validto >= '2012-12-06'
OR icd.validto IS NULL )
AND I.itemname LIKE 'A%'
AND ( I.keyname = 'Assessments' )
ORDER BY I.itemname ASC limit 0,6;
i have index IX_items_itemType_deleteFlag_keyName_itemName on multiple column itemType , deleteFlag ,keyName ,itemName in items table and also have index on other table's column which used in join and where clause.
so how can i improve performance of query ?
Thanks

I would have an index on your items table based on the multiple key columns used for your where clause and order by. I would have the index with the smallest result set in the front position. For example, you are specifically looking for "Assessments". If your table has 1 million records, and 600k of them are of item type "I", but only 5k are "Assessments", then the smallest part up front might be better for your query TO process.
I would have your:
items table indexed on ( keyname, itemtype, deleteflag, itemname )
ItemDetail table, indexed ON ( itemid, propid )
icd table indexed ON ( code, validfrom, validto, ecwstatus )
edi_icdcodes table index ON (itemid)
SELECT
I.itemname,
I.overdue,
D.value,
I.itemid,
icd.ecwstatus AS status,
C.inactiveflag AS inactive,
icd.validfrom,
icd.validto
FROM
items I
JOIN itemdetail D
ON I.itemid = D.itemid
AND D.propid = 13
LEFT OUTER JOIN icd
ON D.value = icd.code
AND ( icd.validfrom <= '2012-12-06'
OR icd.validfrom IS NULL )
AND ( icd.validto >= '2012-12-06'
OR icd.validto IS NULL )
LEFT OUTER JOIN edi_icdcodes C
ON I.itemid = C.itemid
WHERE
I.itemtype = 'I'
AND I.deleteflag = 0
AND I.keyname = 'Assessments'
AND I.itemname LIKE 'A%'
ORDER BY
I.itemname ASC
LIMIT
0,6;
Note... if the ICD table will always have a value for both from/to dates when records are created, you won't need to test for NULL, but do understand why you had that via left-join and putting in the where clause. So, that part might be simplified to
LEFT OUTER JOIN icd
ON D.value = icd.code
AND icd.validfrom <= '2012-12-06'
AND icd.validto >= '2012-12-06'

What you do there, it's that you use a basic table, then you do some 'join' and only after, you do your 'where' requests . To go quicky, try to include your condition in your 'joins'. In this manneer, it selects the different lines in the 'join' request and not after.
SELECT I.itemname,
I.overdue,
D.value,
I.itemid,
icd.ecwstatus AS status,
C.inactiveflag AS inactive,
icd.validfrom,
icd.validto
FROM items I
JOIN itemdetail D
ON I.itemtype = 'I'
AND I.itemid = D.itemid
AND D.propid = 13
AND I.deleteflag = 0
LEFT OUTER JOIN icd
ON icd.code = D.value
AND ( icd.validfrom <= '2012-12-06'
OR icd.validfrom IS NULL )
AND ( icd.validto >= '2012-12-06'
OR icd.validto IS NULL )
LEFT OUTER JOIN edi_icdcodes C
ON I.itemid = C.itemid
AND I.itemname LIKE 'A%'
AND ( I.keyname = 'Assessments' )
ORDER BY I.itemname ASC limit 0,6;

Related

mysql: simpler and more efficient query for getting unique products

I would like to optimize my database query but I am not sure how to do this.
I want to get a list of stores' products opinions, ordered by opinion dates (from newest to oldest ones), but the products need to be unique.
For example, there are 3 users: U1, U2, U3.
There are 2 stores in the city:
S1 (with products P11, P12, P13, P14)
S2 (with products P21, P22, P23, P24)
Users added some opinions (the newest on the top, the oldest on the bottom):
U1: P22
U1: P13
U2: P21
U3: P13
U2: P23
U1: P23
What I want to achieve is:
U1: P22
U1: P13
U2: P21
U2: P23
The query I created is very long and a bit complicated. Could I simplify it somehow?
$sql_query = "
SELECT a.*
, b.name AS 'store_name'
, b.city AS 'store_city'
, c.name AS 'product_name'
FROM `app_products_opinion` AS a
JOIN `app_products_stores` AS b
ON a.store_ID = b.ID
JOIN `app_products` AS c
ON a.product_ID = c.ID
WHERE a.created_on IN
(
SELECT max(created_on) as created_on
FROM app_products_opinion
WHERE show_on_list='1' AND (added_by='".$_SESSION["CMSUserID"]."' OR status = '1')
GROUP by product_ID
ORDER by created_on DESC
)
AND a.show_on_list='1'
AND a.store_ID='".$id_store['ID']."' $addtosql
AND a.photo != ''
AND (a.added_by='".$_SESSION["CMSUserID"]."' OR a.status='1')
ORDER BY a.created_on DESC
";
You could try grouping by product_id and also joining by product_ID and date
(simplified code)
SELECT a.user_id, a.product_ID
from app_products_opinion a
INNER JOIN (
SELECT product_ID, max(created_on) as created_on
FROM app_products_opinion
WHERE show_on_list='1' AND (added_by='".$_SESSION["CMSUserID"]."' OR status = '1')
GROUP by product_ID
ORDER by created_on DESC
) t on a.created_on = t.created_on
AND a.product_ID = t.product_ID
I don't know if you think it's simpler (and ignoring, $addtosql) but you could do this...
SELECT a.*
, b.name AS 'store_name'
, b.city AS 'store_city'
, c.name AS 'product_name'
FROM `app_products_opinion` AS a
JOIN `app_products_stores` AS b
ON a.store_ID = b.ID
JOIN `app_products` AS c
ON a.product_ID = c.ID
JOIN
(
SELECT product_id
, max(created_on) created_on
FROM app_products_opinion
WHERE show_on_list = 1
AND (added_by = 'M' OR status = 1)
GROUP
by product_ID
) x
ON a.created_on = x.created_on
AND a.product_id = x.product_id
AND a.show_on_list = 1
AND a.store_ID = 'N'
AND a.photo != ''
AND (a.added_by = 'Z' OR a.status = 1)

MySql query taking long time to execute when called from VBA

I am trying to retrieve some data from MySql database using Excel Vba. Everything is working fine...but the MySql query is taking too much time to execute.
Here is my code:
SELECT
d.DATE,
c.name,
c.address,
c.state_name,
c.contact_no,
d.AMOUNT,
d.BY_NAME,
d.NARATION,
t.REMARK
FROM
database1.data d
JOIN
(
SELECT DISTINCT
cust_id,
OR_NO
FROM
database1.ordbill
) o ON SUBSTRING_INDEX(database1.d.NARATION,
':',
-1) = o.OR_NO
JOIN
database1.contact c ON o.cust_id = c.id
JOIN
database1.total t ON t.VCH_NO = d.VCH_NO
WHERE
d.PARTY_NAME = 'advance' AND(
d.`BY_NAME` = 'Bank1' OR d.`BY_NAME` = 'CASH' OR d.`BY_NAME` = 'Bank2'
) AND d.DATE BETWEEN '2019-09-01' AND '2019-09-30'
ORDER BY
d.DATE ASC `
Assuming you have already index on table contact pk id an index
SELECT *
FROM `loans`
WHERE `date` >= '2019-11-25'
AND `date`<='2019-11-28'
AND `designation` LIKE '%sdf%'
why does this happen ?
SELECT d.DATE
,c.name
,c.address
,c.state_name
,c.contact_no
, d.AMOUNT
, d.BY_NAME
, d.NARATION
,t.REMARK
FROM database1.data d JOIN (
SELECT DISTINCT cust_id, OR_NO FROM database1.ordbill
) o ON SUBSTRING_INDEX(database1.d.NARATION,':',-1)=o.OR_NO
JOIN database1.contact c on o.cust_id=c.id
JOIN database1.total t on t.VCH_NO=d.VCH_NO
WHERE d.PARTY_NAME = 'advance'
AND (d.`BY_NAME` = 'Bank1' OR d.`BY_NAME` = 'CASH' OR d.`BY_NAME` = 'Bank2')
AND d.DATE BETWEEN '2019-09-01' AND '2019-09-30'
ORDER BY d.DATE ASC
be sure you have also proper composite index on
table data columns(PARTY_NAME, BY_NAME, DATE, VCH_NO )
and a index also
table total column (VCH_NO)

What is slowing down this product search MySQL query?

We are using the OpenCart ecommerce platform running on PHP 7.2 with MySQL 5.7.27, with about 5000 products.
We use an extension to search through products in the admin panel and it takes about 70-80 seconds on average to execute the search query.
Raw query:
SELECT
SQL_CALC_FOUND_ROWS pd.*,
p.*,
(
SELECT
price
FROM
product_special
WHERE
product_id = p.product_id
AND
(
date_start = '0000-00-00'
OR date_start < NOW()
AND
(
date_end = '0000-00-00'
OR date_end > NOW()
)
)
ORDER BY
priority,
price LIMIT 1
)
AS special_price,
IF(p.image IS NOT NULL
AND p.image <> ''
AND p.image <> 'no_image.png', 'Igen', 'Nem') AS image_text,
IF(p.status, 'Engedélyezett', 'Letiltott') AS status_text,
GROUP_CONCAT(DISTINCT CONCAT_WS(' > ', fgd.name, fd.name)
ORDER BY
CONCAT_WS(' > ', fgd.name, fd.name) ASC SEPARATOR '
') AS filter_text, GROUP_CONCAT(DISTINCT fd.filter_id ORDER BY CONCAT_WS(' > ', fgd.name, fd.name) ASC SEPARATOR '_') AS filter, GROUP_CONCAT(DISTINCT cat.name ORDER BY cat.name ASC SEPARATOR ' ') AS category_text, GROUP_CONCAT(DISTINCT cat.category_id ORDER BY cat.name ASC SEPARATOR '_') AS category, GROUP_CONCAT(DISTINCT IF(p2s.store_id = 0, 'ButopĂȘa HU', s.name) SEPARATOR ' ') AS store_text, GROUP_CONCAT(DISTINCT p2s.store_id SEPARATOR '_') AS store FROM product p LEFT JOIN product_description pd ON (p.product_id = pd.product_id AND pd.language_id = '2') LEFT JOIN product_to_category p2c ON (p.product_id = p2c.product_id) LEFT JOIN (SELECT cp.category_id AS category_id, GROUP_CONCAT(cd1.name ORDER BY cp.level SEPARATOR ' > ') AS name FROM category_path cp LEFT JOIN category c ON (cp.path_id = c.category_id) LEFT JOIN category_description cd1 ON (c.category_id = cd1.category_id) LEFT JOIN category_description cd2 ON (cp.category_id = cd2.category_id) WHERE cd1.language_id = '2' AND cd2.language_id = '2' GROUP BY cp.category_id ORDER BY name) AS cat ON (p2c.category_id = cat.category_id) LEFT JOIN product_to_category p2c2 ON (p.product_id = p2c2.product_id) LEFT JOIN product_filter p2f ON (p.product_id = p2f.product_id) LEFT JOIN filter f ON (f.filter_id = p2f.filter_id) LEFT JOIN filter_description fd ON (fd.filter_id = p2f.filter_id AND fd.language_id = '2') LEFT JOIN filter_group_description fgd ON (f.filter_group_id = fgd.filter_group_id AND fgd.language_id = '2')
LEFT JOIN
product_filter p2f2
ON (p.product_id = p2f2.product_id)
LEFT JOIN
product_to_store p2s
ON (p.product_id = p2s.product_id)
LEFT JOIN
store s
ON (s.store_id = p2s.store_id)
LEFT JOIN
product_to_store p2s2
ON (p.product_id = p2s2.product_id)
GROUP BY
p.product_id
ORDER BY
pd.name ASC LIMIT 0,
190
I tried using MySQL's EXPLAIN functionality to see what's going on, but nothing catches my attention right away:
My test environment is running on Intel NVME, 2666 MHz DDR4 RAM, and i7 8th gen. CPU, and yet it's still very slow.
I appreciate any hints as to what is slowing this query down.
It looks like some many-to-many mappings being used in that SELECT. WooCommerce (and the underlying Wordpress) have an inefficient way of implementing such.
Here is my discussion of how to change the schema to improve performance of wp_postmeta and similar tables (product_to_category and product_to_store): http://mysql.rjweb.org/doc.php/index_cookbook_mysql#speeding_up_wp_postmeta
This may be a bug:
( date_start = '0000-00-00'
OR date_start < NOW()
AND ( date_end = '0000-00-00'
OR date_end > NOW() )
)
You probably wanted extra parentheses:
( ( date_start = '0000-00-00'
OR date_start < NOW() )
AND ( date_end = '0000-00-00'
OR date_end > NOW() )
)
Also,
( date_start = '0000-00-00' OR date_start < NOW() )
can be simplified to just
( date_start < NOW() )
And, the query seems to have the explode-implode syndrome where the JOINs expand to generate a large temp table, only to have the GROUP BY collapse down to the original size. The workaround is to turn
GROUP_CONCAT(... foo.x ...) AS blah.
...
LEFT JOIN foo ... ON ...
into
( SELECT GROUP_CONCAT(... foo.x ...) FROM foo WHERE ... ) AS blah,
If that eliminates all the LEFT JOINs, then the GROUP BY p.product_id can also be eliminated.
Do not say LEFT JOIN when the 'right' table is not optional. (Instead, say JOIN.)
LEFT JOIN category_description cd1 ON (cp.category_id = cd1.category_id)
WHERE cd1.language_id = '2' -- this invalidates the `LEFT`
cd2 seems not to be used except for checking that cd2.language_id = '2'. Consider removing references to it.
This requires two temp tables since they are different:
GROUP BY p.product_id
ORDER BY pd.name ASC
Am I correct in saying that pd.name is simply the name for p.product_id in 'language' 2? If so, this may be semantically the same, but faster because of eliminating a temp table and sort):
GROUP BY pd.name
ORDER BY pd.name
Once that is done, it may be better to have INDEX(language_id, name) on pd.
The speedup from LIMIT 0, 190 is mostly eliminated by SQL_CALC_FOUND_ROWS.
Over-normalization led to having the two components of this in separate tables?
CONCAT_WS(' > ', fgd.name, fd.name)

SUM with CASE Statement in mysql

I have following Mysql query
SELECT c.`id`
,c.`category_name`
,c.`category_type`
,c.bookmark_count
,f.category_id cat_id
,f.unfollow_at
,(
CASE WHEN c.id = f.follower_category_id
THEN (
SELECT count(`user_bookmarks`.`id`)
FROM `user_bookmarks`
WHERE (`user_bookmarks`.`category_id` = cat_id)
AND ((`f`.`unfollow_at` > `user_bookmarks`.`created_at`) || (`f`.`unfollow_at` = '0000-00-00 00:00:00'))
)
ELSE 0 END
) counter
,c.id
,f.follower_category_id follow_id
,c.user_id
FROM categories c
LEFT JOIN following_follower_categories f ON f.follower_category_id = c.id
WHERE c.user_id = 26
ORDER BY `category_name` ASC
and here is output what i am getting after execuation
now i just want to count . here i have field id having value 172 against it i have counter 30,3, 2 and Bookmark_count is 4( i need to include only once)
and i am accepting output for id 172 is 30+3+2+4(bookmark_count only once).
I am not sure how to do this.
Can anybody help me out
Thanks a lot
The following may be the most inefficient query for that purpose, but I added a cover to your query in order to hint at grouping the results.
(I removed the second c.id, and my example may have errors since I couldn't try it.)
SELECT `id`,
`category_name`,
`category_type`,
max(`bookmark_count`),
`cat_id`,
`unfollow_at`,
sum(`counter`)+max(`bookmark_count`) counter,
follow_id`, `user_id`
FROM
(SELECT c.`id`
,c.`category_name`
,c.`category_type`
,c.bookmark_count
,f.category_id cat_id
,f.unfollow_at
,(
CASE WHEN c.id = f.follower_category_id
THEN (
SELECT count(`user_bookmarks`.`id`)
FROM `user_bookmarks`
WHERE (`user_bookmarks`.`category_id` = cat_id)
AND ((`f`.`unfollow_at` > `user_bookmarks`.`created_at`) || (`f`.`unfollow_at` = '0000-00-00 00:00:00'))
)
ELSE 0 END
) counter
,f.follower_category_id follow_id
,c.user_id
FROM categories c
LEFT JOIN following_follower_categories f ON f.follower_category_id = c.id
WHERE c.user_id = 26)
GROUP BY `id`, `category_name`, `category_type`, `cat_id`, `unfollow_at`, `follow_id`, `user_id`
ORDER BY `category_name` ASC

Strange SUM() result

I have a strange problem with mysql sum() result.
SELECT `users_limits`.times, SUM( l.times ) AS `result`
FROM `users_limits`
INNER JOIN `users_limits` AS `l`
INNER JOIN `vacation_types` AS `v` ON l.id_vacation_type = v.id_vacation_type
WHERE l.year =2014 AND v.type =0
This query give me result:
times = 10;
result = 30;
But should give me result 10 too, because I have only one record in my db, with these conditions.
you need GROUP BY . try add this
GROUP BY l.times
like that
WHERE l.year =2014 AND v.type =0
GROUP BY l.times
EDIT-:
SELECT `users_limits`.times, SUM( l.times ) AS `result`
FROM `users_limits` l
INNER JOIN `vacation_types` AS `v` ON l.id_vacation_type = v.id_vacation_type
WHERE l.year =2014 AND v.type =0
GROUP BY l.times