Is a way to optimize big mysql query? - mysql

This is my query and it's to slow... I'm looking for the way to optimize it.
SELECT p.id,
Group_concat(pc.cat_id) AS groups,
p.code,
p.NAME,
p.price,
p.thumbnail,
p.image,
mc.queries AS merch_queries,
mc.position AS merch_position,
Group_concat(op.image) AS option_images,
cf_RETAIL.value AS custom_RETAIL,
cf_rating.value AS custom_rating,
cf_reviews.value AS custom_reviews,
cf_sku.value AS custom_sku,
cf_brand.value AS custom_brand,
cf_custom_thumbnail.value AS custom_custom_thumbnail
FROM s01_products AS p
LEFT JOIN s01_categoryxproduct AS pc
ON p.id = pc.product_id
LEFT JOIN (SELECT pv.product_id,
pv.value
FROM s01_cfm_prodfields AS pf
INNER JOIN s01_cfm_prodvalues AS pv
ON pf.id = pv.field_id
WHERE pf.code = 'RETAIL') AS cf_RETAIL
ON p.id = cf_RETAIL.product_id
LEFT JOIN (SELECT pv.product_id,
pv.value
FROM s01_cfm_prodfields AS pf
INNER JOIN s01_cfm_prodvalues AS pv
ON pf.id = pv.field_id
WHERE pf.code = 'rating') AS cf_rating
ON p.id = cf_rating.product_id
LEFT JOIN (SELECT pv.product_id,
pv.value
FROM s01_cfm_prodfields AS pf
INNER JOIN s01_cfm_prodvalues AS pv
ON pf.id = pv.field_id
WHERE pf.code = 'reviews') AS cf_reviews
ON p.id = cf_reviews.product_id
LEFT JOIN (SELECT pv.product_id,
pv.value
FROM s01_cfm_prodfields AS pf
INNER JOIN s01_cfm_prodvalues AS pv
ON pf.id = pv.field_id
WHERE pf.code = 'sku') AS cf_sku
ON p.id = cf_sku.product_id
LEFT JOIN (SELECT pv.product_id,
pv.value
FROM s01_cfm_prodfields AS pf
INNER JOIN s01_cfm_prodvalues AS pv
ON pf.id = pv.field_id
WHERE pf.code = 'brand') AS cf_brand
ON p.id = cf_brand.product_id
LEFT JOIN (SELECT pv.product_id,
pv.value
FROM s01_cfm_prodfields AS pf
INNER JOIN s01_cfm_prodvalues AS pv
ON pf.id = pv.field_id
WHERE pf.code = 'custom_thumbnail') AS cf_custom_thumbnail
ON p.id = cf_custom_thumbnail.product_id
LEFT JOIN (SELECT p.product_id AS product_id,
Group_concat(q.query) AS queries,
Min(p.position) AS position
FROM s01_srch_merchandisingproduct AS p
LEFT JOIN s01_srch_merchandisingquery AS q
ON q.id = p.query_id
GROUP BY p.product_id) AS mc
ON p.id = mc.product_id
LEFT JOIN s01_options AS op
ON p.id = op.product_id
AND op.image <> ''
WHERE p.active = 1
GROUP BY p.id
Thanks for any help!
Updated Tables Schema:
**s01_categoryxproduct**
cat_id,
product_id,
disp_order
**s01_products**
id
catcount
agrpcount
pgrpcount
disp_order
code
name
thumbnail
image
price
cost
descrip
weight
taxable
active
sku
cancat_id
page_id
page_title
dt_created
dt_updated
**s01_CFM_ProdValues**
field_id,
product_id,
value,
value_long
**s01_CFM_ProdFields**
id,
code,
name,
group_id,
fieldtype,
info,
facet
**s01_Options**
id,
product_id,
attr_id,
disp_order,
code,
prompt,
price,
cost,
weight,
image
**s01_SRCH_MerchandisingProduct**
id,
product_id,
query_id,
position
**s01_SRCH_MerchandisingQuery**
id,
query

"Over-normalization" and EAV are the problems.
Put prodfields and prodvalues in the same table. Splitting them into two tables leads to performance-eating overhead.
More on optimizing an EAV table: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#speeding_up_wp_postmeta
More discussion of why EAV is bad: http://mysql.rjweb.org/doc.php/eav

SELECT p.id,
Group_concat(pc.cat_id) AS groups,
p.code,
p.NAME,
p.price,
p.thumbnail,
p.image,
mc.queries AS merch_queries,
mc.position AS merch_position,
Group_concat(op.image) AS option_images,
IF(pf.code = 'RETAIL',pv.value , NULL) AS custom_RETAIL,
IF(pf.code = 'rating',pv.value , NULL) AS custom_rating,
IF(pf.code = 'reviews',pv.value , NULL) AS custom_reviews,
IF(pf.code = 'brand',pv.value , NULL) AS custom_brand,
IF(pf.code = 'custom_thumbnail',pv.value , NULL) AS custom_custom_thumbnail,
pvr.value AS custom_rating,
FROM s01_products AS p
LEFT JOIN s01_categoryxproduct AS pc ON p.id = pc.product_id
LEFT JOIN s01_cfm_prodfields AS cf_RETAIL ON p.id = cf_RETAIL.product_id
LEFT JOIN s01_cfm_prodvalues AS pv ON cf_RETAIL.id = pv.field_id
LEFT JOIN (SELECT p.product_id AS product_id,
Group_concat(q.query) AS queries,
Min(p.position) AS position
FROM s01_srch_merchandisingproduct AS p
LEFT JOIN s01_srch_merchandisingquery AS q
ON q.id = p.query_id
GROUP BY p.product_id) AS mc
ON p.id = mc.product_id
LEFT JOIN s01_options AS op
ON p.id = op.product_id
AND op.image <> ''
WHERE p.active = 1
GROUP BY p.id`;

Related

Complex query containing multiple joins is not executing right outer join

I'm writing a query with multiple joins where I want every user to show entries against all category Types. When I execute the query below only 1 record is returned because the employee u.employee_id = "0079-P" has only worked on 1 project but I want to get data for all the category_types with users workhours displayed as null for the categories he didn't work on.
Select u.employee_id As Employee_ID, u.user_name As UserName, COALESCE(primaryDept.ctd_name, primaryProj.ctd_name) As PrimaryDeptOrProj, region.region_name As Region, categoryType.ctd_id, categoryType.ctd_name, SUM(tsdd.workhours)
From users u
LEFT JOIN category_type_details primaryDept ON u.user_primary_department = primaryDept.ctd_id
LEFT JOIN category_type_details primaryProj ON u.user_primary_project = primaryProj.ctd_id
LEFT JOIN regions region ON u.region_id = region.region_id
LEFT JOIN timesheets ts ON u.user_id = ts.timesheet_user
INNER JOIN timesheet_mr tsmr ON ts.timesheet_caller = tsmr.tsmr_id
INNER JOIN timesheet_details tsd ON ts.timesheet_id = tsd.tsd_timesheet_id
INNER JOIN timesheet_day_details tsdd ON tsd.tsd_id = tsdd.tsd_id
RIGHT OUTER JOIN category_type_details categoryType ON tsd.tsd_category_type_id = categoryType.ctd_id
WHERE tsmr.tsmr_id = 14 and u.employee_id = "0079-P"
GROUP BY u.user_id, tsd.tsd_category_type_id;
I tried this query with variations and it returns 1 record in any case.
You could change your query to this:
SELECT u.employee_id AS Employee_ID, u.user_name AS UserName,
COALESCE(pd.ctd_name, pp.ctd_name) AS PrimaryDeptOrProj,
r.region_name AS Region, ct.ctd_id, ct.ctd_name, SUM(tsdd.workhours)
FROM users u
LEFT JOIN category_type_details pd ON u.user_primary_department = pd.ctd_id
LEFT JOIN category_type_details pp ON u.user_primary_project = pp.ctd_id
LEFT JOIN regions r ON u.region_id = r.region_id
LEFT JOIN timesheets ts ON u.user_id = ts.timesheet_user
INNER JOIN timesheet_mr tsmr ON ts.timesheet_caller = tsmr.tsmr_id AND tsmr.tsmr_id = 14
INNER JOIN timesheet_details tsd ON ts.timesheet_id = tsd.tsd_timesheet_id
INNER JOIN timesheet_day_details tsdd ON tsd.tsd_id = tsdd.tsd_id
RIGHT OUTER JOIN category_type_details ct ON tsd.tsd_category_type_id = ct.ctd_id AND u.employee_id = "0079-P"
GROUP BY ct.ctd_id, u.user_id, u.employee_id, u.user_name,
COALESCE(pd.ctd_name, pp.ctd_name), r.region_name, ct.ctd_name
ORDER BY ct.ctd_id, u.user_id, u.employee_id, u.user_name,
COALESCE(pd.ctd_name, pp.ctd_name), r.region_name, ct.ctd_name;
You only got 1 row because the condition in WHERE clause filter all the NULL user_id rows for other category.
For MySQL, you could omit other columns in GROUP BY clause:
SELECT u.employee_id AS Employee_ID, u.user_name AS UserName,
COALESCE(pd.ctd_name, pp.ctd_name) AS PrimaryDeptOrProj,
r.region_name AS Region, ct.ctd_id, ct.ctd_name, SUM(tsdd.workhours)
FROM users u
LEFT JOIN category_type_details pd ON u.user_primary_department = pd.ctd_id
LEFT JOIN category_type_details pp ON u.user_primary_project = pp.ctd_id
LEFT JOIN regions r ON u.region_id = r.region_id
LEFT JOIN timesheets ts ON u.user_id = ts.timesheet_user
INNER JOIN timesheet_mr tsmr ON ts.timesheet_caller = tsmr.tsmr_id AND tsmr.tsmr_id = 14
INNER JOIN timesheet_details tsd ON ts.timesheet_id = tsd.tsd_timesheet_id
INNER JOIN timesheet_day_details tsdd ON tsd.tsd_id = tsdd.tsd_id
RIGHT OUTER JOIN category_type_details ct ON tsd.tsd_category_type_id = ct.ctd_id AND u.employee_id = "0079-P"
GROUP BY ct.ctd_id, u.user_id
ORDER BY ct.ctd_id, u.user_id;

Select threads from forum based on comments

I've got the following sql that will return a list of forums. Under each forum it will select the thread with the latest comment. This works fine but when a new thread hasn't got any comments, nothing is returned.
How to tackle this problem?
SELECT f.Id AS forum_id,
f.name AS forum_name,
f.slug AS forum_slug,
f.image AS forum_image,
t.Id AS thread_id,
t.title AS thread_topic,
t.unixtime AS thread_timestamp,
p.Id AS post_id,
p.content AS post_content,
p.unixtime AS post_timestamp,
(SELECT COUNT(*) FROM a_comments o WHERE o.forumID=f.Id AND o.teamId = {$teamId}) comments_count,
(SELECT COUNT(*) FROM a_threads w WHERE w.forumID=f.Id AND w.teamId = {$teamId}) threads_count
FROM a_forums f
LEFT JOIN (SELECT t2.forumID, max(COALESCE(p2.unixtime, t2.unixtime)) as ts, COUNT(p2.unixtime) as post_count
FROM a_threads t2
LEFT JOIN a_comments p2 ON p2.threadId = t2.id
GROUP BY t2.forumId) max_p ON f.id = max_p.forumId
LEFT JOIN a_comments p ON max_p.ts = p.unixtime AND p.teamId = {$teamId} AND p.deleted = 0
LEFT JOIN a_threads t ON f.Id = t.forumID AND (max_p.post_count = 0 OR p.threadId = t.ID) AND t.teamId = {$teamId} AND t.deleted = 0
ORDER BY f.id
I think you just have to change the LEFT JOIN in the first subquery to a JOIN. With the LEFT JOIN, you'll get NULL or a non-valid time for the comment. This then throws off the rest of the logic -- I think.
SELECT f.Id AS forum_id, f.name AS forum_name, f.slug AS forum_slug, f.image AS forum_image,
t.Id AS thread_id, t.title AS thread_topic, t.unixtime AS thread_timestamp,
p.Id AS post_id, p.content AS post_content, p.unixtime AS post_timestamp,
(SELECT COUNT(*) FROM a_comments o WHERE o.forumID=f.Id AND o.teamId = {$teamId}) as comments_count,
(SELECT COUNT(*) FROM a_threads w WHERE w.forumID=f.Id AND w.teamId = {$teamId}) as threads_count
FROM a_forums f LEFT JOIN
(SELECT t2.forumID, max(p2.unixtime) as ts,
COUNT(p2.unixtime) as post_count
FROM a_threads t2 JOIN
a_comments p2
ON p2.threadId = t2.id
GROUP BY t2.forumId
) max_p
ON f.id = max_p.forumId LEFT JOIN
a_comments p
ON max_p.ts = p.unixtime AND p.teamId = {$teamId} AND
p.deleted = 0 LEFT JOIN
a_threads t
ON f.Id = t.forumID AND (max_p.post_count = 0 OR p.threadId = t.ID) AND t.teamId = {$teamId} AND t.deleted = 0
ORDER BY f.id

Mysql left join 4 possible values

we're running this query in moodle database:
SELECT l.*,
MAX(l.time) as time,cat.id as catid, cat.name as catname,c.id as courseid,
c.fullname as coursename,
m.name as module,
r.revision as revision,
r.name as resourcename,
sco.name as scormname,
glo.name as glossaryname,
qui.name as quizname
FROM {log} l
INNER JOIN {user} u ON l.userid = u.id
INNER JOIN {course} c ON l.course = c.id
INNER JOIN {course_categories} cat ON c.category = cat.id
INNER JOIN {course_modules} cm ON l.cmid = cm.id
INNER JOIN {modules} m ON cm.module = m.id
LEFT JOIN {resource} r ON cm.instance = r.id and l.module = "resource"
LEFT JOIN {scorm} sco ON cm.instance = sco.id and l.module = "scorm"
LEFT JOIN {glossary} glo ON cm.instance = glo.id and l.module = "glossary"
LEFT JOIN {quiz} qui ON cm.instance = qui.id and l.module = "quiz"
WHERE LOWER(l.action) LIKE LOWER(:modaction) COLLATE utf8_bin AND l.userid = :userid
GROUP BY l.cmid
ORDER BY MAX(l.time) DESC
and i would like something like this:
SELECT l.*,
MAX(l.time) as time,cat.id as catid, cat.name as catname,c.id as courseid,
c.fullname as coursename,
m.name as module,
r.revision as revision,
r.name as resourcename
FROM {log} l
INNER JOIN {user} u ON l.userid = u.id
INNER JOIN {course} c ON l.course = c.id
INNER JOIN {course_categories} cat ON c.category = cat.id
INNER JOIN {course_modules} cm ON l.cmid = cm.id
INNER JOIN {modules} m ON cm.module = m.id
INNER JOIN (
SELECT * {resource} re WHERE cm.instance = r.id OR
SELECT * {scorm} sco WHERE cm.instance = sco.id OR
SELECT * {glossary} WHERE ON cm.instance = glo.id OR
SELECT * {quiz} qui WHERE cm.instance = qui.id OR
) as r
WHERE LOWER(l.action) LIKE LOWER(:modaction) COLLATE utf8_bin AND l.userid = :userid
GROUP BY l.cmid
ORDER BY MAX(l.time) DESC
Results return NULL values in "resource" names (resourcename, scormname, glossaryname or quizname) and only one valid "resource" name. I render results depending on if result is not null.
But i would like to avoid "filter" by not null result in resourcename, scormname, glossaryname and quizname using PHP.
Thanks!
Something like this maybe - use a union
SELECT l.cmid,
MAX(l.time) as logtime,
cat.id as catid,
cat.name as catname,
c.id as courseid,
c.fullname as coursename,
mods.module,
mods.revision,
mods.name as modname
FROM mdl_log l
JOIN mdl_user u ON l.userid = u.id
JOIN mdl_course c ON l.course = c.id
JOIN mdl_course_categories cat ON c.category = cat.id
JOIN (
SELECT cm.id AS cmid, m.name AS module, r.name, r.revision
FROM mdl_course_modules cm
JOIN mdl_modules m ON cm.module = m.id AND m.name = 'resource'
JOIN mdl_resource r ON r.id = cm.instance
UNION
SELECT cm.id AS cmid, m.name as modulentype, s.name, NULL AS revision
FROM mdl_course_modules cm
JOIN mdl_modules m ON cm.module = m.id AND m.name = 'scorm'
JOIN mdl_scorm s ON s.id = cm.instance
UNION
SELECT cm.id AS cmid, m.name as modulentype, g.name, NULL AS revision
FROM mdl_course_modules cm
JOIN mdl_modules m ON cm.module = m.id AND m.name = 'glossary'
JOIN mdl_glossary g ON g.id = cm.instance
UNION
SELECT cm.id AS cmid, m.name as modulentype, q.name, NULL AS revision
FROM mdl_course_modules cm
JOIN mdl_modules m ON cm.module = m.id AND m.name = 'quiz'
JOIN mdl_quiz q ON q.id = cm.instance
) mods ON mods.cmid = l.cmid AND mods.module = l.module
WHERE l.module IN ('resource', 'scorm', 'glossary', 'quiz')
GROUP BY l.cmid
ORDER BY 1 DESC

How write mysql query corresponding this tables

Hi database professionals. I need your help
This is my database tables
I want to get this result
How can I write select query to get that result ?
I write this query , but it works very slow (20 minute)
SELECT c.name, p.title, tbl1.count, tbl1.sum,tbl2.count_1, tbl2.sum_1, tbl3.count_2, tbl3.sum_2
FROM `client` c LEFT JOIN
(SELECT c.name, p.id, p.title,p.unit, o.client_id, COUNT(*) AS `count`, SUM(op.price) AS `sum`
FROM `order` o INNER JOIN order_product op ON o.id = op.order_id
INNER JOIN product p ON op.product_id = p.id
INNER JOIN `client` c ON c.id = o.client_id
WHERE MONTH(o.date) = MONTH('2013-07-01') AND YEAR(o.date) = YEAR('2013-07-01') AND p.category_id = 1
GROUP BY c.id, p.id, YEAR(o.date), MONTH(o.date)) AS tbl1 ON c.id = tbl1.client_id
LEFT JOIN
(SELECT p.id, p.title,p.unit, c.name, o.client_id, COUNT(*) AS `count_1`, SUM(op.price) AS `sum_1`
FROM `order` o INNER JOIN order_product op ON o.id = op.order_id
INNER JOIN product p ON op.product_id = p.id
INNER JOIN `client` c ON c.id = o.client_id
WHERE MONTH(o.date) = MONTH(DATE_ADD('2013-07-01', INTERVAL 1 MONTH)) AND
YEAR(o.date) = YEAR(DATE_ADD('2013-07-01', INTERVAL 1 MONTH)) AND p.category_id = 1
GROUP BY c.id, p.id, YEAR(o.date), MONTH(o.date)) AS tbl2 ON c.id = tbl2.client_id
LEFT JOIN
(SELECT p.id, p.title,p.unit, c.name, o.client_id, COUNT(*) AS `count_2`, SUM(op.price) AS `sum_2`
FROM `order` o INNER JOIN order_product op ON o.id = op.order_id
INNER JOIN product p ON op.product_id = p.id
INNER JOIN `client` c ON c.id = o.client_id
WHERE MONTH(o.date) = MONTH(DATE_ADD('2013-07-01', INTERVAL 2 MONTH)) AND
YEAR(o.date) = YEAR(DATE_ADD('2013-07-01', INTERVAL 2 MONTH)) AND p.category_id = 1
GROUP BY c.id, p.id, YEAR(o.date), MONTH(o.date)) AS tbl3 ON c.id = tbl3.client_id
, product p
WHERE (tbl1.id = p.id OR tbl2.id = p.id OR tbl3.id = p.id)
ORDER BY c.name

Select a product from filters

I have a problem with a query:
I have 3 tables:
products (id, name)
settings (id, name)
product_setting (product_id, setting_id)
for example: I would like to select only the products you have selected filters!
I do this:
SELECT p. *, s.id as setting
FROM Products p
INNER JOIN product_setting p2 ON (p.id = p2.product_id)
INNER JOIN settings s ON (s.id = p2.setting_id)
WHERE s.id IN (1,2)
but I get all products that have the 'setting' id = 1 OR id = 2.
How to get only those products that have those 'setting' (AND)?
thanks!!
SELECT p.*, s.id as setting
FROM Products p
INNER JOIN product_setting p2 ON (p.id = p2.product_id)
INNER JOIN settings s ON (s.id = p2.setting_id)
WHERE s.id IN (1,2)
GROUP BY p.id
HAVING COUNT(*)=2; // size of IN()
This seems like over kill but...
SELECT p. *, s.id as setting
FROM Products p
INNER JOIN product_setting p2 ON (p.id = p2.product_id)
INNER JOIN settings s ON (s.id = p2.setting_id)
INNER JOIN settings s2 ON (s.id = p2.setting_id)
WHERE
s.id = 1
AND s2.id = 2