Query With Both Join and Derived Table - mysql

I'm trying to add row counts to an existing query but am getting an error
unknown column 'pl.GroupNumber' in 'on clause'
I'm apparently going to answer this too in case it helps someone else as, after writing all this here, it occurred to me to reverse the FROM to:
*FROM (SELECT #row_number:=0) AS t, parts_listing pl *
which seemed to do the trick! Don't quite understand why it worked but it did.
SELECT DISTINCT
(#row_number:=#row_number + 1) AS RowNum,
pl.ID,
pn.ID AS SubID,
IF(ListType,(SELECT MAX(ID) FROM parts_notes
WHERE PageNo=429
AND ListType IS NOT NULL AND SubPage IS NULL AND
(BasePart IS NOT NULL AND Models IS NOT NULL
AND (BasePart=pl.PartNo
AND pl.Models LIKE CONCAT('%', Models ,'%')))
OR (BasePart IS NOT NULL AND Models IS NULL
AND BasePart=pl.PartNo) ),NULL) AS SubMax,
ListType,
IndentText,
BaseGroup,
BaseName,
GroupName,
Title,
`Name`,
pl.Models,
pl.PartNo,
pn.PartNo AS SubPartNo,
pl.Quantity,
pn.Quantity AS SubQuantity,
pn.Description AS SubDescription,
ListType,
Column_1,
Column_2,
Column_3,
Column_4,
Column_5,
Column_6,
Column_7,
Column_8,
COALESCE(pl.GroupNumber, pn.GroupNo) AS GroupNo,
COALESCE(pl.Description, pn.Description) AS Description,
COALESCE(pl.PageNo, pn.PageNo) AS PageNo,
COALESCE(pl.SubPage, pn.SubPage) AS SubPage,
COALESCE(pl.RevDate, pn.RevDate) AS RevDate,
COALESCE(pl.Edition, pn.Edition) AS Edition
FROM parts_listing pl, (SELECT #row_number:=0) AS t
LEFT JOIN parts_notes pn ON pl.GroupNumber=pn.GroupNo
AND ((ListType < 5 AND (pl.PartNo=BasePart AND BasePart IS NOT NULL) OR BasePart IS NULL)
OR ((ListType > 4) AND (pl.GroupNumber=pn.GroupNo OR pl.PartNo=BasePart)))
WHERE BaseGroup=30 AND pl.PageNo=429 AND (pl.SubPage IS NULL
AND pn.SubPage IS NULL)
AND (pl.PageNo = pn.PageNo OR pn.PageNo IS NULL)
AND (pl.SubPage = pn.SubPage OR pn.SubPage IS NULL)
ORDER BY pl.ID, pn.ID
LIMIT 150;

Do not use commas in the FROM clause, even when you intend CROSS JOIN. The scoping rules are not what you expect, which is why you get an unknown column.
I put parameter conditions on the end, so I would write this as:
FROM parts_listing pl LEFT JOIN
parts_notes pn
ON pl.GroupNumber = pn.GroupNo AND
((ListType < 5 AND
(pl.PartNo = BasePart AND BasePart IS NOT NULL
) OR BasePart IS NULL
) OR
((ListType > 4) AND
(pl.GroupNumber = pn.GroupNo OR pl.PartNo = BasePart)
)
) CROSS JOIN
(SELECT #row_number := 0) params

Related

Problems with query speed when using a nested query for item count

When I add the nested query for invCount, my query time goes from .03 sec to 14 sec. The query works and I get correct values, but it is very, very slow in comparison. Is that just because I have to many conditions in that query? When I take it out and still have the second nested query, the time is still .03 secs. There is clearly something about the first nested query the database doesn't like, but I am not seeing what it is. I have a foreign key set for all the inner join lines too. Any help or ideas would be appreciated.
SELECT a.*,
f.name,
f.partNumber,
f.showInAdminStore,
f.showInPublicStore,
f.productImage,
r.mastCatID,
(SELECT COUNT(b.inventoryID)
FROM storeInventory b
INNER JOIN events c ON c.eventID = b.eventID
WHERE b.pluID = a.pluID
AND b.listPrice = a.listPrice
AND b.unlimitedQty = a.unlimitedQty
AND (b.packageID = a.packageID OR (b.packageID IS NULL AND a.packageID IS NULL))
AND b.orderID IS NULL
AND c.isOpen = '1'
AND b.paymentTypeID <= '2'
AND (b.inCart < '$cartTime' OR b.inCart IS NULL) ) AS invCount,
(SELECT COUNT(x.inventoryID)
FROM storeInventory x
WHERE x.packageID = a.inventoryID) AS packageCount
FROM storeInventory a
INNER JOIN storePLUs f ON f.pluID = a.pluID
INNER JOIN storeCategories r ON r.catID = f.catID
INNER JOIN events d ON d.eventID = a.eventID
WHERE a.storeFrontID = '1'
AND a.orderID IS NULL
AND a.paymentTypeID <= '2'
AND d.isOpen = '1'
GROUP BY a.packageID, a.unlimitedQty, a.listPrice, a.pluID
Table from query output
UPDATE: 12/12/2022
I changed the line checking the packageID to "AND (b.packageID <=> a.packageID)" as suggested and that cut my query time down to 7.8 seconds from 14 seconds. Thanks for the pointer. I will definitely use that in the future for NULL comparisons.
using "count(*)" took about half a second off. When I take the first nested query out, it drops down to .05 seconds even with the other nested queries in there, so I feel like there is still something causing issues. I tried running it without the other "AND (b.inCart < '$cartTime' OR b.inCart IS NULL)" line and that did take about a second off, but no where what I was hoping for. Is there an operand that includes NULL on a less than comparison? I also tried running it without the inner join in the nested query and that didn't change much at all. Of course removing any of that, throughs the values off and they become incorrect, so I can't run it that way.
Here is my current query setup that still pulls correct values.
SELECT a.*,
f.name,
f.partNumber,
f.showInAdminStore,
f.showInPublicStore,
f.productImage,
r.mastCatID,
(SELECT COUNT(*)
FROM storeInventory b
INNER JOIN events c ON c.eventID = b.eventID
WHERE b.pluID = a.pluID
AND b.listPrice = a.listPrice
AND b.unlimitedQty = a.unlimitedQty
AND (b.packageID <=> a.packageID)
AND b.orderID IS NULL
AND c.isOpen = '1'
AND b.paymentTypeID <= '2'
AND (b.inCart < '$cartTime' OR b.inCart IS NULL) ) AS invCount,
(SELECT COUNT(x.inventoryID)
FROM storeInventory x
WHERE x.packageID = a.inventoryID) AS packageCount
FROM storeInventory a
INNER JOIN storePLUs f ON f.pluID = a.pluID
INNER JOIN storeCategories r ON r.catID = f.catID
INNER JOIN events d ON d.eventID = a.eventID
WHERE a.storeFrontID = '1'
AND a.orderID IS NULL
AND a.paymentTypeID <= '2'
AND d.isOpen = '1'
GROUP BY a.packageID, a.unlimitedQty, a.listPrice, a.pluID
I am not familiar with the term 'Composite indexes' Is that something different than these?
Screenshot of ForeignKeys on Table a
I think
AND (b.packageID = a.packageID
OR (b.packageID IS NULL
AND a.packageID IS NULL)
)
can be simplified to ( https://dev.mysql.com/doc/refman/8.0/en/comparison-operators.html#operator_equal-to ):
AND ( b.packageID <=> a.packageID )
Use COUNT(*) instead of COUNT(x.inventoryID) unless you check for not-NULL.
The subquery to compute packageCount seems strange; you seem to count inventories but join on packages.
The need to reach into another table to check isOpen is part of the performance problem. If eventID is not the PRIMARY KEYforevents, then add INDEX(eventID, isOpen)`.
Some other indexes that may help:
a: INDEX(storeFrontID, orderID, paymentTypeID)
a: INDEX(packageID, unlimitedQty, listPrice, pluID)
b: INDEX(pluID, listPrice, unlimitedQty, orderID)
f: INDEX(pluID, catID)
r: INDEX(catID, mastCatID)
x: INDEX(packageID, inventoryID)
After OP's Update
There is no way to do (x<y OR x IS NULL) except by switching to a UNION. In your case, it is pretty easy to do the conversion. Replace
( SELECT COUNT(*) ... AND ( b.inCart < '$cartTime'
OR b.inCart IS NULL ) ) AS invCount,
with
( SELECT COUNT(*) ... AND b.inCart < '$cartTime' ) +
( SELECT COUNT(*) ... AND b.inCart IS NULL ) AS invCount,
Revised indexes:
storePLUs:
INDEX(pluID, catID)
storeCategories:
INDEX(catID, mastCatID)
events:
INDEX(isOpen, eventID)
storeInventory:
INDEX(pluID, listPrice, unlimitedQty, orderID, packageID)
INDEX(pluID, listPrice, unlimitedQty, orderID, inCart)
INDEX(packageID, inventoryID)
INDEX(storeFrontID, orderID, paymentTypeID)

How can I write this code in MySQL in a shorter and better way?

There are 3 tables in the database and they have no relation. How can I write this code in a short form? Sorry for my bad english already.
MySQL Code;
SELECT
*,
(SELECT COUNT(c2.id) FROM comments AS c2 WHERE c2.phone_number = comments.phone_number AND c2.country_code = comments.country_code AND c2.approve = "0" AND c2.deleted_at IS NULL) AS comment_count,
(SELECT COUNT(f.id) FROM flag_numbers AS f WHERE f.phone_number = comments.phone_number AND f.country_code = comments.country_code AND comments.approve = "0" AND comments.deleted_at IS NULL) AS flag_count
FROM comments
WHERE approve = "0" AND deleted_at IS NULL
ORDER BY id DESC
Your code is pretty good, but the following is a bit simpler:
SELECT c.*,
COUNT(*) OVER (PARTITION BY c.phone_Number, c.country_code) as comment_count,
(SELECT COUNT(f.id)
FROM flag_numbers f
WHERE f.phone_number = c.phone_number AND
f.country_code = c.country_code
) AS flag_count
FROM comments c
WHERE c.approve = '0' AND c.deleted_at IS NULL
ORDER BY id DESC;
If approve is a number, then remove the single quotes. Don't quote number constants.
The differences:
Replacement of the first subquery with a window function for comment_count.
Removal of the redundant comment filters in the flag_count.

MySQL: Getting the results of these two subselects in one

I searched a lot, but did not find a fitting answer, and after trying for several hours its time to ask someone who knows :)
Ok, This is the Query its about (all unneccessary content stripped):
SELECT
`id`,
`name`,
`details`,
`tour_type_id`,
`meeting_point_id`,
`start_date`,
`start_time`,
`end_date`,
`end_time`,
`max_guests`,
(SELECT
MAX(`level`)
FROM
`tour_notification`
WHERE
`model_id` = `tour`.`id`
AND `deactivated` != 1) `notification_level`,
(SELECT
MAX(`level`)
FROM
`guide_notification`
WHERE
(`tour1_id` = `tour`.`id`
OR `tour2_id` = `tour`.`id`)
AND `deactivated` != 1) `guide_notification_level`
FROM
`tour`
What I did not succeed in was to get the 2 MAX(level) in one:
Get the MAX from notification_level and guide_notification_level.
I do not need these values seperately, I just did not succeed getting in in one.
My best attempt was the following approach:
(SELECT
MAX(`level`)
FROM
(SELECT `level` FROM
`tour_notification`
WHERE
`model_id` = `tour`.`id`
AND `deactivated` != 1 UNION DISTINCT SELECT `level`
FROM
`guide_notification`
WHERE
(`tour1_id` = `tour`.`id`
OR `tour2_id` = `tour`.`id`)
AND `deactivated` != 1) as `notifications`) `notification_level`,
but here MySQL complained:
ERROR: Error 1054: Unknown column 'tour.id' in 'where clause'
If I add the tour table into the where clause of these subselects, it always gets the results for any items of tour table, not just the current one.
Anyone can push me in the right direction?
Thanks a lot!
The following might work for you. It provides three different ways of combining the levels into one column. Remove the ones you do not want.
SELECT
a.`id`,
a.`name`,
a.`details`,
a.`tour_type_id`,
a.`meeting_point_id`,
a.`start_date`,
a.`start_time`,
a.`end_date`,
a.`end_time`,
a.`max_guests`,
a.`tour_level`,
a.`guide_level`,
CONCAT(`tour_level`," ",`guide_level`) as `notification_level1`,
a.`tour_level` + a.`guide_level` as `notification_level2`,
GROUP_CONCAT(CONCAT(a.`tour_level`," ",a.`guide_level`) as `notification_level3`
FROM `tour` a
LEFT JOIN `tour_notification` b
ON a.`id` = b.`model_id` AND b.`deactivated` <> 1
LEFT JOIN `guide_notification` c
ON a.`id` = c.`tour2_id` AND c.`deactivated` <> 1
GROUP BY a.`id`
If you need the greatest value of two, you can use the GRATEST() function. But I think a UNION ALL subquery with LIMIT 1 would be more readable:
(
SELECT MAX(`level`) AS `max_level`
FROM `tour_notification`
WHERE `model_id` = `tour`.`id`
AND `deactivated` != 1
UNION ALL
SELECT MAX(`level`) AS `max_level`
FROM `guide_notification`
WHERE (`tour1_id` = `tour`.`id` OR `tour2_id` = `tour`.`id`)
AND `deactivated` != 1
ORDER BY max_level DESC
LIMIT 1
) AS `notification_level`

MySQL Find Most Recent/Largest Record Per Group by order by

MySQL Find Most Recent/Largest Record Per Group by order by, and how can i minimize / shorten this query and every time it returns the first row value of the group, whereas i like to select the last row value of a group, and sort the values based on ja.id ? I know this is a bad query can anyone suggest or provide me solution to shorten this query . I have used all the necessary Column Indexes in all the tables. How to shorten the query without the use of union all .both the queries in union all are same expect in the where statement.
SELECT
a.previous_status,
a.rejected_status,
a.rejection_reason_text,
a.rejection_reason,
a.rjaId,
a.refer_applied_status,
a.title,
a.playerId,
a.gameId,
a.gamePostDate,
a.game_referal_amount,
a.country,
a.country_name,
a.state,
a.location,
a.state_abb,
a.game_type,
a.appliedId,
a.appliedStatus,
a.admin_review,
a.is_req_referal_check,
a.memberId,
a.appliedEmail,
a.game_id,
a.referred_id,
a.memStateAbb,
a.memState,
a.memZipcode,
a.memCity,
a.memCountryNme,
a.memCountry,
a.appliedMemberName,
a.first_name,
a.primary_contact,
a.last_name,
a.addressbookImage,
a.userImage,
a.last_login,
a.user_experience_year,
a.user_experience_month,
a.current_designation,
a.current_player,
a.appliedDate,
a.addressbook_id,
a.joiningdate,
a.gameStatus,
a.gameReferalAmountType,
a.gameFreezeStatus,
a.gameFreezeMsg,
a.app_assign_back_to_rp_count,
a.applied_source,
a.max_id,
a.gamesApplied,
a.gamesAppliedId,
SUM(a.totalgameApplied) AS totalgameApplied,
a.application_assign_to_rp_status,
a.rpAppliedSource,
a.applied_on
FROM
(
(
SELECT
ja.previous_status,
ja.rejected_status,
ja.rejection_reason_text,
ja.rejection_reason,
rja.id AS rjaId,
rja. STATUS AS refer_applied_status,
jp.title,
jp.user_user_id AS playerId,
jp.id AS gameId,
jp.posted_on AS gamePostDate,
jp.game_referal_amount,
jp.country,
jp.country_name,
jp.state,
jp.location,
jp.state_abb,
jp.game_type,
ja.id AS appliedId,
IFNULL(ja. STATUS, '') AS appliedStatus,
IFNULL(ja.admin_review, '') AS admin_review,
ja.is_req_referal_check,
usr.id AS memberId,
rja.email AS appliedEmail,
rja.game_id,
rja.referred_id,
mem.state_abb AS memStateAbb,
mem.state AS memState,
mem.zipcode AS memZipcode,
mem.city AS memCity,
mem.country_name AS memCountryNme,
mem.country_code AS memCountry,
usc. NAME AS appliedMemberName,
usc.first_name,
IFNULL(
mem.primary_contact,
usc.phone_number
) AS primary_contact,
usc.last_name,
usc.profileimage_path AS addressbookImage,
usr.profile_image AS userImage,
usr.last_login,
mem.user_experience_year,
mem.user_experience_month,
mem.current_designation,
mem.current_player,
rja.create_date AS appliedDate,
rja.addressbook_id,
IFNULL(ja.joining_date, '') AS joiningdate,
jp. STATUS AS gameStatus,
jp.games_referal_amount_type AS gameReferalAmountType,
jp.game_freeze_status AS gameFreezeStatus,
jp.game_freeze_message AS gameFreezeMsg,
ja.app_assign_back_to_rp_count,
ja.applied_source,
MAX(rja.id) AS max_id,
GROUP_CONCAT(
jp.title
ORDER BY
rja.create_date DESC
) AS gamesApplied,
GROUP_CONCAT(DISTINCT(jp.id)) AS gamesAppliedId,
COUNT(DISTINCT(jp.id)) totalgameApplied,
ja.application_assign_to_rp_status,
1 AS rpAppliedSource,
rja.create_date AS applied_on
FROM
(`refer_gameapplied` AS rja)
JOIN `games_post` AS jp ON `jp`.`id` = `rja`.`game_id`
JOIN `user_socialconnections` AS usc ON `rja`.`addressbook_id` = `usc`.`id`
LEFT JOIN `user_user` AS usr ON `usr`.`email` = `rja`.`email`
LEFT JOIN `user_member` AS mem ON `mem`.`user_id` = `usr`.`id`
LEFT JOIN `game_applied` AS ja ON `ja`.`id` = `rja`.`applied_id`
WHERE
`rja`.`referby_id` = '2389'
GROUP BY
`rja`.`email`
)
UNION ALL
(
SELECT
ja.previous_status,
ja.rejected_status,
ja.rejection_reason_text,
ja.rejection_reason,
jr.id AS rjaId,
jrtm. STATUS AS refer_applied_status,
jp.title,
jp.user_user_id AS playerId,
jp.id AS gameId,
jp.posted_on AS gamePostDate,
jp.game_referal_amount,
jp.country,
jp.country_name,
jp.state,
jp.location,
jp.state_abb,
jp.game_type,
ja.id AS appliedId,
IFNULL(ja. STATUS, '') AS appliedStatus,
IFNULL(ja.admin_review, '') AS admin_review,
ja.is_req_referal_check,
usr.id AS memberId,
jrtm.referto_email AS refappliedEmail,
jr.game_id,
jrtm.id,
mem.state_abb AS memStateAbb,
mem.state AS memState,
mem.zipcode AS memZipcode,
mem.city AS memCity,
mem.country_name AS memCountryNme,
mem.country_code AS memCountry,
usc. NAME AS appliedMemberName,
usc.first_name,
IFNULL(
mem.primary_contact,
usc.phone_number
) AS primary_contact,
usc.last_name,
usc.profileimage_path AS addressbookImage,
usr.profile_image AS userImage,
usr.last_login,
mem.user_experience_year,
mem.user_experience_month,
mem.current_designation,
mem.current_player,
jrtm.refer_on AS appliedDate,
jrtm.referto_addressbookid,
IFNULL(ja.joining_date, '') AS joiningdate,
jp. STATUS AS gameStatus,
jp.games_referal_amount_type AS gameReferalAmountType,
jp.game_freeze_status AS gameFreezeStatus,
jp.game_freeze_message AS gameFreezeMsg,
ja.app_assign_back_to_rp_count,
ja.applied_source,
MAX(jrtm.id) AS max_id,
GROUP_CONCAT(
jp.title
ORDER BY
jr.refer_on DESC
) AS gamesApplied,
GROUP_CONCAT(DISTINCT(jp.id)) AS gamesAppliedId,
COUNT(DISTINCT(jp.id)) totalgameApplied,
ja.application_assign_to_rp_status,
2 AS rpAppliedSource,
jrtm.refer_on AS applied_on
FROM
(`game_refer` AS jr)
JOIN `game_refer_to_member` AS jrtm ON `jrtm`.`rid` = `jr`.`id`
JOIN `games_post` AS jp ON `jp`.`id` = `jr`.`game_id`
JOIN `user_socialconnections` AS usc ON `jrtm`.`referto_addressbookid` = `usc`.`id`
LEFT JOIN `user_user` AS usr ON `usr`.`email` = `jrtm`.`referto_email`
LEFT JOIN `user_member` AS mem ON `mem`.`user_id` = `usr`.`id`
LEFT JOIN `game_applied` AS ja ON `ja`.`referred_by` = `jrtm`.`id`
WHERE
`jrtm`.`status` = '1'
AND `jr`.`referby_user_id` = '2389'
AND `jrtm`.`refer_source` NOT IN ('4')
GROUP BY
`jrtm`.`referto_email`
)
) a
GROUP BY
a.appliedEmail
ORDER BY
a.gamesAppliedId DESC
It sounds like a "groupwise-max" problem. I added a tag that you should research.
At least get rid of the columns that are not relevant to the question.
Try tossing the tables other than jrtm and jr to see if the performance problem persists. (I'm thinking that the LEFT JOINs may be red herrings.)
Try with one part of the UNION, then with the other. This may identify which of the two is more of a burden.
Some indexes to add:
rja: (referby_id, create_date)
jrtm: (status, referto_email)
jrtm: (rid, status, referto_email)
jr: (referby_user_id, refer_on)
DISTINCT is not a function. Don't use parents in DISTINCT(jp.id).

MySQL - How can I do a conditional WHERE and OR clause Query?

Problem 1:
I need to add a field in a WHERE clause with a condition in my stored procedure, so what is the right syntax for the below query in CASE condition?
Problem 2:
OR condition is giving me all records with values 0,1,2,3,4 for below condition (I need tqm.is_imp = 1 if not found then tqm.is_imp IN (0,2,3,4)). What should I do? Please help.
tqm.is_imp = 1 OR tqm.is_imp IN (0,2,3,4)
My Query:
SELECT tqm.id INTO selectedQuestionNumber
FROM tc_question_master tqm
INNER JOIN tc_question_mapping tqmap ON tqmap.tc_question_id = tqm.id
WHERE tqmap.syllabus_chapter_details_id IN (SELECT cms_id from topic where chapter_id IN (SELECT id from MyChapters))
AND tqm.marks = CurMark
CASE WHEN CurQuestionType > 0 THEN
AND tqm.question_type = CurQuestionType
END
AND (tqm.level_of_question = 'Easy' OR tqm.level_of_question
IN ('Moderate','Difficult','Very difficult'))
AND (tqm.is_imp = 1 OR tqm.is_imp IN (0,2,3,4))
AND tqm.id NOT IN (SELECT q.question_id FROM QBDetails q)
ORDER BY RAND() LIMIT 1;
The upper part is just a reformat, but lower as you approach the logic that was a case expression, most of that is pure guess.
SELECT
tqm.id INTO selectedQuestionNumber
FROM tc_question_master tqm
INNER JOIN tc_question_mapping tqmap ON tqmap.tc_question_id = tqm.id
WHERE tqmap.syllabus_chapter_details_id IN (
SELECT
cms_id
FROM topic
WHERE chapter_id IN (
SELECT
id
FROM MyChapters
)
)
AND (
tqm.level_of_question = 'Easy'
OR tqm.level_of_question IN ('Moderate', 'Difficult', 'Very difficult')
)
AND (
tqm.is_imp = 1
OR tqm.is_imp IN (0, 2, 3, 4)
)
AND tqm.id NOT IN (
SELECT
q.question_id
FROM QBDetails q
)
## now it is really vague
AND (
tqm.marks = CurMark
OR
(CurQuestionType > 0 AND tqm.question_type = CurQuestionType)
)
## end of guesswork
ORDER BY
RAND()
LIMIT 1;