SQL Inner Query WHERE clause access to Outer Query tables - mysql

Good morning -
This is my first post here, after many years using SO as a very useful resource.
I've run into a problem with a complex (for me) query I'm pulling together for a wordpress site running woocommerce to process orders. I'm trying to add a filter to the order list which filters orders which contain products in a particular product category.
I'm afraid I've gotten in over my head with this query which joins a variety of meta tables on inner queries in order to get at the information I need in order to determine the product's category.
The problem is that I can't get the scoping rules to work in order to access required outer table information in the inner queries.
The query is:
SELECT SQL_CALC_FOUND_ROWS
wp_ot6q6i_posts.ID
FROM
wp_ot6q6i_posts
WHERE
1 = 1 AND YEAR(wp_ot6q6i_posts.post_date) = 2015 AND MONTH(wp_ot6q6i_posts.post_date) = 12 AND wp_ot6q6i_posts.post_type = 'shop_order' AND(
(
wp_ot6q6i_posts.post_status = 'wc-pending' OR wp_ot6q6i_posts.post_status = 'wc-processing' OR wp_ot6q6i_posts.post_status = 'wc-on-hold' OR wp_ot6q6i_posts.post_status = 'wc-completed' OR wp_ot6q6i_posts.post_status = 'wc-cancelled' OR wp_ot6q6i_posts.post_status = 'wc-refunded' OR wp_ot6q6i_posts.post_status = 'wc-failed'
)
) AND EXISTS(
SELECT
t2.PROD_ID
FROM
(
SELECT
wp_ot6q6i_woocommerce_order_itemmeta.meta_value AS PROD_ID
FROM
wp_ot6q6i_woocommerce_order_items
LEFT JOIN
wp_ot6q6i_woocommerce_order_itemmeta
ON
wp_ot6q6i_woocommerce_order_itemmeta.order_item_id = wp_ot6q6i_woocommerce_order_items.order_item_id
WHERE
wp_ot6q6i_woocommerce_order_items.order_item_type = 'line_item' AND wp_ot6q6i_woocommerce_order_itemmeta.meta_key = '_product_id' AND wp_ot6q6i_posts.ID = wp_ot6q6i_woocommerce_order_items.order_id
) t1
INNER JOIN
(
SELECT DISTINCT
wposts.ID AS PROD_ID
FROM
wp_ot6q6i_posts wposts
LEFT JOIN
wp_ot6q6i_postmeta wpostmeta
ON
wposts.ID = wpostmeta.post_id
LEFT JOIN
wp_ot6q6i_term_relationships
ON
(
wposts.ID = wp_ot6q6i_term_relationships.object_id
)
LEFT JOIN
wp_ot6q6i_term_taxonomy
ON
(
wp_ot6q6i_term_relationships.term_taxonomy_id = wp_ot6q6i_term_taxonomy.term_taxonomy_id
)
WHERE
wp_ot6q6i_term_taxonomy.taxonomy = 'product_cat' AND wp_ot6q6i_term_taxonomy.term_id IN(
SELECT
term_id
FROM
`wp_ot6q6i_terms`
WHERE
slug = 'preorder'
)
ORDER BY
wpostmeta.meta_value
) t2
ON
t1.PROD_ID = t2.PROD_ID
)
ORDER BY
wp_ot6q6i_posts.post_date
DESC
LIMIT 0, 20
And the error I'm getting is:
1054 - Unknown column 'wp_ot6q6i_posts.ID' in 'where clause'

Thanks all for your help. I ended up going in a different direction to solve this problem, one I'm more comfortable with as a dev...I'm pulling the fixed list of items from the last join and building a query in code that has a series of more simple queries in the where clause, thereby avoiding the whole Exists approach.
Thanks again for your help.

Related

Modifying MySQL Query for WpDataTables

I am building a table in WpDataTables that involves two custom post types: 'applicants' and 'reviews'. I am trying to create a table that shows all of the applicants that have either been reviewed by the current user (reviewer) who is viewing the WpDataTable. Each applicant can have one or more reviews.
This query runs exactly like I want it to in MySQL, however, WpDataTables does not like it for some reason. I believe it is because of the subquery.
SELECT a.ID, r.post_title, a.post_title, a.guid, r.post_type, a.post_type, r.post_author,
b.meta_value review_app_score,
c.meta_value review_app_comment
FROM (SELECT * FROM `wppm_2_posts` WHERE post_author = 1 AND post_type = 'reviews' AND post_status = 'publish') as r
RIGHT JOIN `wppm_2_posts`as a ON r.post_title = a.ID
LEFT JOIN `wppm_2_postmeta` b ON r.ID = b.post_id AND b.meta_key='review_app_score'
LEFT JOIN `wppm_2_postmeta` c ON r.ID = c.post_id AND c.meta_key='review_app_comment'
WHERE a.post_type = 'applicants'
AND a.post_status = 'publish'
Here is what the MySQL results:
https://i.stack.imgur.com/KzvHt.png
This is exactly what I am needing, except I need it to work in WpDataTables.
The reason is that I am looking to take advantage of their dynamic placeholder (%CURRENT_USER_ID%) instead of the '1' in the subquery above for post_author.
Such as:
(SELECT * FROM `wppm_2_posts` WHERE post_author = %CURRENT_USER_ID% AND post_type = 'reviews' AND post_status = 'publish')
Is there another way I could write this without using a subquery? Or there is another way to build this without needing the dynamic placeholder such as PHP?
Any help is appreciated.
Here is the professional way (AKA it wasn't me) of writing the query. Not sure if it will help anyone else but hopefully it will.
SELECT
APPLICANTS.ID AS applicants_id
,APPLICANTS.post_title AS applicants_post_title
,APPLICANTS.guid AS applicants_guid
,APPLICANTS.post_type AS applicants_post_type
,REVIEWS.post_author AS reviews_author_id
,REVIEWS.post_type AS reviews_post_type
,REVIEWS.post_title AS reviews_post_title
,R_APP_SCORES.meta_value AS review_app_score
,R_APP_COMMENTS.meta_value AS review_app_comment
FROM
`%WPDB%posts` AS APPLICANTS
LEFT JOIN
`%WPDB%posts` AS REVIEWS ON
(REVIEWS.post_author = %CURRENT_USER_ID%) AND
(REVIEWS.post_title = APPLICANTS.ID) AND
(REVIEWS.post_type = 'reviews') AND
(REVIEWS.post_status = 'publish')
LEFT JOIN
`%WPDB%postmeta` R_APP_SCORES ON
(R_APP_SCORES.post_id = REVIEWS.ID) AND
(R_APP_SCORES.meta_key = 'review_app_score')
LEFT JOIN
`%WPDB%postmeta` R_APP_COMMENTS ON
(R_APP_COMMENTS.post_id = REVIEWS.ID) AND
(R_APP_COMMENTS.meta_key = 'review_app_comment')
WHERE
(APPLICANTS.post_type = 'applicants') AND
(APPLICANTS.post_status = 'publish')

How to Make This SQL Query More Efficient?

I'm not sure how to make the following SQL query more efficient. Right now, the query is taking 8 - 12 seconds on a pretty fast server, but that's not close to fast enough for a Website when users are trying to load a page with this code on it. It's looking through tables with many rows, for instance the "Post" table has 717,873 rows. Basically, the query lists all Posts related to what the user is following (newest to oldest).
Is there a way to make it faster by only getting the last 20 results total based on PostTimeOrder?
Any help would be much appreciated or insight on anything that can be done to improve this situation. Thank you.
Here's the full SQL query (lots of nesting):
SELECT DISTINCT p.Id, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime, p.Content AS Content, p.Bu AS Bu, p.Se AS Se, UNIX_TIMESTAMP(p.PostCreationTime) AS PostTimeOrder
FROM Post p
WHERE (p.Id IN (SELECT pc.PostId
FROM PostCreator pc
WHERE (pc.UserId IN (SELECT uf.FollowedId
FROM UserFollowing uf
WHERE uf.FollowingId = '100')
OR pc.UserId = '100')
))
OR (p.Id IN (SELECT pum.PostId
FROM PostUserMentions pum
WHERE (pum.UserId IN (SELECT uf.FollowedId
FROM UserFollowing uf
WHERE uf.FollowingId = '100')
OR pum.UserId = '100')
))
OR (p.Id IN (SELECT ssp.PostId
FROM SStreamPost ssp
WHERE (ssp.SStreamId IN (SELECT ssf.SStreamId
FROM SStreamFollowing ssf
WHERE ssf.UserId = '100'))
))
OR (p.Id IN (SELECT psm.PostId
FROM PostSMentions psm
WHERE (psm.StockId IN (SELECT sf.StockId
FROM StockFollowing sf
WHERE sf.UserId = '100' ))
))
UNION ALL
SELECT DISTINCT p.Id AS Id, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime, p.Content AS Content, p.Bu AS Bu, p.Se AS Se, UNIX_TIMESTAMP(upe.PostEchoTime) AS PostTimeOrder
FROM Post p
INNER JOIN UserPostE upe
on p.Id = upe.PostId
INNER JOIN UserFollowing uf
on (upe.UserId = uf.FollowedId AND (uf.FollowingId = '100' OR upe.UserId = '100'))
ORDER BY PostTimeOrder DESC;
Changing your p.ID in (...) predicates to existence predicates with correlated subqueries may help. Also since both halves of your union all query are pulling from the Post table and possibly returning nearly identical records you might be able to combine the two into one query by left outer joining to UserPostE and adding upe.PostID is not null as an OR condition in the WHERE clause. UserFollowing will still inner join to UPE. If you want the same Post record twice once with upe.PostEchoTime and once with p.PostCreationTime as the PostTimeOrder you'll need keep the UNION ALL
SELECT
DISTINCT -- <<=- May not be needed
p.Id
, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime
, p.Content AS Content
, p.Bu AS Bu
, p.Se AS Se
, UNIX_TIMESTAMP(coalesce( upe.PostEchoTime
, p.PostCreationTime)) AS PostTimeOrder
FROM Post p
LEFT JOIN UserPostE upe
INNER JOIN UserFollowing uf
on (upe.UserId = uf.FollowedId AND
(uf.FollowingId = '100' OR
upe.UserId = '100'))
on p.Id = upe.PostId
WHERE upe.PostID is not null
or exists (SELECT 1
FROM PostCreator pc
WHERE pc.PostId = p.ID
and pc.UserId = '100'
or exists (SELECT 1
FROM UserFollowing uf
WHERE uf.FollowedId = pc.UserID
and uf.FollowingId = '100')
)
OR exists (SELECT 1
FROM PostUserMentions pum
WHERE pum.PostId = p.ID
and pum.UserId = '100'
or exists (SELECT 1
FROM UserFollowing uf
WHERE uf.FollowedId = pum.UserId
and uf.FollowingId = '100')
)
OR exists (SELECT 1
FROM SStreamPost ssp
WHERE ssp.PostId = p.ID
and exists (SELECT 1
FROM SStreamFollowing ssf
WHERE ssf.SStreamId = ssp.SStreamId
and ssf.UserId = '100')
)
OR exists (SELECT 1
FROM PostSMentions psm
WHERE psm.PostId = p.ID
and exists (SELECT
FROM StockFollowing sf
WHERE sf.StockId = psm.StockId
and sf.UserId = '100' )
)
ORDER BY PostTimeOrder DESC
The from section could alternatively be rewritten to also use an existence clause with a correlated sub query:
FROM Post p
LEFT JOIN UserPostE upe
on p.Id = upe.PostId
and ( upe.UserId = '100'
or exists (select 1
from UserFollowing uf
where uf.FollwedID = upe.UserID
and uf.FollowingId = '100'))
Turn IN ( SELECT ... ) into a JOIN .. ON ... (see below)
Turn OR into UNION (see below)
Some the tables are many:many mappings? Such as SStreamFollowing? Follow the tips in http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
Example of IN:
SELECT ssp.PostId
FROM SStreamPost ssp
WHERE (ssp.SStreamId IN (
SELECT ssf.SStreamId
FROM SStreamFollowing ssf
WHERE ssf.UserId = '100' ))
-->
SELECT ssp.PostId
FROM SStreamPost ssp
JOIN SStreamFollowing ssf ON ssp.SStreamId = ssf.SStreamId
WHERE ssf.UserId = '100'
The big WHERE with all the INs becomes something like
JOIN ( ( SELECT pc.PostId AS id ... )
UNION ( SELECT pum.PostId ... )
UNION ( SELECT ssp.PostId ... )
UNION ( SELECT psm.PostId ... ) )
Get what you can done of that those suggestions, then come back for more advice if you still need it. And bring SHOW CREATE TABLE with you.

MySQL Query, (access mainquery from subquery)

I have a problem that I need a WHERE clause in a subquery that depends on the results of the main Query, otherwise my results would be wrong and the query takes too long / is not executeable.
The circumstances that I need this query to create a view which I need for a search server support the problem that I cannot split this into two queries, nor process it with a script dynamically.
The problem occurs with the following query:
SELECT `s`.`id` AS `seminar_id`, (SUM( `sub`.`seminar_rate` ) / COUNT( `sub`.`seminar_id` )) AS `total_rate`
FROM
(
SELECT (SUM( value ) / COUNT( * )) AS `seminar_rate` , `r`.`seminar_id`
FROM `rating` r
INNER JOIN `rating_item` ri ON `r`.`id` = `ri`.`rating_id`
WHERE `r`.`seminar_id` = `s`.`id`/* <- Here is my problem, this is inacessible */
GROUP BY `r`.`seminar_id`
) AS sub,
`seminar` s
INNER JOIN `date` d
ON `s`.`id` = `d`.`seminar_id`
INNER JOIN `date_unit` du
ON `d`.`id` = `du`.`date_id`
LEFT JOIN `seminar_subject` su
ON `s`.`id` = `su`.`seminar_id`
LEFT JOIN `subject` suj
ON `su`.`subject_id` = `suj`.`id`
INNER JOIN `user` u
ON `s`.`user_id` = `u`.`id`
INNER JOIN `company` c
ON `u`.`company_id` = `c`.`id`
GROUP BY `du`.`date_id`, `sub`.`seminar_id`
This query should calculate a total rate out of ratings for each Seminar.
However my ratings are stored in my "rating" table and should be processed live.
(Sidenote: If you wonder about all the joins: This query has alooot more SELECT'ed fields, I just removed them because they are not nesessary to solve the problem and to make the query look less complicated [I know it still is >.>]...)
The reason is that I want this results to be sortable by my search engine later depending
on the users sort parameters, thatswhy I need it inside this query.
The problem itself is pretty obvious:
ERROR 1054 (42S22): Unknown column 's.id' in 'where clause'
The subselect doesnt know about the results of the main query, is there a solution to bypass this?
Could someone give me a hint to get this working?
Thanks in advance.
Using your subquery in the JOIN you can eliminate the WHERE clause and achieve nearly the same result. Here is your modified query. Hope this solves your problem.
SELECT `s`.`id` AS `seminar_id`, (SUM( `sub`.`seminar_rate` ) / COUNT( `sub`.`seminar_id` )) AS `total_rate`
FROM `seminar` s
INNER JOIN
(
SELECT (SUM( value ) / COUNT( * )) AS `seminar_rate` , `r`.`seminar_id`
FROM `rating` r
INNER JOIN `rating_item` ri ON `r`.`id` = `ri`.`rating_id`
/*WHERE `r`.`seminar_id` = `s`.`id` <- Here is my problem, this is inacessible */
GROUP BY `r`.`seminar_id`
) AS sub ON s.id = sub.`seminar_id`
INNER JOIN `date` d
ON `s`.`id` = `d`.`seminar_id`
INNER JOIN `date_unit` du
ON `d`.`id` = `du`.`date_id`
LEFT JOIN `seminar_subject` su
ON `s`.`id` = `su`.`seminar_id`
LEFT JOIN `subject` suj
ON `su`.`subject_id` = `suj`.`id`
INNER JOIN `user` u
ON `s`.`user_id` = `u`.`id`
INNER JOIN `company` c
ON `u`.`company_id` = `c`.`id`
GROUP BY `du`.`date_id`, `sub`.`seminar_id`

Count number of occurrences based on user id

I'm very new to SQL/MySQL and Stackoverflow for that matter, and I'm trying to create a query through iReport (though I don't have to use iReport) for SugarCRM CE. What I need is to create a report that displays the number of "Referrals", "Voicemails", "Emails", and "Call_ins" that are linked to a specific "user" (employee). The query I currently have set up works; however it is running through the data multiple times generating a report that is 200+ pages. This is the code that I am currently using:
SELECT
( SELECT COUNT(*) FROM `leads` INNER JOIN `leads_cstm` ON `leads`.`id` = `leads_cstm`.`id_c` WHERE (leadtype_c = 'Referral' AND users.`id` = leads.`assigned_user_id`) ),
( SELECT COUNT(*) FROM `leads` INNER JOIN `leads_cstm` ON `leads`.`id` = `leads_cstm`.`id_c` WHERE (leadtype_c = 'VM' AND users.`id` = leads.`assigned_user_id`) ),
( SELECT COUNT(*) FROM `leads` INNER JOIN `leads_cstm` ON `leads`.`id` = `leads_cstm`.`id_c` WHERE (leadtype_c = 'Email' AND users.`id` = leads.`assigned_user_id`) ),
users.`first_name`,users.`last_name`
FROM
`users` users,
`leads` leads
I would appreciate any guidance!
You want to use conditional summation. The following uses MySQL syntax:
SELECT sum(leadtype_c = 'Referral') as Referrals,
sum(leadtype_c = 'VM') as VMs,
sum(leadtype_c = 'Email') as Emails,
users.`first_name`, users.`last_name`
FROM users join
`leads`
on users.`id` = leads.`assigned_user_id` INNER JOIN
`leads_cstm`
ON `leads`.`id` = `leads_cstm`.`id_c`
group by users.id;
You can use COUNT with CASE for this:
SELECT u.first_name,
u.last_name,
count(case when leadtype_c = 'Referral' then 1 end),
count(case when leadtype_c = 'VM' then 1 end),
count(case when leadtype_c = 'Email' then 1 end)
FROM users u
JOIN leads l ON u.id = l.assigned_user_id
JOIN leads_cstm lc ON l.id = lc.id_c
GROUP BY u.id
To match your exact results, you should probably use an OUTER JOIN instead, but this gives you the idea.
A Visual Explanation of SQL Joins

MySql on what cols should I put indexes?

I have this query:
SELECT Concat(f.name, ' ', f.parent_names) AS FullName,
stts.name AS 'Status',
u.name AS Unit,
city.name AS City,
(SELECT Group_concat(c.mobile1)
FROM contacts c
WHERE c.id = f.husband_id
OR c.id = f.wife_id) AS MobilePhones,
f.phone AS HomePhone,
f.contact_initiation_date AS InitDate,
f.status_change_date AS StatusChangeDate,
cmt.created_at AS CommentDate,
cmt.comment AS LastComment,
f.reconnection_date AS ReconnectionDate,
(SELECT Group_concat(t.name, ' ')
FROM taggings tgs
JOIN tags t
ON tgs.tag_id = t.id
WHERE tgs.taggable_type = 'family'
AND tgs.taggable_id = f.id) AS HandlingStatus
FROM families f
JOIN categories stts
ON f.family_status_cat_id = stts.id
JOIN units u
ON f.unit_id = u.id
JOIN categories city
ON f.main_city_cat_id = city.id
LEFT JOIN comments cmt
ON f.last_comment_id = cmt.id
WHERE 1 = 0
OR ( u.is_busy = 1 )
OR ( f.family_status_cat_id = 1423 )
OR ( f.family_status_cat_id = 1422
AND f.status_change_date BETWEEN '2011-03-21' AND '2012-03-13' )
My problem is very specific. It is regarding the line:
SELECT GROUP_CONCAT( c.mobile1 )
FROM contacts c
WHERE c.id = f.husband_id
OR c.id = f.wife_id
) AS MobilePhones
When I use EXPLAIN, it seems that this query is bad. I get for this table (c = contacts): 38307 rows.
On what columns should I put the index according to the query?
I tried mobile1 - but no improvement (BTW - family_id is indexed in the contacts table).
I attach the image of the explain result:
Or maybe someone can help me optimize the query...
Any column you'll be searching on, to speed up the process. Keep in mind that keys are already indexed.
Well, it seems that using the GROUP_CONCAT is the problem.
I just seperated the wife and husband mobile to be 2 different columns.
First, I thought that using the GROUP_CONCAT will be faster, but it proved to be VERY WRONG.
Just out of my curiosity, what is the performance of the query
SELECT GROUP_CONCAT( c.mobile1 )
FROM contacts c
WHERE c.id IN(f.husband_id, f.wife_id)
) AS MobilePhones