Limit a Left Join on the first table - mysql

I have two tables: gems and gemdetail that are left joined. I am trying to limit the LEFT JOIN to 10 records in the gems table. There are 2 other tables joined (gemreply and users) as well, but they do not contribute to the problem. The following does not work:
SELECT
gems.gemid,
gems.title,
r.tot,
gemdetail.filename
FROM
(
(
gems
LEFT JOIN(
SELECT
gemid,
COUNT(*) AS tot
FROM
gemreply
GROUP BY
gemid
) AS r
ON
gems.gemid = r.gemid
)
LEFT JOIN gemdetail ON gems.gemid = gemdetail.gemid
)
LEFT JOIN users ON gems.userid = users.userid
WHERE
gems.grade = '7'
ORDER BY
gems.gemid
LIMIT 0, 10;
This limits the total number of returned rows to 10, but since there are multiple detail records for each gem, I am left with fewer than 10 gem records. I've read every "LIMIT" post, but haven't found this situation.
This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery'.

Something like that
SELECT * FROM A
INNER JOIN ( SELECT * FROM A WHERE A.FIELD1='X' ORDER BY A.FIELD2 LIMIT 10) X
ON (A.KEYFIELD=X.KEYFIELD)
LEFT JOIN B ON (A.FIELD = B.FIELD)
LEFT JOIN C ON (A.FIELD = C.FIELD)

Try this:
SELECT g.gemid, g.title, r.tot, gemdetail.filename
FROM (SELECT * FROM gems WHERE grade = '7' LIMIT 10) g
LEFT JOIN (SELECT gemid, COUNT(*) AS tot FROM gemreply GROUP BY gemid) r
ON r.gemid = g.gemid
LEFT JOIN gemdetail ON g.gemid = gemdetail.gemid
LEFT JOIN users ON g.userid = users.userid
ORDER BY g.gemid;
This should work.

Ten years later, I can LIMIT records within a nested query on the left table of a one to many JOIN where more than one record will be returned for the left table--even when there is a WHERE or ORDER BY clause on fields on the right side tables of the join.
In the inner query, have all the tables with joins necessary to satisfy the WHERE clause. Apply the limit there.
In the outer query, rejoin to all the same tables to add the many to many fields
SELECT
alias.field_from_A,
alias.field_from_B
FROM (
SELECT DISTINCT
/* ONLY fields from LEFT table (or tables
with a one to one relationship with it) will
allow LIMIT to apply only to the LEFT table. DISTINCT is needed due to JOIN*/
A.field_from_A
FROM
A
/* JOIN to your hearts content on tables
which need to satisfy WHERE and ORDER BY */
LEFT JOIN B ON A.field_from_A = B.field_from_B
WHERE A.field_from_A = 'value' AND B.field_from_B = 'value 2'
ORDER BY A.field_from_A, B.field_from_B
LIMIT 0, 10
) alias
/* JOIN the one to many tables here to add the field info */
LEFT JOIN B ON A.field_from_A = B.field_from_B
/* the WHERE and ORDER BY need to be applied again */
WHERE alias.field_from_A = 'value' AND B.field_from_B = 'value 2'
ORDER BY alias.field_from_A, B.field_from_B

WITH last_orders AS (
SELECT *
FROM orders
WHERE id IN (
SELECT MAX(id)
FROM orders
GROUP BY customer_id
)
)
SELECT customers.id, customers.first_name, customers.last_name,
last_orders.order_date, last_orders.order_status
FROM customers
JOIN last_orders
ON customers.id = last_orders.customer_id
ORDER BY customer_id;

Related

Multiple inner-join statements combined with subquery

Following query shows the results I need from table pointsList (latest records per user grouped by column idMatch)
select * from pointsList p
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
order by p.idMatch ASC
But now I want to also select some additional information about the user (from table users). My naive way to tackle this was by making a new inner join like this:
select * from pointsList p, users u
inner join users
on u.idUser = p.idUser
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
order by p.idMatch ASC
But I get the error message "unknown column 'p.idUser' in on clause". I tried using users.idUser and pointsList.idUser and other combinations (renaming pointsList in the on-clouse) but I always get the unknown column error (pointsList.idUser really does exist). Anyone could explain what I am doing wrong? I would like to extend this query to another table as well.
Thank you in advance!
Comma , priority in FROM clause is less than JOIN priority. And your query acts as:
select * from pointsList p,
(
users u
inner join users
on u.idUser = p.idUser
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
)
order by p.idMatch ASC
Of course, table pointsList AS p is not accessible within the parenthesis.
Use CROSS JOIN instead of comma:
select * from pointsList p
CROSS JOIN users u
inner join users
on u.idUser = p.idUser
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
order by p.idMatch ASC

MySQL LEFT JOIN only last row painfully slow

I have three tables, which looks like this
forts
|id|lat|lon|
fort_sightings
|id|fort_id|team|
fort_raids
|id|fort_id|raid_level|
I need a query that fetches all the rows from forts and then select the latest information from fort_sightings and fort_raids, if any. There might be several rows where fort_id has the same value, so I need the latest information.
Currently, I have this, which might not be the prettiest
SELECT
*
FROM
forts c
LEFT JOIN fort_sightings o ON o.id = (
SELECT
id
FROM
fort_sightings
WHERE
fort_id = c.id
ORDER BY
id DESC
LIMIT 1
)
LEFT JOIN fort_raids r ON r.id = (
SELECT
id
FROM
fort_raids
WHERE
fort_id = c.id
ORDER BY
id DESC
LIMIT 1
)
But it's painfully slow, the query takes over 10 seconds. There's only ~350 rows in forts, so it really shouldn't take this long. I believe it's from all the SELECT queries in the JOIN, but I don't know any alternative.
EXPLAIN
This is your query:
SELECT *
FROM forts c LEFT JOIN
fort_sightings o
ON o.id = (SELECT fs2.id
FROM fort_sightings fs2
WHERE fs.fort_id = c.id
ORDER BY fs2.id DESC
LIMIT 1
) LEFT JOIN
fort_raids r
ON r.id = (SELECT fr2.id
FROM fort_raids fr2
WHERE fr2.fort_id = c.id
ORDER BY fr2.id DESC
LIMIT 1
);
I find it strangely structured. But, see if this works:
fort_sightings(fort_id, id)
fort_raids(fort_id, id)
It is important that the fort_id be the first key in the indexes.
If this doesn't work, then you might need to modify the query.
Aside from the index recommendation from Gordon, you are doing a correlated subquery which means that for every record in the forts table, you are re-running the query to the sightings and raids. I would change slightly to pull all max() per fort from each THEN join... like
SELECT
c.id,
c.lat,
c.lon,
fs2.team,
fr2.raid_level
FROM
forts c
LEFT JOIN
( select fs.fort_id, max( fs.id ) as MaxSightID
from fort_sightings fs
group by fs.fort_id ) S
on c.id = s.fort_id
LEFT JOIN fort_sightings fs2
on s.MaxSightID = fs2.id
LEFT JOIN
( select fr.fort_id, max( fr.id ) as MaxRaidID
from fort_raids fs
group by fr.fort_id ) R
on c.ID = r.fort_id
LEFT JOIN fort_raids fr2
on r.MaxRaidID = fr2.id
The second part of the left-joins goes back to the original raid and sighting tables to pull the corresponding team and raid level in the final results if any such are found.
If you need the most efficient query, you should add columns latest_fort_sighting_id latest_fort_raid_id to table forts. MySQL does not have powerful features such as Materialized Views or Hash Joins like PostgreSQL, we need to handle them manually. Do not forget using transaction for updates.
If you limit range of forts, alternatively, you can run optimized query only using LEFT JOIN.
select - SQL join: selecting the last records in a one-to-many relationship - Stack Overflow
SELECT forts.*, fs1.team, fr1.raid_level FROM forts
LEFT JOIN fort_sightings fs1 ON fs1.fort_id = forts.id
LEFT JOIN fort_sightings fs2 ON fs2.fort_id = forts.id AND fs1.id < fs2.id
LEFT JOIN fort_raids fr1 ON fr1.fort_id = forts.id
LEFT JOIN fort_raids fr2 ON fr2.fort_id = forts.id AND fr1.id < fr2.id
WHERE fs2.id IS NULL AND fr2.id IS NULL AND forts.id > 5 ORDER BY forts.id LIMIT 5;

Getting max record on varchar field

I have this query
SELECT
s.account_number,
a.id AS 'ASPIRION ID',
a.patient_first_name,
a.patient_last_name,
s.admission_date,
s.total_charge,
astat.name AS 'STATUS',
astat.definition,
latest_note.content AS 'LAST NOTE',
a.insurance_company
FROM
accounts a
INNER JOIN
services s ON a.id = s.account_id
INNER JOIN
facilities f ON f.id = a.facility_id
INNER JOIN
account_statuses astat ON astat.id = a.account_status_id
INNER JOIN
(SELECT
account_id, MAX(content) content, MAX(created)
FROM
notes
GROUP BY account_id) latest_note ON latest_note.account_id = a.id
WHERE
a.facility_id = 56
My problem comes from
(SELECT
account_id, MAX(content) content, MAX(created)
FROM
notes
GROUP BY account_id)
Content is a varchar field and I am needed to get the most recent record. I now understand that MAX will not work on a varchar field the way that I want it. I am not sure how to be able to get the corresponding content with the MAX id and group that by account id on in this join.
What would be the best way to do this?
My notes table looks like this...
id account_id content created
1 1 This is a test 2011-03-16 02:06:40
2 1 More test 2012-03-16 02:06:40
Here are two choices. If your content is not very long and don't have funky characters, you can use the substring_index()/group_concat() trick:
(SELECT account_id,
SUBSTRING_INDEX(GROUP_CONCAT(content ORDER BY created desc SEPARATOR '|'
), 1, '|') as content
FROM notes
GROUP BY account_id
) latest_note
ON latest_note.account_id = a.id
Given the names of the columns and tables, that is likely not to work. Then you need an additional join or a correlated subquery in the from clause. I think that might be easiest in this case:
select . . .,
(select n.content
from notes n
where n.account_id = a.id
order by created desc
limit 1
) as latest_note
from . . .
The advantage to this method is that it only gets the notes for the rows you need. And, you don't need a left join to keep all the rows. For performance, you want an index on notes(account_id, created).
SELECT
s.account_number,
a.id AS 'ASPIRION ID',
a.patient_first_name,
a.patient_last_name,
s.admission_date,
s.total_charge,
astat.name AS 'STATUS',
astat.definition,
latest_note.content AS 'LAST NOTE',
a.insurance_company
FROM
accounts a
INNER JOIN services s ON a.id = s.account_id
INNER JOIN facilities f ON f.id = a.facility_id
INNER JOIN account_statuses astat ON astat.id = a.account_status_id
INNER JOIN
(SELECT account_id, MAX(created) mxcreated
FROM notes GROUP BY account_id) latest_note ON latest_note.account_id = a.id and
latest_note.mxcreated = --datetime column from any of the other tables being used
WHERE a.facility_id = 56
You have to join on the max(created) which would give the latest content.
Or you can change the query to
SELECT account_id, content, MAX(created) mxcreated
FROM notes GROUP BY account_id
as mysql allows you even if you don't include all non-aggregated columns in group by clause. However, unless you join on the max date you wouldn't get the correct results.
The last created record is the one for which does not exist a newer one. Hence:
SELECT
s.account_number,
a.id AS "ASPIRION ID",
a.patient_first_name,
a.patient_last_name,
s.admission_date,
s.total_charge,
astat.name AS "STATUS",
astat.definition,
latest_note.content AS "LAST NOTE",
a.insurance_company
FROM accounts a
INNER JOIN services s ON a.id = s.account_id
INNER JOIN facilities f ON f.id = a.facility_id
INNER JOIN account_statuses astat ON astat.id = a.account_status_id
INNER JOIN
(
SELECT account_id, content
FROM notes
WHERE NOT EXISTS
(
SELECT *
FROM notes newer
WHERE newer.account_id = notes.account_id
AND newer.created > notes.created
)
) latest_note ON latest_note.account_id = a.id
WHERE a.facility_id = 56;

mysql query optimization steps or how to optimze query

I don't know much about query optimization but I know the order in which queries get executed
FROM clause
WHERE clause
GROUP BY clause
HAVING clause
SELECT clause
ORDER BY clause
This the query I had written
SELECT
`main_table`.forum_id,
my_topics.topic_id,
(
SELECT MAX(my_posts.post_id) FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id
) AS `maxpostid`,
(
SELECT my_posts.admin_user_id FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id ORDER BY my_posts.post_id DESC LIMIT 1
) AS `admin_user_id`,
(
SELECT my_posts.user_id FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id ORDER BY my_posts.post_id DESC LIMIT 1
) AS `user_id`,
(
SELECT COUNT(my_topics.topic_id) FROM my_topics WHERE my_topics.forum_id = main_table.forum_id ORDER BY my_topics.forum_id DESC LIMIT 1
) AS `topicscount`,
(
SELECT COUNT(my_posts.post_id) FROM my_posts WHERE my_topics.topic_id = my_posts.topic_id ORDER BY my_topics.topic_id DESC LIMIT 1
) AS `postcount`,
(
SELECT CONCAT(admin_user.firstname,' ',admin_user.lastname) FROM admin_user INNER JOIN my_posts ON my_posts.admin_user_id = admin_user.user_id WHERE my_posts.post_id = maxpostid ORDER BY my_posts.post_id DESC LIMIT 1
) AS `adminname`,
(
SELECT forum_user.nick_name FROM forum_user INNER JOIN my_posts ON my_posts.user_id = forum_user.user_id WHERE my_posts.post_id = maxpostid ORDER BY my_posts.post_id DESC LIMIT 1
) AS `nickname`,
(
SELECT CONCAT(ce1.value,' ',ce2.value) AS fullname FROM my_posts INNER JOIN customer_entity_varchar AS ce1 ON ce1.entity_id = my_posts.user_id INNER JOIN customer_entity_varchar AS ce2 ON ce2.entity_id=my_posts.user_id WHERE (ce1.attribute_id = 1) AND (ce2.attribute_id = 2) AND my_posts.post_id = maxpostid ORDER BY my_posts.post_id DESC LIMIT 1
) AS `fullname`
FROM `my_forums` AS `main_table`
LEFT JOIN `my_topics` ON main_table.forum_id = my_topics.forum_id
WHERE (forum_status = '1')
And now I want to know if there is any way to optimize it ? Because all the logic is written in Select section not From, but I don't know how to write the same logic in From section of the query ?
Does it make any difference or both are same ?
Thanks
Correlated subqueries should really be a last resort, they often end up being executed RBAR, and given that a number of your subqueries are very similar, trying to get the same result using joins is going to result in a lot less table scans.
The first thing I note is that all of your subqueries include the table my_posts, and most contain ORDER BY my_posts.post_id DESC LIMIT 1, those that don't have a count with no group by so the order and limit are redundant anyway, so my first step would be to join to my_posts:
SELECT *
FROM my_forums AS f
LEFT JOIN my_topics AS t
ON f.forum_id = t.forum_id
LEFT JOIN
( SELECT topic_id, MAX(post_id) AS post_id
FROM my_posts
GROUP BY topic_id
) AS Maxp
ON Maxp.topic_id = t.topic_id
LEFT JOIN my_posts AS p
ON p.post_id = Maxp.post_id
WHERE forum_status = '1';
Here the subquery just ensures you get the latest post per topic_id. I have shortened your table aliases here for my convenience, I am not sure why you would use a table alias that is longer than the actual table name?
Now you have the bulk of your query you can start adding in your columns, in order to get the post count, I have added a count to the subquery Maxp, I have also had to add a few more joins to get some of the detail out, such as names:
SELECT f.forum_id,
t.topic_id,
p.post_id AS `maxpostid`,
p.admin_user_id,
p.user_id,
t2.topicscount,
maxp.postcount,
CONCAT(au.firstname,' ',au.lastname) AS adminname,
fu.nick_name AS nickname
CONCAT(ce1.value,' ',ce2.value) AS fullname
FROM my_forums AS f
LEFT JOIN my_topics AS t
ON f.forum_id = t.forum_id
LEFT JOIN
( SELECT topic_id,
MAX(post_id) AS post_id,
COUNT(*) AS postcount
FROM my_posts
GROUP BY topic_id
) AS Maxp
ON Maxp.topic_id = t.topic_id
LEFT JOIN my_posts AS p
ON p.post_id = Maxp.post_id
LEFT JOIN admin_user AS au
ON au.admin_user_id = p.admin_user_id
LEFT JOIN forum_user AS fu
ON fu.user_id = p.user_id
LEFT JOIN customer_entity_varchar AS ce1
ON ce1.entity_id = p.user_id
AND ce1.attribute_id = 1
LEFT JOIN customer_entity_varchar AS ce2
ON ce2.entity_id = p.user_id
AND ce2.attribute_id = 2
LEFT JOIN
( SELECT forum_id, COUNT(*) AS topicscount
FROM my_topics
GROUP BY forum_id
) AS t2
ON t2.forum_id = f.forum_id
WHERE forum_status = '1';
I am not familiar with your schema so the above may need some tweaking, but the principal remains - use JOINs over sub-selects.
The next stage of optimisation I would do is to get rid of your customer_entity_varchar table, or at least stop using it to store things as basic as first name and last name. The Entity-Attribute-Value model is an SQL antipattern, if you added two columns, FirstName and LastName to your forum_user table you would immediately lose two joins from your query. I won't get too involved in the EAV vs Relational debate as this has been extensively discussed a number of times, and I have nothing more to add.
The final stage would be to add appropriate indexes, you are in the best decision to decide what is appropriate, I'd suggest you probably want indexes on at least the foreign keys in each table, possibly more.
EDIT
To get one row per forum_id you would need to use the following:
SELECT f.forum_id,
t.topic_id,
p.post_id AS `maxpostid`,
p.admin_user_id,
p.user_id,
MaxT.topicscount,
maxp.postcount,
CONCAT(au.firstname,' ',au.lastname) AS adminname,
fu.nick_name AS nickname
CONCAT(ce1.value,' ',ce2.value) AS fullname
FROM my_forums AS f
LEFT JOIN
( SELECT t.forum_id,
COUNT(DISTINCT t.topic_id) AS topicscount,
COUNT(*) AS postCount,
MAX(t.topic_ID) AS topic_id
FROM my_topics AS t
INNER JOIN my_posts AS p
ON p.topic_id = p.topic_id
GROUP BY t.forum_id
) AS MaxT
ON MaxT.forum_id = f.forum_id
LEFT JOIN my_topics AS t
ON t.topic_ID = Maxt.topic_ID
LEFT JOIN
( SELECT topic_id, MAX(post_id) AS post_id
FROM my_posts
GROUP BY topic_id
) AS Maxp
ON Maxp.topic_id = t.topic_id
LEFT JOIN my_posts AS p
ON p.post_id = Maxp.post_id
LEFT JOIN admin_user AS au
ON au.admin_user_id = p.admin_user_id
LEFT JOIN forum_user AS fu
ON fu.user_id = p.user_id
LEFT JOIN customer_entity_varchar AS ce1
ON ce1.entity_id = p.user_id
AND ce1.attribute_id = 1
LEFT JOIN customer_entity_varchar AS ce2
ON ce2.entity_id = p.user_id
AND ce2.attribute_id = 2
WHERE forum_status = '1';

My Odd SubSelect, Need a LEFT JOIN Improvement

Here is a sample SQL dump: https://gist.github.com/JREAM/99287d033320b2978728
I have a SELECT that grabs a bundle of users.
I then do a foreach loop to attach all the associated tree_processes to that user.
So I end up doing X Queries: users * tree.
Wouldn't it be much more efficient to fetch the two together?
I've thought about doing a LEFT JOIN Subselect, but I'm having a hard time getting it correct.
Below I've done a query to select the correct data in the SELECT, however I would have to do this for all 15 rows and it seems like a TERRIBLE waste of memory.
This is my dirty Ateempt:
-
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
(
SELECT tp.id FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS newest_tree_id,
#
# Don't want to have to do this below for every row
(
SELECT t.type FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS tree_type
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
This is my LEFT JOIN attempt, but I am having trouble getting the SELECT values
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
freshness.id,
# freshness.subscribers_id < -- Cant get multiples out of the LEFT join
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
LEFT JOIN ( SELECT tp.id, tp.subscribers_id AS tp FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
ORDER BY tp.id DESC
LIMIT 1 ) AS freshness
ON (
s.id = subscribers_id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
In the LEFT JOIN you are using 'freshness' as the table alias. This in you select you need to additionally state what column(s) you want from it. Since there is only one column (id) you need to add:
freshness.id
to the select clause.
Your ON clause of the left join looks pretty dodgy too. Maybe freshness.id = ss.subscribers_id?
Cheers -