Nested Join in Subquery and failing correlation - mysql

I have 3 tables sc_user, sc_cube, sc_cube_sent
I wand to join to a user query ( sc_user) one distinct random message/cube ( from sc_cube ), that has not been sent to that user before ( sc_cube_sent), so each row in the result set has a disctinct user id and a random cubeid from sc_cube that is not part of sc_cube_sent with that user id associated there.
I am facing the problem that I seem not to be able to use a correlation id for the case that I need the u.id of the outer query in the inner On clause. I would need the commented section to make it work.
# get one random idcube per user not already sent to that user
SELECT u.id, sub.idcube
FROM sc_user as u
LEFT JOIN (
SELECT c.idcube, sent.idreceiver FROM sc_cube c
LEFT JOIN sc_cube_sent sent ON ( c.idcube = sent.idcube /* AND sent.idreceiver = u.id <-- "unknown column u.id in on clause" */ )
WHERE sent.idcube IS NULL
ORDER BY RAND()
LIMIT 1
) as sub
ON 1
I added a fiddle with some data : http://sqlfiddle.com/#!9/7b0bc/1
new cubeids ( sc_cube ) that should show for user 1 are the following : 2150, 2151, 2152, 2153
Edit>>
I could do it with another subquery instead of a join, but that has a huge performance impact and is not feasible ( 30 secs+ on couple of thousand rows on each table with reasonably implemented keys ), so I am still looking for a way to use the solution with JOIN.
SELECT
u.id,
(SELECT sc_cube.idcube
FROM sc_cube
WHERE NOT EXISTS(
SELECT sc_cube.idcube FROM sc_cube_sent WHERE sc_cube_sent.idcube = sc_cube.idcube AND sc_cube_sent.idreceiver = u.id
)
ORDER BY RAND() LIMIT 0,1
) as idcube
FROM sc_user u

without being able to test this, I would say you need to include your sc_user in the subquery because you have lost the scope
LEFT JOIN
( SELECT c.idcube, sent.idreceiver
FROM sc_user u
JOIN sc_cube c ON c.whatever_your_join_column_is = u.whatever_your_join_column_is
LEFT JOIN sc_cube_sent sent ON ( c.idcube = sent.idcube AND sent.idreceiver = u.id )
WHERE sent.idcube IS NULL
ORDER BY RAND()
LIMIT 1
) sub

If you want to get messagges ids that has not been sent to the particular user, then why use a join or left join at all ?
Just do:
SELECT sent.idcube
FROM sc_cube_sent sent
WHERE sent.idreceiver <> u.id
Then the query may look like this:
SELECT u.id,
/* sub.idcube */
( SELECT sent.idcube
FROM sc_cube_sent sent
WHERE sent.idreceiver <> u.id
ORDER BY RAND()
LIMIT 1
) as idcube
FROM sc_user as u

Got it working with NOT IN subselect in the on clause. Whereas the correlation link u.id is not given within the LEFT JOIN scope, it is for the scope of the ON clause. Here is how it works:
SELECT u.id, sub.idcube
FROM sc_user as u
LEFT JOIN (
SELECT idcube FROM sc_cube c ORDER BY RAND()
) sub ON (
sub.idcube NOT IN (
SELECT s.idcube FROM sc_cube_sent s WHERE s.idreceiver = u.id
)
)
GROUP BY u.id
Fiddle : http://sqlfiddle.com/#!9/7b0bc/48

Related

Multiple inner-join statements combined with subquery

Following query shows the results I need from table pointsList (latest records per user grouped by column idMatch)
select * from pointsList p
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
order by p.idMatch ASC
But now I want to also select some additional information about the user (from table users). My naive way to tackle this was by making a new inner join like this:
select * from pointsList p, users u
inner join users
on u.idUser = p.idUser
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
order by p.idMatch ASC
But I get the error message "unknown column 'p.idUser' in on clause". I tried using users.idUser and pointsList.idUser and other combinations (renaming pointsList in the on-clouse) but I always get the unknown column error (pointsList.idUser really does exist). Anyone could explain what I am doing wrong? I would like to extend this query to another table as well.
Thank you in advance!
Comma , priority in FROM clause is less than JOIN priority. And your query acts as:
select * from pointsList p,
(
users u
inner join users
on u.idUser = p.idUser
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
)
order by p.idMatch ASC
Of course, table pointsList AS p is not accessible within the parenthesis.
Use CROSS JOIN instead of comma:
select * from pointsList p
CROSS JOIN users u
inner join users
on u.idUser = p.idUser
inner join ( select idMatch, max(datePointsCalculated ) as MaxDate from pointsList group by idMatch ) tm
on p.idMatch = tm.idMatch and p.datePointsCalculated = tm.MaxDate
order by p.idMatch ASC

mysql query takes to much time

I want to optimize this query becouse it takes to much time to return records
SELECT
u.*,
s.legal_name AS structure_name,
ui.id AS userinfo_id,
ui.structure_id AS structure_id,
ui.lrn_user,
ui.gender,
ui.fiscal_code,
ui.prov,
ui.phone,
ui.school_name,
ui.school_codice_meccanografico,
us.status, us.date AS status_date,
CONCAT(u.lastname,' ',u.firstname) AS fullname,
CONCAT(u.firstname,' ',u.lastname) AS display_name,
uu.username AS created_by_name,
g.group_names,
IF(u.website_id = 0,'Sito Web principale', w.name) AS website_name
FROM fcf_users AS u
LEFT JOIN (
SELECT
gu.user_id,
GROUP_CONCAT(gg.name SEPARATOR ', ') AS group_names
FROM fcf_user_user_groups gu
JOIN fcf_user_groups gg ON gg.id = gu.group_id
GROUP BY user_id
) g ON g.user_id = u.id
LEFT JOIN fcf_users_userinfo AS ui ON ui.user_id = u.id
LEFT JOIN fcf_users_user_statuses AS us ON us.user_id = u.id
LEFT JOIN fcf_structures_structures AS s ON s.id = ui.structure_id
LEFT JOIN fcf_users AS uu ON uu.id = u.created_by
LEFT JOIN fcf_websites AS w ON w.id = u.website_id
WHERE
u.id IN (SELECT user_id FROM fcf_user_user_groups WHERE group_id = '8')
AND u.id IN (SELECT user_id FROM fcf_user_user_groups WHERE group_id = '8')
AND ui.lrn_user = '0'
ORDER BY fullname ASC
LIMIT 0,25
If anyone can help, thanks
Turn it inside-out. That is, first use a 'derived' table to locate 25 users you want. Then gather the rest of the info.
What you have gathers all the info (including all the JOIN work) for all the users, then sorts and peels off 25.
It will be something like:
SELECT -- lots of stuff
FROM ( SELECT u.id,
CONCAT(u.lastname,' ',u.firstname) AS fullname
FROM fcf_users AS u
JOIN fcf_user_user_groups AS ug ON ...
JOIN fcf_users_userinfo AS ui ON ui.user_id = u.id
WHERE ug.group_id = '8'
AND ui.lrn_user = '0'
ORDER BY u.lastname, u.firstname -- now sargeable
LIMIT 25
) AS u25
JOIN .... -- whatever tables are needed to get the rest of the columns
ORDER BY u25.fullname -- yes, again, but now using the CONCAT
-- no limit here
Also:
u: INDEX(lastname, firstname, id)
user_user_group is a "many-t0=many mapping" table? If so, follow the indexing advice here: http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
Ditto for any other many:many tables.
Note how I put into the derived table only the tables needed to achieve the LIMIT.

MYSQL: Handling Multiple LEFT JOINS

I have a query with one LEFT JOIN that works fine. When I add a second LEFT JOIN to a table with multiple records per field in the first table, however, I am getting the product of the results in the two tables ie books x publishers returned. How can I prevent this from happening?
SELECT a.*,b.*,p.*, group_concat(b.id as `bids`)
FROM authors `a`
LEFT JOIN books `b`
ON b.authorid = a.id
LEFT JOIN publishers `p`
on p.authorid = a.id
GROUP by a.id
EDIT:
Figured it out. The way to do this is to use subqueries as in this answer:
SELECT u.id
, u.account_balance
, g.grocery_visits
, f.fishmarket_visits
FROM users u
LEFT JOIN (
SELECT user_id, count(*) AS grocery_visits
FROM grocery
GROUP BY user_id
) g ON g.user_id = u.id
LEFT JOIN (
SELECT user_id, count(*) AS fishmarket_visits
FROM fishmarket
GROUP BY user_id
) f ON f.user_id = u.id
ORDER BY u.id;
If you do multiple LEFT Joins, your query will return a cartesian product of the results. To avoid this and get only one copy of fields you desire, do a subquery for each table you wish to join as below. Hope this helps someone in the future.
SELECT u.id
, u.account_balance
, g.grocery_visits
, f.fishmarket_visits
FROM users u
LEFT JOIN (
SELECT user_id, count(*) AS grocery_visits
FROM grocery
GROUP BY user_id
) g ON g.user_id = u.id
LEFT JOIN (
SELECT user_id, count(*) AS fishmarket_visits
FROM fishmarket
GROUP BY user_id
) f ON f.user_id = u.id
ORDER BY u.id;

Join two different tables ordered by common value (hotness)

I'm trying to select results from two different unrelated tables, showcase and questions to appear in a feed. They should be ordered by the common column hotness which is a float value.
SELECT s.id,s.date,s.title,s.views,s.image,s.hidpi,s.width,s.description,u.display_name,u.avatar
FROM showcase AS s
INNER JOIN users AS u ON s.user_id = u.id
UNION
SELECT q.id,q.date,q.title,q.views,q.text,u.display_name,u.avatar,0,0,0
FROM questions AS q
INNER JOIN users AS u ON q.user_id = u.id
ORDER BY hotness DESC
LIMIT 10
I've tried UNION, but I have no idea how I should be using it here and get this error unknown column hotness
You need to select the value in order for the ORDER BY to recognize it:
SELECT s.id,s.date,s.title,s.views,s.image,s.hidpi,s.width,s.description,u.display_name,u.avatar, s.hotness
FROM showcase AS s
INNER JOIN users AS u ON s.user_id = u.id
UNION ALL
SELECT q.id,q.date,q.title,q.views,q.text,u.display_name,u.avatar,0,0,0, q.hotness
FROM questions AS q
INNER JOIN users AS u ON q.user_id = u.id
ORDER BY hotness DESC;
Note that I also changed the UNION to UNION ALL. Unless you intend to remove duplicates, there is no reason to incur the extra processing for doing that.
You can try this query:
SELECT r.* FROM (
SELECT s.id,s.date,s.title,s.views,s.image,s.hidpi,s.width,s.description,u.display_name,u.avatar, s.hotness
FROM showcase AS s
INNER JOIN users AS u ON s.user_id = u.id
UNION
SELECT q.id,q.date,q.title,q.views,q.text,u.display_name,u.avatar,0,0,0, q.hotness
FROM questions AS q
INNER JOIN users AS u ON q.user_id = u.id
) as r
ORDER BY r.hotness DESC
LIMIT 10
You need to merge Union result in subquery to apply Order by on the result. I also added hotness in select clause, please check I take field from good table.

My Odd SubSelect, Need a LEFT JOIN Improvement

Here is a sample SQL dump: https://gist.github.com/JREAM/99287d033320b2978728
I have a SELECT that grabs a bundle of users.
I then do a foreach loop to attach all the associated tree_processes to that user.
So I end up doing X Queries: users * tree.
Wouldn't it be much more efficient to fetch the two together?
I've thought about doing a LEFT JOIN Subselect, but I'm having a hard time getting it correct.
Below I've done a query to select the correct data in the SELECT, however I would have to do this for all 15 rows and it seems like a TERRIBLE waste of memory.
This is my dirty Ateempt:
-
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
(
SELECT tp.id FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS newest_tree_id,
#
# Don't want to have to do this below for every row
(
SELECT t.type FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
WHERE subscribers_id = s.id
ORDER BY tp.id DESC
LIMIT 1
) AS tree_type
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
This is my LEFT JOIN attempt, but I am having trouble getting the SELECT values
SELECT
s.id,
s.firstname,
s.lastname,
s.email,
freshness.id,
# freshness.subscribers_id < -- Cant get multiples out of the LEFT join
FROM subscribers AS s
INNER JOIN scenario_subscriptions AS ss ON (
ss.subscribers_id = s.id
)
LEFT JOIN ( SELECT tp.id, tp.subscribers_id AS tp FROM tree_processes AS tp
JOIN tree AS t ON (
t.id = tp.tree_id
)
ORDER BY tp.id DESC
LIMIT 1 ) AS freshness
ON (
s.id = subscribers_id
)
WHERE ss.scenarios_id = 1
AND ss.completed != 1
AND ss.purchased_exit != 1
AND deleted != 1
GROUP BY s.id
LIMIT 0, 100
In the LEFT JOIN you are using 'freshness' as the table alias. This in you select you need to additionally state what column(s) you want from it. Since there is only one column (id) you need to add:
freshness.id
to the select clause.
Your ON clause of the left join looks pretty dodgy too. Maybe freshness.id = ss.subscribers_id?
Cheers -