My question is why the ORDER BY and LIMIT was not working in the concerned query? Sometimes, it returns 1 row which is correct and sometimes it returns all rows (by join). Is there any known issues of MySQL regarding this?
select t1.*, t2.c_value
from table1 t1
inner join table2 t2 on t1.id = t2.f_id
where table1.user_id = 139
order by table1.id desc
limit 0,1
Related
select distinct column1
from Table1
where Table1id in ((select T2.Table1id
from Table2 T2
where (conditions)
order by T2.column)
limit 2
);
I cannot use limit inside the In operator. Do we have any other way to limit inside IN operator? Or do we have any other way without using IN and also without using any joins?
Error (while using limit inside the In Operator):-
Error Code: 1235. This version of MySQL doesn't yet support 'LIMIT & IN/ALL/ANY/SOME subquery'
For non-specific MySQL versions, Ergest gave you a good solution of using JOIN. Here is another workaround in which an outer layer is used on top of the derived table.
select distinct column1
from Table1
where Table1id in (select id
from
(select T2.Table1id as id
from Table2 T2
where (conditions)
order by T2.column
limit 2) tb);
PS: this trick can be used to bypass the ERROR 1093 (HY000): You can't specify target table 'terms' for update in FROM clause
What a weird restriction. Well, you can simply use an ad-hoc view (aka. WITH clause or CTE):
with limited as
(
select T2.Table1id
from Table2 T2
where (conditions)
order by T2.column
limit 2
)
select distinct column1
from Table1
where Table1id in (select Table1id from limited);
Demo: https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=cc148fae3a1089324446ec792e1476e2
Let's turn it inside out:
select distinct T1.column1
FROM
( SELECT T2.Table1id
from Table2 T2
where (conditions)
order by T2.column
limit 2
) AS x
JOIN Table1 AS T1 ON T1.table1id = x.Table1id
The inner (derived) table may need DISTINCT.
I have read here that MySQL processes ordering before applying limits. However, I receive different results when applying a LIMIT parameter in conjunction with a JOIN subquery. Here is my query:
SELECT
t1.id,
(t2.counts / c.matches)
FROM
table_one t1
JOIN
table_two t2 ON t1.id = t2.id
JOIN
(
SELECT
t1.id, COUNT(DISTINCT t1.id) AS matches
FROM
table_one t1
JOIN table_two t2 ON t1.id = t2.id
WHERE
t1.id IN (3390 , 3236, 148, 2811, 829, 137)
AND t2.value_one <= 30
AND t2.value_two < 2
GROUP BY t1.id
ORDER BY (t2.counts / matches)
LIMIT 0, 50 -- PROBLEM IS HERE (I think)
) c ON c.id = t1.id
ORDER BY (t2.counts / c.matches), t1.id;
Here is a rough description of what I think is happening:
The sub-query selects a bunch of ids from table_one that meet the criteria
These are ordered by (t2.counts / matches)
The top 50 (in ascending order) are fashioned into a table
This resulting table is then joined on the the id column
Results are returned from the top level JOIN - without a GROUP BY clause this time. table_one is a reference table so this will return many rows with the same ID.
I appreciate that some of these joins don't make a lot of sense but I have stripped down my query for readability - it's normally quite chunky .
The problem is that when, I include the LIMIT parameter I get a different set of results and not just the top 50. What I want to do is get the top results from the subquery and use these to join onto a bunch of other tables based on the reference table.
Here is what I have tried so far:
LIMIT on the outer query (this is undesirable as this cuts off important information).
Trying different LIMIT tables and values.
Any idea what is going wrong, or what else I could try?
I have found a solution to my problem. It seems as if my matches column name does can't be used in my ORDER BY clause - which is weird since I don't get an error. Either way, this solves the problem:
SELECT
t1.id,
(t2.counts / c.matches)
FROM
table_one t1
JOIN
table_two t2 ON t1.id = t2.id
JOIN
(
SELECT
t1.id, COUNT(DISTINCT t1.id) AS matches
FROM
table_one t1
JOIN table_two t2 ON t1.id = t2.id
WHERE
t1.id IN (3390 , 3236, 148, 2811, 829, 137)
AND t2.value_one <= 30
AND t2.value_two < 2
GROUP BY t1.id
ORDER BY (t2.counts / COUNT(DISTINCT t1.id)) -- This line is changed
LIMIT 0, 50
) c ON c.id = t1.id
ORDER BY (t2.counts / c.matches), t1.id;
I try to explain a very high level
I have two complex SELECT queries(for the sake of example I reduce the queries to the following):
SELECT id, t3_id FROM t1;
SELECT t3_id, MAX(added) as last FROM t2 GROUP BY t3_id;
query 1 returns 16k rows and query 2 returns 15k
each queries individually takes less than 1 second to compute
However what I need is to sort the results using column added of query 2, when I try to use LEFT join
SELECT
t1.id, t1.t3_Id
FROM
t1
LEFT JOIN
(SELECT t3_id, MAX(added) as last FROM t2 GROUP BY t3_id) AS t_t2
ON t_t2.t3_id = t1.t3_id
GROUP BY t1.t3_id
ORDER BY t_t2.last
However, the execution time goes up to over a 1 minute.
I like to understand the reason
what is the cause of such a huge explosion?
NOTE:
ALL the used columns on every table have been indexed
e.g. :
table t1 has index on id,t3_Id
table t2 has index on t3_id and added
EDIT1
after #Tim Biegeleisen suggestion, I change the query to the following now the query is executing in about 16 seconds. If I remove the ORDER BY it query gets executed in less than 1 seconds. The problem is that ORDER BY the sole reason for this.
SELECT
t1.id, t1.t3_Id
FROM
t1
LEFT JOIN
t2 ON t2.t3_id = t1.t3_id
GROUP BY t1.t3_id
ORDER BY MAX(t2.added)
Even though table t2 has an index on column t3_id, when you join t1 you are actually joining to a derived table, which either can't use the index, or can't use it completely effectively. Since t1 has 16K rows and you are doing a LEFT JOIN, this means the database engine will need to scan the entire derived table for each record in t1.
You should use MySQL's EXPLAIN to see what the exact execution strategy is, but my suspicion is that the derived table is what is slowing you down.
The correct query should be:
SELECT
t1.id,
t1.t3_Id,
MAX(t2.added) as last
FROM t1
LEFT JOIN t2 on t1.t3_Id = t2.t3_Id
GROUP BY t2.t3_id
ORDER BY last;
This is happen because a temp table is generating on each record.
I think you could try to order everything after the records are available. Maybe:
select * from (
select * from
(select t3_id,max(t1_id) from t1 group by t3_id) as t1
left join (select t3_id,max(added) as last from t2 group by t3_id) as t2
on t1.t3_id = t2.t3_id ) as xx
order by last
I have a slow-running update statement, and I was curious if moving the where condition to the join clause would improve performance. Here's the query:
update T1 inner join (select ID, GROUP_CONCAT(x) as X from T3 group by ID) as T2
on T1.ID=T2.ID set T1.X=T2.X where T1.TYPE='something';
Now... for a very big table (millions of records), would it be faster to do this?
update T1 inner join (select ID, GROUP_CONCAT(x) as X from T3 group by ID) as T2
on T1.ID=T2.ID and T1.TYPE='something' set T1.X=T2.X;
The query is simple enough that both approaches should be optimized identically.
Both approaches might also be sub-optimal because the inner query isn't correlated to the outer query. Your query is creating an implicit temporary table containing all possible rows for derived table T2 -- exactly the same result as if you just ran the query select ID, GROUP_CONCAT(x) as X from T3 group by ID by itself -- and then the server is discarding the ones that can't be joined to T1 and using the rest to do the update.
This is more than likely not the optimum path.
Unless t1.TYPE = 'something' involves a large percentage of the rows in T1, it should be more efficient to do this:
UPDATE t1
SET t1.x = (SELECT GROUP_CONCAT(x) FROM T3 WHERE T3.id = T1.id GROUP BY T3.id)
WHERE t1.TYPE = 'something';
The inner subquery is correlated to the outer subquery, and only executed for the rows in T1 that are matched by the WHERE clause.
I have the following LEFT JOIN statement (though probably also applies to even a simpler SELECT statement):
SELECT * FROM t1 LEFT JOIN t2 ON t2.c = t1.c WHERE t1.m LIKE 'captain%' GROUP BY
t1.c ORDER BY t2.date DESC LIMIT 0,10
The results get returned but they are not ordered by t2.date DESC...I imagine this is due to having the GROUP BY statement in there. How can I group the results AND order them?
thx
Put your query in a subquery and then use ORDER BY.
SELECT *
FROM
(SELECT *
FROM t1 LEFT JOIN t2 ON t2.c = t1.c
WHERE t1.m LIKE 'captain%'
GROUP BY t1.c DESC LIMIT 0,10) l1
ORDER BY date
In normal SQL it would not even be allowed to ORDER BY t2.date because you aren't grouping on it and you are not using an aggregate function on it, so you can't select it.
Anyway, as MySQL just picks values from the rows in an undefined manner and afterwards sorts them, the t2.date column in your results should be sorted. Please show some actual output.