MySQL ORDER BY, GROUP BY within a JOIN - mysql

I've been searching for a while and couln't find an example that worked. Hopefully you can spot the glaring mistake!
SELECT
Timestamp
, i_currency.Code AS Code
, Conversion
FROM
i_convert(
SELECT
Timestamp
, Conversion
FROM
i_convert
ORDER BY Timestamp DESC
)
JOIN i_currency
ON i_convert.CurrencyID = i_currency.CurrencyID
GROUP BY Code
I'm not sure wherer the JOIN should be, should it be in the parentheses, outside or both? I've tried all three with no luck, keep getting:
You have an error... ...near '( SELECT Timestamp , Conversion FROM i_convert ORDER '

The original query doesn't make sense. It is using GROUP BY, but attempting to select columns that are neither grouping columns nor aggregate functions of the groups. Which Timestamp and which Conversion are expected for any given result row?
I suppose the objective is to select the most recent conversion for each currency. That might look more like this:
SELECT
latest.Timestamp AS Timestamp
, i_currency.Code AS Code
, i_convert.Conversion AS Conversion
FROM i_currency
JOIN i_convert
ON i_convert.CurrencyID = i_currency.CurrencyID
JOIN (
SELECT MAX(Timestamp) as Timestamp, CurrencyId
FROM i_convert
GROUP BY CurrencyId
) latest
ON i_convert.Timestamp = latest.Timestamp
AND i_convert.CurrencyId = latest.CurrencyId

Related

How to select max / distinct record in MySQL using a deleted_at column

I am trying to select distinct rows under the following two rules:
If its deleted_at date is null then it is the most recent record, select it
If it is the latest deleted_at date (and there's not a record with a NULL), it is also the most recent record, select it
Consider this table:
The result I am looking for would be:
I'm using MySQL mariaDB v10.1.33 which does not have all the functions I am use to.
NULL was being ignored so I use a
coalesce(fc.deleted_at, CURRENT_TIMESTAMP())
to trick it into being the latest date. That way I can use max() function to select it. However, when I use this it is mismatching the data in the rows! i.e. this:
SELECT max(coalesce(fc.deleted_at, CURRENT_TIMESTAMP())), folder_id, code
FROM folder_code fc
WHERE fc.folder_id = 5683
returns:
I did some reading and this is a common problem where it seems to be ordering and selecting the max of each column independent of the row it is associated with and there are suggestions to use group by and order by to overcome it. However when I do this I get the same result i.e. this also returns the same as above:
SELECT max(coalesce(fc.deleted_at, CURRENT_TIMESTAMP())) as maxdeleteddate, fc.folder_id, fc.code
FROM folder_code fc
WHERE fc.folder_id = 5683
GROUP BY fc.folder_id
ORDER BY maxdeleteddate desc
How to I achieve my desired result?
Thank you
This is how I would do it:
SELECT f1.*
FROM folder f1
INNER JOIN (
SELECT folder_id,
NULLIF(MAX(IF(deleted_at IS NULL,NOW(),deleted_at)),NOW()) AS deleted_at
FROM folder
GROUP BY folder_id
) f2 ON f2.folder_id = f1.folder_id AND f2.deleted_at <=> f1.deleted_at
And here's a fiddle: https://www.db-fiddle.com/f/wzCYktpavBNnJu2uejPpe9/1
The idea is to get the groupwise-max, then join your table against itself. If you simply group the rows, you are not guaranteed to get the correct values for non-aggregated columns.
There is also a trick with deleted_at column, using NOW() if it's null, then using NULLIF() to set it back to NULL for the join.
This approach also benefits from the fact that it potentially uses indexes if they exist.
If you are using MySQL 8+, then you may use ROW_NUMBER here:
WITH cte AS (
SELECT *, ROW_NUMBER() OVER (PARTITION BY folder_id
ORDER BY -ISNULL(deleted_at), deleted_at DESC) rn
FROM folder_code
)
SELECT folder_id, code, deleted_at
FROM cte
WHERE rn = 1;
Demo
The ORDER BY clause used in the call to ROW_NUMBER places all records having a NULL deletion date after those records have a date, for each group of folder_id records. Then, the second level of sorting places more recent deletion date records first. This means that for those folders have a NULL record, it would appear first, otherwise the most recent record would appear first.
Here is an old school solution which might also work:
SELECT f1.folder_id, f1.code, f1.deleted_at
FROM folder_code f1
INNER JOIN
(
SELECT folder_id,
CASE WHEN COUNT(*) = COUNT(deleted_at)
THEN MAX(deleted_at) END AS max_deleted_at
FROM folder_code
GROUP BY folder_id
) f2
ON f1.folder_id = f2.folder_id AND
(f1.deleted_at = f2.max_deleted_at OR
(f1.deleted_at IS NULL AND f2.max_deleted_at IS NULL));
Demo
One way to get the latest date is to make sure there is no later date. Your approach to replace NULL with a high date is good and can be used for this.
select *
from folder_code fc
where not exists
(
select *
from folder_code fc2
where fc2.folder_id = fc.folder_id
and coalesce(fc2.deleted_at, date '9999-12-31') > coalesce(fc.deleted_at, date '9999-12-31')
);
You can try below - using correlated subquery
DEMO
select * from t1 a
where coalesce(deleted_at,CURRENT_TIMESTAMP()) =
(select max(coalesce(deleted_at,CURRENT_TIMESTAMP())) from t1 a1 where a.folder_id=a1.folder_id)
OUTPUT:
older_id code deleted_at
5333 12VA1 2019-09-27
5683 12SR1-X

MySQL: Update with nested subquery

I'm trying to make this query work. The aim is to get the minimum date between a list of minimum dates.
But I'm getting: Unknown column 'di.device_id' in 'where clause'
How can I expose the device_id field to be visible inside the second subquery?
update device_information di
set first_seen=(
select min(date) from (
select min(insert_date) date from product_state_change_event where device_id=di.device_id
union select min(insert_date) date from sensor_range_switch_change_event where device_id=di.device_id
) as dates
);
Edited
In fact, I have 19 tables to union them.
I would expect a query such as this:
update device_information di join
(select min(insert_date) as minid
from product_state_change_event psce
group by psce.device_id
) psce
on psce.device_id = di.device_id
set first_seen = least(psce.insert_date, psce.minid);
This assumes that the dates are never NULL. That is easily incorporated into the answer (by adding more logic to the least()).
If anyone want to know how I solved it:
update device_information di
join (
select identifier, min(date) date from (
select client_var_name identifier, min(insert_date) date from product_state_change_event group by client_var_name
union all select client_var_name identifier, min(insert_date) date from sensor_range_switch_change_event group by client_var_name
...
) as unique_dates group by identifier
) as dates on di.identifier = dates.identifier
set di.first_seen = dates.date;

MySQL Order By timestamp using IN clause

I have the below query and it works fine at returning results based on the set timestamp.
SELECT * FROM catalog WHERE part IN (SELECT part FROM query_data WHERE timestamp >= '2015-02-02') LIMIT 10
What I would like to do is get the results from the above but ORDER BY timestamp in DESC order. Something like this, but it doesn't work. The same values are returned, but not in DESC order based on the timestamp.
SELECT * FROM catalog WHERE part IN (SELECT part FROM query_data WHERE timestamp >= '2015-02-02' ORDER BY timestamp DESC) LIMIT 10
Thoughts? The timestamp column is only found in the query_data table. Not sure if this is causing the issue or not?
I believe this will work:
SELECT * FROM catalog c INNER JOIN query_data q ON c.part = q.part WHERE q.timestamp >= '2015-02-02' ORDER BY timestamp DESC;
The main problem with your approach is that you are ordering the subquery. Using a join and "order by" outside should fix it.
I would encourage you to watch this link (subselect order timestamp) The problem you have is as you thought, even if the subselect is ordered then it gets out of place on the main query, a Join would be useful for this cases.
Hope it helps you.

Incorrect group by and order by merge

I have couple tables joined in MySQL - one has many others.
And try to select items from one, ordered by min values from another table.
Without grouping in seems to be like this:
Code:
select `catalog_products`.id
, `catalog_products`.alias
, `tmpKits`.`minPrice`
from `catalog_products`
left join `product_kits` on `product_kits`.`product_id` = `catalog_products`.`id`
left join (
SELECT MIN(new_price) AS minPrice, id FROM product_kits GROUP BY id
) AS tmpKits on `tmpKits`.`id` = `product_kits`.`id`
where `category_id` in ('62')
order by product_kits.new_price ASC
Result:
But when I add group by, I get this:
Code:
select `catalog_products`.id
, `catalog_products`.alias
, `tmpKits`.`minPrice`
from `catalog_products`
left join `product_kits` on `product_kits`.`product_id` = `catalog_products`.`id`
left join (
SELECT MIN(new_price) AS minPrice, id FROM product_kits GROUP BY id
) AS tmpKits on `tmpKits`.`id` = `product_kits`.`id`
where `category_id` in ('62')
group by `catalog_products`.`id`
order by product_kits.new_price ASC
Result:
And this is incorrect sorting!
Somehow when I group this results, I get id 280 before 281!
But I need to get:
281|1600.00
280|2340.00
So, grouping breaks existing ordering!
For one, when you apply the GROUP BY to only one column, there is no guarantee that the values in the other columns will be consistently correct. Unfortunately, MySQL allows this type of SELECT/GROUPing to happen other products don't. Two, the syntax of using an ORDER BY in a subquery while allowed in MySQL is not allowed in other database products including SQL Server. You should use a solution that will return the proper result each time it is executed.
So the query will be:
For one, when you apply the GROUP BY to only one column, there is no guarantee that the values in the other columns will be consistently correct. Unfortunately, MySQL allows this type of SELECT/GROUPing to happen other products don't. Two, the syntax of using an ORDER BY in a subquery while allowed in MySQL is not allowed in other database products including SQL Server. You should use a solution that will return the proper result each time it is executed.
So the query will be:
select CP.`id`, CP.`alias`, TK.`minPrice`
from catalog_products CP
left join `product_kits` PK on PK.`product_id` = CP.`id`
left join (
SELECT MIN(`new_price`) AS "minPrice", `id` FROM product_kits GROUP BY `id`
) AS TK on TK.`id` = PK.`id`
where CP.`category_id` IN ('62')
order by PK.`new_price` ASC
group by CP.`id`
The thing is that group by does not recognize order by in MySQL.
Actually, what I was doing is really bad practice.
In this case you should use distinct and by catalog_products.*
In my opinion, group by is really useful when you need group result of agregated functions.
Otherwise you should not use it to get unique values.

Mysql Distinct function nested within other function in select

Is there a way we can use disctinct for below cd.date_id? It's getting concatenated but I would also like distinct function aswell so it's like GROUP_CONCAT( CAST( distinct(cd.date_id) AS CHAR ) ) As date_id but it does not seem to work...
SELECT b.bar_id, b.bar_name, b.bar_image, b.bar_lat, b.bar_lng, b.bar_address,
b.bar_phone, b.bus_web, b.bar_open_hours, c.coupon_id, c.coupon_text,
bc.coupon_start_time, bc.coupon_exp_time,
GROUP_CONCAT( CAST( cd.date_id AS CHAR ) ) As date_id,
d.calender_date, bc.bc_id, bc.priority As priority
FROM bars b
JOIN bars_coupons bc ON (bc.bar_id = b.bar_id)
JOIN coupons c ON (bc.coupon_id = c.coupon_id)
JOIN calendardates cd ON (cd.bc_id = bc.bc_id)
JOIN date d ON (d.date_id = cd.date_id)
GROUP BY cd.bc_id
DISTINCT inside there would not make any sense. What do you want to do here exactly? It seems you want only one row per distinct cd.date_id.
If that's the case just add cd.date_id to your GROUP BY clause at the end. Like this:
GROUP BY cd.bc_id, cd.date_id
And as you are using MySQL, you need not worry about using aggregate functions on all selected columns, if they would have the same value in all rows Grouped by the Group By clause, if not MySQL will just pick the first one
UPDATE
Looking at documentation for GROUP_CONCAT, you might be able to do it like this
GROUP_CONCAT( DISTINCT CAST(cd.date_id AS CHAR) ) As date_id