Optimize mysql query with UNION - mysql

I have a requirement of getting next date and previous date
the table structure is as follows
| auto_id | id | next_date | next_activity |
| 1 | 1 | 22-12-2012 | - |
| 2 | 1 | 25-12-2012 | - |
| 3 | 1 | 26-12-2012 | - |
| 4 | 1 | 28-12-2012 | - |
so i need next_day and previous_day
next_day = next_date after current day
previous_day = next_date before current_date
(SELECT * FROM `activity` WHERE id = 1 and next_date > CURDATE() order by next_date asc limit 1)
UNION
(SELECT * FROM `activity` WHERE id = 1 and next_date = CURDATE() )
UNION
(SELECT * FROM `activity` WHERE id = 1 and next_date < CURDATE() order by next_date desc limit 1)
ORDER BY next_date desc limit 2
Other way to do it self join the table...
Is there a way to optimize the table

Here is another way:
SELECT next_date, date_diff
FROM (SELECT *,
#dateDiff := datediff(next_date, curdate()) AS date_diff,
#pDateDiff :=
IF((#dateDiff < 0 AND #dateDiff > #pDateDiff),
#dateDiff,
#pDateDiff)
AS pDateDiff,
#nDateDiff :=
IF((#dateDiff > 0 AND #dateDiff < #nDateDiff),
#dateDiff,
#nDateDiff)
AS nDateDiff
FROM activity, (SELECT #pDateDiff := -9999, #nDateDiff := 9999) tmp
WHERE id = 1) aView
WHERE date_diff IN (#pDateDiff, 0, #nDateDiff)
ORDER BY next_date;
date_diff value gives perspective of prev and next dates.
pick all dates where id = 1
find the date difference between next_date and curdate() & store in an user defined variable #dateDiff.
#pDateDiff is another variable which tracks maximum among negative #dateDiff values (our previous date)
#nDateDiff is yet another variable which tracks minimum among positive #dateDiff values (our next date)
in the end, select only those dates which are in (-ve max, 0, +ve min).
PS: if you've duplicate date entries then query may return all of them.

Related

How to count concurrently bookings in sql in time interval per minute?

If I have a start and stop time for a booking, how can I calculate the number of bookings there are each minute? I made a simplified version of my database table looks like here:
Start time | End time | booking |
--------------------------------------------------
2020-09-01 10:00 | 2020-09-01 10:10 | Booking 1 |
2020-09-01 10:00 | 2020-09-01 10:05 | Booking 2 |
2020-09-01 10:05 | 2020-09-01 10:10 | Booking 3 |
2020-09-01 10:09 | 2020-09-01 10:10 | Booking 4 |
I want to have the bookings between a given time interval like 10:02 - 10:09. It should be something like this as result:
Desired result
Time | count
-----------
10:02 | 2 |
10:03 | 2 |
10:04 | 2 |
10:05 | 3 |
10:06 | 2 |
10:07 | 2 |
10:08 | 2 |
10:09 | 3 |
Question
How can this be achieved? Today I export it to python however I think it should be possible to achieve directly in SQL.
You can use a recursive CTE directly on your data:
with recursive cte as (
select start_time, end_time
from t
union all
select start_time + interval 1 minute, end_time
from cte
where start_time < end_time
)
select start_time, count(*)
from cte
group by start_time
order by start_time;
Here is a db<>fiddle.
EDIT:
In earlier versions of MySQL, it helps to have a tally table. You can create one on the fly, using something like:
(select #rn := #rn + 1 as n
from t cross join
(select #rn := 0) params
) tally
You need enough numbers for your maximum span, but then you can do:
select t.start_time + interval tally.n hour, count(*)
from t join
(select #rn := #rn + 1 as n
from t cross join
(select #rn := -1) params -- so it starts from 0
limit 100
) tally
on t.start_time + interval tally.n hour <= t.end_time
group by t.start_time + interval tally.n hour;
You can use a recursive query to generate the timestamp range, then unpivot the table and join:
with recursive dates (ts) as(
select '2020-09-01'
union all
select ts + interval 1 minute
from dates
where ts + itnerval 1 minute < '2020-09-02'
)
select d.ts, sum(t.cnt) over(order by d.ts) cnt
from dates d
left join (
select start_time ts, 1 cnt from mytable
union all select end_time, -1 from mytable
) t on t.ts <= d.ts
If you are going to run this repeatedly and/or against large time periods, you would better materialize the date ranges in a calendar table rather than use a recursive query. The calendar table has one row per minute over a large period of dates - assuming a table called date_calendar, you would do:
select d.ts, sum(t.cnt) over(order by d.ts) cnt
from date_calendar d
left join (
select start_time ts, 1 cnt from mytable
union all select end_time, -1 from mytable
) t on t.ts <= d.ts
where d.ts >= '2020-09-01' and d.ts < '2020-09-02'

Get DateTime corresponding to the lowest Time

I'd like to get the Date & ID which corresponds to the lowest and Largest Time, respectively the extreme rows in the table below with ID 5 & 4.
Please note the following:
Dates are stored as values in ms
The ID reflects the Order By Date ASC
Below I have split the Time to make it clear
* indicates the two rows to return.
Values should be returns as columns, i.e: SELECT minID, minDate, maxID, maxDate FROM myTable
| ID | Date | TimeOnly |
|----|---------------------|-----------|
| 5 | 14/11/2019 10:01:29 | 10:01:29* |
| 10 | 15/11/2019 10:01:29 | 10:01:29 |
| 6 | 14/11/2019 10:03:41 | 10:03:41 |
| 7 | 14/11/2019 10:07:09 | 10:07:09 |
| 11 | 15/11/2019 12:01:43 | 12:01:43 |
| 8 | 14/11/2019 14:37:16 | 14:37:16 |
| 1 | 12/11/2019 15:04:50 | 15:04:50 |
| 9 | 14/11/2019 15:04:50 | 15:04:50 |
| 2 | 13/11/2019 18:10:41 | 18:10:41 |
| 3 | 13/11/2019 18:10:56 | 18:10:56 |
| 4 | 13/11/2019 18:11:03 | 18:11:03* |
In earlier versions of MySQL, you can use couple of inline queries. This is a straight-forward option that could be quite efficient here:
select
(select ID from mytable order by TimeOnlylimit 1) minID,
(select Date from mytable order by TimeOnly limit 1) minDate,
(select ID from mytable order by TimeOnly desc limit 1) maxID,
(select Date from mytable order by TimeOnly desc limit 1) maxDate
One option for MySQL 8+, using ROW_NUMBER with pivoting logic:
WITH cte AS (
SELECT *, ROW_NUMBER() OVER (ORDER BY TimeOnly) rn_min,
ROW_NUMBER() OVER (ORDER BY Date TimeOnly) rn_max
FROM yourTable
)
SELECT
MAX(CASE WHEN rn_min = 1 THEN ID END) AS minID,
MAX(CASE WHEN rn_min = 1 THEN Date END) AS minDate
MAX(CASE WHEN rn_max = 1 THEN ID END) AS maxID,
MAX(CASE WHEN rn_max = 1 THEN Date END) AS maxDate
FROM cte;
Here is an option for MySQL 5.7 or earlier:
SELECT
MAX(CASE WHEN pos = 1 THEN ID END) AS minID,
MAX(CASE WHEN pos = 1 THEN Date END) AS minDate
MAX(CASE WHEN pos = 2 THEN ID END) AS maxID,
MAX(CASE WHEN pos = 2 THEN Date END) AS maxDate
FROM
(
SELECT ID, Date, 1 AS pos FROM yourTable
WHERE TimeOnly = (SELECT MIN(TimeOnly) FROM yourTable)
UNION ALL
SELECT ID, Date, 2 FROM yourTable
WHERE TimeOnly = (SELECT MAX(TimeOnly) FROM yourTable)
) t;
This second 5.7 option uses similar pivoting logic, but instead of ROW_NUMBER is uses subqueries to identify the min and max records. These records are brought together using a union, along with an identifier to keep track of which record be min/max.
You could simply do this:
SELECT minval.ID, minval.Date, maxval.ID, maxval.Date
FROM (
SELECT ID, Date
FROM t
ORDER BY CAST(Date AS TIME)
LIMIT 1
) AS minval
CROSS JOIN (
SELECT ID, Date
FROM t
ORDER BY CAST(Date AS TIME) DESC
LIMIT 1
) AS maxval
If you want two rows then change CROSS JOIN query to a UNION ALL query.
Demo on db<>fiddle

SQL how to get the max price of the 33% cheapests products

I need to to get the max price of the 33% cheapests products. My idea is like this. Of course, this code is just an example. I need to use subqueries.
select max((select price from products order by preco limit 33% )) as result from products
For example
product_id price
1 10
2 50
3 100
4 400
5 900
6 8999
I need I query that returns 50, since 33% of the rows are 2, and the max value of the 2(33%) of the rows is 50.
In MySQL 8+, you would use window functions:
select avg(precio)
from (select p.*, row_number() over (order by precio) as seqnum,
count(*) over () as cnt
from products p
) p
where seqnum <= 0.33 * cnt;
Obviously there are multiple approaches to this but here is how I would do it.
Simply get a count on the table. This will let me pick the max price of the cheapest 33% of products. Let's say it returned n records. Third of that would be n/3. Here you can either round up or down but needs to be rounded in case of a fraction.
Then my query would be something like SELECT * FROM products ORDER BY price ASC LIMIT 1 OFFSET n/3. This would return me a single record with minimal calculations and look ups on MySQL side.
For MySQL versions under MySQL 8.0 you can use MySQL's user variables to simulate/emulate a ROW_NUMBER()
Query
SELECT
t.product_id
, t.price
, (#ROW_NUMBER := #ROW_NUMBER + 1) AS ROW_NUMBER
FROM
t
CROSS JOIN (SELECT #ROW_NUMBER := 0) AS init_user_variable
ORDER BY
t.price ASC
Result
| product_id | price | ROW_NUMBER |
| ---------- | ----- | ---------- |
| 1 | 10 | 1 |
| 2 | 50 | 2 |
| 3 | 100 | 3 |
| 4 | 400 | 4 |
| 5 | 900 | 5 |
| 6 | 8999 | 6 |
When we get the ROW_NUMBER we can use that in combination with ROW_NUMBER <= CEIL(((SELECT COUNT(*) FROM t) * 0.33));
Which works like this
(SELECT COUNT(*) FROM t) => Counts and returns 6
(SELECT COUNT(*) FROM t) * 0.33) Calculates 33% from 6 which is 1.98 and returns it
CEIL(..) Return the smallest integer value that is greater than or equal to 1.98 which is 2 in this case
ROW_NUMBER <= 2 So the last filter is this.
Query
SELECT
a.product_id
, a.price
FROM (
SELECT
t.product_id
, t.price
, (#ROW_NUMBER := #ROW_NUMBER + 1) AS ROW_NUMBER
FROM
t
CROSS JOIN (SELECT #ROW_NUMBER := 0) AS init_user_variable
ORDER BY
t.price ASC
) AS a
WHERE
ROW_NUMBER <= CEIL(((SELECT COUNT(*) FROM t) * 0.33));
Result
| product_id | price |
| ---------- | ----- |
| 1 | 10 |
| 2 | 50 |
see demo
To get get the max it's just as simple as adding ORDER BY a.price DESC LIMIT 1
Query
SELECT
a.product_id
, a.price
FROM (
SELECT
t.product_id
, t.price
, (#ROW_NUMBER := #ROW_NUMBER + 1) AS ROW_NUMBER
FROM
t
CROSS JOIN (SELECT #ROW_NUMBER := 0) AS init_user_variable
ORDER BY
t.price ASC
) AS a
WHERE
ROW_NUMBER <= CEIL(((SELECT COUNT(*) FROM t) * 0.33))
ORDER BY
a.price DESC
LIMIT 1;
Result
| product_id | price |
| ---------- | ----- |
| 2 | 50 |
see demo
If your version supports window functions, you can use NTILE(3) to divide the rows into three groups ordered by price. The first group will contain (about) "33%" of lowest prices. Then you just need to select the MAX value from that group:
with cte as (
select price, ntile(3) over (order by price) as ntl
from products
)
select max(price)
from cte
where ntl = 1
Demo
Prior to MySQL 8.0 I would use a temprary table with an AUTO_INCREMENT column:
create temporary table tmp (
rn int auto_increment primary key,
price decimal(10,2)
);
insert into tmp(price)
select price from products order by price;
set #max_rn = (select max(rn) from tmp);
select price
from tmp
where rn <= #max_rn / 3
order by rn desc
limit 1;
Demo

count of every day (mysql)

I want get something like this
Mysql data
(dat_reg)
1.1.2000
1.1.2000
1.1.2000
2.1.2000
2.1.2000
3.1.2000
I want to get:
(dat_reg) (count)
1.1.2000 - 3
2.1.2000 - 5
3.1.2000 - 6
What I tried is this:
SELECT COUNT( * ) as a , DATE_FORMAT( dat_reg, '%d.%m.%Y' ) AS dat
FROM members
WHERE (dat_reg > DATE_SUB(NOW() , INTERVAL 5 DAY))
GROUP BY DATE_FORMAT(dat_reg, '%d.%m.%Y')
ORDER BY dat_reg
but I get:
1.1.2000 - 3 | 2.1.2000 - 2 | 3.1.2000 - 1
Some tips how create query for this?
I would suggest using variables in MySQL:
SELECT d.*, (#sumc := #sumc + cnt) as running_cnt
FROM (SELECT DATE_FORMAT(dat_reg, '%d.%m.%Y') as dat, COUNT(*) as cnt
FROM members
WHERE dat_reg > DATE_SUB(NOW() , INTERVAL 5 DAY)
GROUP BY dat
ORDER BY dat_reg
) d CROSS JOIN
(SELECT #sumc := 0) params;
If you want an accumulative from the beginning of time, then you need an additional subquery:
SELECT d.*
FROM (SELECT d.*, (#sumc := #sumc + cnt) as running_cnt
FROM (SELECT DATE_FORMAT(dat_reg, '%d.%m.%Y') as dat, dat_reg, COUNT(*) as cnt
FROM members
GROUP BY dat
ORDER BY dat_reg
) d CROSS JOIN
(SELECT #sumc := 0) params
) d
WHERE dat_reg > DATE_SUB(NOW() , INTERVAL 5 DAY)
A subquery counting the rows where the registration date is less than or equal to the current registration date could help you out.
SELECT m2.dat_reg,
(SELECT count(*)
FROM members m3
WHERE m3.dat_reg <= m2.dat_reg) count
FROM (SELECT DISTINCT m1.dat_reg
FROM m1.members
WHERE m1.dat_reg > date_sub(now(), INTERVAL 5 DAY)) m2
ORDER BY m2.dat_reg;
(If you got days, on which no one registered and don't want to have gaps in the result, you need to replace the subquery aliased m2 with a table or subquery, that has all days in the respective range.)
I believe you can use the window functions to do the work:
mysql> SELECT employee, sale, date, SUM(sale) OVER (PARTITION by employee ORDER BY date) AS cum_sales FROM sales;
+----------+------+------------+-----------+
| employee | sale | date | cum_sales |
+----------+------+------------+-----------+
| odin | 200 | 2017-03-01 | 200 |
| odin | 300 | 2017-04-01 | 500 |
| odin | 400 | 2017-05-01 | 900 |
| thor | 400 | 2017-03-01 | 400 |
| thor | 300 | 2017-04-01 | 700 |
| thor | 500 | 2017-05-01 | 1200 |
+----------+------+------------+-----------+
In your case you already have the right groups, it is only a matter of specifying the order in which you want the data the be aggregated.
Source: https://mysqlserverteam.com/mysql-8-0-2-introducing-window-functions/
Cheers
Here is a solution using rank and a continuous count variable:
WITH ranked AS (
SELECT m.*
,ROW_NUMBER() OVER (PARTITION BY m.dat_reg ORDER BY m.id DESC) AS rn
FROM (
select id, dat_reg
,#cnt := #cnt + 1 AS ccount from members
,(SELECT #cnt := 0) var
WHERE (dat_reg > DATE_SUB(NOW(), INTERVAL 5 DAY))
) AS m
)
SELECT DATE_FORMAT(dat_reg, '%d.%m.%Y') as dat, ccount FROM ranked WHERE rn = 1;
DB-Fiddle

Update row order by date

I've asked this question in the morning, but did not get any answer for this. So I delete previous one, and ask this again cause I am getting stuck for a long while. Hope you guys can help me.
I have a table named overtime like following:
| total | remain | t_date |
|-------|--------|---------------------|
| 3 | 0 | 2016-01-01 12:20:00 |
| 4 | 0 | 2016-02-01 13:10:00 |
| 2 | 0 | 2016-03-01 14:40:00 |
| 3 | 0 | 2016-04-01 10:20:00 |
| 5 | 2 | 2016-05-01 17:20:00 |
I want to update column remain order by t_date desc, also I have an input parameter, assume it is $h = 9, the expected result is:
| total | remain | t_date |
|-------|--------|---------------------|
| 5 | 5 | 2016-05-01 17:20:00 | -- remain will be updated to 5 cause total = 5, then $h(6) = $h(9) - (total(5) - remain(2))
| 3 | 3 | 2016-04-01 10:20:00 | -- remain will be updated to 3 cause total = 3, then $h(3) = $h(6) - (total(3) - remain(0))
| 2 | 2 | 2016-03-01 14:40:00 | -- remain will be updated to 2 cause total = 2, then $h(1) = $h(3) - (total(2) - remain(0))
| 4 | 1 | 2016-02-01 13:10:00 | -- remain will be updated to 1 cause $h only has 1, then $h will be 0
| 3 | 0 | 2016-01-01 12:20:00 | -- cause $h = 0, this row has no need to be updated
Edited:
The sample data is like above, what I want to do is update column remain, and remain'value bases on total and an input parameter(just assume it is 9):
update order is order by t_date desc. For example, I must update 2016-05-01 17:20:00 row first, then 2016-04-01 10:20:00, then 2016-03-01 14:40:00 and so on...
Parameter is 9, it will be allocated to every row, and remain should be updated to total's value. For example, the first row 2016-05-01 17:20:00, total = 5 and remain = 2, so remain will be updated to 5, and parameter will minus total - remain, it will be 6 and do the next row's allocation, till row 2016-02-01 13:10:00, the paramter is 1, so this row's remain only needs to updated to 1. And another row will have no need to updated.
Here is SQLFiddle demo.
If there is any unclear point for question, please leave a comment, I can explain that.
Any help is appreciated. Thanks in advance.
SELECT QUERY:
SELECT
*,
IF(
(IF(#h <= 0,0,IF(#h >= total,total, #h)) + remain) <= total,
(IF(#h <= 0,0,IF(#h >= total,total, #h)) + remain),
(IF(#h <= 0,0,IF(#h >= total,total, #h)))
) allocated,
#h := #h - (total - remain)
FROM overtime
CROSS JOIN (SELECT #h := 9) var
ORDER BY t_date DESC
Demo of SELECT
UPDATE QUERY:
UPDATE
overtime OT
INNER JOIN
(
SELECT
*,
IF(
(IF(#h <= 0,0,IF(#h >= total,total, #h)) + remain) <= total,
(IF(#h <= 0,0,IF(#h >= total,total, #h)) + remain),
(IF(#h <= 0,0,IF(#h >= total,total, #h)))
) allocated,
#h := #h - (total - remain)
FROM overtime
CROSS JOIN (SELECT #h := 9) var
ORDER BY t_date DESC
) AS t
ON OT.t_date = t.t_date
SET OT.remain = t.allocated;
WORKING DEMO
Demo shows the table data sorted by descending order of date after being updated by the above update query.
More:
See Demo for h=2
You must use a sub query to do this, like following:
update overtime t1
join (
select overtime.*,
total - remain, IF(#h > (total - remain), total, #h + remain) as h,
#h := IF(#h > (total - remain), #h - (total - remain), 0)
from overtime
cross join (
select #h := 9
) t
order by t_date desc
) t2 on t1.t_date = t2.t_date
set t1.remain = t2.h;
Demo Here
You could use a sub-query that makes the calculations using a variable #h:
update overtime
join (select t_date,
least(#h,total)
+ if(least(#h,total) + remain <= total, remain, 0) new_remain,
#h := greatest(0, #h - (total - remain)) h
from overtime,
(select #h := 9) init
order by t_date desc
) as calc
on overtime.t_date = calc.t_date
set overtime.remain := calc.new_remain;
See this SQL fiddle