Expand query result in select where statement - mysql

I need to get all the values between two dates on the same table, but also, include the value immediately before the earliest one:
To be more clear, I have the following table (Table 1. with aprox. 46 million rows):
Table 1. (Table I have)
updatetime | value
. .
. .
2018-01-01 08:32:02 | 2
2018-01-01 09:23:12 | 5
2018-01-01 10:45:00 | 8 * value to include on the result.
2018-01-01 11:10:44 | 10 * <-- earliest date.
2018-01-01 13:11:54 | 12 *
2018-01-01 16:14:57 | 16 *
2018-01-01 20:00:55 | 22 *
2018-01-01 22:34:43 | 23 *
2018-01-01 23:55:23 | 23 * <-- latest date.
2018-01-02 01:03:57 | 25
2018-01-02 03:39:07 | 28
. .
. .
I have accumulative values, so, the next value will always be equal or greater than the one before. I need to get all values between 2018-01-01 11:10:44 and 2018-01-02 01:03:57, but, I also need the value before the earliest date, that means that I need this result: (Table 2.)
Table 2. (Result I need)
2018-01-01 10:45:00 | 8
2018-01-01 11:10:44 | 10
2018-01-01 13:11:54 | 12
2018-01-01 16:14:57 | 16
2018-01-01 20:00:55 | 22
2018-01-01 22:34:43 | 23
2018-01-01 23:55:23 | 23
As you could see in Table 1, date spaces are not the same, so, previous date of the earliest date is unknown to me, and value incrementation is also random.
I already tried theese two queries, but I have doubts about performance:
The first one is just two queries made one, very bad performance because of the change of order of the table.
SELECT * FROM his
WHERE updatetime>=(
SELECT updatetime
FROM definition
WHERE updatetime<"2018-01-01 11:10:44"
ORDER BY updatetime DESC
LIMIT 1
) and updatetime<="2018-01-02 01:03:57";
The second should be slower than the first one, but if I define the column as incremental to make MySQL walk the table from the last value of the condition and not to walk all over it?
SELECT * FROM his
WHERE updatetime>=(
SELECT MAX(updatetime)
FROM definition
WHERE updatetime<"2018-01-01 11:10:44"
LIMIT 1
) AND updatetime<="2018-01-02 01:03:57";

First of all, for these queries to perform efficiently, you would need an index on column updatetime :
CREATE INDEX idx_updatetime ON mytable(updatetime);
One way to do it would to use a correlated subquery in the WHERE clause that selects the previous record :
SELECT t.*
FROM mytable t
WHERE
t.updatetime < '2018-01-02 01:03:57'
AND NOT EXISTS (
SELECT 1
FROM mytable t1
WHERE t1.updatetime < '2018-01-01 11:10:44' AND t1.updatetime > t.updatetime
)
Demo on DB Fiddle :
| updatetime | value |
| ------------------- | ----- |
| 2018-01-01 11:10:44 | 10 |
| 2018-01-01 13:11:54 | 12 |
| 2018-01-01 16:14:57 | 16 |
| 2018-01-01 20:00:55 | 22 |
| 2018-01-01 22:34:43 | 23 |
| 2018-01-01 23:55:23 | 23 |
Another option is to compute the updatetime of the previous record in a subquery, and then use it to filter the table. This might perform better than the first query.
SELECT t.*
FROM mytable t
INNER JOIN (
SELECT MAX(updatetime) updatetime
FROM mytable
WHERE updatetime < '2018-01-01 11:10:44'
) x ON t.updatetime >= x.updatetime AND t.updatetime < '2018-01-02 01:03:57';
Demo on DB Fiddle

Related

How to create an occurrence counter in MySQL ,each time the counter find the id it occurrence increases?

I'd like to count how many time each id occurred in the database using MYSQL
for the first time it reads the ID 100 it gives it 1 and for the fifth time it gives it 5 so each occurrence for each id is different
ID | occurrence
100 | 1
200 | 1
300 | 1
200 | 2
500 | 1
100 | 2
200 | 3
600 | 1
.
You ideally should have a third column which maintains the ordering of data you seem to be assuming above. Assuming that a timestamp column ts exists, you could use COUNT here as an analytic function:
SELECT ID, COUNT(*) OVER (PARTITION BY ID ORDER by ts) occurrence
FROM yourTable
ORDER BY ts;
This assumes your data would look something like:
ID | ts
100 | 2021-01-31 12:00:00
200 | 2021-01-31 13:00:00
300 | 2021-01-31 14:00:00
200 | 2021-01-31 15:00:00
500 | 2021-01-31 16:00:00
100 | 2021-01-31 17:00:00
200 | 2021-01-31 18:00:00
600 | 2021-01-31 19:00:00
This answer also assumes you are running MySQL 8+, and if not, consider upgrading.
Here is a version for MySQL 5.7:
SELECT
ID,
(SELECT COUNT(*) FROM yourTable t2
WHERE t2.ID = t1.ID AND t2.ts <= t1.ts) occurrence
FROM yourTable t1
ORDER BY ts;

Selecting the last record of yesterday and all records of today in single query

I know it's possible to get yesterday records, most common way using SUBDATE(CURDATE(), 1) or maybe simply use CURDATE() - 1 and use LIMIT and ORDER to retrieve the last record of yesterday.
But here, I need to get the last record of yesterday in the first row and the rest will be all records of today. I need to run this within single query.
For example, I have following records in one of my table:
--------------------------------------------------
| value | created_at |
--------------------------------------------------
| 70 | 1/1/2017 |
| 300 | 1/1/2017 |
| 100 | 1/1/2017 |
| 235 | 1/2/2017 |
| 45 | 1/2/2017 |
--------------------------------------------------
The created_at column is a timestamp, if today is 1/2/2017 (2th January 2017) then the result of the query should be:
--------------------------------------------------
| value | created_at |
--------------------------------------------------
| 100 | 1/1/2017 |
| 235 | 1/2/2017 |
| 45 | 1/2/2017 |
--------------------------------------------------
So far, I only able to retrieve the records of today with following query:
SELECT * FROM my_table WHERE created_at >= CURDATE();
What query I need to accomplish this?
Hoping you have id as primary key
select * from
(select
*
from
tbl
where date(created_at) =date(DATE_ADD(now(), INTERVAL -1 DAY))
order by id desc limit 0,1
)tmp
UNION
select * from tbl where date(created_at)=date(now())

UPDATE + SET + WHERE - Dynamic minimum value

This is a follow-up to:
Dynamic minimum value for specfic range (mysql)
I do have the query to fetch the third column (lowest of the last 3 days) "Low_3_days" via SELECT command:
-----------------------------------------
| Date | Unit_ | Lowest_in_last_|
| | price | 3_days |
|----------------------------------------
| 2015-01-01 | 15 | 15 |
| 2015-01-02 | 17 | 15 |
| 2015-01-03 | 21 | 15 |
| 2015-01-04 | 18 | 17 |
| 2015-01-05 | 12 | 12 |
| 2015-01-06 | 14 | 12 |
| 2015-01-07 | 16 | 12 |
|----------------------------------------
select S.Date,Unit_price,
(select S.Date, Unit_price,
(SELECT min(s2.Unit_Price)
FROM table s2
WHERE s2.DATE BETWEEN s.DATE - interval 3 day and
s.DATE - interval 1 day
) as min_price_3_days
FROM table S;
My new challenge is - what is the best way to use UPDATE-SET-WHERE so I could add the ("Lowest_in_last_3_days") values to a new column in a table (instead of having temporary results displayed to me via SELECT).
By following the UPDATE-SET-WHERE syntax, the query would be:
UPDATE table
SET min_price_3_days =
(select S.Date, Unit_price,
(SELECT min(s2.Unit_Price)
FROM table s2
WHERE s2.DATE BETWEEN s.DATE - interval 3 day and
s.DATE - interval 1 day
) as min_price_3_days
but I have difficulties constructing the correct query.
What would be the correct approach to this? I do recognize this one is a tough one to solve.
Your UPDATE should look like:
update table set low_3_days=
(SELECT min(Unit_Price)
FROM (select unit_price, date as date2 from table) as s2
WHERE s2.date2 BETWEEN date - interval 3 day and date - interval 1 day
);
You can check it in SQLFiddle
In Fiddle I used different names for table and column. I prefer not to use SQL keywords as names

Select difference between row dates in MySQL

I want to calculate the difference in unique date fields between different rows in the same table.
For instance, given the following data:
id | date
---+------------
1 | 2011-01-01
2 | 2011-01-02
3 | 2011-01-15
4 | 2011-01-20
5 | 2011-01-10
6 | 2011-01-30
7 | 2011-01-03
I would like to generate a query that produces the following:
id | date | days_since_last
---+------------+-----------------
1 | 2011-01-01 |
2 | 2011-01-02 | 1
7 | 2011-01-03 | 1
5 | 2011-01-10 | 7
3 | 2011-01-15 | 5
4 | 2011-01-20 | 5
6 | 2011-01-30 | 10
Any suggestions for what date functions I would use in MySQL, or is there a subselect that would do this?
(Of course, I don't mind putting WHERE date > '2011-01-01' to ignore the first row.)
A correlated subquery could be of help:
SELECT
id,
date,
DATEDIFF(
(SELECT MAX(date) FROM atable WHERE date < t.date),
date
) AS days_since_last
FROM atable AS t
Something like this should work :
SELECT mytable.id, mytable.date, DATEDIFF(mytable.date, t2.date)
FROM mytable
LEFT JOIN mytable AS t2 ON t2.id = table.id - 1
However, this imply that your id are continuous in your table, otherwise this won't work at all. And maybe MySQL will complain for the first row since t2.date will be null but I don't have the time to check now.

MySQL Query for obtaining count per hour

I need to obtain a count of how many actions occur on an hourly basis.
My database keeps a log by timestamp of the actions.
I understand that I could do a
SELECT table.time COUNT(table.time) from table t group by t.time
However, there are periods of time where no actions take place. For example if I have 10 actions during 8:00AM, no actions during 9:00AM and 4 actions during 10:00AM,
That query would return:
8:00 10
10:00 4
Skipping 9:00AM because it has no entries.
How can I make a query that will take into account 0-count entries.
I also need to make the same query for entries by days of the week, but I assume that by answering the first question I can easily handle the other.
Thanks in advance!
you can solve this by creating a table that will contain 24 values for hours (00:00, 01:00 etc) and perform a left (or right) join with it and your table allowing nulls so you will have all 24 rows even if your table contains 0 rows at all, then group by should work fine.
Dont forget to truncate everything but hour from your table when you perform join so result of func you call & perform join on can be equal to value of this help table.
you can use following query to do the job after populating testtime table with 24 test_time values
select test_time,sum(sign(coalesce(idFromYourTable,0))) as count from testtime
left join yourTable on test_time=hour(yourTableTime)
group by test_time
This will provide 0 as count if there are no values matching row from test table, while having count(*) will provide 24 rows with 1s instead of 0s even if your table is empty, also if there is just 1 row in your table it is impossible to distinguish the difference between 0 rows cause results will look the same for following 2 different rows
23 NULL
23 1
cause will both provide same result row count equal to 1 , while sum technique treats this rows differently
A simple way to do the same without creating any table would be as follows
SELECT
HOUR(time) 'hr', COUNT(DISTINCT id)
FROM schema.table
WHERE time BETWEEN '2016-01-23 00:00:00' AND '2016-01-24 00:00:00'
GROUP BY hr;
Hour function in mysql gives the hour from a datetime or timestamp data type. This query is grouping them based on particular hour withing the date range. Distinct is not mandatory but if you are looking for unique order or id in the time range per hour. This is the query.
You can use Valentin's solution but without the need to create a table of time slots. The idea is to generate the time slots on the fly and then JOIN them to your table as he suggests.
WITH RECURSIVE timeSlots (t) AS (
SELECT 0
UNION ALL
SELECT t + 3600 FROM timeSlots WHERE t < (23 * 3600)
)
SELECT t, TIME_FORMAT(SEC_TO_TIME(t), '%H:%i:%s') FROM timeSlots;
Gives:
+-------+----------+
| t | Time |
+-------+----------+
| 0 | 00:00:00 |
| 3600 | 01:00:00 |
| 7200 | 02:00:00 |
| 10800 | 03:00:00 |
| 14400 | 04:00:00 |
| 18000 | 05:00:00 |
| 21600 | 06:00:00 |
| 25200 | 07:00:00 |
| 28800 | 08:00:00 |
| 32400 | 09:00:00 |
| 36000 | 10:00:00 |
| 39600 | 11:00:00 |
| 43200 | 12:00:00 |
| 46800 | 13:00:00 |
| 50400 | 14:00:00 |
| 54000 | 15:00:00 |
| 57600 | 16:00:00 |
| 61200 | 17:00:00 |
| 64800 | 18:00:00 |
| 68400 | 19:00:00 |
| 72000 | 20:00:00 |
| 75600 | 21:00:00 |
| 79200 | 22:00:00 |
| 82800 | 23:00:00 |
+-------+----------+
If you want to change your time slot buckets you can just fiddle with the generator arithmetic rather than having to create another table.
An alternative is to use LIMIT:
WITH RECURSIVE timeSlots (t) AS (
SELECT 0
UNION ALL
SELECT t + 3600 FROM timeSlots LIMIT 24
)
SELECT t, TIME_FORMAT(SEC_TO_TIME(t), '%H:%i:%s') AS Time FROM timeSlots;