I'm trying to get amount of record based on two dates in MySql, example table :
id
data
datetime_1
datetime_2.
1
row data 1
2021-06-28 10:00:00
2021-07-02 10:00:00
and I want the result like this :
id
data
datetime
1
row data 1
2021-06-28 10:00:00
2
row data 1
2021-06-29 10:00:00
3
row data 1
2021-06-30 10:00:00
4
row data 1
2021-07-01 10:00:00
5
row data 1
2021-07-02 10:00:00
is that possible ?
You would typically handle this requirement via a calendar table, which is a table containing a sequence of dates which you expect to need in your query. Here is an example, where I have used an inline union subquery in place of a formal calendar table:
WITH dates AS (
SELECT '2021-06-28 10:00:00' AS dt UNION ALL
SELECT '2021-06-29 10:00:00' UNION ALL
SELECT '2021-06-30 10:00:00' UNION ALL
SELECT '2021-07-01 10:00:00' UNION ALL
SELECT '2021-07-02 10:00:00'
)
SELECT
ROW_NUMBER() OVER (ORDER BY t2.dt) AS id,
t1.data,
t2.dt AS datetime
FROM yourTable t1
INNER JOIN dates t2
ON t2.dt BETWEEN t1.datetime_1 AND t1.datetime_2
ORDER BY
t2.dt;
WITH RECURSIVE date_ranges AS (
SELECT datetime_1 AS dt1, datetime_2 dt2 FROM mytable
UNION ALL
SELECT dt1+INTERVAL 1 DAY, dt2 FROM date_ranges WHERE dt1 < dt2)
SELECT ROW_NUMBER() OVER (ORDER BY dt1) AS id,
b.data, dt1 as datetime FROM date_ranges a JOIN
mytable b ON a.dt1 BETWEEN datetime_1 AND b.datetime_2;
You can use this method to generate a date range based on your existing data. The final query might need other functions like MAX() etc. depending on your data but with the current sample that you've provided, this should suffice.
Fiddle with additional scenario
Related
I have the following sql table:
id time
1 2018-12-30
1 2018-12-31
1 2018-01-03
2 2018-12-15
2 2018-12-30
I want to make a query which will result in following data:
id start_time end_time
1 2018-12-30 2018-12-31
1 2018-12-31 2018-01-03
2 2018-12-15 2018-12-30
Is this even possible to do with sql in reasonable amount of time or it is better to do this with other means?
Failed approach (it takes too much time):
SELECT id, time as start_time, (
SELECT MIN(time)
FROM table as T2
WHERE T2.id = T1.id
AND T2.time < T1.time
) as end_time
FROM table as T1
I have dates in my db, each of them have non unique id. I want to calculate time range between closest dates for each id. So transformation should be performed on each id separately and should not affect other ids. We can even forget about ids, and just imagine that I have only one column in my DB which is dates. I want to sort my dates and perform sliding window with step 1 and capacity 2. So if I have 10 dates, I want to have in a result 9 time ranges, which are should be in increasing order. Assume we have four dates: D1 < D2 < D3 < D4. Result should be (D1,D2), (D2,D3), (D3,D4)
In MySQL 8.x you can use the LEAD() function to peek at the next row:
with x as (
select
id,
time as start_time,
lead(time) over(partition by id order by time) as end_time
from my_table
)
select * from x where end_time is not null
I am working on a project where I need to write validation query to validate data.
so I have two tables 1. Input table(raw data) 2. Output table(Harmonized data)
Currently, as a validation query, I am using below two queries to fetch results & then copy both results into excel file to validate if there is any difference in data or not.
1 Query
Select Date,sum(Val),sum(Vol)
From Input_table
Group by Date
2 Query
Select Date,sum(Val),sum(Vol)
From Output_table
Group by Date
Is there any way where I can put both these results in one query and also create one calculated column like.... (sum(Input_table.VAL)-sum(Output_table.VAL)) as Validation_Check.
So output will be like:
Date | sum(Input_table.Val) | sum(Output_table.Val) | Validation_Check
thanks.
It looks like you need to full join your results like:
select
ifnull(I.[Date], O.[Date]) as Date,
I.Val as Input_Val,
O.Val as Output_Val,
ifnull(I.Val, 0) - ifnull(O.Val, 0) as Validation_Check
from
(
Select Date,sum(Val) as Val,sum(Vol) as Vol
From Input_table
Group by Date
) as I
full outer join
(
Select Date,sum(Val) as Val,sum(Vol) as Vol
From Output_table
Group by Date
) as O on O.[Date] = I.[Date]
Use UNION. This will join two query on the condition that the two query have the same datatypes in the columns.
Use this Statement:
select Date, sum(Val), sum(Vol) from (
Select Date,Val,Vol
From Input_table
union
Select Date,Val,Vol
From Input_table
)
Group by Date
This will concat the data of both tables in the inner select and then Group it to one result
SELECT Date, SUM(VAL) as SUM_VAL, SUM(VOL) as SUM_VOL, SUM(VAL-VOL) as Validation_Check from
(Select Date,val,vol
From Input_table
UNION ALL
Select Date,val, vol
From Output_table
) X
group by Date
I suggest using a UNION ALL instead of a UNION here since there may be similar results fetched from both queries.
For example, your query 1 has a result like
May 01, 2017 | 5 | 5
and your query 2 has a result with the same values
May 01, 2017 | 5 | 5
If you use union, you'd only get 1 instance of
May 01, 2017 | 5 | 5
instead of 2 instances of
May 01, 2017 | 5 | 5
May 01, 2017 | 5 | 5
If your MySQL Supports FULL JOIN then you can use
SELECT
IFNULL(a.Date, b.Date) AS Date,
SUM(IFNULL(a.Val, 0)) AS Input_Val_Sum,
SUM(IFNULL(b.Val, 0)) AS Output_Val_Sum,
SUM(IFNULL(a.Val, 0) - IFNULL(b.Val, 0)) AS Validation_Check
FROM Input_table AS a
FULL OUTER JOIN Output_table AS b
ON a.Date = b.Date
GROUP BY IFNULL(a.Date, b.Date)
I have sqlite table like so:
CREATE TABLE "table" (
`id` INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT UNIQUE,
`param` REAL NOT NULL,
`date` INTEGER NOT NULL
);
INSERT INTO table (param , date) VALUES (123.3, 1427824800 );
INSERT INTO table (param , date) VALUES (122.3, 1427825800 );
INSERT INTO table (param , date) VALUES (125.0, 1427652000 );
INSERT INTO table (param , date) VALUES (123.9, 1427652900);
|id| param | date |
|==|=======|============|
| 1| 123.3 | 1427824800 |
| 2| 122.3 | 1427825800 |
| 3| 125 | 1427652000 |
| 4| 123.9 | 1427652900 |
And get row with max date for each day like(BETWEEN startDay AND endDay) - I need to understand how to group it at least by day, but if there are any way to group via custom period(week, month) it would be great
SELECT id, param, MAX(date) FROM table WHERE date BETWEEN 1427652000 AND 1427824799
SELECT id, param, MAX(date) FROM table WHERE date BETWEEN 1427824800 AND 1427911199
But trully I got much more data for nearly year and 1000+ rows and make 365 queries not an option I think, but I don't know how to optimize it
UPD
After all I think it is real to get one query to get result like this:
|id| param | date |
|==|=======|============|
| 2| 122.3 | 1427825800 |
| 4| 123.9 | 1427652900 |
Here's what you want. You'll use GROUP BY and you'll use the DATE() function to extract the day from each item. UNIX_TIMESTAMP() and FROM_UNIXTIME() are helpful for flipping back and forth between the TIMESTAMP and DATETIME representations. This is necessary because the date arithmetic stuff works on DATETIME values. CURDATE() means today.
SELECT DATE(FROM_UNIXTIME(`date`)) AS day,
MAX(`date`) AS latest_timestamp_in_day
FROM table
WHERE `date` >= UNIX_TIMESTAMP(CURDATE() - INTERVAL 366 DAY)
AND `date` < UNIX_TIMESTAMP(CURDATE())
GROUP BY DATE(FROM_UNIXTIME(`date`))
ORDER BY DATE(FROM_UNIXTIME(`date`)) DESC
This works because
DATE(FROM_UNIXTIME(`date`))
takes each timestamp and returns the DATETIME value of the first moment of the calendar day in which it falls.
The first WHERE clause picks up dates on or after a year and a day ago. The second one excludes today's dates; presumably the MAX operation doesn't make much sense on a not-yet-completed day.
It isn't clear from your question whether you also want to display the param value associated with the last timestamp in each calendar day. If you do, that's a little harder. You first need to get the latest timestamp in each day, then you need to pull out the detail record. That requires a join operation. You'll treat the above query as a subquery, and join it to your table. Like this.
SELECT summary.day, detail.`date`, detail.param
FROM table AS detail
JOIN (
SELECT DATE(FROM_UNIXTIME(`date`)) AS day,
MAX(`date`) AS latest_timestamp_in_day
FROM table
WHERE `date` >= UNIX_TIMESTAMP(CURDATE() - INTERVAL 366 DAY)
AND `date` < UNIX_TIMESTAMP(CURDATE())
GROUP BY DATE(FROM_UNIXTIME(`date`))
) AS summary ON detail.`date` = summary.latest_timestamp_in_day
ORDER BY summary.day DESC
Careful, though. the DATETIME arithmetic is done in the local time zone. This can lead to bizarre results on the days when local time changes from daylight savings to standard and back.
Notice that your column named date is in backticks. It's the same as a reserved word in MySQL's query language, so the backticks help disambiguate.
Here's a more detailed exposition of this business of grouping by date. http://www.plumislandmedia.net/mysql/sql-reporting-time-intervals/
You can use a generator to generate each year day and then join with your table:
SELECT r0 as start_day, MAX(date)
(
SELECT #row as r0, #row := #row + (24*60*60) as r1 FROM
(select 0 union all select 1 union all select 3 union all
select 4 union all select 5 union all select 6 union all
select 6 union all select 7 union all select 8 union all select 9) st1,
(select 0 union all select 1 union all select 3 union all
select 4 union all select 5 union all select 6 union all
select 6 union all select 7 union all select 8 union all select 9) st2,
(select 0 union all select 1 union all select 3 union all select 4 ) st3,
(SELECT #row:=1420070400) st00
) S
left outer join T
on T.date BETWEEN S.r0 AND S.r1
where r1 <= 1451520000
Currently trying to create a query that shows how many accounts have paid month on month but on a cumulative basis (penetration). So as an example I have a table with Month paid and account number, which shows what month that account paid.
Month | AccountNo
Jan-14 | 123456
Feb-14 | 321654
So using the above the result set would show
Month | Payers
Jan-14 | 1
Feb-14 | 2
being because one account paid in Jan, then one in Feb meaning that there have been by the end of Feb 2 payments overall, but only one in Jan. Tried a few inner joins back onto the table itself with a t1.Month >= t2.Month as i would for a normal cumulative query but the result is always out.
Any questions please ask, unsure if the above will be clear to anyone but me.
If you have date in the table then you can try the following query.
SELECT [Month]
,(SELECT COUNT(AccountNo)
FROM theTable i
-- This is to make sure to add until the last day of the current month.
WHERE i.[Date] <= DATEADD(s,-1,DATEADD(mm, DATEDIFF(m,0,o.[Date])+1,0)) AS CumulativeCount
FROM theTable o
Ok, several things. You need to have an actual date field, as you can't order by the month column you have.
You need to consider there may be gaps in the months - i.e. some months where there is no payment (not sure if that is true or not)
I'd recommend a recursive common table expression to do the actual aggregation
Heres how it works out:
-- setup
DECLARE #t TABLE ([Month] NCHAR(6), AccountNo INT)
INSERT #t ( [Month], AccountNo )
VALUES ( 'Jan-14',123456),('Feb-14',456789),('Apr-14',567890)
-- assume no payments in march
; WITH
t2 AS -- get a date column we can sort on
(
SELECT [Month],
CONVERT(DATETIME, '01 ' + REPLACE([Month], '-',' '), 6) AS MonthStart,
AccountNo
FROM #t
),
t3 AS -- group by to get the number of payments in each month
(
SELECT [Month], MonthStart, COUNT(1) AS PaymentCount FROM t2
GROUP BY t2.[Month], t2.MonthStart
),
t4 AS -- get a row number column to order by (accounting for gaps)
(
SELECT [Month], MonthStart, PaymentCount,
ROW_NUMBER() OVER (ORDER BY MonthStart) AS rn FROM t3
),
t5 AS -- recursive common table expression to aggregate subsequent rows
(
SELECT [Month], MonthStart, PaymentCount AS CumulativePaymentCount, rn
FROM t4 WHERE rn = 1
UNION ALL
SELECT t4.[Month], t4.MonthStart,
t4.PaymentCount + t5.CumulativePaymentCount AS CumulativePaymentCount, t4.rn
FROM t5 JOIN t4 ON t5.rn + 1 = t4.rn
)
SELECT [Month], CumulativePaymentCount FROM t5 -- select desired results
and the results...
Month CumulativePaymentCount
Jan-14 1
Feb-14 2
Apr-14 3
If your month column is date type then its easy to work on else you need some additional conversion for it. Here the query goes...
create table example (
MONTHS datetime,
AccountNo INT
)
GO
insert into example values ('01/Jan/2009',300345)
insert into example values ('01/Feb/2009',300346)
insert into example values ('01/Feb/2009',300347)
insert into example values ('01/Mar/2009',300348)
insert into example values ('01/Feb/2009',300349)
insert into example values ('01/Mar/2009',300350)
SELECT distinct datepart (m,months),
(SELECT count(accountno)
FROM example b
WHERE datepart (m,b.MONTHS) <= datepart (m,a.MONTHS)) AS Total FROM example a
I want to make a MySQL to get daily differential values from a table who looks like this:
Date | VALUE
--------------------------------
"2011-01-14 19:30" | 5
"2011-01-15 13:30" | 6
"2011-01-15 23:50" | 9
"2011-01-16 9:30" | 10
"2011-01-16 18:30" | 15
I have made two subqueries. The first one is to get the last daily value, because I want to compute the difference values from this data:
SELECT r.Date, r.VALUE
FROM table AS r
JOIN (
SELECT DISTINCT max(t.Date) AS Date
FROM table AS t
WHERE t.Date < CURDATE()
GROUP BY DATE(t.Date)
) AS x USING (Date)
The second one is made to get the differential values from the result of the first one (I show it with "table" name):
SELECT Date, VALUE - IFNULL(
(SELECT MAX( VALUE )
FROM table
WHERE Date < t1.table) , 0) AS diff
FROM table AS t1
ORDER BY Date
At first, I tried to save the result of first query in a temporary table but it's not possible to use temporary tables with the second query. If I use the first query inside the FROM of second one between () with an alias, the server complaints about table alias doesn't exist. How can get a something like this:
Date | VALUE
---------------------------
"2011-01-15 00:00" | 4
"2011-01-16 00:00" | 6
Try this query -
SELECT
t1.dt AS date,
t1.value - t2.value AS value
FROM
(SELECT DATE(date) dt, MAX(value) value FROM table GROUP BY dt) t1
JOIN
(SELECT DATE(date) dt, MAX(value) value FROM table GROUP BY dt) t2
ON t1.dt = t2.dt + INTERVAL 1 DAY