I need some help with a mysql query. I've got db table that has data from Jan 1, 2011 thru April 30, 2011. There should be a record for each date. I need to find out whether any date is missing from the table.
So for example, let's say that Feb 2, 2011 has no data. How do I find that date?
I've got the dates stored in a column called reportdatetime. The dates are stored in the format: 2011-05-10 0:00:00, which is May 5, 2011 12:00:00 am.
Any suggestions?
This is a second answer, I'll post it separately.
SELECT DATE(r1.reportdate) + INTERVAL 1 DAY AS missing_date
FROM Reports r1
LEFT OUTER JOIN Reports r2 ON DATE(r1.reportdate) = DATE(r2.reportdate) - INTERVAL 1 DAY
WHERE r1.reportdate BETWEEN '2011-01-01' AND '2011-04-30' AND r2.reportdate IS NULL;
This is a self-join that reports a date such that no row exists with the date following.
This will find the first day in a gap, but if there are runs of multiple days missing it won't report all the dates in the gap.
CREATE TABLE Days (day DATE PRIMARY KEY);
Fill Days with all the days you're looking for.
mysql> INSERT INTO Days VALUES ('2011-01-01');
mysql> SET #offset := 1;
mysql> INSERT INTO Days SELECT day + INTERVAL #offset DAY FROM Days; SET #offset := #offset * 2;
Then up-arrow and repeat the INSERT as many times as needed. It doubles the number of rows each time, so you can get four month's worth of rows in seven INSERTs.
Do an exclusion join to find the dates for which there is no match in your reports table:
SELECT d.day FROM Days d
LEFT OUTER JOIN Reports r ON d.day = DATE(r.reportdatetime)
WHERE d.day BETWEEN '2011-01-01' AND '2011-04-30'
AND r.reportdatetime IS NULL;`
It could be done with a more complicated single query, but I'll show a pseudo code with temp table just for illustration:
Get all dates for which we have records:
CREATE TEMP TABLE AllUsedDates
SELECT DISTINCT reportdatetime
INTO AllUsedDates;
now add May 1st so we track 04-30
INSERT INTO AllUsedData ('2011-05-01')
If there's no "next day", we found a gap:
SELECT A.NEXT_DAY
FROM
(SELECT reportdatetime AS TODAY, DATEADD(reportdatetime, 1) AS NEXT_DAY FROM AllUsed Dates) AS A
WHERE
(A.NEXT_DATE NOT IN (SELECT reportdatetime FROM AllUsedDates)
AND
A.TODAY <> '2011-05-01') --exclude the last day
If you mean reportdatetime has the entry of "Feb 2, 2011" but other fields associated to that date are not present like below table snap
reportdate col1 col2
5/10/2011 abc xyz
2/2/2011
1/1/2011 bnv oda
then this query works fine
select reportdate from dtdiff where reportdate not in (select df1.reportdate from dtdiff df1, dtdiff df2 where df1.col1 = df2.col1)
Try this
SELECT DATE(t1.datefield) + INTERVAL 1 DAY AS missing_date FROM table t1 LEFT OUTER JOIN table t2 ON DATE(t1.datefield) = DATE(t2.datefield) - INTERVAL 1 DAY WHERE DATE(t1.datefield) BETWEEN '2020-01-01' AND '2020-01-31' AND DATE(t2.datefield) IS NULL;
If you want to get missing dates in a datetime field use this.
SELECT CAST(t1.datetime_field as DATE) + INTERVAL 1 DAY AS missing_date FROM table t1 LEFT OUTER JOIN table t2 ON CAST(t1.datetime_field as DATE) = CAST(t2.datetime_field as DATE) - INTERVAL 1 DAY WHERE CAST(t1.datetime_field as DATE) BETWEEN '2020-01-01' AND '2020-07-31' AND CAST(t2.datetime_field as DATE) IS NULL;
The solutions above seem to work, but they seem EXTREMELY slow (taking possibly hours, I waited for 30 min only) at least in my database.
This clause takes less than a second in same database (of course you need to repeat it manually dozen times and possibly change function names to find the actual dates). pvm = my datetime, WEATHER = my table.
mysql> select year(pvm) as _year,count(distinct(date(pvm))) as _days from WEATHER where year(pvm)>=2000 and month(pvm)=1 group by _year order by _year asc;
--ako
Related
I am using the Graph Reports for the select below. The MySQL database only has the active records in the database, so if no records are in the database from X hours till Y hours that select does not return anything. So in my case, I need that select return Paypal zero values as well even the no activity was in the database. And I do not understand how to use the UNION function or re-create select in order to get the zero values if nothing was recorded in the database in time interval. Could you please help?
select STR_TO_DATE ( DATE_FORMAT(`acctstarttime`,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', count(*) as `Active Paid Accounts`
from radacct_history where `paymentmethod` = 'PayPal'
group by DATE_FORMAT(`#date`,'%y-%m-%d %H')
When I run the select the output is:
Current Output
But I need if there are no values between 2016-07-27 07:00:00 and 2016-07-28 11:00:00, then in every hour it should show zero active accounts Like that:
Needed output with no values every hour
I have created such select below , but it not put to every hour the zero value like i need. showing the big gap between the 12 Sep and 13 Sep anyway, but there should be the zero values every hour
(select STR_TO_DATE ( DATE_FORMAT(acctstarttime,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', count(paymentmethod) as Active Paid Accounts
from radacct_history where paymentmethod <> 'PayPal'
group by DATE_FORMAT(#date,'%y-%m-%d %H'))
union ALL
(select STR_TO_DATE ( DATE_FORMAT(acctstarttime,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', 0 as Active Paid Accounts
from radacct_history where paymentmethod <> 'PayPal'
group by DATE_FORMAT(#date,'%y-%m-%d %H')) ;
I guess, you want to return 0 if there is no matching rows in MySQL. Here is an example:
(SELECT Col1,Col2,Col3 FROM ExampleTable WHERE ID='1234')
UNION (SELECT 'Def Val' AS Col1,'none' AS Col2,'' AS Col3) LIMIT 1;
Updated the post: You are trying to retrieve data that aren't present in the table, I guess in reference to the output provided. So in this case, you have to maintain a date table to show the date that aren't in the table. Please refer to this and it's little bit tricky - SQL query that returns all dates not used in a table
You need an artificial table with all necessary time intervals. E.g. if you need daily data create a table and add all day dates e.g. start from 1970 till 2100.
Then you can use the table and LEFT JOIN your radacct_history. So for each desired interval you will have group item (group by should be based on the intervals table.
I am trying to nest a few queries but so far am getting back error 1242: Subquery returns more than 1 row. I want more than one row, as I am working on a number of records.
I have 2 tables. One has a commencement date stored in 3 columns; yr_comm, mth_comm, day_comm. The 2nd table has a period of service (in years) for a number of users which is expressed as an integer (2.71, 3.45, etc).
I need to take this start date (from table 1), and add on the period of service (from table 2) to obtain an end date, but I only need to display the year.
I have 2 queries which work just fine when seperate, they result in the required values, however I am having trouble combining the queries to get the desired end result.
Query 1: Concatenate the 3 commencement values into date format
SELECT concat_ws('-', yr_comm, mth_comm, day_comm) AS date_comm
FROM table 1
Query 2: Convert the integer yrs_service into days
SELECT format(yrs_served * 365, 0) AS days_served
FROM table 2
Query 3: Use date_add function to add the days service to the commencement date
SELECT date_add(date_comm, INTERVAL days_served DAY) AS date_left
Can anyone suggest how I can achieve the above? Many thanks in advance.
EDIT - Here is the full query I am working on:
SELECT prime_minister.pm_name, yr_comm, party, ADDDATE(
(SELECT CONCAT_WS('-', yr_comm, mth_comm, day_comm) FROM ministry), INTERVAL
(SELECT FORMAT(yrs_served * 365, 0) FROM prime_minister) YEAR) AS date_left
FROM ministry JOIN prime_minister USING (pm_name)
WHERE party NOT LIKE '%labor%'
AND prime_minister.pm_name = ministry.pm_name
ORDER BY pm_name;
you can use user variables
SET #date = CONCAT_WS('-', 2012,1,1); -- paste your query here
SET #toAdd = (SELECT MONTH(CURDATE())); -- paste your query here
SELECT DATE_ADD(#date, INTERVAL #toAdd DAY) AS date_left
SQLFiddle Demo
which is the same as
SET #date = CONCAT_WS('-', 2012,1,1); -- paste your query here
SET #toAdd = (SELECT MONTH(CURDATE())); -- paste your query here
SELECT #date + INTERVAL #toAdd DAY AS date_left
SQLFiddle Demo
or without using variable, which is more longer,
SELECT (CONCAT_WS('-', 2012,1,1)) + INTERVAL (SELECT MONTH(CURDATE())) DAY AS date_left
SQLFiddle Demo
I have a query that looks like (I've tried to strip out non-relevant fields/joins for clarity):
SET #num = -1;
SELECT
*,
CAST(DATE_ADD( '2012-04-01', interval #num := #num+1 day)AS DATE) AS date_sequence,
DAYOFWEEK(DATE_ADD('2012-04-01', interval #num+1 day)) AS day_week
FROM batch AS b1
left join (
select
batch.`startedDate` AS batch_startedDate,
epiRun.`runType` AS epiRun_runType
.... other fields selected.........
from batch
left join `epiRun` epiRun ON epiRun.`batchID`= batch.`keyID`
.......other joins........
WHERE batch.`startedDate` >= '2012-04-01' AND batch.`startedDate` <= '2012-04-18'
ORDER BY batch.`startedDate`ASC)
AS b2 ON cast((b2.`batch_startedDate`) AS DATE)=CAST(DATE_ADD('2012-04-01', interval #num+1
day)AS DATE)
WHERE
(DATE_ADD('2012-04-01', interval #num+1 day) <= '2012-04-18')
The nested select query performs as I expect when run by itself. This query has a couple of problems:
-Every field in from the batch table is selected, but since this is for an iReport it's not too much of a problem
-I get the list of dates from 1st April to 18th April, but if I have multiple batches on a day then I only get one displayed - ideally I'd like multiple identical entries in the date column with a unique entry for each batch. It is important that I can see when there are days with no batches.
Example of table I have:
Date Batch
01/04/2012 TS01
02/04/2012 TS03
03/04/2012 null
and what I'd like to generate:
Date Batch
01/04/2012 TS01
01/04/2012 TS02
02/04/2012 TS03
02/04/2012 TS04
03/04/2012 null
I personally would create a separate "dates" table and populate it with all the dates from say 1/1/2000 through 12/31/2050 (or whatever date range will cover all potential queries) and then do a left join from that table to the batch and epiRun tables.
I think that this is a much cleaner way to do what you are looking for and will give you exactly the results you desire.
I have a table that tracks emails sent. It is pretty simple.
ID | DATETIME | E-MAIL | SUBJECT | MESSAGE
I have been collecting data for several years. Some days I don't have any entries in the table.
query1:
SELECT COUNT(ID) FROM emails
WHERE DATE(datetime) >= 'XXXX-XX-XX'
AND DATE(datetime) is <= 'ZZZZ-ZZ-ZZ'
GROUP BY DATE(datetime)
I then use a some php to get one year prior for both XXXX and YYYY and run the second query which is the same as the first...
query2:
SELECT COUNT(ID) from emails
WHERE DATE(datetime) >= 'XXXX-XX-XX'
AND DATE(datetime) is <= 'ZZZZ-ZZ-ZZ'
GROUP BY DATE(datetime)
I am using a charting package to compare how many emails I got for a date range and then I overlay how many emails I got for the same range only one year prior. This is two queries right now and I chart the results.
The issue is where mysql does not have any emails for 2011 for a day in question, but has a few in 2012 for the same day.
Combining the results and graphing them skews the results since I am missing a date and a 0 value for last year for that day, effectively making all my values no longer match up.
2011-03-01 10 2012-03-01 4
2011-03-02 4 2012-03-02 2
2011-03-03 6 2012-03-04 1 <---- see where the two queries
end up diverging? (I had nothing
logged for 2012-03-03 so naturally
it was not in the results.
Is there a way I can get mysql to output the data I need including dates where value appear in one year but not another OR if no values appear in either year (still need date and 0) so my chart works?
I cannot seem to figure out how to do this...
Thanks!
There are a few different ways to get the results for a contiguous set of dates. My favourite one is to create the full set that is required using a dummy table or an existing contiguous set of ids from an AI PK. Something like this -
SELECT '2011-01-01' + INTERVAL (id -1) DAY
FROM dummy
WHERE id BETWEEN 1 AND 365
This will return a full set of days for 2011 which can then be LEFT JOINed to your emails table to get the counts -
SELECT `dates`.`date`, COUNT(emails.id)
FROM (
SELECT '2011-01-01' + INTERVAL (id - 1) DAY AS `date`, '2011-01-01 23:59:59' + INTERVAL (id - 1) DAY AS `end_of_day`
FROM dummy
WHERE id BETWEEN 1 AND 365
) `dates`
LEFT JOIN emails
ON `emails`.`datetime` BETWEEN `dates`.`date` AND `dates`.`end_of_day`
GROUP BY `dates`.`date`
To populate your dummy / seq table you can insert the first ten values manually and then use INSERT ... SELECT to add the rest -
CREATE TABLE dummy (id INTEGER NOT NULL PRIMARY KEY);
INSERT INTO dummy VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9),(10);
SET #tmp := (SELECT MAX(id) FROM dummy) + 1;
INSERT INTO dummy
SELECT #tmp + id
FROM dummy;
You need to execute the SET query before each run of the INSERT ... SELECT query.
I have some problems when coding SQL group by week.
I have a MySQL table named order.
In this entity, there are several attributes, called 'order_id', 'order_date', 'amount', etc.
I want to make a table to show the statistics of past 7 days order sales amount.
I think first I should get the today value.
Since I use Java Server Page, the code like this:
Calendar cal = Calendar.getInstance();
int day = cal.get(Calendar.DATE);
int Month = cal.get(Calendar.MONTH) + 1;
int year = cal.get(Calendar.YEAR);
String today = year + "-" + Month + "-" + day;
then, I need to use group by statement to calculate the SUM of past 7 day total sales amount.
like this:
ResultSet rs=statement.executeQuery("select order_date, SUM(amount) " +
"from `testing`.`order` GROUP BY order_date");
I have problem here. In my SQL, all order_date will be displayed.
How can I modify this SQL so that only display past seven days order sale amount?
Besides that, I discover a problem in my original SQL.
That is, if there is no sales on that day, no results would be displayed.
OF course, I know the ResultSet does not allow return null values in my SQL.
I just want to know if I need the past 7 order sales even the amount is 0 dollars,
Can I have other methods to show the 0?
Please kindly give me advices if you have idea.
Thank you.
Usually it occurs to create with a script or with a stored procedure a calendar table with all dates.
However if you prefer you can create a table with few dates (in your case dates of last week) with a single query.
This is an example:
create table orders(
id int not null auto_increment primary key,
dorder date,
amount int
) engine = myisam;
insert into orders (dorder,amount)
values (curdate(),100),
(curdate(),200),
('2011-02-24',50),
('2011-02-24',150),
('2011-02-22',10),
('2011-02-22',20),
('2011-02-22',30),
('2011-02-22',5),
('2011-02-19',10);
select t.cdate,sum(coalesce(o.amount,0)) as total
from (
select curdate() -
interval tmp.digit * 1 day as `cdate`
from (
select 0 as digit union all
select 1 union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7 ) as tmp) as t
left join orders as o
on t.cdate = o.dorder and o.dorder >= curdate() - interval 7 day
group by t.cdate
order by t.cdate desc
Hope that it helps. Regards.
To answer your question "How can I modify this SQL so that only display past seven days order sale amount?"
Modify the SQL statement by adding a where clause to it:
Where order_date >= #date_7days_ago
The value for this #date_7days_ago date variable can be set before your statement:
Select #date_7days_ago = dateadd(dd,-7,getdate())
Adding that where clause to your query will return only those records which order date is in the last seven days.
Hope this helps.
You can try using this:
ResultSet rs = statement.executeQuery(
"SELECT IFNULL(SUM(amount),0)
FROM table `testing`.`order`
WHERE order_date >= DATE_SUB('" + today + "', INTERVAL 7 DAY)"
);
This will get you the number of orders made in the last 7 days, and 0 if there were none.