I'm trying create an SQL query to resolve my problem.
I use mysqk5.7.
My Table:
|project_id|start |end |cost(per month)|
|1 |2018-05-01|2018-06-30|1000 |
|2 |2018-06-01|2018-07-31|2000 |
I want to generate date-columns by start and end columns.
like this:
|date |project_id|cost|
|2018-05|1 |1000|
|2018-06|1 |1000|
|2018-06|2 |2000|
|2018-07|2 |2000|
Create a table and populate it with first day of each month. You can programmatically do that or even use Excel to generate data and port it to MySQL.
create table dates (
start_date date
);
insert into dates values
('2018-04-01'),
('2018-05-01'),
('2018-06-01'),
('2018-07-01'),
('2018-8-01');
Then, you can run a query like so:
Query
select
date_format(start_date, '%Y-%m') as `Date`,
a.project_id,
a.cost
from projects a
inner join dates b on b.start_date between a.start and a.end;
Result
Date project_id cost
2018-05 1 1000
2018-06 1 1000
2018-06 2 2000
2018-07 2 2000
Example
http://rextester.com/JRIUZ98116
Alternative
The other alternative is to create a stored procedure that creates a temporary table containing dates so that you don't have to generate a table. Minimum start date and maximum end date from the table can be extracted to create the temporary table of dates.
Then, the stored procedure can do the same join as above to generate a resultset.
This is one of those places where a separate date table will make life much easier. If you have a table with something like this:
create table DateTable(ThisDate date, Month varchar(7))
adding whatever other columns you might need (isWeekday etc) and populate it in a loop then you will be able to re-use this for all sorts of things, including this query. For example you can create a view on it to get month, startdate, enddate, and then join from that back into your table looking for dates that are between the start and end date.
This and lots of other queries will become simple.
create table DateTable(ThisDate date, Month varchar(7))
--- populate the table just once, re-use for all future queries
create view MonthView as
select Month,
min(ThisDate) as StartOfMonth,
max(ThisDate) as EndOfMonth
from DateTable
select Month, ProjectID, Cost
from MonthView
join MyTab on myTab.Start<=EndOfmonth and myTab.End>=StartofMonth
Related
I have information to be collected monthly. same data columns but different content of course. I'm asking about which are the best way to make the user insert this data, should I make a database table for each month with the same columns, or should I make one table with one column to determine the month.
For example:
table: July
id|program_name|program_date|program_result
table: June
id|program_name|program_date|program_result
Or:
table: monthly_info
id|program_name|program_date|program_result|month
I'm asking which way is more efficient than the other.
Thank you
Create one table to save all your data with date.
table: monthly_info
id|program_name|program_date|program_result|date
Then you can query monthly data as below.
If your condition month parameter is integer. Use this query. (this will return all data matches to month August)
SELECT * FROM monthly_info WHERE MONTH(date) = 8
If your condition month parameter is string. Use this query.
SELECT * FROM monthly_info WHERE DATENAME(mm, date) = 'August'
I am using the Graph Reports for the select below. The MySQL database only has the active records in the database, so if no records are in the database from X hours till Y hours that select does not return anything. So in my case, I need that select return Paypal zero values as well even the no activity was in the database. And I do not understand how to use the UNION function or re-create select in order to get the zero values if nothing was recorded in the database in time interval. Could you please help?
select STR_TO_DATE ( DATE_FORMAT(`acctstarttime`,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', count(*) as `Active Paid Accounts`
from radacct_history where `paymentmethod` = 'PayPal'
group by DATE_FORMAT(`#date`,'%y-%m-%d %H')
When I run the select the output is:
Current Output
But I need if there are no values between 2016-07-27 07:00:00 and 2016-07-28 11:00:00, then in every hour it should show zero active accounts Like that:
Needed output with no values every hour
I have created such select below , but it not put to every hour the zero value like i need. showing the big gap between the 12 Sep and 13 Sep anyway, but there should be the zero values every hour
(select STR_TO_DATE ( DATE_FORMAT(acctstarttime,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', count(paymentmethod) as Active Paid Accounts
from radacct_history where paymentmethod <> 'PayPal'
group by DATE_FORMAT(#date,'%y-%m-%d %H'))
union ALL
(select STR_TO_DATE ( DATE_FORMAT(acctstarttime,'%y-%m-%d %H'),'%y-%m-%d %H')
as '#date', 0 as Active Paid Accounts
from radacct_history where paymentmethod <> 'PayPal'
group by DATE_FORMAT(#date,'%y-%m-%d %H')) ;
I guess, you want to return 0 if there is no matching rows in MySQL. Here is an example:
(SELECT Col1,Col2,Col3 FROM ExampleTable WHERE ID='1234')
UNION (SELECT 'Def Val' AS Col1,'none' AS Col2,'' AS Col3) LIMIT 1;
Updated the post: You are trying to retrieve data that aren't present in the table, I guess in reference to the output provided. So in this case, you have to maintain a date table to show the date that aren't in the table. Please refer to this and it's little bit tricky - SQL query that returns all dates not used in a table
You need an artificial table with all necessary time intervals. E.g. if you need daily data create a table and add all day dates e.g. start from 1970 till 2100.
Then you can use the table and LEFT JOIN your radacct_history. So for each desired interval you will have group item (group by should be based on the intervals table.
I am working with MySQL database and I have to generate date intervals for specified period(specified by start and stop date) with specified step(for example one day).
I have written a stored procedure to generate intervals, to create a temporary table and to populate this table with intervals.
DELIMITER $$
CREATE PROCEDURE showu(IN start date, IN stop date)
BEGIN
CREATE TEMPORARY TABLE intervals(single_day DATE);
next_date: LOOP
IF start>stop THEN
LEAVE next_date;
END IF;
INSERT INTO intervals(single_day) VALUES(start);
SET start = DATE_ADD(start, INTERVAL 1 DAY);
END LOOP next_date;
END$$
DELIMITER ;
I want to use this temporary table in join queries. However I faced with a problem. When I call procedure call showu('2008-01-09', '2010-02-09'); it is executing approximately 30 seconds. The question why it is executing so long? Is it possible to improve it? If this solution is wrong how can I resolve my problem in different way?
From comments:
2 big problems: 1. I don't know exactly value of step(one day or one month or one hour).
Create one big table like this once (not temporary):
full_date | year | month | day | full_time | hour | minute | is_weekend | whatever
----------------------------------------------------------------------------------
...
Create as much indexes as needed and you will have a very performant a powerful swiss knife for all sorts of reports.
Note: You might consider not having time and date in the same table. This is just to simplify the example.
Your second problem
I will clog my database with not model data.
is no problem. Databases are there to hold data. That's it. If you have problems with space or whatever, the solution is to get more space, not to limit your ability to work efficiently.
That being said, here's some examples how to use this table.
You need dates:
SELECT
*
FROM
(
SELECT full_date AS your_step
FROM your_new_swiss_army_knife
WHERE `year` = 2012
GROUP BY full_date
) dates
LEFT JOIN your_tables_that_you_want_to_build_a_report_on y ON dates.your_step = y.date
Same with months:
SELECT
*
FROM
(
SELECT CONCAT(year, '-', month) AS your_step
FROM your_new_swiss_army_knife
WHERE full_date BETWEEN this AND that
GROUP BY year, month
) dates
LEFT JOIN your_tables_that_you_want_to_build_a_report_on y ON dates.your_step = CONCAT(YEAR(y.date), '-', MONTH(y.date))
So lets say I have some records that look like:
2011-01-01 Cat
2011-01-02 Dog
2011-01-04 Horse
2011-01-06 Lion
How can I construct a query that will return 2011-01-03 and 2011-01-05, ie the unused dates. I postdate blogs into the future and I want a query that will show me the days I don't have anything posted yet. It would look from the current date to 2 weeks into the future.
Update:
I am not too excited about building a permanent table of dates. After thinking about it though it seems like the solution might be to make a small stored procedure that creates a temp table. Something like:
CREATE PROCEDURE MISSING_DATES()
BEGIN
CREATE TABLE TEMPORARY DATES (FUTURE DATETIME NULL)
INSERT INTO DATES (FUTURE) VALUES (CURDATE())
INSERT INTO DATES (FUTURE) VALUES (ADDDATE(CURDATE(), INTERVAL 1 DAY))
...
INSERT INTO DATES (FUTURE) VALUES (ADDDATE(CURDATE(), INTERVAL 14 DAY))
SELECT FUTURE FROM DATES WHERE FUTURE NOT IN (SELECT POSTDATE FROM POSTS)
DROP TABLE TEMPORARY DATES
END
I guess it just isn't possible to select the absence of data.
You're right — SQL does not make it easy to identify missing data. The usual technique is to join your sequence (with gaps) against a complete sequence, and select those elements in the latter sequence without a corresponding partner in your data.
So, #BenHoffstein's suggestion to maintain a permanent date table is a good one.
Short of that, you can dynamically create that date range with an integers table. Assuming the integers table has a column i with numbers at least 0 – 13, and that your table has its date column named datestamp:
SELECT candidate_date AS missing
FROM (SELECT CURRENT_DATE + INTERVAL i DAY AS candidate_date
FROM integers
WHERE i < 14) AS next_two_weeks
LEFT JOIN my_table ON candidate_date = datestamp
WHERE datestamp is NULL;
One solution would be to create a separate table with one column to hold all dates from now until eternity (or whenever you expect to stop blogging). For example:
CREATE TABLE Dates (dt DATE);
INSERT INTO Dates VALUES ('2011-01-01');
INSERT INTO Dates VALUES ('2011-01-02');
...etc...
INSERT INTO Dates VALUES ('2099-12-31');
Once this reference table is set up, you can simply outer join to determine the unused dates like so:
SELECT d.dt
FROM Dates d LEFT JOIN Blogs b ON d.dt = b.dt
WHERE b.dt IS NULL
If you want to limit the search to two weeks in the future, you could add this to the WHERE clause:
AND d.dt BETWEEN NOW() AND ADDDATE(NOW(), INTERVAL 14 DAY)
The way to extract rows from the mysql database is via SELECT. Thus you cannot select rows that do not exist.
What I would do is fill my blog table with all possible dates (for a year, then repeat the process)
create table blog (
thedate date not null,
thetext text null,
primary key (thedate));
doing a loop to create all dates entries for 2011 (using a program, eg $mydate is the date you want to insert)
insert IGNORE into blog (thedate,thetext) values ($mydate, null);
(the IGNORE keyword to not create an error (thedate is a primary key) if thedate exists already).
Then you insert the values normally
insert into blog (thedate,thetext) values ($mydate, "newtext")
on duplicate key update thetext="newtext";
Finally to select empty entries, you just have to
select thedate from blog where thetext is null;
You probably not going to like this:
select '2011-01-03', count(*) from TABLE where postdate='2011-01-03'
having count(*)=0 union
select '2011-01-04', count(*) from TABLE where postdate='2011-01-04'
having count(*)=0 union
select '2011-01-05', count(*) from TABLE where postdate='2011-01-05'
having count(*)=0 union
... repeat for 2 weeks
OR
create a table with all days in 2011, then do a left join, like
select a.days_2011
from all_days_2011
left join TABLE on a.days_2011=TABLE.postdate
where a.days_2011 between date(now()) and date(date_add(now(), interval 2 week))
and TABLE.postdate is null;
Is there an easy way to do a GROUP BY DATE(timestamp) that includes all days in a period of time, regardless of whether there are any records associated with that date?
Basically, I need to generate a report like this:
24 Dec - 0 orders
23 Dec - 10 orders
22 Dec - 8 orders
21 Dec - 2 orders
20 Dec - 0 orders
Assuming you have more orders than dates something like this could work:
select date, count(id) as orders
from
(
SELECT DATE_ADD('2008-01-01', INTERVAL #rn:=#rn+1 DAY) as date from (select #rn:=-1)t, `order` limit 365
) d left outer join `order` using (date)
group by date
One method is to create a calendar table and join against it.
I would create it permanently, and then create a task that will insert new dates, it could be done weekly, daily, monthly, etc.
Note, that I am assuming that you are converting your timestamp into a date.
Instead of using GROUP BY, make a table (perhaps a temporary table) which contains the specific dates you want, for example:
24 Dec
23 Dec
22 Dec
21 Dec
20 Dec
Then, join that table to the Orders table.
you need to generate an intermediate result set with all the dates in it that you want included in the output...
if you're doing this in a stored proc, then you could create a temp table or table variable (I don't knoiw MySQL's capabilities), but once you have all the dates in a table or resultset of some kind
Just join to the real dataa from the temp table, using an outer join
In SQL Server it would be like this
Declare #Dates Table (aDate DateTime Not Null)
Declare #StartDt DateTime Set #StartDt = 'Dec 1 2008'
Declare #EndDt DateTime Set #EndDt = 'Dec 31 2008'
While #StartDt < #EndDt Begin
Insert #Dates(aDate) Values(#StartDt)
Set #StartDt = DateAdd(Day, 1, #StartDt)
End
Select D.aDate, Count(O.*) Orders
From #Dates D Left Join
OrderTable O On O.OrderDate = D.aDate
Group By D.aDate
In a data warehouse, the method taken is to create a table that contains all dates and create a foreign key between your data and the date table. I'm not saying that this is the best way to go in your case, just that it is the best practice in cases where large amounts of data need to be rolled up in numerous ways for reporting purposes.
If you are using a reporting layer over SQL Server, you could just write some logic to insert the missing dates within the range of interest after the data returns and before rendering your report.
If you are creating your reports directly from SQL Server and you do not already have a data warehouse and there isn't the time or need to create one right now, I would create a date table and join to it. The formatting necessary to do the join and get the output you want may be a bit wonky, but it will get the job done.
There's a pretty straightforward way to do this… except that I can't remember it. But I adapted this query from this thread:
SELECT
DISTINCT(LEFT(date_field,11)) AS `Date`,
COUNT(LEFT(date_field,11)) AS `Number of events`
FROM events_table
GROUP BY `Date`
It works in MySQL too