SQL query to bring months that doesn't exist in a column - mysql

I want to count the number of rows with respect to the month that they've written in database. My database has a column named created_date. you can see what i've done bellow :
select month(created_date) as "Month", count(created_date) as "Count" from transactions group by month(created_date)
what this query returns is something like this :
{'Month':1,'Count':10}
this happens because i only have one month in my database, while i need to have all months in results, including months that doesn't exist in database, like this :
{'Month':1,'Count':10}
{'Month':2,'Count':0}
{'Month':3,'Count':0}
{'Month':4,'Count':0}
{'Month':5,'Count':0}
{'Month':6,'Count':0}
{'Month':7,'Count':0}
{'Month':8,'Count':0}
{'Month':9,'Count':0}
{'Month':10,'Count':0}
{'Month':11,'Count':0}
{'Month':12,'Count':0}
how should i do it?

You could JOIN to a list of months so you can get a row even for months that don't exist in your table:
SELECT m.month,
COUNT(t.created_date)
FROM (SELECT 1 AS month UNION ALL SELECT 2 UNION ALL SELECT 3
UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6
UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9
UNION ALL SELECT 10 UNION ALL SELECT 11 UNION ALL SELECT 12) m
LEFT JOIN transactions t ON MONTH(t.created_date) = m.month
GROUP BY m.month

SELECT * from
(
SELECT 1 as month UNION ALL
SELECT 2 as month UNION ALL
..
SELECT 12 as month
) as months
left outer join
(
select month(created_date) as "Month", count(created_date) as "Count" from transactions group by month(created_date)
) as data
on (months.month=data.month)
ORDER BY months.month

Related

MySQL query to count zero value using group by in the same table

Here's my "customers" table:
To get number of enquiries per for a particular month and year, I'm using following query:
SELECT YEAR(customer_date) AS Year, MONTH(customer_date) AS Month, COUNT(customer_id) AS Count FROM customers WHERE customer_product = 6 GROUP BY YEAR(customer_date), MONTH(customer_date)
I get following result:
You can see that as there is no enquery in the April month, so no row fetched for month number 4. But I want 0 value in Count column if there is no record found in that particular month and year.
This is what I want:
One option uses a calendar table to represent all months and years, even those which do not appear in your data set:
SELECT
t1.year,
t2.month,
COUNT(c.customer_id) AS Count
FROM
(
SELECT 2017 AS year UNION ALL
SELECT 2018
) t1
CROSS JOIN
(
SELECT 1 AS month UNION ALL
SELECT 2 UNION ALL
SELECT 3 UNION ALL
SELECT 4 UNION ALL
SELECT 5 UNION ALL
SELECT 6 UNION ALL
SELECT 7 UNION ALL
SELECT 8 UNION ALL
SELECT 9 UNION ALL
SELECT 10 UNION ALL
SELECT 11 UNION ALL
SELECT 12
) t2
LEFT JOIN customers c
ON t1.year = YEAR(c.customer_date) AND
t2.month = MONTH(c.customer_date)
WHERE
c.customer_product = 6
GROUP BY
t1.year,
t2.month
ORDER BY
t1.year,
t2.month;
Note: The above query can probably be made faster by actually creating dedicated calendar tables in your MySQL schema.
The following index on the customers table might help:
CREATE INDEX idx ON customers(customer_product, customer_id);
This might make the join between the calendar tables and customers faster, assuming that the customer_product = 6 condition is restrictive.

MySQL avg count per month for current year (include months with no data)

I am trying to create a query for a bar-chart which displays a monthly overview of the number of orders.
The query I am using is correctly providing me with the breakdown per month but when I skipped a month, it is not providing a 0 for that month, just doesnt add it at all.
Since this chart is expecting 12 numbers, everything after the missing month would be off an inaccurate.
Current Attempt:
select Month(dateCreated) as monthID,
Monthname(dateCreated) as monthName,
count(dateCreated) as totalRewards
from reward
where Year(dateCreated) = '2018'
GROUP BY monthID
If we were to assume that it is currently May 2018, I would like to see Jan - May, current counts even if the month had no orders (April = 0).
Whats the best way to include all months that have happened so far in the provided year and then their appropriate count?
You can mock a months table, then LEFT JOIN the reward table against it. To ensure you only get valid results, it's best to use a SUM() where not null rather than a COUNT() aggregate:
SELECT
months.id as monthID,
MONTHNAME(CONCAT('2018-',months.id,'-01')) as monthName,
SUM(CASE WHEN dateCreated IS NULL THEN 0 ELSE 1 END) as totalRewards
FROM
(
SELECT 1 AS id
UNION SELECT 2
UNION SELECT 3
UNION SELECT 4
UNION SELECT 5
UNION SELECT 6
UNION SELECT 7
UNION SELECT 8
UNION SELECT 9
UNION SELECT 10
UNION SELECT 11
UNION SELECT 12
) as months
LEFT JOIN reward
ON MONTH(reward.dateCreated) = months.id
AND YEAR(dateCreated) = '2018'
GROUP BY monthID, monthName
ORDER BY monthID;
SQL Fiddle

SQL trick count number of rows by months?

I have an elementary SQL query:
SELECT MONTH(created_at), COUNT(id) as total FROM `clients` GROUP BY MONTH(created_at)
It returns me data groupped by month like as:
MONTH(created_at) | total
09 1
10 2
How to fill remaining months to zero? So, in result I need to get all months:
MONTH(created_at) | total
09 1
.. 2
12 5
I tried this way:
SELECT months.id, COUNT(clients.id) as total FROM `months` LEFT JOIN `clients` ON MONTH(created_at) = months.id GROUP BY MONTH(created_at)
Use a derived table with all month numbers and left join your table on to that.
SELECT mths.mth, COUNT(c.id) as total
FROM (select 1 as mth union select 2 union select 3 union
select 4 union select 5 union select 6 union select 7 union
select 8 union select 9 union select 10 union select 11 union select 12) mths
LEFT JOIN `clients` c on mths.mth=month(c.created_at)
GROUP BY mths.mth
The best practice is to have CALENDAR table, from which you can query period you need and then left join table with data.
Or you can simply generate list of periods you need. In case you have a small period, you can use derived table. The fastest way in this case would be excel-generated list.
Thank all for answers, especially Alex, this way works:
SELECT months.id, COUNT(clients.id) as total FROM `months` LEFT JOIN `clients` ON months.id = MONTH(created_at) GROUP BY months.id;

How to find daily average over a time period in mysql?

I've a table where there's two column:
MARKS
CREAT_TS
I want to daily average marks for between two date range (e.g. startDate & endDate)
I've made the following query:
select SUM(MARKS)/ COUNT(date(CREAT_TS)) AS DAILY_AVG_MARKS,
date(CREAT_TS) AS DATE
from TABLENAME
group by date(CREAT_TS)
With this query I can get the daily average only if there's a row in the database for the date. But my requirement is that even if there's no row, I want to show 0 for that date.
I mean I want the query to return X rows if there are X days between (startDate, endDate)
Can anyone help me. :(
You need to create a set of integers that you can add to the dates. The following will give you an idea:
select thedate, avg(Marks) as DAILY_AVG_MARKS
from (select startdate+ interval num day as thedate
from (select d1.d + 10 * d2.d + 100*d3.d as num
from (select 0 as d union select 1 union select 2 union select 3 union select 4 union
select 5 union select 6 union select 7 union select 8 union select 9
) d1 cross join
(select 0 as d union select 1 union select 2 union select 3 union select 4 union
select 5 union select 6 union select 7 union select 8 union select 9
) d2 cross join
(select 0 as d union select 1 union select 2 union select 3 union select 4 union
select 5 union select 6 union select 7 union select 8 union select 9
) d3
) n cross join
(select XXX as startdate, YYY as enddate) const
where startdate + num <= enddate
) left outer join
tablename t
on date(CREAT_TS) = thedate
group by thedate
All the complication is in creating a set of sequential dates for the report. If you have a numbers table or a calendar table, then the SQL looks much simpler.
How does this work? The first big subquery has two parts. The first just generates the numbers from 0 to 999 by cross joining the digits 0-9 and doing some arithmetic. The second joins this to the two dates, startdate and enddate -- you need to put the correct values in for XXX and YYY. With this table, you have all the dates between the two values. If you need more than 999 days, just add in another cross join.
This is the left joined to your data table. The result is that all dates appear for the group by.
In terms of reporting, there are advantages and disadvantages to doing this in the presentation layer. Basically, the advantage to doing it in SQL is that the report layer is simpler. The advantage to doing it in the reporting layer is that the SQL is simpler. It is hard for an outsider to make that judgement.
My suggestion would be to create a numbers table that you can just use in reports like this. Then the query will look simpler and you won't have to change the reporting layer.

mysql skips certain months

Been trying to sort this one out for a while. I'd really appreciate any help.
I've got this table where I'm getting 2 columns with date and int values respectively. The problem is that mysql skips the date values wherever the int value is null.
Here the sql statement
SELECT DATE_FORMAT(sales_date_sold, '%b \'%y')
AS sale_date, sales_amount_sold
AS sale_amt
FROM yearly_sales
WHERE sales_date_sold BETWEEN DATE_SUB(SYSDATE(), INTERVAL 2 YEAR) AND SYSDATE()
GROUP BY YEAR(sales_date_sold), MONTH(sales_date_sold)
ORDER BY YEAR(sales_date_sold), MONTH(sales_date_sold) ASC;
There aren't any values for feb 2011 so that month gets skipped, along with a few others. Coalesce and if_null don't work too.
You need a row source that provides values for all of the months in the dimension, and then left join your yearly_sales table to that.
You are doing a GROUP BY, you most likely want an aggregate on your measure (sales_amount_sold), or you don't want a GROUP BY. (The query in your question is going to return a value from sales_amount_sold for only one row in a given month. That may be what you want, but its a very odd resultset to return.)
One approach is to have a "calendar_month" table that contains DATE values all of the months you want returned. (There are other ways to generate this, existing answers to questions elsewhere on stackoverflow)
SELECT m.month AS sale_date
, IFNULL(SUM(s.sales_amount_sold),0) AS sale_amt
FROM calendar_months m
LEFT
JOIN yearly_sales s
ON s.sales_date_sold >= m.month
AND s.sales_date_sold < DATE_ADD(m.month,INTERVAL 1 MONTH)
WHERE m.month BETWEEN DATE_SUB(SYSDATE(), INTERVAL 2 YEAR) AND SYSDATE()
GROUP BY m.month
ORDER BY m.month
This query returns a slightly different result, you are only going to get rows in groups of "whole months", rather than including partial months, as in your original query, because the WHERE clause on sale_date references two years before the current date and time, rather than the "first of the month" two years before.
A calendar_months table is not necessarily required; this could be replaced with a query that returns the row source. In that case, the predicate on the month value could be moved from the outer query into the subquery.
Addendum: if you use a calendar_month table as a rowsource, you'll need to populate it with every possible "month" value you want to return.
CREATE TABLE calendar_month
(`month` DATE NOT NULL PRIMARY KEY COMMENT 'one row for first day of each month');
INSERT INTO calendar_month(`month`) VALUES ('2011-01-01'),('2011-02-01'),('2011-03-01')
As an alternative, you can specify a dynamically generated rowsource, as an inline view, rather than a table reference. (You could use a similar query to quickly populate a calendar_months table.)
You can wrap this query in parenthesis, and paste it between FROM and calendar_months in the previous query I provided.
SELECT DATE_ADD('1990-01-01',INTERVAL 1000*thousands.digit + 100*hundreds.digit + 10*tens.digit + ones.digit MONTH) AS `month`
FROM ( SELECT 0 AS digit UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 ) ones
JOIN ( SELECT 0 AS digit UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 ) tens
JOIN ( SELECT 0 AS digit UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 ) hundreds
JOIN ( SELECT 0 AS digit UNION ALL SELECT 1 UNION ALL SELECT 2 UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5 UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8 UNION ALL SELECT 9 ) thousands
The problem is not that the value is NULL, the problem is that you are selecting data off your Database. If you don't have data for a specific month, MySQL has no way of selecting data that is not there.
The only way to solve this completely in MySQL is already answered in a very similar question
I have had this problem before with timestamps. The solution I used was to create a reference table with all of your months. This could be a table with just the numbers 1-12 (12 rows) or you could go one step further and put the month names. Then you can left join your yearly_sales table to the 1_through_12 table to get every month.
Why don't you just use 0 instead of NULL?