mysql group by year diffrent from group by year and month - mysql

I am trying to make some reports regarding our orders and the products sold, revenue and the total weight, but when im grouping our orders together I get different results in some of my queries that should return the same.
The following is my queries:
Monthly
SELECT
MONTH(orders.date_purchased) as date,
YEAR(orders.date_purchased) as year,
count(DISTINCT orders.orders_id) AS total_orders,
categories.fields_23 as currency
FROM
orders_shipping_products
Inner Join orders ON orders.orders_id = orders_shipping_products.orders_id
Inner Join categories ON categories.fields_6 = orders.shopping_store_category_id
WHERE
orders.orders_status NOT IN (0, 1, 99)
GROUP BY
date,
year,
categories.fields_23
ORDER by
YEAR(orders.date_purchased),
MONTH(orders.date_purchased) ASC,
categories.fields_23
This returns for 2012 the following table, a total of 353:
+------+------+--------------+----------+
| date | year | total_orders | currency |
+------+------+--------------+----------+
| 11 | 2012 | 86 | EUR |
| 12 | 2012 | 267 | EUR |
+------+------+--------------+----------+
Yearly
SELECT
YEAR(orders.date_purchased) as year,
count(DISTINCT orders.orders_id) AS total_orders,
categories.fields_23 as currency
FROM
orders_shipping_products
Inner Join orders ON orders.orders_id = orders_shipping_products.orders_id
Inner Join categories ON categories.fields_6 = orders.shopping_store_category_id
WHERE
orders.orders_status NOT IN (0, 1, 99)
GROUP BY
year,
categories.fields_23
ORDER by
YEAR(orders.date_purchased),
categories.fields_23
it returns for 2012 the following
+------+--------------+----------+
| year | total_orders | currency |
+------+--------------+----------+
| 2012 | 351 | EUR |
+------+--------------+----------+
The only thing that changes is the total_orders, total amount of products, the revenue and weight is the same. I just get two more orders when checking per month. I also checked with a selection and grouping by QUARTER, YEAR that returns the following:
+---------+------+--------------+----------+
| quarter | year | total_orders | currency |
+---------+------+--------------+----------+
| 4 | 2012 | 351 | EUR |
+---------+------+--------------+----------+
This make me thinks, that i might be doing something wrong with my selections when i want to generate a report per month
WITH ROLLUP
Daniel asked me to try add WITH ROLLUP on my query, follow is what i did and got in return
SELECT
MONTH(orders.date_purchased) as date,
YEAR(orders.date_purchased) as year,
count(DISTINCT orders.orders_id) AS total_orders,
categories.fields_23 as currency
FROM
orders_shipping_products
Inner Join orders ON orders.orders_id = orders_shipping_products.orders_id
Inner Join categories ON categories.fields_6 = orders.shopping_store_category_id
WHERE
orders.orders_status NOT IN (0, 1, 99)
GROUP BY
year,
date,
categories.fields_23 WITH ROLLUP
This is the returned data:
+------+------+--------------+----------+
| date | year | total_orders | currency |
+------+------+--------------+----------+
| 11 | 2012 | 86 | EUR |
| 11 | 2012 | 86 | NULL |
| 12 | 2012 | 267 | EUR |
| 12 | 2012 | 267 | NULL |
| NULL | 2012 | 351 | NULL |
+------+------+--------------+----------+

I rechecked orders_id and it where not unique, in fact there where 2-3 the same orders id and some of them where located in different months. Found that a combination of orders_id and another column made it up for the real unique id. adding this to my join, solved everything.

Related

MySQL : calculate total and percentage compared to previous year

I have this table structure
+------------+--------+
| date | amount |
+------------+--------+
| 2020-05-01 | 10 |
| 2020-10-01 | 15 |
| 2021-03-01 | 9 |
| 2021-09-01 | 10 |
| 2021-12-01 | 15 |
| 2022-06-01 | 19 |
| 2022-08-01 | 25 |
| 2022-09-01 | 13 |
+---------------------+
I would like to calculate the total for each year, and also the percentage from the previous year
SELECT YEAR(p.date) AS year, SUM(p.amount) AS year_total, SUM(p1.amount) AS prev_year, ROUND(SUM(p1.amount)/SUM(p.amount)*100) AS percentage
FROM payments p
LEFT JOIN payments p1
ON YEAR(p1.date) = YEAR(p.date)-1
GROUP BY year
ORDER BY year DESC;
But with this query the result goes crazy... The totals are not correct except for the first year.
+------+------------+-----------+------------+
| year | year_total | prev_year | percentage |
+------+------------+-----------+------------+
| 2022 | 171 | 102 | 60 |
| 2021 | 68 | 75 | 110 |
| 2020 | 25 | NULL | NULL |
+------+------------+-----------+------------+
I guess I have a group by issue but I can't find a solution.
Edit: forgot to mention I am using MariaDB 10
Your query joins each record of one year with each record of the previous year. Afterwards the sum grouped by year is taken. The simplest solution is to use a window function like LAG as Tim Biegeleisen did. If yoh have to use an older version of MySQL, you'll have to calculate the sums before joining.
Sketch:
SELECT c.year, c.total, p.total
FROM (SELECT year(date) AS year, sum(amount) AS total FROM payments GROUP BY year(date)) c
LEFT JOIN (SELECT year(date) AS year, sum(amount) AS total FROM payments GROUP BY year(date)) p
ON p.year = c.year - 1
On MySQL 8+, we can make use of LAG() here:
SELECT YEAR(date) AS year, SUM(amount) AS year_total,
LAG(SUM(amount)) OVER (ORDER BY YEAR(date)) AS prev_year,
100.0 * LAG(SUM(amount)) OVER (ORDER BY YEAR(date)) / SUM(amount) AS percentage
FROM payments
GROUP BY YEAR(date)
ORDER BY YEAR(date) DESC;

How to join two tables with average function and where clause? SQL

I have two tables below with the following information
project.analytics
| proj_id | list_date | state
| 1 | 03/05/10 | CA
| 2 | 04/05/10 | WA
| 3 | 03/05/10 | WA
| 4 | 04/05/10 | CA
| 5 | 03/05/10 | WA
| 6 | 04/05/10 | CA
employees.analytics
| employee_id | proj_id | worked_date
| 20 | 1 | 3/12/10
| 30 | 1 | 3/11/10
| 40 | 2 | 4/15/10
| 50 | 3 | 3/16/10
| 60 | 3 | 3/17/10
| 70 | 4 | 4/18/10
What query can I write to determine the average number of unique employees who have worked on the project in the first 7 days that it was listed by month and state?
Desired output:
| list_date | state | # Unique Employees of projects first 7 day list
| March | CA | 1
| April | WA | 2
| July | WA | 2
| August | CA | 1
My Attempt
select
month(list_date),
state_name,
count(*) as Projects,
from projects
group by
month(list_date),
state_name;
I understand the next steps are to subtract the worked_date - list_date and if value is <7 then average count of employees from the 2nd table but I'm not sure what query functions to use.
You could use a CASE with a DISTINCT to COUNT the unique employees that worked within the first 7 days of the list_date.
Once you have that total of employees per project, then you can calculate those averages per month & state.
SELECT
MONTHNAME(list_date) as `ListMonth`,
state,
AVG(TotalUniqEmp7Days) AS `Average Unique Employees of projects first 7 day list`
FROM
(
SELECT
proj.proj_id,
proj.list_date,
proj.state,
COUNT(DISTINCT CASE
WHEN emp.worked_date BETWEEN proj.list_date and DATE_ADD(proj.list_date, INTERVAL 6 DAY)
THEN emp.employee_id
END) AS TotalUniqEmp7Days
-- , COUNT(DISTINCT emp.employee_id) AS TotalUniqEmp
FROM project.analytics proj
LEFT JOIN employees.analytics emp ON emp.proj_id = proj.proj_id
GROUP BY proj.proj_id, proj.list_date, proj.state
) AS ProjectTotals
GROUP BY YEAR(list_date), MONTH(list_date), MONTHNAME(list_date), state;
A Sql Fiddle test can be found here
I think this is the code that you want
select
p.list_date, p.state,
emp.no_of_unique_emp
from project.analytics p
inner join (
select
t.project_id,
count(t.employee_id) as no_of_unique_emp
from (
select distinct employee_id, project_id
from employees.analytics
) t
group by t.project_id
) emp
on emp.project_id = p.project_id
where datediff (p.list_date, getdate()) <= 7

MySQL perform subquery if count is zero

I have three tables: monthly_revenue, currencies and foreign_exchange.
monthly_revenue table
|------------------------------------------------------|
| id | product_id | currency_id | value | month | year |
|------------------------------------------------------|
| 1 | 1 | 1 | 100 | 1 | 2015 |
| 2 | 1 | 2 | 125 | 1 | 2015 |
| 3 | 1 | 3 | 115 | 1 | 2015 |
| 4 | 1 | 1 | 100 | 2 | 2015 |
| 5 | 1 | 2 | 125 | 2 | 2015 |
| 6 | 1 | 3 | 115 | 2 | 2015 |
|------------------------------------------------------|
foreign_exchange table
|---------------------------------------|
| id | base | target | rate | rate_date |
|---------------------------------------|
| 1 | GBP | USD | 1.6 |2015-01-01 |
| 2 | GBP | USD | 1.62 |2015-01-15 |
| 3 | GBP | USD | 1.61 |2015-01-31 |
| 4 | EUR | USD | 1.2 |2015-01-01 |
| 5 | EUR | USD | 1.4 |2015-01-15 |
| 6 | EUR | USD | 1.4 |2015-01-31 |
| 7 | GBP | EUR | 1.4 |2015-01-01 |
| 8 | GBP | EUR | 1.45 |2015-01-15 |
| 9 | GBP | EUR | 1.44 |2015-01-31 |
|---------------------------------------|
From this, we can see the average fx rates:
GBP > USD in January is 1.61
EUR > USD in January is 1.33
GBP > EUR in January is 1.43
No rates are available for USD as a base currency, and no rates are available for February.
currencies table
|-----------|
| id | name |
|-----------|
| 1 | GBP |
| 2 | USD |
| 3 | EUR |
|-----------|
What i'm trying to achieve
Each row within the monthly_revenue table can have a different currency_id, as orders are placed is different currencies. I want to see all revenue for a given month, in a common currency. So, rather than looking at all revenue in January in GBP, and then separately looking at all revenue in January in USD, I'd like to get one value for all revenue in January - converted to USD (for example).
This can be calculated for each row, using the following (using January for this example):
revenue value x average fx rate for January between base and target currency
If I have 50 orders in January, in 4 different currencies, this let's me see all revenue in any single currency.
Example - get all revenue in January, in USD
This should return:
|------------------------------------------------------|
| id | product_id | currency_id | value | month | year |
|------------------------------------------------------|
| 1 | 1 | 1 | 100 | 1 | 2015 |
| 2 | 1 | 2 | 125 | 1 | 2015 |
| 3 | 1 | 3 | 115 | 1 | 2015 |
|------------------------------------------------------|
However, rows 1 and 3 are not in USD (these are GBP, and EUR respectively).
What I'd like to see is each row returned with the average FX rate that is being converted to, and a converted column. For example:
|-------------------------------------------------------------------------|
| id | prod_id | currency_id | value | month | year | fx_avg | converted |
|-------------------------------------------------------------------------|
| 1 | 1 | 1 | 100 | 1 | 2015 | 1.61 | 161 |
| 2 | 1 | 2 | 125 | 1 | 2015 | 1 | 125 |
| 3 | 1 | 3 | 115 | 1 | 2015 | 1.33 | 152.95 |
|-------------------------------------------------------------------------|
Where I'm at
I can currently get the basic calculation done using the query below, but a couple of key features are lacking:
If there is no FX rate available (for example for future dates where of course an FX rate isn't available) then the entire row is ignored. What I'd like in this instance is for the latest month's average to be used.
If the calculation is being performed where the target currency is the same as the base currency, the entire row is ignored (as there is no record in the FX table where the base equals the target). In this instance, the rate should be hard defined as 1.
Query so far
SELECT
r.value * IFNULL(AVG(fx.rate),1) as converted, AVG(fx.rate) as averageFx,
r.*, fx.*
FROM
foreign_exchange fx, monthly_revenue r, order_headers h
WHERE
fx.base IN (SELECT name FROM currencies WHERE id = r.currency_id) AND
r.order_header_id = h.id AND
fx.target = 'USD' AND
MONTH(fx.rate_date) = r.month AND
YEAR(fx.rate_date) = r.year AND
r.year = 2015
GROUP BY r.id
ORDER BY month ASC
If there are no records available for FX, it looks like a separate subquery should be performed to get the average of the latest month's rates.
Any input would be appreciated. If any further info is required, please post a comment.
Thanks.
Edit Here is a SQFiddle which has the example schemas and the code which highlights the issue.
Here is an approximation of a function that computes your exchange for a given currency and start of month:
DELIMITER //
CREATE FUNCTION MonthRate(IN _curr CHAR(3) CHARACTER SET ascii,
IN _date DATE)
RETURNS FLOAT
DETERMINISTIC
BEGIN
-- Note: _date must be the first of some month, such as '2015-02-01'
DECLARE _avg FLOAT;
DECLARE _prev FLOAT;
-- First, try to get the average for the month:
SELECT AVG(rate) INTO _avg FROM foreign_exchange
WHERE base = _curr
AND target = 'USD'
AND rate_date >= _date
AND rate_date < _date + INTERVAL 1 MONTH;
IF _avg IS NOT NULL THEN
RETURN _avg;
END;
-- Fall back onto the last rate before the month:
SELECT rate INTO _prev
FROM foreign_exchange
WHERE base = _curr
AND target = 'USD'
AND rate_date < _date
ORDER BY _date
LIMIT 1;
IF _prev IS NOT NULL THEN
RETURN _prev;
END;
SELECT "Could not get value -- ran off start of Rates table";
END;
DELIMITER ;
There are probably syntax errors, etc. But hopefully you can work with it.
It should be easy to call the function from the rest of the code.
For performance, this would be beneficial:
INDEX(base, target, rate_date, rate)
Create a view :
create view avg_rate as
select base, target, year(fx.rate_date) year, month(fx.rate_date) month,
avg(rate) avg_rate
from foreign_exchange group by base, target
Join it twice, once for current month, and once for previous
select r.id, r.month,
r.value * avg(coalesce(cr.avg_rate, pr.avg_rate, 1)) converted,
avg(coalesce(cr.avg_rate, pr.avg_rate), 0) rate
from monthly_revenue r, avg_rate cr, avg_rate pr, order_headers h
where
r.year = 2015 and
cr.year = r.year and cr.month = r.month and cr.target='USD' and
pr.year = r.year and pr.month = r.month - 1 and pr.target='USD' and
r.order_header_id = h.id
group by r.id
order by r.month
Also I personally don't like this way of writing query and prefer to using explicit joins as you group conditions logically and don't have a mess in where clause. i.e.:
...
from monthly_revenue r
inner join order_headers h on r.order_header_id = h.id
left join avg_rate cr on cr.year = r.year and cr.month = r.month and cr.target='USD'
left join avg_rate pr on pr.year = r.year and pr.month = r.month - 1 and pr.target='USD'
where r.year = 2015
http://sqlfiddle.com/#!9/6a41a/1
This fiddle based on your original one but I added some rate values for February and March to test and show how it works.
SELECT t.*,
IF(#first=t.id, #flag := #flag+1,#flag:=1) `flag`,
#first:=t.id
FROM
(SELECT
coalesce(fx.rate,1) `rate`, (r.value * coalesce(fx.rate,1)) as converted,
r.*, fx.base,fx.target, fx.avg_date, fx.rate frate
FROM
monthly_revenue r
LEFT JOIN
currencies
ON r.currency_id = currencies.id
LEFT JOIN
(SELECT AVG(rate) `rate`,
`base`,
`target`,
STR_TO_DATE(CONCAT('1/',MONTH(rate_date),'/',YEAR(rate_date)), '%d/%m/%Y') avg_date
FROM foreign_exchange
GROUP BY `base`, `target`, `avg_date`
) fx
ON currencies.name = fx.base
AND fx.target = 'USD'
AND fx.avg_date <= STR_TO_DATE(CONCAT('1/',r.month,'/',r.year), '%d/%m/%Y')
ORDER BY r.id, fx.avg_date DESC) t
HAVING `flag` = 1
and if you need records just for specific month you can add WHERE before ORDER like this:
WHERE r.month = 1 and r.year = 2015
ORDER BY r.id, fx.avg_date DESC) t
You may test this query on the fiddle link you provided : http://sqlfiddle.com/#!9/33def/2
select id,product_id,currency_id,currency_name,
value,month,year,#prev_fx_avg:=ifnull(fx_avg,#prev_fx_avg) fx_avg,
value*#prev_fx_avg as converted
from (SELECT
r.id,r.product_id,r.currency_id,c.name as currency_name,
r.value,r.month,r.year,if(c.name="USD",1,temp.avg_rate) as fx_avg
FROM
monthly_revenue r
left join currencies c on r.currency_id=c.id
left join
(select base , avg(rate) as avg_rate, MONTH(fx.rate_date) month,
YEAR(fx.rate_date) year
from foreign_exchange fx
where target="USD"
group by base,target,MONTH(fx.rate_date),
YEAR(fx.rate_date)) temp on(r.month=temp.month and r.year=temp.year and c.name=temp.base)
group by r.id
order by r.currency_id,r.month ASC, r.year ASC) final,(select #prev_fx_avg:=-1) temp2;

Using MySQL group by clause with where clause

I have two tables, one that store product information and one that stores reviews for the products.
I am now trying to get the number of reviews submitted for the products between two dates but for some reason I get the same results regardless of the dates i put.
This is my query:
SELECT
productName,
COUNT(*) as `count`,
avg(rating) as `rating`
FROM `Reviews`
LEFT JOIN `Products` using(`productID`)
WHERE `date` BETWEEN '2015-07-20' AND '2015-07-30'
GROUP BY
`productName`
ORDER BY `count` DESC, `rating` DESC;
This returns:
+------------+---------------------+
| productName| count|rating |
+------------+------+--------------+
| productA | 23 | 4.3333333 |
| productB | 17 | 4.25 |
| productC | 10 | 3.5 |
+------------+---------------------+
Products table:
+---------+-------------+
|productID | productName|
+---------+-------------+
| 1 | productA |
| 2 | productB |
| 3 | productC |
+---------+-------------+
Reviews table
+---------+-----------+--------+---------------------+
|reviewID | productID | rating | date |
+---------+-----------+--------+---------------------+
| 1 | 1 | 4.5 | 2015-07-27 17:47:01|
| 2 | 1 | 3.5 | 2015-07-27 18:54:22|
| 3 | 3 | 2 | 2015-07-28 13:28:37|
| 4 | 1 | 5 | 2015-07-28 18:33:14|
| 5 | 2 | 1.5 | 2015-07-29 11:58:17|
| 6 | 2 | 3.5 | 2015-07-30 15:04:25|
| 7 | 2 | 2.5 | 2015-07-30 18:11:11|
| 8 | 1 | 3 | 2015-07-30 18:26:23|
| 9 | 1 | 3 | 2015-07-30 21:35:05|
| 10 | 1 | 4.5 | 2015-07-31 14:25:47|
| 11 | 3 | 0.5 | 2015-07-31 14:47:48|
+---------+-----------+--------+---------------------+
when I put two random dates that I do know for sure they not on the date column, I will still get the same results. Even when I want to retrieve records only on a certain day, I get the same results.
You should not use left join, because by doing so you retrieve all the data from one table. What you should use is something like :
select
productName,
count(*) as `count`,
avg(rating) as `rating`
from
products p,
reviews r
where
p.productID = r.productID
and `date` between '2015-07-20' and '2015-07-30'
group by productName
order by count desc, rating desc;
If the result, given your sample data, that you're looking for is:
| productName | count | rating |
|-------------|-------|--------|
| productA | 5 | 4 |
| productB | 3 | 3 |
| productC | 1 | 2 |
This is the count and average of reviews made on any date between 2015-07-20 and 2015-07-30 inclusive.
Then the there are two issues with your query. First, you need to change the join to a inner join instead of a left join, but more importantly you need to change the date condition as you are currently excluding reviews that fall on the last date on the range, but after midnight.
This happens because your between clause compares datetime values with date values so the comparison ends up being date between '2015-07-20 00:00:00' and '2015-07-30 00:00:00' which clearly excludes some dates at the end.
The fix is to either change the date condition so that the end is a day later:
where date >= '2015-07-20' and date < '2015-07-31'
or cast the date column to a date value, which will remove the time part:
where date(date) between '2015-07-20' and '2015-07-30'
Sample SQL Fiddle
You are using a LEFT JOIN between your reviews and your products tables. This will result in all the rows of reviews being shown with some rows having all product columns left empty.
You should use INNER JOIN, as this will filter only the wanted results.
(In the end I can only guess, since I don't even know which column belongs to which table ...)
The full query (very similar to Angelo Giannis's solution):
select
productName,
count(*) as `count`,
avg(rating) as `rating`
from
products INNER JOIN reviews USING(productId)
where date between '2015-07-20' and '2015-07-30'
group by productName
order by count desc, rating desc;
Here a fiddle with my and Angelo's solution (they both work).

Sort sql results by Month with unix

here's what im trying to achive:
i have 2 sql tables:
transactions and payplans
bellow is the structures of 2 tables:
transactions
uid | plan | date | payid | status
------------------------------------
12 | 3 | 1388534400 | 334 | 1
699 | 4 | 1388214400 | 335 | 1
payplans:
plan | plan_price
-------------------
3 | 9.99
4 | 19.99
with this query:
SELECT SUM(plan_price)
FROM transations AS t
INNER JOIN payplans AS p
ON t.plan = p.plan
WHERE t.status = '1'
i was able to calculate total sum of all "plan_price" rows,
but i would like to have the price sum for every month starting jan 2013
for example:
jan-13 | 9.99
feb-13 | 29.99
etc.
For MySQL
SELECT date_format(FROM_UNIXTIME(t.date), '%b-%y') as mnth,
SUM(plan_price)
FROM transations AS t
INNER JOIN payplans AS p
ON t.plan = p.plan
WHERE t.status = '1'
GROUP BY mnth;
SQLFiddle
You converting unix_timestamp to date using FROM_UNIXTIME
formatting it into 'MON-YY' format with DATE_FORMAT
then grouping by month.