How can these SQL-queries to extract statistics from my database be combined for better performance?
$total= mysql_query("SELECT COUNT(*) as number, SUM(order_total) as sum FROM history");
$month = mysql_query("SELECT COUNT(*) as number, SUM(order_total) as sum FROM history WHERE date >= UNIX_TIMESTAMP(DATE_ADD(CURDATE(),INTERVAL -30 DAY))");
$day = mysql_query("SELECT COUNT(*) as number, SUM(order_total) as sum FROM history WHERE date >= UNIX_TIMESTAMP(CURDATE())");
If you want to all the data in a single query, you have two choices:
Use a UNION query (as sugested by bishop in his answer)
Tweak a query to get what you need in a single row
I'll show option 2 (option 1 has been already covered).
Note: I'm using user variables (that stuff in the init subquery) to avoid writing the expressions again and again. Also, to filter the aggregate data, I'm using case ... end expressions.
select
-- Your first query:
count(*) as number, sum(order_total) as `sum`
-- Your second query:
, sum(case when `date` <= #prev_date then 1 else 0 end) as number_prev
, sum(case when `date` <= #prev_date then order_total else 0 end) as sum_prev
-- Your third query:
, sum(case when `date` <= #cur_date then 1 else 0 end) as number_cur
, sum(case when `date` <= #cur_date then order_total else 0 end) as sum_cur
from (
select #cur_date := unix_timestamp(curdate())
, #prev_date := unix_timestamp(date_add(curdate(), interval -30 day))
) as init
, history;
Hope this helps
Since the queries have the same column structure, you can ask MySQL to combine them with the UNION operation:
(SELECT 'total' AS kind, COUNT(*) as number, SUM(order_total) as sum FROM history~
UNION
(SELECT 'by-month' AS kind, COUNT(*) as number, SUM(order_total) as sum FROM history WHERE date <= UNIX_TIMESTAMP(DATE_ADD(CURDATE(),INTERVAL -30 DAY)))
UNION
(SELECT 'by-day' AS kind, COUNT(*) as number, SUM(order_total) as sum FROM history WHERE date <= UNIX_TIMESTAMP(CURDATE()))
Related
I'm new to SQL and trying to create a total summary of a working SQL query. It's listing the total results from one month of data.
Now I need the total values of the outcome of the query.
So I created a 'query in a query' piece of SQL, but it ain't working because my lack of SQL knowledge. I guess it's an easy fix for you pro's :-)
The working SQL query with the daily outcome of one month:
SELECT
DATE_FORMAT(date, '%d/%m/%y') AS Datum,
COUNT(*) AS Berichten,
SUM(CASE WHEN virusinfected>0 THEN 1 ELSE 0 END) AS Virus,
SUM(CASE WHEN (virusinfected=0 OR virusinfected IS NULL) AND isspam>0 THEN 1 ELSE 0 END) AS Ongewenst,
SUM(CASE WHEN (virusinfected=0 OR virusinfected IS NULL) AND (isspam=1) AND isrblspam>0 THEN 1 ELSE 0 END) AS RBL,
SUM(size) AS Grootte
FROM
maillog
WHERE
1=1
AND (1=1)
AND
date < '2017-04-01'
AND
date >= '2017-03-01'
AND
to_domain = 'domain1.nl'
OR
date < '2017-04-01'
AND
date >= '2017-03-01'
AND
to_domain = 'domain2.nl'
GROUP BY
Datum
ORDER BY
date
The incorrect query trying to create the monthly totals:
SELECT Datum,
SUM(Berichten) AS Berichten,
SUM(Virus) AS Virus,
SUM(Ongewenst) AS Ongewenst,
SUM(RBL) AS RBL,
SUM(Grootte) AS Grootte,
FROM ( SELECT
DATE_FORMAT(date, '%d/%m/%y') AS Datum,
COUNT(*) AS Berichten,
SUM(CASE WHEN virusinfected>0 THEN 1 ELSE 0 END) AS Virus,
SUM(CASE WHEN (virusinfected=0 OR virusinfected IS NULL) AND isspam>0 THEN 1 ELSE 0 END) AS Ongewenst,
SUM(CASE WHEN (virusinfected=0 OR virusinfected IS NULL) AND (isspam=1) AND isrblspam>0 THEN 1 ELSE 0 END) AS RBL,
SUM(size) AS Grootte
FROM
maillog
WHERE
1=1
AND (1=1)
AND
date < '2017-04-01'
AND
date >= '2017-03-01'
AND
to_domain = 'domain1.nl'
OR
date < '2017-04-01'
AND
date >= '2017-03-01'
AND
to_domain = 'domain2.nl'
GROUP BY
Datum
ORDER BY
date
) t
GROUP BY Datum;
Thanks in advance.
What you want can be done with just a little addition to your first SQL statement: add with rollup after the group by clause:
GROUP BY Datum WITH ROLLUP
It will run more efficiently than the version with sub-query, although it could work that way, but you should then remove the outer group by clause and not select Datum there, since you don't want the totals per date any more, but overall.
Still, you will lose the details and only get the overall totals then. You would have to use a union with your original query to get both levels of totals. You can imagine that the with rollup modifier will do the job more efficiently.
I have sales table and have two different date ranges.
i.e, I have total sales between (2016-12-21 - 2016-12-30) is 100 and for period (2016-12-11 - 2016-12-20) is 85.
Now the result I want is
100 (sales of 2016-12-21 - 2016-12-30), 85 (sales of 2016-12-11 - 2016-12-20), 15 (difference of both periods) through single query.
What I am thinking is
select *, (a.sales - b.sales) as diff
from (select id, sum(sales) as sales from salestable where date >= '2016-12-21' and date <= '2016-12-30') a
join (select id, sum(sales) as sales from salestable where date >= '2016-12-11' and date <= '2016-12-20') b
on a.id = b.id;
Is there any other better way to do this?
You can use conditional aggregation:
select sum(case when date >= '2016-12-21' and date <= '2016-12-30' then sales else 0
end) as sales_a,
sum(case when date >= '2016-12-11' and date <= '2016-12-20' then sales else 0
end) as sales_b,
sum(case when date >= '2016-12-21' and date <= '2016-12-30'
then sales else 0
when date >= '2016-12-11' and date <= '2016-12-20'
then -sales
else 0
end) as sales_diff
from salestable;
If you want the overall sum by id (as suggested by your inclusion of id), then add id to the select and add group by id.
You can use case to do a conditional sum like this:
select id,
sum_21_to_30,
sum_11_to_20,
sum_21_to_30 - sum_11_to_20 diff
from (select id,
sum(case when date >= '2016-12-21' and date <= '2016-12-30' then sales else 0 end) sum_21_to_30,
sum(case when date >= '2016-12-11' and date <= '2016-12-20' then sales else 0 end) sum_11_to_20
from table group by id) t;
I am trying to get the total sum of a column and the sum of the same column between 2 dates in one query. is this possible?
My table looks like this:
uid|amount|date
The two queries i am trying to make one of:
SELECT sum(amount) as `keys` FROM tbl_keys WHERE uid = 1
SELECT sum(amount) as `keys` FROM tbl_keys WHERE uid = 1 AND YEAR(`date`) = YEAR(CURRENT_DATE)
AND MONTH(`date`) = MONTH(CURRENT_DATE)
You could use a UNION query:
SELECT 'All' AS cnt, sum(amount) as `keys` FROM tbl_keys WHERE uid = 1
UNION ALL
SELECT 'Current_month' AS cnt, sum(amount) as `keys`
FROM tbl_keys
WHERE
uid = 1
AND `date`<= last_day(current_date)
`date`>= current_date - interval (day(current_date)-1) day
(I prefer to use >= and <= on the date column, as it can make use of an index if present, while functions like MONTH() or YEAR() cannot, also I assume that date is a date columnd and that it doesn't contain time informations).
If you want the result in one row, you could use an inline query:
SELECT
(SELECT sum(amount) as `keys` FROM tbl_keys WHERE uid = 1) AS total,
(SELECT sum(amount) as `keys`
FROM tbl_keys
WHERE
uid = 1
AND `date`<= last_day(current_date)
`date`>= current_date - interval (day(current_date)-1) day
) AS current_month
Something like this:
SELECT sum(amount) as `keys`,
(
SELECT sum(t.amount)
FROM tbl_keys as t
WHERE t.uid = tbl_keys.uid AND YEAR(t.`date`) = YEAR(CURRENT_DATE)
AND MONTH(t.`date`) = MONTH(CURRENT_DATE)
) as `keys2`
FROM tbl_keys
WHERE uid = 1
SELECT sum(amount) AS `keys`
FROM (
SELECT amount FROM tbl_keys
UNION ALL
SELECT amount FROM tbl_keys
WHERE uid = 1
AND YEAR(`date`) = YEAR(CURRENT_DATE)
AND MONTH(`date`) = MONTH(CURRENT_DATE)
) AS new_table;
Using a UNION clause, you will get the desired output you want.
Use CASE to count only the amount for the specified date:
SELECT SUM(amount) AS `keys`,
SUM(CASE WHEN YEAR(`date`) = YEAR(CURRENT_DATE)
AND MONTH(`date`) = MONTH(CURRENT_DATE) THEN amount ELSE 0 END) AS 'keys2'
FROM tbl_keys
WHERE uid = 1
;
My guess is that this will run more efficient than a solution using UNION SELECT.
Hi all i execute this query to get a table where there's statistics of some database information.. i'd like to intialise the fields that don't exist ( because the query is executed in different dates and sometimes there's a day where there's nothing ) so i'd like it to return 0 and NULL ( in TOP column )
SELECT
SUM(IF(`TOP` = 'one',`Nb`,0)) as first_one,
SUM(IF(`TOP` = 'two',`Nb`,0)) as second_one,
SUM(IF(`TOP` = 'three',`Nb`,0)) as thrid_one,
SUM(IF(`TOP` NOT IN ('three','two','one'),`Nb`,0)) as forth_one,
GROUP_CONCAT(IF(`TOP` NOT IN ('three','two','one'),`TOP`,'') SEPARATOR '') as `OR`
FROM (
SELECT
COUNT(*) as Nb,
'one' as `TOP`
FROM
mytable
WHERE
TYPE = 'MSS'
AND YEAR(date) = YEAR(CURDATE())
AND MONTH(date) = MONTH(CURDATE())
UNION ALL
SELECT
COUNT(*) as Nb,
'two' as `TOP`
FROM
mytable
WHERE
TYPE = 'MSS'
AND S=0
AND YEAR(date) = YEAR(CURDATE())
AND MONTH(date) = MONTH(CURDATE())
UNION ALL
SELECT
COUNT(*) as Nb,
'three' as `TOP`
FROM
mytable
WHERE
TYPE = 'MSS'
AND S<>0
AND YEAR(date) = YEAR(CURDATE())
AND MONTH(date) = MONTH(CURDATE())
UNION ALL
SELECT
`Nb`,
`TOP`
FROM(
SELECT
COUNT(*) as Nb ,
`OR` as `TOP`
FROM
mytable
WHERE
TYPE = 'MSS'
AND YEAR(date) = YEAR(CURDATE())
AND MONTH(date) = MONTH(CURDATE())
GROUP BY
`OR`
ORDER BY
Nb DESC
LIMIT 1
) as tmp
)as tmp1
Assuming that in tmp1 you have data you need but with "gaps" (days when there were no data at all) you could RIGHT JOIN tmp1 to table tmp2 using day (I assume that you have such column in tmp1 table). So tmp2 would be just list of days:
SELECT '2013-05-17' as day UNION SELECT '2013-05-18' UNION SELECT ...
I could elaborate my answer if you'd like to provide your DB schema.
You can replace each subquery with:
SELECT
IFNULL(tmp.Nb,0) as Nb,
IFNULL(tmp.`TOP`, 'value') as `TOP`
FROM (
--subquery
) as tmp
Example for the first subquery:
SELECT
IFNULL(tmp.Nb,0) as Nb,
IFNULL(tmp.`TOP`, 'one') as `TOP`
FROM (
SELECT
COUNT(*) as Nb,
'one' as `TOP`
FROM
mytable
WHERE
TYPE = 'MSS'
AND YEAR(date) = YEAR(CURDATE())
AND MONTH(date) = MONTH(CURDATE())
) as tmp
SQL is good at grouping existing entities into categories, but bad at "creating" entities itself. I would advise either a generic number table (really just the numbers from 0 to a few hundredthousand) if you have also non-date categories or as Wiktor suggested a date-Table which gets filled every now and then and has the next few years as well as the time since your program is working.
With a date table
list_dates (
id int(11) not null primary key auto_increment,
dateval date not null
)
you could start your queries from that table (with a reasonable range, of course) and count every thing else:
select list_dates.dateval as date, count(*) as cnt
from list_dates
left join actions on actions.actiontime >= (cast list_dates.date_val as datetime)
and actions.actiontime < (cast list_dates.date_val `interval 1 day as datetime)
where list_dates.dateval between '$fromDate' and '$toDate'
group by list_dates.dateval
;
or starting with a number table numbers
select $fromDate + interval numbers.number day as date, count(*) as cnt
from numbers
left join actions
on actions.actiontime >= (cast $fromDate + interval numbers.number day as datetime)
and actions.actiontime < (cast $fromDate + interval (1 + numbers.number) day as datetime)
where numbers.number >= 0 and numbers.number < $countDates
group by numbers.number
;
One Day
If you really want just that one day (today) then you can of course use a anonymous subselect- Table instead, so it becomes
select list_dates.dateval as date, count(*) as cnt
from ( select curdate() as dateval ) as list_dates
left join actions on actions.actiontime >= (cast list_dates.date_val as datetime)
and actions.actiontime < (cast list_dates.date_val `interval 1 day as datetime)
where list_dates.dateval between '$fromDate' and '$toDate'
group by list_dates.dateval
;
I have this quite long query that should give me some information about shipments, and it works, but it's performing terribly bad. It takes about 4500ms to load.
SELECT
DATE(paid_at) AS day,
COUNT(*) as order_count,
(
SELECT COUNT(*) FROM line_items
WHERE order_id IN (SELECT id from orders WHERE DATE(paid_at) = day)
) as product_count,
(
SELECT COUNT(*) FROM orders
WHERE shipping_method = 'colissimo'
AND DATE(paid_at) = day
AND state IN ('paid','shipped','completed')
) as orders_co,
(
SELECT COUNT(*) FROM orders
WHERE shipping_method = 'colissimo'
AND DATE(paid_at) = day
AND state IN ('paid','shipped','completed')
AND paid_amount < 70
) as co_less_70,
(
SELECT COUNT(*) FROM orders
WHERE shipping_method = 'colissimo'
AND DATE(paid_at) = day
AND state IN ('paid','shipped','completed')
AND paid_amount >= 70
) as co_plus_70,
(
SELECT COUNT(*) FROM orders
WHERE shipping_method = 'mondial_relais'
AND DATE(paid_at) = day
AND state IN ('paid','shipped','completed')
) as orders_mr,
(
SELECT COUNT(*) FROM orders
WHERE shipping_method = 'mondial_relais'
AND DATE(paid_at) = day
AND state IN ('paid','shipped','completed')
AND paid_amount < 70
) as mr_less_70,
(
SELECT COUNT(*) FROM orders
WHERE shipping_method = 'mondial_relais'
AND DATE(paid_at) = day
AND state IN ('paid','shipped','completed')
AND paid_amount >= 70
) as mr_plus_70
FROM orders
WHERE MONTH(paid_at) = 11
AND YEAR(paid_at) = 2011
AND state IN ('paid','shipped','completed')
GROUP BY day;
Any idea what I could be doing wrong or what I could be doing better? I have other queries of similar length that don't take as much time to load as this. I thought this would be faster than for example having an individual query for each day (in my programming instead of the SQL query).
It is because you are using sub-queries where you don't need them.
As a general rule, where you have a sub-query within a main SELECT clause, that sub-query will query the tables within it once for each row in the main SELECT clause - so if you have 7 subqueries and are selecting a date range of 30 days, you will effectively be running 210 separate subqueries (plus your main query).
(Some query optimisers can resolve sub-queries into the main query under some circumstances, but as a general rule you can't rely on this.)
In this case, you don't need any of the orders sub-queries, because all the orders data you require is included in the main query - so you can rewrite this as:
SELECT
DATE(paid_at) AS day,
COUNT(*) as order_count,
(
SELECT COUNT(*) FROM line_items
WHERE order_id IN (SELECT id from orders WHERE DATE(paid_at) = day)
) as product_count,
sum(case when shipping_method = 'colissimo' then 1 end) as orders_co,
sum(case when shipping_method = 'colissimo' AND
paid_amount < 70 then 1 end) as co_less_70,
sum(case when shipping_method = 'colissimo' AND
paid_amount >= 70 then 1 end) as co_plus_70,
sum(case when shipping_method = 'mondial_relais' then 1 end) as orders_mr,
sum(case when shipping_method = 'mondial_relais' AND
paid_amount < 70 then 1 end) as mr_less_70,
sum(case when shipping_method = 'mondial_relais' AND
paid_amount >= 70 then 1 end) as mr_plus_70
FROM orders
WHERE MONTH(paid_at) = 11
AND YEAR(paid_at) = 2011
AND state IN ('paid','shipped','completed')
GROUP BY day;
The problem in your query is that scans the same table over and over. All scans (selects in your case) of ORDER table can be transformed to multiple SUM+CASE or COUNT+CASE as in SQL query with count and case statement.