where clause sql and display NULL values - mysql

I have a database with information about several servers :
mysql> select DATE_FORMAT(timestamp,'%M %Y') as 'Mois',AVG(value_perf) as 'moy' from info WHERE server = 'pi4' GROUP BY MONTH(timestamp);
+----------------+-----------+
| Mois | moy |
+----------------+-----------+
| June 2014 | 98.465500 |
| July 2014 | 98.854516 |
| August 2014 | 98.227097 |
| September 2014 | 95.008667 |
| October 2014 | 77.880000 |
+----------------+-----------+
5 rows in set (0.00 sec)
However, each pi server has different starting date ( pi4 stats in june, but pi1 stats in january)
Example :
mysql> select DISTINCT DATE_FORMAT(timestamp,'%M %Y') as 'Mois' from info ORDER BY timestamp;
+----------------+
| Mois |
+----------------+
| January 2014 |
| February 2014 |
| March 2014 |
| April 2014 |
| May 2014 |
| June 2014 |
| July 2014 |
| August 2014 |
| September 2014 |
| October 2014 |
+----------------+
And i would like a query to get the average score per server, BUT with the most large view, including NULL.
Example for pi4 :
+----------------+-----------+
| Mois | moy |
+----------------+-----------+
| January 2014 | NULL |
| February 2014 | NULL |
| March 2014 | NULL |
| April 2014 | NULL |
| May 2014 | NULL |
| June 2014 | 98.465500 |
| July 2014 | 98.854516 |
| August 2014 | 98.227097 |
| September 2014 | 95.008667 |
| October 2014 | 77.880000 |
+----------------+-----------+
Ps : Here is an example of the database structure
mysql> select * from info limit 10;
+----+------------+-----------+------------+-------------+
| id | timestamp | server | value_perf | value_avail |
+----+------------+-----------+------------+-------------+
| 45 | 2014-06-11 | pi4 | 98.33 | 99.91 |
| 46 | 2014-06-12 | pi4 | 97.92 | 100.00 |
| 52 | 2014-06-18 | pi4 | 98.15 | 99.97 |
| 54 | 2014-06-20 | pi | 98.33 | 99.94 |
+----+------------+-----------+------------+-------------+
and i want the average per month, with NULL value for unavailable months.
How can i get this ?
Thanks

The simplest way to achieve what you want is to create a date lookup table. Something like
CREATE TABLE `month_lookup` (
`month_year` varchar(30),
`st` datetime,
`en` datetime,
PRIMARY KEY (`month_year`),
KEY `st` (`st`)
KEY `end` (`end`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Then insert data for each month's start and end date. Go ahead and populate it for 1970 to 2070 or something. This table can come in handy. An example row would look like
month-year st en
Jan-2014 2014-01-01 00:00:00 2014-01-31 23:59:59
Then what you can do is something like this:
SELECT a.month_year, avg(b.value_perf) AS moy
FROM month_lookup AS a
LEFT JOIN info AS b
ON b.timestamp BETWEEN a.st AND b.en
GROUP BY a.month_year, b.server;
The indices on the month_lookup table might be better if you did st_end (st, en) instead of st and en separately, but you'd have to test that... I don't recall off the top of my head if a compound index would help you with with a join ... between clause or not
Either way, create a month_lookup table and use it in a left join... probably your best option. Otherwise you'll be doing unions ond subqueries and it'll just get hard to manage.

Related

Get month-wise count of records in mySQL starting with the current month

Below is my mysql table:
mysql> DESCRIBE mytable;
+---------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------+--------------+------+-----+---------+-------+
| url_timestamp | timestamp | NO | | NULL | |
| cr_num | varchar(50) | NO | | NULL | |
| status | varchar(255) | NO | | NULL | |
| rollback_date | timestamp | YES | | NULL | |
| rollback_user | varchar(255) | YES | | NULL | |
+---------------+--------------+------+-----+---------+-------+
5 rows in set (0.00 sec)
I wish to get the month-wise count for the cr_num which is primary key for the past 12 months.
mysql> SELECT MONTHNAME(url_timestamp) , COUNT(url_timestamp) FROM mytable WHERE status='PRODUCTION' and url_timestamp >= NOW() - INTERVAL 1 YEAR GROUP BY MONTHNAME(url_timestamp);
+--------------------------+----------------------+
| MONTHNAME(url_timestamp) | COUNT(url_timestamp) |
+--------------------------+----------------------+
| November | 43 |
| December | 69 |
| January | 220 |
| October | 225 |
| February | 209 |
| March | 123 |
| April | 93 |
| May | 113 |
| June | 217 |
| July | 129 |
| August | 185 |
| September | 415 |
+--------------------------+----------------------+
12 rows in set (0.01 sec)
There are three issues with the output i get.
I want the output sorted starting with the current month listed first i.e October 2020 all the way back to November 2019.
Months November and December are from previous year i.e. 2019; so i would like to display the year next to the each month.
I would like to get the data only for this Year 2020 thus, November and December records should not show.
I'm not from the database background, so I did not deep dive too much into what I could have tried.
You can use functions LAST_DAY() to group by month and DATE_FORMAT() to format the month with the year:
SELECT DATE_FORMAT(LAST_DAY(url_timestamp), '%M %Y') month,
COUNT(url_timestamp) counter
FROM mytable
WHERE status='PRODUCTION' and url_timestamp >= NOW() - INTERVAL 1 YEAR
GROUP BY month;
To get the rows sorted correctly you need to also GROUP BY LAST_DAY(url_timestamp) so you can use it in the ORDER BY clause:
SELECT DATE_FORMAT(LAST_DAY(url_timestamp), '%M %Y') month,
COUNT(url_timestamp) counter
FROM mytable
WHERE status='PRODUCTION' and url_timestamp >= NOW() - INTERVAL 1 YEAR
GROUP BY month, LAST_DAY(url_timestamp)
ORDER BY LAST_DAY(url_timestamp);
Your conditions (2) and (3) are incompatible. To get months only from this year:
SELECT MONTHNAME(url_timestamp), COUNT(*)
FROM mytable
WHERE status = 'PRODUCTION' AND
YEAR(url_timestamp) = YEAR(NOW())
GROUP BY MONTHNAME(url_timestamp)
ORDER BY MIN(url_timestamp)

Is it possible to add to an existing value in a cell in mysql?

I have a table which saves monthly values, but also a value for the complete year. Is is possible to add to the yearly value whenever I insert a value for a month?
I want to avoid loading the value first, adding to it in the server-code and writing it again.
You can write a trigger and insert value in the years table when any value is inserted in the Month table like
CREATE TRIGGER tr_month ON monthly_table
AFTER INSERT
AS
BEGIN
UPDATE year_table
SET // insert your values here
FROM inserted
WHERE monthly.id = inserted.id; // something like that, I am not sure about your structure thats why cannot add exact syntax
END
GO
Your best approach to this is avoiding redundant data in your table. When you need year totals, SELECT them.
You didn't tell us your table definition, so I will guess. The table months contains
year int (for example, 2019)
month int (1-12)
value number
You can get the details of this the obvious way: `
SELECT year, month, value FROM months;
You can get the details and the yearly sums this way
SELECT year, month, SUM(value) value
FROM months
GROUP BY year, month WITH ROLLUP;
The result set for this query looks like the other result set, but also contains sums. It looks like this:
| year | month | value |
| ---- | ----- | ----- |
| 2018 | 1 | 100 | detail month values...
| 2018 | 2 | 140 |
| 2018 | 3 | 130 |
| 2018 | 4 | 190 |
| 2018 | 5 | 120 |
| 2018 | 6 | 180 |
| 2018 | 7 | 130 |
| 2018 | 8 | 140 |
| 2018 | 9 | 150 |
| 2018 | 10 | 200 |
| 2018 | 11 | 230 |
| 2018 | 12 | 300 |
| 2018 | | 2010 | yearly sum for 2018 (month is NULL)
| 2019 | 1 | 100 |
| 2019 | 2 | 130 |
| 2019 | 3 | 160 |
| 2019 | 4 | 140 |
| 2019 | 5 | 190 |
| 2019 | 6 | 240 |
| 2019 | | 960 | yearly sum for 2019 (month is NULL)
| | | 2970 | total sum (both month and year are NULL)
View on DB Fiddle
Why is this a good process?
you need to store no extra data.
it works correctly even if you update or delete rows in your table.
it's fast: SQL is made to do this kind of thing.
Just solved it by adding the values client side, this also saves computing time on the server.

MYSQL: Left JOIN from two SELECT to "fill gaps" in dates

Let's say I have a table "calendar"
+------------+
| day_date |
+------------+
| 2015-01-01 |
| 2015-01-02 |
| 2015-01-03 |
| .......... |
| 2015-07-14 |
| 2015-07-15 |
+------------+
With this query I can select the WEEK (that I need)
SELECT WEEK(day_date,1) AS NUM_WEEK,
YEAR(day_date) AS YEAR,
STR_TO_DATE(CONCAT(YEAR(day_date),WEEK(day_date,1),' Monday'), '%X%V %W') AS date_start
FROM calendar
GROUP BY NUM_WEEK
And this is the result:
+----------+------+------------+
| NUM_WEEK | YEAR | date_start |
+----------+------+------------+
| 29 | 2015 | 2015-07-20 |
| 30 | 2015 | 2015-07-27 |
| 31 | 2015 | 2015-08-03 |
| 32 | 2015 | 2015-08-10 |
| 33 | 2015 | 2015-08-17 |
| 34 | 2015 | 2015-08-24 |
| 35 | 2015 | 2015-08-31 |
| 36 | 2015 | 2015-09-07 |
| 37 | 2015 | 2015-09-14 |
| 38 | 2015 | 2015-09-21 |
| 39 | 2015 | 2015-09-28 |
| 40 | 2015 | 2015-10-05 |
| 41 | 2015 | 2015-10-12 |
| 42 | 2015 | 2015-10-19 |
| 43 | 2015 | 2015-10-26 |
+----------+------+------------+
Now I have another table:
+----+------------+--------+---------------------+
| id | id_account | amount | date_transaction |
+----+------------+--------+---------------------+
| 1 | 283 | 150 | 2015-06-21 15:50:47 |
| 2 | 283 | 47.74 | 2015-07-23 15:55:44 |
| 3 | 281 | 21.55 | 2015-08-24 12:27:11 |
| 4 | 283 | 11.22 | 2015-08-25 10:00:54 |
+----+------------+--------+---------------------+
They are gaps in date.
With a similar query:
SELECT WEEK(date_transaction,1) AS NUM_WEEK,
YEAR(date_transaction) AS YEAR,
STR_TO_DATE(CONCAT(YEAR(date_transaction),WEEK(date_transaction,1),' Monday'), '%X%V %W')
AS date_start,
transaction.id_account,
SUM(amount) as total FROM transaction
INNER JOIN account ON account.id_account = transaction.id_account
WHERE amount > 0 AND transaction.id_account
IN ( SELECT id_account FROM account WHERE id_customer = 12 )
GROUP BY id_account, WEEK(date_transaction,1)
I obtain this result (probably data are not accurate, referring to previous tables, just to explain).
+----------+------+------------+-----------+----------+
| NUM_WEEK | YEAR | date_start | idAccount | total |
+----------+------+------------+-----------+----------+
| 29 | 2015 | 2015-07-20 | 281 | 22377.00 |
| 30 | 2015 | 2015-07-27 | 281 | 11550.00 |
| 32 | 2015 | 2015-08-04 | 281 | 4500.00 |
| 30 | 2015 | 2015-07-27 | 283 | 1500 |
+----------+------+------------+-----------+----------+
What I would, RIGHT (or LEFT) JOINING the two tables?
The min (and max) WEEK, so I can... (see 2)
Fill the gaps with missing WEEKS with NULL VALUES.
E.g., in a more complicated resultset:
+----------+------+------------+-----------+----------+
| NUM_WEEK | YEAR | date_start | idAccount | total |
+----------+------+------------+-----------+----------+
| 29 | 2015 | 2015-07-20 | 281 | 22377.00 |
| 30 | 2015 | 2015-07-27 | 281 | 11550.00 |
| 31 | 2015 | 2015-07-02 | 281 | NULL |
| 32 | 2015 | 2015-08-09 | 281 | 4500.00 |
| 29 | 2015 | 2015-08-09 | 283 | NULL |
| 30 | 2015 | 2015-07-16 | 283 | 1500 |
| 31 | 2015 | 2015-07-16 | 283 | NULL |
| 32 | 2015 | 2015-07-16 | 283 | NULL |
+----------+------+------------+-----------+----------+
Note, for example, that id=283 now has NULL at WEEK 29, 31 and 32, for example, like id=281 has NULL in WEEK 31.
I prepared also SQLFiddle here: http://sqlfiddle.com/#!9/a8fdc/3
Thank you very much.
I take a look on your question and i came up with this solution. Here is how your query could look like:
SELECT t1.NUM_WEEK, t1.`YEAR`, t1.date_start, t1.id_account, t2.total
FROM (SELECT c.NUM_WEEK, c.`YEAR`, c.date_start, a.id_account
FROM (SELECT WEEK(day_date,1) AS NUM_WEEK,
YEAR(day_date) AS `YEAR`,
STR_TO_DATE(CONCAT(YEAR(day_date),WEEK(day_date,1),' Monday'), '%X%V %W') AS date_start,
(SELECT GROUP_CONCAT(id_account) FROM account WHERE id_customer=12) AS accounts_id
FROM calendar
GROUP BY NUM_WEEK) c
INNER JOIN account a
ON FIND_IN_SET(a.id_account, c.accounts_id)
ORDER BY a.id_account, c.NUM_WEEK) t1
LEFT JOIN
(SELECT WEEK(t.date_transaction,1) AS NUM_WEEK,
YEAR(t.date_transaction) AS `YEAR`,
STR_TO_DATE(CONCAT(YEAR(t.date_transaction),WEEK(t.date_transaction,1),' Monday'), '%X%V %W') AS date_start,
t.id_account, SUM(t.amount) AS total
FROM `transaction` t
INNER JOIN account a
ON a.id_account = t.id_account
WHERE t.amount > 0 AND
t.id_account IN (SELECT id_account FROM account WHERE id_customer = 12)
GROUP BY id_account, WEEK(date_transaction,1)) t2
ON t1.NUM_WEEK = t2.NUM_WEEK AND t1.YEAR = t2.YEAR AND t1.id_account = t2.id_account;
Here is SQL Fiddle for that so you can check up result. Hope that is what are you looking for.
Little explanation:
First think i done is that I little modified your first query where you extract data from table calendar and add there one new column called accounts_id. That query now look's like this:
SELECT WEEK(day_date,1) AS NUM_WEEK,
YEAR(day_date) AS `YEAR`,
STR_TO_DATE(CONCAT(YEAR(day_date),WEEK(day_date,1),' Monday'), '%X%V %W') AS date_start,
(SELECT GROUP_CONCAT(id_account) FROM account WHERE id_customer=12) AS accounts_id
FROM calendar
GROUP BY NUM_WEEK
Please pay attention on this line in SELECT statement
(SELECT GROUP_CONCAT(id_account) FROM account WHERE id_customer=12) AS accounts_id
Note that when you select for specific customer you need to change customer ID in this line too!!!
Here is Fiddle so you can check result that this query produce.
This is necessary because we need to connect each week with each account to get desired result.
Next step is to extend previous query so we could separate accounts_id column (look result of previous query) so we could get row for each value in that column. Extended query look like this:
SELECT c.NUM_WEEK, c.`YEAR`, c.date_start, a.id_account
FROM (SELECT WEEK(day_date,1) AS NUM_WEEK,
YEAR(day_date) AS `YEAR`,
STR_TO_DATE(CONCAT(YEAR(day_date),WEEK(day_date,1),' Monday'), '%X%V %W') AS date_start,
(SELECT GROUP_CONCAT(id_account) FROM account WHERE id_customer=12) AS accounts_id
FROM calendar
GROUP BY NUM_WEEK) c
INNER JOIN account a
ON FIND_IN_SET(a.id_account, c.accounts_id)
ORDER BY a.id_account, c.NUM_WEEK
and output you can see in this Fiddle
After that all we need to do is to make left join between this query and query you already wrote in your question (last query).
There might be a better solution or even this one maybe can be improved a little, but I don't have much time now to deal with that and this is the first think that cross my mind...
GL!
P. S. pay attention when you use reserved word in MySQL like YEAR, TRANSACTION etc for column name (as column_name).. that can cause you a treble if have to use them in name of column or table use backquote () to mark them (asyear`)...

Incorrect results for year,month grouping for mysql count

I am having an issue with incorrect counts for a query which is as follows:
mysql> SELECT DATE_FORMAT(FROM_UNIXTIME(timestamp), '%M') AS month,
YEAR(FROM_UNIXTIME(timestamp)) AS year, count(*) AS count FROM table1 WHERE
result LIKE '%created%' GROUP BY YEAR(FROM_UNIXTIME(timestamp)),
MONTH(FROM_UNIXTIME(timestamp)) DESC;
+-----------+------+-------+
| month | year | count |
+-----------+------+-------+
| September | 2011 | 1 |
| December | 2013 | 393 |
| November | 2013 | 70 |
| September | 2014 | 233 |
| August | 2014 | 739 |
| July | 2014 | 691 |
| June | 2014 | 618 |
| May | 2014 | 120 |
| March | 2014 | 272 |
| February | 2014 | 528 |
| January | 2014 | 607 |
+-----------+------+-------+
11 rows in set (0.40 sec)
However, when I check my work by using another syntax I get different results:
mysql> select count(*) from table1 where result like '%created%' and
timestamp >= unix_timestamp('2014-07-01') and timestamp <= unix_timestamp('2014-07-31');
+----------+
| count(*) |
+----------+
| 662 |
+----------+
1 row in set (0.39 sec)
What is wrong with the syntax?
In the second query use < unix_timestamp('2014-08-01'). I think you filter out everything that happened after 2014-07-31 00:00:00 am. There are still 24 hours in that day that shouldn't be filtered.

Getting the missing period through mmysql query or procedure..?

I have a table customer_order as follows
mysql> select * from customer_order;
+---------+---------+-----------+------------------+----------------+
| cust_id | orderno | region_cd | order_start_date |order_del_date |
+---------+---------+-----------+------------------+----------------+
| CU_082 | ONO_001 | reg1 | 2012-04-25 | 2012-08-25 |
| CU_082 | ONO_002 | reg1 | 2012-04-28 | 2012-11-28 |
| CU_083 | ONO_002 | reg2 | 2012-04-28 | 2012-11-28 |
| CU_082 | ONO_003 | reg1 | 2012-04-25 | 2012-08-25 |
| CU_084 | ONO_004 | reg4 | 2012-04-25 | 2012-10-25 |
I need a table like this...which i get....as follows
mysql> select order_start_date,order_del_date,orderno,cust_id from customer_order wh
ere order_start_date >= '2012-04-25' AND order_del_date <='2012-12-28' and cust_i
d IN ('36082','36088') order by cust_id ;
+------------------+----------------+---------+---------+
| order_start_date | order_del_date | pid | emp_id |
+------------------+----------------+---------+---------+
| 2012-04-25 | 2012-05-25 | ONO_001 | CU_082 |
| 2012-08-22 | 2012-12-28 | ONO_004 | CU_082 |
| 2012-06-22 | 2012-08-28 | ONO_003 | CU_082 |
| 2012-05-27 | 2012-06-25 | ONO_002 | CU_082 |
| 2012-04-25 | 2012-05-25 | ONO_001 | CU_082 |
| 2012-05-27 | 2012-06-25 | ONO_001 | CU_082 |
| 2012-04-30 | 2012-06-25 | ONO_001 | CU_088 |
| 2012-06-28 | 2012-07-15 | ONO_002 | CU_088 |
| 2012-07-28 | 2012-08-25 | ONO_003 | CU_088 |
| 2012-07-16 | 2012-09-25 | ONO_004 | CU_088 |
+------------------+----------------+---------+---------+
now i need to query on this table ...
to get
for each customer here we get data for the period from wat date to wat date his order processing details in the above table..
now for each customer i shld find the period for which thr is no order processing...
eg cust_id =CU_088
he has his order processed from 30 apr to 25 june
den from 28th june to 15 july
(here thr is a diff that is thr is no order taken or any processing done from 26th to 27th june..this is wat is the required result)
**one more imp consideration is...
in the next entry we find thr is a order process from 28th july to 25th aug
w.r.t previous entry i.e, 28th june to 15july we find that for this customer thr is no order taken or processed from 16th to 27th july..
but with the last entry tat is 16th july to 25th sept he has an order with different order_no thrfore the gap 16th july to 27th july is filled here so this kind of a condition also needs to be checked...
I need to get the output as something like this..
+------------------+----------------+---------+---------+
| order_start_date | order_del_date | pid | emp_id |
+------------------+----------------+---------+---------+
| 2012-06-26 | 2012-06-27 | ONO_001 | CU_088 |
+------------------+----------------+---------+---------+
that is either the query or procedure which is more efficient should give me the period wer in thr was no action done for customer...
help me write the query which does the all the above things.
I m new to db queries..so please help me out..
To fetch every 'gap' between orders, you can use a self-join:
SELECT o1.cust_id,
o1.order_del_date + INTERVAL 1 DAY AS gap_begin,
MIN(o2.order_start_date) - INTERVAL 1 DAY AS gap_end
FROM customer_order o1
JOIN customer_order o2 ON o1.cust_id = o2.cust_id
AND o1.order_del_date <= o2.order_start_date
GROUP BY o1.cust_id, o1.order_del_date
HAVING gap_begin < gap_end