MySQl - SQL - Top 5 records per day - mysql

With respect to the data set below, I'm trying to get the top 5 records per day on a MySQL database. It's a table of web page visits & my aim is to find out the 5 most visited pages.
I'm comfortable getting just the top 10 in a given date range, but, have not been able to manage to get a query going for the topic in question.
I did try the below
select
VISIT_DATE,
group_concat(PAGE_ID order by NUM_VISITS desc separator ',') as pagehits
from
PAGEVISITS
where
VISIT_DATE >= '2015-07-01' and VISIT_DATE <= '2015-07-15'
group by
VISIT_DATE
but I can't get SUM(NUM_VISITS) int here & I couldn't get group byVISIT_DATE` which makes it pretty useless. This apart, this is how far I've got
select
VISIT_DATE,
PAGE_ID,
SUM(NUM_VISITS) as pagehits
from
PAGEVISITS
where
VISIT_DATE >= '2015-01-01' and VISIT_DATE <= '2015-03-15'
group by
VISIT_DATE,
PAGE_ID
order by
pagehits desc
limit 5;
which obviously is not top 5 per day. Also, there could be more than one page that can end up having the same number of page hits obviously & may also end up appearing as one of the top 5 which is why I tried using group concat to display all those PAGE IDs whose number of page hits is in the top 5 page hit count for that day.
I'm not a seasoned SQL coder. Could I please request assistance to get this working. If I've not sounded clear anywhere, please do let me know.
CREATE TABLE PAGEVISITS
(`VISIT_DATE` date, `PAGE_ID` varchar(20), `SERVER_NAME` varchar(50), `NUM_VISITS` int)
;
INSERT INTO PAGEVISITS
(`VISIT_DATE`, `PAGE_ID`, `SERVER_NAME`, `NUM_VISITS`)
VALUES
('2015-01-01','2015A12123','A',10),
('2015-01-01','2015A12123','B',10),
('2015-01-01','2015A12124','A',30),
('2015-01-01','2015A12124','B',30),
('2015-01-01','2015A12125','A',40),
('2015-01-01','2015A12125','B',40),
('2015-01-01','2015A12126','A',1),
('2015-01-01','2015A12126','B',1),
('2015-01-01','2015A12127','A',0),
('2015-01-01','2015A12127','B',1),
('2015-01-01','2015A12128','A',40),
('2015-01-01','2015A12129','A',30),
('2015-01-01','2015A12134','A',45),
('2015-01-01','2015A12126','A',56),
('2015-01-01','2015A12167','A',23),
('2015-01-01','2015A12145','A',17),
('2015-01-01','2015A121289','A',12),
('2015-01-01','2015A121289','B',5),
('2015-01-02','2015A12123','A',3),
('2015-01-02','2015A12124','A',10),
('2015-01-02','2015A12125','A',70),
('2015-01-02','2015A12126','A',10),
('2015-01-02','2015A12127','A',100),
('2015-01-02','2015A12128','A',3),
('2015-01-02','2015A12128','B',2),
('2015-01-02','2015A12129','A',10),
('2015-01-02','2015A12134','A',5),
('2015-01-02','2015A12126','A',6),
('2015-01-02','2015A12167','A',3),
('2015-01-02','2015A12145','A',170),
('2015-01-02','2015A121289','A',34),
('2015-01-03','2015A12123','A',34),
('2015-01-03','2015A12124','A',14),
('2015-01-03','2015A12125','A',37),
('2015-01-03','2015A12126','A',23),
('2015-01-03','2015A12127','A',234),
('2015-01-03','2015A12128','A',47),
('2015-01-03','2015A12129','A',67),
('2015-01-03','2015A12134','A',89),
('2015-01-03','2015A12134','B',1),
('2015-01-03','2015A12126','A',97),
('2015-01-03','2015A12167','A',35),
('2015-01-03','2015A12145','A',0),
('2015-01-03','2015A121289','A',19),
('2015-01-04','2015A12123','A',115),
('2015-01-04','2015A12124','A',149),
('2015-01-04','2015A12125','A',370),
('2015-01-04','2015A12126','A',34),
('2015-01-04','2015A12127','A',4),
('2015-01-04','2015A12128','A',70),
('2015-01-04','2015A12129','B',70),
('2015-01-04','2015A12134','A',70),
('2015-01-04','2015A12126','B',64),
('2015-01-04','2015A12167','A',33),
('2015-01-04','2015A12145','A',10);
ANTICIPATED OUTPUT
Fiddle here

If this is going to be used daily, then you should consider to create a separate table and fill the data in it using procedure. There is still better way to do this(using merge). This is just for your reference.
create table daily_results
(`VISIT_DATE` date, `PAGE_ID` varchar(20), `SERVER_NAME` varchar(50), `NUM_VISITS` int);
CREATE PROCEDURE proc_loop_test( IN startdate date, in enddate date)
BEGIN
WHILE(startdate < enddate) DO
insert into daily_results (select * from PAGEVISITS where VISIT_DATE=startdate order by NUM_VISITS desc limit 5);
SET startdate = date_add(startdate, INTERVAL 1 DAY);
end WHILE;
END;
call it using
call proc_loop_test(`2015-01-01`,`2015-03-15`);
select * from daily_results;

The query should be
select sub.*,
CASE WHEN #vd!=VISIT_DATE THEN #rn:=0 ELSE #rn:=#rn+1 END as row_num,
#vd:=VISIT_DATE
from (
select
VISIT_DATE,
PAGE_ID,
SUM(NUM_VISITS) as pagehits
from
PAGEVISITS
where
VISIT_DATE >= '2015-01-01' and VISIT_DATE <= '2015-03-15'
group by
VISIT_DATE,
PAGE_ID
order by
VISIT_DATE, pagehits desc) sub
having row_num<5
Somehow the SQL fiddle shows kind of internal error when the query is executed.

Related

MySQL Query Group By Date (12 hours Interval)

I wrote a sql query for getting number of users created in a period of time for plotting graph (grafana or chart js) , and my sql query is
SELECT
date(user.created_date) as "time",
count(distinct user.id) as Number Of User,
status as status
FROM user
WHERE
created_date >= FROM_UNIXTIME(1649635200) AND
created_date < FROM_UNIXTIME(1649894399)
GROUP BY user.status, date(user.created_date)
ORDER BY date(user.created_date)
Here in this query created date is passed dynamically from front-end,
Now i am getting the result like,
Now whenever i select the date filter from last 24 hours/12 hours some of the result is not there,
Is there is any way to modify my sql query to group by created_date with 12 hour interval
For Example, Now query result is 11/04/2022 - 5 Users(Application Created) I want query result like this 11/04/2022 00:00:00 2 - 2 users created 11/04/2022 12:00:00 - 3 users created
In grafana there is a filed $__timeFrom() and $__timeTo()
On the basis of this I rewrite my query:
SELECT
(CASE
WHEN HOUR(TIMEDIFF($__timeFrom(), $__timeTo())) <= 24
THEN user.created_date
ELSE date(user.created_date) end) AS "time",
count(distinct user.id) as Users,
FROM user
WHERE
user.created_date >= $__timeFrom() AND
user.created_date < $__timeTo() AND
GROUP BY CASE
when HOUR(TIMEDIFF($__timeFrom(), $__timeTo())) <= 24
then user.created_date
else date(created_date) end
ORDER BY CASE
when HOUR(TIMEDIFF($__timeFrom(), $__timeTo())) <= 24
then user.created_date
else date(created_date) end;
If you use this expresion in your GROUP BY, you'll get a 12-hour grouping.
DATE(created_date) + INTERVAL (HOUR(created_date) - HOUR(created_date) MOD 12) HOUR
You can, if you have the priv, declare a stored function to make this easier to read.
DELIMITER $$
DROP FUNCTION IF EXISTS TRUNC_HALFDAY$$
CREATE
FUNCTION TRUNC_HALFDAY(datestamp DATETIME)
RETURNS DATETIME DETERMINISTIC NO SQL
COMMENT 'truncate to 12 hour boundary. Returns the nearest
preceding half-day (noon, or midnight)'
RETURN DATE(datestamp) +
INTERVAL (HOUR(datestamp) -
HOUR(datestamp) MOD 12) HOUR$$
DELIMITER ;
Then you can do
SELECT
TRUNC_HALFDAY(user.created_date) as "time",
count(distinct user.id) as Number Of User,
status as status
FROM user
WHERE
created_date >= whatever AND
created_date < whatever
GROUP BY user.status, TRUNC_HALFDAY(user.created_date)
ORDER BY TRUNC_HALFDAY(user.created_date)
Even though the function appears three times in your query, because it's declared DETERMINISTIC it only gets called once per row.
More complete writeup here.

Fetch values between two dates - MariaDB version: 10.1.37

I'm trying to fetch values between two dates, specifically 24hrs
SELECT *
FROM `transactions`
WHERE accnum = '1534610376'
AND tdate BETWEEN 20190311 AND 20190312
This query works fine but, i don't want it for a constant date, i have checked and seen many format but none seems to work. please help
If you "want records from today alone" - a simple way would be:
WHERE accnum = '1534610376'
AND DATE(tdate) = CURRENT_DATE()
However - To utilize an index, a column should not be wrapped into a function. So an efficient way would be
WHERE accnum = '1534610376'
AND tdate >= CURRENT_DATE()
AND tdate < CURRENT_DATE() + INTERVAL 1 DAY
A good index for this query would be INDEX(accnum, tdate).
I suggest you to put your date between quots like this:
SELECT *
FROM `transactions`
WHERE accnum = '1534610376'
AND tdate BETWEEN '20190311' AND '20190312'
After, you can define a user defined function like this :
CREATE FUNCTION BetweenDate(#toCompare VARCHAR(30), #rightDate DATE, #leftDate DATE)
RETURNS TABLE
AS
BEGIN
RETURN (
SELECT *
FROM transactions
WHERE accum = #toCompare AND tdate BETWEEN #rightDATE AND #leftDate
)
END;

MySQL: Undesired result with max function on a timestamp

I use a Mantis Bug Database (that uses MySQL) and I want to query which bugs had a change in their severity within the last 2 weeks, however only the last severity change of the bug should be indicated.
The problem is, that I get multiple entries per bugID (which is the primary key), which is not my desired result since I want to have only the latest change per bug. This means that somehow I am using the max function and the group by clause wrongfully.
Here you can see my query:
SELECT `bug_id`,
max(date_format(from_unixtime(`mantis_bug_history_table`.`date_modified`),'%Y-%m-%d %h:%i:%s')) AS `Severity_changed`,
`mantis_bug_history_table`.`old_value`,
`mantis_bug_history_table`.`new_value`
from `prepared_bug_list`
join `mantis_bug_history_table` on `prepared_bug_list`.`bug_id` = `mantis_bug_history_table`.`bug_id`
where (`mantis_bug_history_table`.`field_name` like 'severity')
group by `bug_id`,`old_value`,`.`new_value`
having (`Severity_modified` >= (now() - interval 2 week))
order by bug_id` ASC
For the bug with the id 8 for example I get three entries with this query. The bug with the id 8 had indeed three severity changes within the last 2 weeks but I only want to get the latest severity change.
What could be the problem with my query?
max() is an aggregation function and it does not appear to be suitable for what you are trying to do.
I have feeling that what you are trying to do is to get the latest out of all the applicable bug_id in mantis_bug_history_table . If that is true, then I would rewrite the query as the following -- I would write a sub-query getLatest and join it with prepared_bug_list
Updated answer
Caution: I don't have access to the actual DB tables so this query may have bugs
select
`getLatest`.`last_bug_id`
, `mantis_bug_history_table`.`date_modified`
, `mantis_bug_history_table`.`old_value`
, `mantis_bug_history_table`.`new_value`
from
(
select
(
select
`bug_id`
from
`mantis_bug_history_table`
where
`date_modified` > unix_timestamp() - 14*24*3600 -- two weeks
and `field_name` like 'severity'
and `bug_id` = `prepared_bug_list`.`bug_id`
order by
`date_modified` desc
limit 1
) as `last_bug_id`
from
`prepared_bug_list`
) as `getLatest`
inner join `mantis_bug_history_table`
on `prepared_bug_list`.`bug_id` = `getLatest`.`last_bug_id`
order by `getLatest`.`bug_id` ASC
I finally have a solution! I friend of mine helped me and one part of the solution was to include the Primary key of the mantis bug history table, which is not the bug_id, but the column id, which is a consecutive number.
Another part of the solution was the subquery in the where clause:
select `prepared_bug_list`.`bug_id` AS `bug_id`,
`mantis_bug_history_table`.`old_value` AS `old_value`,
`mantis_bug_history_table`.`new_value` AS `new_value`,
`mantis_bug_history_table`.`type` AS `type`,
date_format(from_unixtime(`mantis_bug_history_table`.`date_modified`),'%Y-%m-%d %H:%i:%s') AS `date_modified`
FROM `prepared_bug_list`
JOIN mantis_import.mantis_bug_history_table
ON `prepared_bug_list`.`bug_id` = mantis_bug_history_table.bug_id
where (mantis_bug_history_table.id = -- id = that is the id of every history entry, not confuse with bug_id
(select `mantis_bug_history_table`.`id` from `mantis_bug_history_table`
where ((`mantis_bug_history_table`.`field_name` = 'severity')
and (`mantis_bug_history_table`.`bug_id` = `prepared_bug_list`.`bug_id`))
order by `mantis_bug_history_table`.`date_modified` desc limit 1)
and `date_modified` > unix_timestamp() - 14*24*3600 )
order by `prepared_bug_list`.`bug_id`,`mantis_bug_history_table`.`date_modified` desc

Mysql time spread

Sorry, I have difficulty explaining my question and search for a previous answer. This is my problem -- I have a MySQL table with events
CREATE TABLE events {
id INT,
event INT,
date DATETIME
}
Data is being added a few times a week or month. I would like to see the statistical spread of time between two adjacent events. Something like:
Time difference between two events
1 day appart - 4 occurances
2 days apart - 2 occurances
n days apart - x occurances
It should be something like this, I guess, but calculating the time difference between events.
SELECT COUNT('id') AS 'no', ??? AS 'delta' GROUP BY FLOOR( 'delta' )
This piece of SQL code did it:
SET #old = NOW();
SELECT COUNT(`id`) AS `no`, query1.`delta` FROM
( SELECT `id`, `date`, DATEDIFF( #old, `date` ) AS `delta`, #old := `date` AS `old`
FROM `life`
ORDER BY `date`DESC ) query1
GROUP BY `delta`
ORDER BY `delta`

Some questions about SQL group by week

I have some problems when coding SQL group by week.
I have a MySQL table named order.
In this entity, there are several attributes, called 'order_id', 'order_date', 'amount', etc.
I want to make a table to show the statistics of past 7 days order sales amount.
I think first I should get the today value.
Since I use Java Server Page, the code like this:
Calendar cal = Calendar.getInstance();
int day = cal.get(Calendar.DATE);
int Month = cal.get(Calendar.MONTH) + 1;
int year = cal.get(Calendar.YEAR);
String today = year + "-" + Month + "-" + day;
then, I need to use group by statement to calculate the SUM of past 7 day total sales amount.
like this:
ResultSet rs=statement.executeQuery("select order_date, SUM(amount) " +
"from `testing`.`order` GROUP BY order_date");
I have problem here. In my SQL, all order_date will be displayed.
How can I modify this SQL so that only display past seven days order sale amount?
Besides that, I discover a problem in my original SQL.
That is, if there is no sales on that day, no results would be displayed.
OF course, I know the ResultSet does not allow return null values in my SQL.
I just want to know if I need the past 7 order sales even the amount is 0 dollars,
Can I have other methods to show the 0?
Please kindly give me advices if you have idea.
Thank you.
Usually it occurs to create with a script or with a stored procedure a calendar table with all dates.
However if you prefer you can create a table with few dates (in your case dates of last week) with a single query.
This is an example:
create table orders(
id int not null auto_increment primary key,
dorder date,
amount int
) engine = myisam;
insert into orders (dorder,amount)
values (curdate(),100),
(curdate(),200),
('2011-02-24',50),
('2011-02-24',150),
('2011-02-22',10),
('2011-02-22',20),
('2011-02-22',30),
('2011-02-22',5),
('2011-02-19',10);
select t.cdate,sum(coalesce(o.amount,0)) as total
from (
select curdate() -
interval tmp.digit * 1 day as `cdate`
from (
select 0 as digit union all
select 1 union all
select 2 union all
select 3 union all
select 4 union all
select 5 union all
select 6 union all
select 7 ) as tmp) as t
left join orders as o
on t.cdate = o.dorder and o.dorder >= curdate() - interval 7 day
group by t.cdate
order by t.cdate desc
Hope that it helps. Regards.
To answer your question "How can I modify this SQL so that only display past seven days order sale amount?"
Modify the SQL statement by adding a where clause to it:
Where order_date >= #date_7days_ago
The value for this #date_7days_ago date variable can be set before your statement:
Select #date_7days_ago = dateadd(dd,-7,getdate())
Adding that where clause to your query will return only those records which order date is in the last seven days.
Hope this helps.
You can try using this:
ResultSet rs = statement.executeQuery(
"SELECT IFNULL(SUM(amount),0)
FROM table `testing`.`order`
WHERE order_date >= DATE_SUB('" + today + "', INTERVAL 7 DAY)"
);
This will get you the number of orders made in the last 7 days, and 0 if there were none.