Return multiple copies slightly modified - mysql

In my events table there are records that have a daily flag that indicate that this event must be repeated each day:
CREATE TABLE IF NOT EXISTS `events` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`start` datetime DEFAULT NULL,
`title` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`description` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`daily` tinyint(1) NOT NULL,
) ENGINE=InnoDB;
When I try to fill a calendar with this data I need a row for each item, it is, if the calendar ask for all events this week, daily flaged events must return 7 events (one for each week day) with the same data (title, description, etc), but with a different start day.
Is it possible to do from MySQL?

Usually I would use a table that contains each of the single date.
Example
create table daily
(
/* FYI, date is not a reserved keyword */
date date
);
insert into daily values ('2011-09-11'), ('2011-09-12'),
('2011-09-13'), ('2011-09-14'), ('2011-09-15'),
('2011-09-16'), ('2011-09-17');
alter table daily add index (date);
select daily.date as daily_date,
weekly.start, weekly.id, weekly.title, weekly.description
from daily
inner join
(
select events.id, events.title,
events.description, events.start, events.daily
from events
where events.start between '2011-09-11' and '2011-09-17'
) as weekly
on date(weekly.start)=daily.date
union
/* query for recurr */
select daily.date as daily_date,
recurr.start, recurr.id, recurr.title, recurr.description
from daily
inner join
(
select events.id, events.title,
events.description, events.start, events.daily
from events
where events.daily=1 AND events.start between '2011-09-11' and '2011-09-17'
)as recurr
on date(recurr.start)<=daily.date
order by daily_date;
It would be more efficient if you break the start column to two column (date, time).
The drawback of this method is on the pre-create of table daily and loaded it with lots of day value.
This should be one time cost and can easily inserted via a for.
Alternatively, you can do a repeat of (from a week start to week end) on the query for recurr

Related

Is there a better approach in SQL to group actions by day/week/month/year/alltime?

I'm thinking my way to do that is a little archaic, not optimized... i don't need super detailed statistics, lets say i want the number of clicks on a link on a blog post per (actual) day/week/month/year nothing more, don't want the hour of click, just a number for each corresponding times (day/month/year).
I've created this table :
CREATE TABLE `clicks` (
`file_id` bigint(20) unsigned NOT NULL,
`filename` varchar(100) NOT NULL,
`day` int(10) unsigned DEFAULT 0,
`week` int(10) unsigned DEFAULT 0,
`month` int(10) unsigned DEFAULT 0,
`year` int(10) unsigned DEFAULT 0,
`all` int(10) unsigned DEFAULT 0,
PRIMARY KEY (`file_id`)
)
And each time there's a click, i update every column of a row by +1.
UPDATE clicks SET day = day+1, week = week+1 [..] WHERE file_id = $id
And at every end of day/week/month/year there's a cronjob who will reset the corresponding column for every file. For each day end it will be :
UPDATE clicks SET day = 0 [No WHERE Clause]
And when there's new click on a file tomorrow, it'll increment the day column again.
I have a small VPS server, small storage space, small RAM etc.. i just need how many times a file has been clicked this day only (not yesterday), this week (not the week before) etc.. and i'm trying to not have big & slow queries by having a line for each click and having millions of them.
Is my way of doing seems ok, or is there a better approach for what i want ?
Thanks everyone for the help.
You could create a table just storing the clicks, something like this:
CREATE TABLE clicks (
file_id INT NOT NULL,
filename VARCHAR(100) NOT NULL,
click_time TIMESTAMP NOT NULL
);
Then you just need to use the group by, to extract the clicks. For Example:
-- group clicks by day
SELECT DATE(click_time) AS day, COUNT(*) AS clicks
FROM clicks
GROUP BY day;
-- group clicks by week
SELECT YEARWEEK(click_time) AS week, COUNT(*) AS clicks
FROM clicks
GROUP BY week;
This is quite more efficient and requires less storage
Build and maintain a "Summary table" by date. Only summarize by DAY, then sum up the counts to get "by week", etc. That also lets you get the tallies for arbitrary ranges of days.
More on Summary Tables

MySQL subquery count with calendar table slow

I have a sales table in MySQL (InnoDB). It's +- 1 million records big. I would like to show some nice charts. Fetching the right data is not a problem. Fetching it fast is...
So I like to count the amount of sales in table A grouped per day (later on also month, and year) for PERIOD A till Z. Concrete; for the last 30 days I like to know for each day how many sales records we have in the DB.
So MySQL would have to return something like this:
I like to achieve that MySQL returns the data like this:
date, count
2017-04-01, 2482
2017-04-02, 1934
2017-04-03, 2701
...
The structure of the Sales basically like this:
CREATE TABLE `sales` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`created_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`deleted_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `contacts_created_at_index` (`created_at`),
KEY `contacts_deleted_at_index` (`deleted_at`),
KEY `ind_created_at_deleted_at` (`created_at`,`deleted_at`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Some days (datapoints) might not have any results, but I don't like to have gaps in the data. So I also have some 'calendar' table.
CREATE TABLE `time_dimension` (
`id` int(11) NOT NULL,
`db_date` date NOT NULL,
`year` int(11) NOT NULL,
`month` int(11) NOT NULL,
`day` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `td_ymd_idx` (`year`,`month`,`day`),
UNIQUE KEY `td_dbdate_idx` (`db_date`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Fetching 30 rows (30 days) with a count per day takes 30 secs...
This is the first query I tried:
SELECT
`db_date` AS `date`,
(SELECT
COUNT(1)
FROM
sales
WHERE
DATE(created_at) = db_date) AS count
FROM
`time_dimension`
WHERE
`db_date` >= '2017-04-11'
AND `db_date` <= '2017-04-25'
ORDER BY `db_date` ASC
But like I said it's really slow (11.9 secs). I tried al kinds of other approaches, but without luck. For example:
SELECT time_dimension.db_date AS DATE,
COUNT(1) AS count
FROM sales RIGHT JOIN time_dimension ON (DATE(sales.created_at) =
time_dimension.db_date)
WHERE
(time_dimension.db_date BETWEEN '2017-03-11' AND '2017-04-11')
GROUP BY
DATE
A query for just 1 datapoint takes only 5.4ms:
SELECT COUNT(1) FROM sales WHERE created_at BETWEEN '2017-04-11 00:00:00' AND '2017-04-25 23:59:59'
I haven't checked innodb_buffer_poolsize on my local machine. I will check that as well. Any ideas on how to make queries like this fast? In the future I would even need to where clauses and joins, to filter the set of sales records..
Thanks.
Nick
You could try to count sale data first, then join count result with your calendar table.
SELECT time_dimension.db_date AS date,
by_date.sale_count
FROM time_dimension
LEFT JOIN (SELECT DATE(sales.created_at) sale_date,
COUNT(1) AS sale_count
FROM sales
WHERE created_at BETWEEN '2017-03-11 00:00:00' AND
'2017-04-11 23:59:59'
GROUP BY DATE(sales.created_at)) by_date
ON time_dimension.db_date = by_date.sale_date
WHERE time_dimension.db_date BETWEEN '2017-03-11' AND '2017-04-11'
The problematic part of your query is the data type conversion DATE(created_at), which effectively prevents Mysql from using the index at created_at.
Your 1 datapoint query avoids that, and that is why it is working fast.
To fix this you should check if created_at is within a range of specific day, like that:
created_at BETWEEN db_date AND DATE_ADD(db_date,INTERVAL 1 DAY)
This way Mysql will be able to make use of index on it (do a range lookup), as appropriate.
WHERE DATE(created_at) = db_date)
-->
WHERE created_at >= db_date
AND created_at < db_date + INTERVAL 1 DAY
This avoids including midnight of second day (as BETWEEN does)
Work for all flavors: DATE, DATETIME, DATETIME(6)
Does not hid the created_at inside a function where the index cannot see it.
For time_dimension, get rid of PRIMARY KEY (id) and change UNIQUE(db_date) to the PK.
After making these changes, your original subquery may be competitive with the LEFT JOIN ( SELECT ... ). (It depends on which version of MySQL.)

Mysql Subquery between specific dates

I'm a newbie to MySQL querying and need some assistance with the subqueries.
I am using ASP .NET charting control that retrieves data from MySQL.I want to display a drill down chart and need some help on MySQL subquery.
Below is my table:
CREATE TABLE IF NOT EXISTS `data` (
`runtime` smallint(6) NOT NULL,
`app` varchar(60) NOT NULL,
`process` varchar(40) NOT NULL,
`username` varchar(51) NOT NULL,
`time` time NOT NULL,
`date` date NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Step 1 :
Showing a pie chart of Top 10 users with highest time between 2 dates.
I get the top 10 users used between 2 dates using the below query:
SELECT username ,SUM(runtime) as Runtime,
process,ROUND(SUM(runtime/201600),2) as 'Total Time',
role ,
date
FROM data
WHERE `date` BETWEEN 'date1' AND 'date2'
Group BY process LIMIT 10.
Step 2:
When user clicks on the individual user in chartArea, I wan to display the top 10 apps/process between specific dates.

Auto add days to datetime as each days passes in mysql

I want to make a column in mysql database that when user login first time in system, it stores that datetime in mysql table. And since that day in other column days will add according to his register date. Like 1, 2, 3,....and so on. So, is there any way I can achieve the results? Please guide me soon.
You can do this with just one column (to hold the registration / first login date) and the DATEDIFF function:
CREATE TABLE users (
ID int(11) NOT NULL AUTO_INCREMENT,
name varchar(20) NOT NULL,
registered_at datetime NOT NULL,
PRIMARY KEY (ID)
);
INSERT INTO users SET
name = 'myname',
registered_at = NOW();
SELECT registered_at, DATEDIFF(NOW(), registered_at) AS days_since
FROM users
WHERE name = 'myname';

Counting year and month entries from datetime fields

I have a problem constructing a mysql query:
I have this table "tSubscribers" were I store the subscribers for my newsletter mailing list.
The table looks like this (simplified):
--
-- Table structure for tSubscriber
--
CREATE TABLE tSubscriber (
fId INT UNSIGNED NOT NULL AUTO_INCREMENT,
fSubscriberGroupId INT UNSIGNED NOT NULL,
fEmail VARCHAR(255) NOT NULL DEFAULT '',
fDateConfirmed DATETIME NOT NULL DEFAULT '0000-00-00 00:00:00',
fDateUnsubscribed TIMESTAMP NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (fId),
INDEX (fSubscriberGroupId),
) ENGINE=MyISAM;
Now what I want to accomplish is to have a diagram showing the subscriptions and unsubscriptions per month per subscriber group.
So I need to extract the year and months from the fDateConfirmed, fDateUnsubscribed dates, count them and show the count sorted by month and year for a subscriber group.
I think this sql query gets quite complex and I just can't get my head around it. Is this even possible with one query.
You will need two separate queries, one for subscriptions and other for unsubscriptions.
SELECT COUNT(*), YEAR(fDateConfirmed), MONTH(fDateConfirmed) FROM tSubscriber GROUP BY YEAR(fDateConfirmed), MONTH(fDateConfirmed)
SELECT COUNT(*), YEAR(fDateUnsubscribed), MONTH(fDateUnsubscribe ) FROM tSubscriber GROUP BY YEAR(fDateUnsubscribed), MONTH(fDateUnsubscribed)