I want to fetch the videos that top viewed from the last week,
my client has the following table :
CREATE TABLE `videos` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`album` int(10) unsigned NOT NULL DEFAULT '0',
`name` varchar(225) NOT NULL,
`title` varchar(255) NOT NULL DEFAULT '',
`uploading_user` int(8) NOT NULL DEFAULT '2715',
`host` tinyint(1) unsigned NOT NULL DEFAULT '0',
`host_url` varchar(255) NOT NULL DEFAULT '',
`active` tinyint(1) unsigned NOT NULL DEFAULT '0',
`featured` tinyint(1) unsigned NOT NULL DEFAULT '0',
`date_added` date NOT NULL DEFAULT '0000-00-00',
`view` int(125) NOT NULL,
`rating` int(125) NOT NULL,
`rating_count` int(125) NOT NULL,
`category` varchar(225) NOT NULL,
`genre` varchar(225) NOT NULL,
`playlist` varchar(225) NOT NULL,
`video_image` varchar(225) NOT NULL,
`votecount` int(5) DEFAULT NULL,
`banner_image` varchar(255) NOT NULL DEFAULT 'IB_header_solo4.jpg',
`bg_image` varchar(255) NOT NULL DEFAULT 'IB_bck_hd_sect.jpg',
`bg_color` varchar(255) NOT NULL DEFAULT '#000000',
`user_video` int(1) NOT NULL DEFAULT '0',
`description` varchar(1000) DEFAULT NULL,
`country` varchar(30) DEFAULT NULL,
`city` varchar(50) DEFAULT NULL,
`location` varchar(50) DEFAULT NULL,
`reported` int(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=1531 DEFAULT CHARSET=latin1
in this table date_added field for on which date we have added this video,
and i have updated the table on every video view in the view field, means i have the total views of the particular video.
now what i want, how can i fetch the result of top views videos from last week?
can i have to add another date field on which i add the id of the viewed videos?
or any other alternate solution?
If you want to get a list of videos with the number of views in the last week, you need to store views separately. The views column in the table would only give you access to the total number of views but it does not give any indication as to when those views were generated.
I would use a table with video_id, datetime and number of views to store the views. By using a DATETIME column and the requirement that the combination video_id and DATETIME is unique, you can store views down to the second and create more statistics later.
I think it's best you keep a separate table to store who viewed the videos, adding it to this table you will be limited with the number of people you can associate as viewers. Regarding your query. It would be better if you keep a seperate field that stores weekly views.
assuming you had a field called week_views
SELECT required_fields FROM videos ORDER BY week_views DESC
SELECT * (or required fields) FROM videos
WHERE date_added >= CURDATE() - INTERVAL WEEKDAY(CURDATE()) +7 DAY
AND date_added < CURDATE() - INTERVAL WEEKDAY(CURDATE()) DAY
ORDER BY view DESC
Something like that should select the videos from last week, Monday to Sunday (not necessarily last 7 days.)
Related
I had an mysql event and runs eery day at 9:45 AM.
Begin
SET #v_ym :=(SELECT extract(year_month from DATE_SUB(SYSDATE(),INTERVAL 1 DAY)));
SELECT CAST(#ym AS CHAR);
select ssaname,extract(year_month from date_sub(sysdate(),interval 1 day)) ym,
omcr.btscount_ssa(ssaname) btscount,sum(case when duration>30 then duration else 0 end) dur_30 from
btsoutage.bts_faults
where ym=#v_ym and ssaname is not null
group by ssaname;
END;
in the query [ym is yearmonth and ym is indexed] when i substitute with variable #v_ym it is taking full table scan and the table is locked for further inserts. where as when i given the value directly it is using index and the output is fast.
The table contains more than 10 million records.
Create table is
CREATE TABLE `bts_faults` (
`bts_name` varchar(250) DEFAULT NULL,
`make` varchar(10) DEFAULT NULL,
`occuredtime` datetime DEFAULT NULL,
`clearedtime` datetime DEFAULT NULL,
`duration` int(10) DEFAULT NULL,
`reason` varchar(100) DEFAULT NULL,
`site_type` varchar(10) DEFAULT NULL,
`tech` varchar(5) DEFAULT NULL,
`fault_id` bigint(20) NOT NULL AUTO_INCREMENT,
`ssaname` varchar(20) DEFAULT NULL,
`fault_type` int(1) DEFAULT '0',
`remarks` varchar(250) DEFAULT NULL,
`bts_section` varchar(100) DEFAULT NULL,
`vendor` varchar(50) DEFAULT NULL,
`occureddate` date DEFAULT NULL,
`cleareddate` date DEFAULT NULL,
`ym` varchar(6) DEFAULT NULL,
`updatedate` datetime DEFAULT NULL,
`USERNAME` varchar(100) DEFAULT NULL,
`mask` int(1) DEFAULT '0',
`mask_cat` varchar(10) DEFAULT NULL,
`outage_cat` varchar(20) DEFAULT NULL,
`site_category` varchar(50) DEFAULT NULL,
`escalated_time` datetime DEFAULT NULL,
`zone` varchar(20) DEFAULT NULL,
`zone_fault_reason` varchar(500) DEFAULT NULL,
`zone_fault_remarks` varchar(500) DEFAULT NULL,
`zone_username` varchar(20) DEFAULT NULL,
`zone_updatetime` datetime DEFAULT NULL,
`zone_fault_duration` int(11) DEFAULT NULL,
`fault_category` varchar(250) DEFAULT NULL,
`remarks_1` varchar(2500) DEFAULT NULL,
PRIMARY KEY (`fault_id`),
UNIQUE KEY `UIDX_BTS_FAULTS` (`bts_name`,`occuredtime`),
KEY `indx_btsfaults_ym` (`ym`),
KEY `indx_btsfaults_cleareddate` (`cleareddate`),
KEY `Index_btsfaults_btsname` (`bts_name`),
KEY `index_btsfaults_ssaname` (`ssaname`),
KEY `indx_btsfaults_occureddate` (`occureddate`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=3807469710 DEFAULT CHARSET=latin1
The Explain Plan for the 2 type are
What percentage of the table is in the "current month"? If that is more than something like 20%, then there is no fix -- a table scan is likely to be faster. If it is less than 20%, then, as you suspect, #variables may be the villain. In that case, change the test to be
WHERE ym = CAST(
extract(year_month from DATE_SUB(SYSDATE(),INTERVAL 1 DAY))
AS CHAR)
AND ...
Much faster would be to build and maintain a Summary Table with a PRIMARY KEY of day and ssaname. This would have the subtotals for each day. It would be maintained either as the data is INSERTed or each night after midnight.
Then the 9:45 query becomes very fast. Maybe so fast that you don't even need to do it just once a day, but instead "on-demand".
More discussion: http://mysql.rjweb.org/doc.php/summarytables
I suggest you use NOW() instead of SYSDATE() -- The former is constant throughout a statement; the latter is not.
bts_faults looks like it might be a terabyte in size. If so, you probably don't want to here ways to make is smaller.
If the Auto_inc value is at 3.8B, yet there are only 10M rows, does this mean that you are purging 'old' data? Do you want to discuss speeding up the Deletes? (Start a new Question if you do.)
I have a MySQL query I wrote that displays the data I want it to, but it takes at least 30 secs - 1 min to run.
I researched to find out how to created the nested SELECT query with the COUNT that I needed in order to display the data I required. The SQL is also part of a web page I have, and when I go from page to page it takes the same amount of time to load. I am sure there is a more efficient way to write the query so it loads fast, as there are only about 1,500 records in the ttb_shows table and about 11k in the ttb_books table. Below is the query.
-- DDL
CREATE TABLE `ttb_books` (
`book_id` int(11) NOT NULL,
`book_name` varchar(255) NOT NULL DEFAULT '',
`cover_image` varchar(255) DEFAULT NULL,
`show_id` int(11) NOT NULL DEFAULT '0',
`state_id` int(7) NOT NULL DEFAULT '0',
`notes` text,
`year` varchar(255) DEFAULT NULL,
`publisher` varchar(255) DEFAULT NULL,
`status_id` int(7) NOT NULL DEFAULT '0',
`no_pages` varchar(255) DEFAULT NULL,
`footer` text,
`opt1` varchar(255) NOT NULL DEFAULT '$5-10',
`opt2` varchar(255) DEFAULT NULL,
`opt3` varchar(255) DEFAULT NULL,
`opt4` varchar(255) DEFAULT NULL,
`opt5` varchar(255) DEFAULT NULL,
`owned` int(1) DEFAULT '0'
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
CREATE TABLE `ttb_shows` (
`show_id` int(11) NOT NULL,
`show_name` varchar(255) NOT NULL DEFAULT '',
`date_added` datetime NOT NULL DEFAULT '0000-00-00 00:00:00'
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
-- QUERY
SELECT ttb_shows.show_id, ttb_shows.show_name, ttb_shows.date_added,
COUNT(ttb_books.book_id) AS books,
(SELECT COUNT(ttb_books.owned) AS owned FROM ttb_books WHERE (owned=1 AND ttb_books.show_id = ttb_shows.show_id))
FROM ttb_shows LEFT JOIN ttb_books ON ttb_shows.show_id = ttb_books.show_id
GROUP BY ttb_shows.show_id, ttb_shows.show_name, ttb_shows.date_added
Thank you to all who are able to help with this. It is really appreciated!
You could avoid the subquery for owner using sum based on case
SELECT ttb_shows.show_id
, ttb_shows.show_name
, ttb_shows.date_added
, COUNT(ttb_books.book_id) AS books
, sum( case when ttb_books.owned = 1 then 1 else 0 end) AS owned
FROM ttb_shows
LEFT JOIN ttb_books ON ttb_shows.show_id = ttb_books.show_id
GROUP BY ttb_shows.show_id, ttb_shows.show_name, ttb_shows.date_added
Your query can be optimized by you only. It seems that so many left outer will obviously slow the output. If you can either avoid so many left outers or make small chunks of cases to fetch out data and then fetch the Final output.
Important updade (explanation):
I realized that my query having single DESC order is 10 times slower that the same query with ASC order. The ordered field has an index. Is it normal behavior?
Original question with queries:
I have a mysql table with a few hundred of product items. It's suprising (for me) how 2 similar sql queries differs in terms of performance. I don't know why. Can you please give me a hint or explain why the difference is so huge?
This query takes 3ms:
SELECT
*
FROM
`product_items`
WHERE
(product_items.shop_active = 1)
AND (product_items.active = 1)
AND (product_items.active_category_id is not null)
AND (has_picture is not null)
AND (price_orig is not null)
AND (category_min_discount IS NOT NULL)
AND (product_items.slug is not null)
AND `product_items`.`active_category_id` IN (6797, 5926, 5806, 6852)
ORDER BY
price asc
LIMIT 1
But the following query takes already 169ms... Only difference is that the order clause contains 2 columns. "Price" value has each product, while "price top" has roughly only 1% of products.
SELECT
*
FROM
`product_items`
WHERE
(product_items.shop_active = 1)
AND (product_items.active = 1)
AND (product_items.active_category_id is not null)
AND (has_picture is not null)
AND (price_orig is not null)
AND (category_min_discount IS NOT NULL)
AND (product_items.slug is not null)
AND `product_items`.`active_category_id` IN (6797, 5926, 5806, 6852)
ORDER BY
price asc,
price_top desc
LIMIT 1
The table structure looks like this:
CREATE TABLE `product_items` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`shop_id` int(11) DEFAULT NULL,
`item_id` varchar(255) DEFAULT NULL,
`productname` varchar(255) DEFAULT NULL,
`description` text,
`url` text,
`url_hash` varchar(255) DEFAULT NULL,
`img_url` text,
`price` decimal(10,2) DEFAULT NULL,
`price_orig` decimal(10,2) DEFAULT NULL,
`discount` decimal(10,2) DEFAULT NULL,
`discount_percent` decimal(10,2) DEFAULT NULL,
`manufacturer` varchar(255) DEFAULT NULL,
`delivery_date` varchar(255) DEFAULT NULL,
`categorytext` text,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
`active_category_id` int(11) DEFAULT NULL,
`shop_active` int(11) DEFAULT NULL,
`active` int(11) DEFAULT '0',
`price_top` decimal(10,2) NOT NULL DEFAULT '0.00',
`attention_priority` int(11) DEFAULT NULL,
`attention_priority_over` int(11) DEFAULT NULL,
`has_picture` varchar(255) DEFAULT NULL,
`size` varchar(255) DEFAULT NULL,
`category_min_discount` int(11) DEFAULT NULL,
`slug` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `index_product_items_on_url_hash` (`url_hash`),
KEY `index_product_items_on_shop_id` (`shop_id`),
KEY `index_product_items_on_active_category_id` (`active_category_id`),
KEY `index_product_items_on_productname` (`productname`),
KEY `index_product_items_on_price` (`price`),
KEY `index_product_items_on_discount_percent` (`discount_percent`),
KEY `index_product_items_on_price_top` (`price_top`)
) ENGINE=InnoDB AUTO_INCREMENT=1715708 DEFAULT CHARSET=utf8;
UPDATE
I realized that the difference is mainly in the type of ordering: if I use asc+asc for both columns the query takes around 6ms, if I use asc+desc or desc+asc, the query takes around 160ms..
Thank you.
If creating an index to help ORDER BY doesn't help, try creating an index that helps both WHERE and ORDER BY:
CREATE INDEX product_items_i1 ON product_items (
shop_active,
active,
active_category_id,
has_picture,
price_orig,
category_min_discount,
slug,
price,
price_top DESC
)
Obviously, this is a bit clunky, and you'll have to balance the performance gain for the query with the price of maintaining the index.
I have a start date and an end date. For each day between start and end date I like to get a result, even if there are no results within my stats table.
The best suggestion to solve this is a stored procedure what creates a temporary table for each date (day) so that I could create a join.
BEGIN
declare d datetime;
create temporary table joindates (d date not null);
SET d = fkstartdate;
WHILE d <= fkenddate DO
insert into joindates (d) values (d);
set d = date_add(d, interval 1 day);
END WHILE;
SELECT * FROM `joindates` AS tempt LEFT JOIN `radacct` AS stats
ON stats.acctstarttime <= tempt.d AND (stats.acctstoptime >= tempt.d OR stats.acctstoptime IS NULL) AND calledstationid = calledstationid
ORDER BY tempt.d ASC;
drop temporary table joindates;
END
The join is not finalized now (just for testing). But as you see, the join would be perfect if the "radacct" table would use just one date col. Than I could easily create a JOIN by using the = operator for ON.
Now the "radacct" however is using acctstarttime and acctstoptime for there entries (I can not modify this table, is given by the radius server) both are datetime types.
So in my query I need all entries between my start and stop date where acctstarttime has started in the past or within my time period. And acctstoptime is within my time period, after my time period or NULL (active session).
The problem with my JOIN solution: Its slow as hell
Do you may have any faster query ideas please?
Thanks
Appendix1: It is very important that I get a result for each entry on each day that matches my period of days, even if acctstarttime to acctstoptime goes over multiple days. So that I always get all users online.
Appendix2:
As requested, the full table structure of "radacct" – the accounting table of the freeradius MySql module …
CREATE TABLE `radacct` (
`radacctid` bigint(21) NOT NULL AUTO_INCREMENT,
`acctsessionid` varchar(64) NOT NULL DEFAULT '',
`acctuniqueid` varchar(32) NOT NULL DEFAULT '',
`username` varchar(64) NOT NULL DEFAULT '',
`groupname` varchar(64) NOT NULL DEFAULT '',
`realm` varchar(64) DEFAULT '',
`nasipaddress` varchar(15) NOT NULL DEFAULT '',
`nasportid` varchar(15) DEFAULT NULL,
`nasporttype` varchar(32) DEFAULT NULL,
`acctstarttime` datetime DEFAULT NULL,
`acctstoptime` datetime DEFAULT NULL,
`acctsessiontime` int(12) unsigned DEFAULT NULL,
`acctauthentic` varchar(32) DEFAULT NULL,
`connectinfo_start` varchar(50) DEFAULT NULL,
`connectinfo_stop` varchar(50) DEFAULT NULL,
`acctinputoctets` bigint(20) DEFAULT NULL,
`acctoutputoctets` bigint(20) DEFAULT NULL,
`calledstationid` varchar(50) NOT NULL DEFAULT '',
`callingstationid` varchar(50) NOT NULL DEFAULT '',
`acctterminatecause` varchar(32) NOT NULL DEFAULT '',
`servicetype` varchar(32) DEFAULT NULL,
`framedprotocol` varchar(32) DEFAULT NULL,
`framedipaddress` varchar(15) NOT NULL DEFAULT '',
`acctstartdelay` int(12) unsigned DEFAULT NULL,
`acctstopdelay` int(12) unsigned DEFAULT NULL,
`xascendsessionsvrkey` varchar(10) DEFAULT NULL,
PRIMARY KEY (`radacctid`),
UNIQUE KEY `acctuniqueid` (`acctuniqueid`),
KEY `username` (`username`),
KEY `framedipaddress` (`framedipaddress`),
KEY `acctsessionid` (`acctsessionid`),
KEY `acctsessiontime` (`acctsessiontime`),
KEY `acctstarttime` (`acctstarttime`),
KEY `acctstoptime` (`acctstoptime`),
KEY `nasipaddress` (`nasipaddress`)
) ENGINE=InnoDB AUTO_INCREMENT=2041894 DEFAULT CHARSET=latin1
I was asked to pull records from Match table from 2 Given dates.
Say 2013/01/01 and 2013/02/29.
For 1 Match there are many possible bets when relationship is applied.
Say for a Match that Started around 2013/01/05, the bets relating to it
are bets that was made on or before 2013/01/05
What's weird is my boss would like to get also the records before the Match Date occured. Coz I already made a page that allows him to view the bet of a certain match.
coz in the first place the page is a Match page, so the search should be within the Match data right?
My idea is use something like
SELECT * from betdb
WHERE DateTime <= '2013/01/01 00:00:00' AND DateTime <= '2013/02/29 23:59:00'
But it just yields only 1 record that has a Date
1931-01-29 00:00:00
which was possibly typo error.
Is there a better way on pulling those kind of records that occured before the supplied DateTime?
Match table definition
CREATE TABLE IF NOT EXISTS `matchdb` (
`MatchID` int(11) NOT NULL AUTO_INCREMENT,
`BookID` int(11) NOT NULL,
`Category` varchar(40) NOT NULL,
`MatchName` varchar(60) NOT NULL,
`StartTime` datetime NOT NULL,
`Result1` varchar(20) DEFAULT NULL,
`Result2` varchar(20) DEFAULT NULL,
`Result3` varchar(20) DEFAULT NULL,
PRIMARY KEY (`MatchID`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 AUTO_INCREMENT=1 ;
Bet table definition
CREATE TABLE IF NOT EXISTS `betdb` (
`BetID` int(11) NOT NULL,
`BookID` int(10) NOT NULL,
`PlayerID` int(10) NOT NULL,
`DateTime` datetime NOT NULL,
`MatchID` int(10) NOT NULL,
`BetType` varchar(40) NOT NULL,
`Bet` varchar(40) NOT NULL,
`BetAmount` float NOT NULL,
`Odds` float NOT NULL,
`BetResult` varchar(40) NOT NULL,
`Payout` float NOT NULL,
PRIMARY KEY (`BetID`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 AUTO_INCREMENT=1;