fix Using index, Using temporary, Using filesort - mysql

i have such 3 tables. Im trying to Left Join them
SELECT `t`.`title` AS `category_title`,`t`.`id` AS `category_id`, `st`.`title` AS
`subcategory_title`, `st`.`id` AS `subcategory_id`, `st`.`parent_id` AS
`subcategory_parent`, `n`.`title` AS `news_title`,`n`.`id` AS `news_id` FROM
`t_categories` `t` LEFT JOIN t_categories AS `st` ON `st`.`parent_id`=t.`id` LEFT JOIN
t_newsrelations AS `nr` ON `nr`.`category_id`=st.`id` LEFT JOIN t_news AS `n` ON
`n`.`id`=nr.`news_id` WHERE `t`.`enabled` = 1 AND `n`.`enabled` = 1 AND `n`.`type`!=1 AND
`n`.`type`!=5 ORDER BY `t`.`position`,`st`.`position`,`n`.`position` ASC
here is the structure of tables
CREATE TABLE IF NOT EXISTS `t_categories` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`parent_id` int(11) NOT NULL,
`title` tinytext NOT NULL,
`position` tinyint(4) unsigned NOT NULL DEFAULT '0',
`type` tinyint(1) unsigned NOT NULL,
`enabled` tinyint(1) unsigned NOT NULL DEFAULT '1',
UNIQUE KEY `id` (`id`),
KEY `type` (`type`),
KEY `parent_id` (`parent_id`),
KEY `enabled` (`enabled`),
KEY `id_parent_position_enabled` (`id`,`parent_id`,`position`,`enabled`),
KEY `position` (`position`),
KEY `parent_id_2` (`parent_id`,`enabled`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `t_news` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`title` tinytext NOT NULL,
`m_title` tinytext NOT NULL,
`url` varchar(2000) NOT NULL,
`keywords` text NOT NULL,
`description` text NOT NULL,
`body` longtext NOT NULL,
`position` tinyint(4) unsigned NOT NULL DEFAULT '0',
`type` tinyint(1) unsigned NOT NULL,
`city_id` int(4) NOT NULL,
`quickmenu_enabled` tinyint(1) unsigned NOT NULL DEFAULT '0',
`quickmenu` text NOT NULL,
`enabled` tinyint(1) unsigned NOT NULL DEFAULT '1',
PRIMARY KEY (`id`),
UNIQUE KEY `id` (`id`),
KEY `position` (`position`),
KEY `type` (`type`),
KEY `city_id` (`city_id`),
KEY `url` (`url`(333)),
KEY `quickmenu_enabled` (`quickmenu_enabled`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `t_newsrelations` (
`category_id` int(11) NOT NULL,
`news_id` int(11) unsigned NOT NULL,
KEY `category_id` (`category_id`),
KEY `news_id` (`news_id`),
KEY `category_id_2` (`category_id`,`news_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
And SELECT EXPLAIN shows me
t_newsrelations is intermediate table. Table t_categories contains categories and subcategories linked by parent_id column. Each item from t_news can be a member of more than one subcategories thats why they linked through t_newsrelations
how to optimise a query? Why it shows Using index, Using temporary, Using filesort?

ORDER BY `t`.`position`,`st`.`position`,`n`.`position` ASC
You can't eliminate the temp table and filesort in this query, given the tables you have, because you're sorting on columns from multiple tables. Optimizing sorting means using an index so that the query fetches rows in the order you want them. But there's no way in MySQL to create an index that spans multiple tables.
The only way to fix this is to denormalize until all three columns are in a single table, and create one index over the three columns. But denormalization comes with its own downsides.

Related

MySql Using filesort when i using a group by

I have a little problem with optimizing a query, I have 2 tables, one which records the participation (participation) in a quiz, and the other which records the answer to each question (participation_rep), participation is linked to the campaign table.
SELECT count(DISTINCT p.id) as number_of_participation
FROM participation_rep prep
INNER JOIN participation p
ON p.id = prep.id_participation
AND p.trash <> 1
WHERE prep.id_question IN (780,787,794,801,809)
AND prep.trash <> 1
GROUP BY pp.id_campaign
Explain of the query
And the problem is that this request is very heavy to execute when there is a lot of data which is concerned by the request and I do not know how to optimize it.
This query take 30-50ms to execute.
Structure of table participation :
CREATE TABLE IF NOT EXISTS `participation` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_campagne` int(11) NOT NULL,
`id_identifiant` int(11) DEFAULT NULL,
`firstname` varchar(255) DEFAULT NULL,
`surname` varchar(255) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`date_p` date NOT NULL,
`hour_p` time NOT NULL,
`comment` text,
`trash` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Structure of table participation_rep :
CREATE TABLE IF NOT EXISTS `participation_rep` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`id_participation` int(11) NOT NULL,
`id_question` int(11) NOT NULL,
`id_rep` int(11) NOT NULL,
`trash` tinyint(1) DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `id_participation` (`id_participation`,`id_question`,`id_reponse`),
KEY `id_question` (`id_question`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;

Min, MAx SQL query optimization

I have the following query and I really need help optimizing it:
SELECT min(p.price) as min_price, max(p.price) as max_price
FROM product p
INNER JOIN product_category pc
ON p.id_product = pc.id_product AND
p.id_project = 1 AND
pc.id_category = 2 AND
p.active = 1
The tables are:
CREATE TABLE `product` (
`id_product` bigint(10) UNSIGNED NOT NULL,
`id_project` int(11) NOT NULL,
`reference` varchar(50) NOT NULL,
`reference_internal` varchar(125) NOT NULL,
`sku` varchar(50) NOT NULL,
`price` float(12,2) NOT NULL,
`old_price` float NOT NULL,
`reduction_amount` float NOT NULL,
`reduction_percent` float NOT NULL,
`is_reduced` int(1) NOT NULL,
`id_manufacturer` int(10) NOT NULL,
`id_supplier` int(10) NOT NULL,
`is_new` int(1) NOT NULL,
`popularity` int(1) NOT NULL,
`quantity` int(1) NOT NULL,
`active` int(1) NOT NULL,
`date_add` datetime NOT NULL,
`date_upd` datetime NOT NULL,
`id_category` int(11) NOT NULL,
`indexed` int(1) NOT NULL,
`id_color` int(13) NOT NULL,
`rating` int(1) NOT NULL,
`rating_count` int(11) NOT NULL,
`viewed` int(5) NOT NULL,
`a_id_product` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
ALTER TABLE `product`
ADD PRIMARY KEY (`id_product`),
ADD KEY `id_product` (`id_product`),
ADD KEY `reference` (`reference`),
ADD KEY `reference_internal` (`reference_internal`),
ADD KEY `sku` (`sku`),
ADD KEY `reduction_amount` (`reduction_amount`),
ADD KEY `reduction_percent` (`reduction_percent`),
ADD KEY `is_reduced` (`is_reduced`),
ADD KEY `id_manufacturer` (`id_manufacturer`),
ADD KEY `id_supplier` (`id_supplier`),
ADD KEY `is_new` (`is_new`),
ADD KEY `popularity` (`popularity`),
ADD KEY `date_add` (`date_add`),
ADD KEY `date_upd` (`date_upd`),
ADD KEY `id_category` (`id_category`),
ADD KEY `indexed` (`indexed`),
ADD KEY `id_color` (`id_color`),
ADD KEY `price` (`price`),
ADD KEY `rating` (`rating`),
ADD KEY `a_id_product` (`a_id_product`),
ADD KEY `t7` (`price`,`id_product`,`id_project`,`active`) USING BTREE;
-- AUTO_INCREMENT for dumped tables
-- AUTO_INCREMENT for table `product`
ALTER TABLE `product`
MODIFY `id_product` bigint(10) UNSIGNED NOT NULL AUTO_INCREMENT;
COMMIT;
There product_categories table:
CREATE TABLE `product_category` (
`id_product` int(10) DEFAULT NULL,
`id_project` int(11) NOT NULL,
`id_category` int(5) NOT NULL,
`menu_order` int(5) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
--
-- Indexes for dumped tables
--
--
-- Indexes for table `product_category`
--
ALTER TABLE `product_category`
ADD KEY `id_product` (`id_product`),
ADD KEY `id_project` (`id_project`),
ADD KEY `id_category` (`id_category`),
ADD KEY `menu_order` (`menu_order`),
ADD KEY `t1` (`id_product`,`id_category`),
ADD KEY `t2` (`id_product`,`id_project`,`id_category`);
COMMIT;
The explain statement https://snag.gy/8ClFnh.jpg
I have 45 MS execution time but the database is not very large, i have around 20.000 products but when it's live i will have million of products and the 0.0550 seconds execution time seems very long for this number of products, anyone can help with an optimization suggestion?
I have mysql 5.7.
Your query should be pretty easy to optimize, if the number of rows you are selecting is low (0.5% or less of the total).
The indexes below should help:
create index ix1_product on product (id_project, active, price);
create index ix2_procat on product_category (id_product, id_category);

Large MySQL table slow to query column with unique index

I have a large MySQL table (36 million rows, 120 GB) that is unable to handle a simple query on an column with a UNIQUE KEY. Ex:
select * from items where item_id = 12345;
Is there some reason why the index isn't helping here or am I just way beyond what MySQL can handle in terms of table size? Any pointers?
Edit: My table create statement:
CREATE TABLE `items` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`product_sku` int(11) DEFAULT NULL,
`item_id` varchar(19) NOT NULL DEFAULT '',
`title` tinytext NOT NULL,
`subtitle` tinytext,
`description` text,
`category_id` varchar(10) NOT NULL DEFAULT '',
`created_at` datetime NOT NULL,
`updated_at` datetime DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `itemId` (`item_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;

Perf of select mysql query is really bad

I'm not sure why this query is taking 4 minutes to complete:
SELECT
su.sid,u.uid,u.display_name,u.locale
FROM user u
LEFT JOIN subscription_user su ON su.uid = u.uid
ORDER BY u.display_name DESC
LIMIT 0,25;
Well, I know it's due to the order, remove it and it's very fast. If I change to using INNER JOIN instead it's fast but the issue is not all users may be in the subscription_user table.
CREATE TABLE `user` (
`uid` int(11) NOT NULL AUTO_INCREMENT,
`password` varchar(100) DEFAULT NULL,
`user_type` varchar(10) NOT NULL DEFAULT 'user',
`display_name` varchar(50) NOT NULL,
`email` varchar(100) NOT NULL,
`locale` varchar(8) DEFAULT 'en',
`last_login` datetime DEFAULT NULL,
`auth_type` varchar(10) DEFAULT NULL,
`auth_data` varchar(500) DEFAULT NULL,
`inactive` tinyint(4) NOT NULL DEFAULT '0',
`receive_email` tinyint(4) NOT NULL DEFAULT '1',
`stateid` int(10) DEFAULT NULL,
`owner_group_id` int(11) DEFAULT NULL,
`signature` varchar(500) DEFAULT NULL,
`raw_signature` varchar(500) DEFAULT NULL,
`round_robin` smallint(5) unsigned NOT NULL DEFAULT '0',
PRIMARY KEY (`uid`),
UNIQUE KEY `email` (`email`),
KEY `stateid` (`stateid`) USING BTREE,
KEY `user_type` (`user_type`) USING BTREE,
KEY `name` (`display_name`)
) ENGINE=InnoDB AUTO_INCREMENT=28343 DEFAULT CHARSET=latin1;
CREATE TABLE `subscription_user` (
`sid` varchar(50) NOT NULL,
`uid` int(11) NOT NULL,
`deleted` tinyint(4) NOT NULL DEFAULT '0',
`forum_user` varchar(50) NOT NULL,
PRIMARY KEY (`sid`,`uid`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
When you have an SQL query, the index can only really help you if the first column in the index is part of the query.
Your query joins su.uid = u.uid and the optimizer will not be able to use that to reference the first column in the subscription primary key index.
You should either reverse the order of the columns in the primary key, or alternatively, you should add a foreign key index, or an independent index on the uid

SQL query; inner join on 4 tables

Is this the most efficient way of joining these 4 tables? Also is it possible to only have some rows of each tables selected? I tried changing * to a name of a column but only the columns from studentlist are allowed.
SELECT c.classID, c.instrument, c.grade, u.ID, u.firstname, u.lastname, u.lastsongplayed, u.title
FROM studentlist s
INNER JOIN classlist c ON s.listID = c.classID
INNER JOIN (
SELECT *
FROM users u
INNER JOIN library l ON u.lastsongplayed = l.fileID
)
u ON s.studentID = u.ID
WHERE teacherID =3
ORDER BY classID
LIMIT 0 , 30
Database structure:
CREATE TABLE IF NOT EXISTS `classlist` (
`classID` int(11) NOT NULL AUTO_INCREMENT,
`teacherID` int(11) NOT NULL,
`instrument` text,
`grade` int(11) DEFAULT NULL,
PRIMARY KEY (`classID`),
KEY `teacherID_2` (`teacherID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=27 ;
CREATE TABLE IF NOT EXISTS `studentlist` (
`listID` int(11) NOT NULL,
`studentID` int(11) NOT NULL,
KEY `teacherID` (`studentID`),
KEY `studentID` (`studentID`),
KEY `listID` (`listID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `users` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`email` varchar(60) NOT NULL,
`password` varchar(60) NOT NULL,
`firstname` text NOT NULL,
`lastname` text NOT NULL,
`sessionID` varchar(60) DEFAULT NULL,
`lastlogin` time DEFAULT NULL,
`registerdate` date NOT NULL,
`isteacher` tinyint(1) DEFAULT NULL,
`isstudent` tinyint(1) DEFAULT NULL,
`iscomposer` tinyint(1) DEFAULT NULL,
`lastsongplayed` int(11) NOT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `ID` (`ID`),
UNIQUE KEY `email` (`email`,`sessionID`),
KEY `ID_2` (`ID`),
KEY `ID_3` (`ID`),
KEY `lastsongplayed` (`lastsongplayed`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=63 ;
CREATE TABLE IF NOT EXISTS `library` (
`fileID` int(11) NOT NULL AUTO_INCREMENT,
`userID` int(11) NOT NULL,
`uploaddate` datetime NOT NULL,
`title` varchar(60) NOT NULL,
`OrigComposer` varchar(60) NOT NULL,
`composer` varchar(60) NOT NULL,
`genre` varchar(60) DEFAULT NULL,
`year` year(4) DEFAULT NULL,
`arrangement` varchar(60) DEFAULT NULL,
PRIMARY KEY (`fileID`),
KEY `userID` (`userID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=77 ;
Is this the most efficient way of joining these 3 tables?
Your JOIN looks correct and you are joining on your keys. So this should be efficient. However, I would encourage you to analyze your query with EXPLAIN to determine additional optimizations.
Is it possible to only have some rows of each tables selected?
Yes. Change * to be the columns from each table you want. I encourage you to explicitly prefix them with the originating table. Depending on the columns you select, this could also make your query more performant.
SELECT studentlist.studentID, users.email FROM ...