MySQL query optimization, VERY SLOW - mysql

This is what I would like to achieve:
Show 50 last students and their cheapest products.
If product doesn't exist present empty values.
Here is the SELECT query:
SELECT
students.*,
cs.cheapest_id,
cs.cheapest_price
FROM students
LEFT JOIN (SELECT iqs.* FROM (
SELECT
student_id,
id AS cheapest_id,
price AS cheapest_price
FROM products
ORDER BY price ASC
) AS iqs
GROUP BY iqs.student_id) AS cs ON cs.student_id = students.id
ORDER BY students.name DESC
LIMIT 50;
Creating tables:
CREATE TABLE `students` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=3 DEFAULT CHARSET=utf8;
INSERT INTO `students` VALUES ('1', 'Mark');
INSERT INTO `students` VALUES ('2', 'Chris');
CREATE TABLE `products` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`student_id` int(11) DEFAULT NULL,
`price` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `student_id` (`student_id`)
) ENGINE=MyISAM AUTO_INCREMENT=4 DEFAULT CHARSET=utf8;
INSERT INTO `products` VALUES ('1', '1', '2');
INSERT INTO `products` VALUES ('2', '1', '3');
Result:
id name cheapest_id cheapest_price
1 Mark 1 2
2 Chris (NULL) (NULL)
Here is the problem:
If there are many records in both tables query is very slow(minutes).
If I use INNER JOIN instead of LEFT JOIN or if I remove "ORDER BY students.name DESC" query is fast.
I have set index on student_id but still it is very slow.
Can anyone please help? I've been struggling with this for days...
Edit: The result of EXPLAIN
1 PRIMARY students ALL 3 Using temporary; Using filesort
1 PRIMARY <derived2> ALL 2
2 DERIVED <derived3> ALL 4 Using temporary; Using filesort
3 DERIVED products ALL 4 Using filesort

Try adding these indexes:
ALTER TABLE `students` ADD INDEX (`id`, `name`);
ALTER TABLE `products` ADD INDEX (`student_id`, `id`, `price`);
and write your query like this:
SELECT students.*,
MIN(products.id) AS cheapest_id,
MIN(products.price) AS cheapest_price
FROM students
LEFT JOIN products ON student_id = students.id
GROUP BY students.id
ORDER BY students.name DESC
LIMIT 50;
You should definately read about the EXPLAIN SELECT syntax and how to index properly:
https://dev.mysql.com/doc/refman/5.6/en/explain.html
https://dev.mysql.com/doc/refman/5.5/en/optimization-indexes.html
With EXPLAIN SELET %YOUR QUERY% you can see how MySQL is optimizing the query and how the engine is actually reading your datasets.

Related

MySQL: JOIN and WHERE with multiple matches

I would like to select products from products table with the attributes with id 2 and 5 using the following query:
SELECT `products`.`title`, `products`.`price`
FROM `products`
LEFT JOIN `products_attributes_mapping`
ON `products`.`id` = `products_attributes_mapping`.`product_id`
WHERE
`products_attributes_mapping`.`attribute_value_id` IN (2)
AND `products_attributes_mapping`.`attribute_value_id` IN (5)
GROUP BY `products`.`id`
I expect the product 'Example product 1, blue, size 1' to be returned. But I don't get any result, even though the product with id 1 has attribute_value_id 2 and 5 assigned in the products_attributes_mapping table.
I use IN because I would like to be able to provide multiple attributes, I simplified it only for the example.
SQL fiddle: http://sqlfiddle.com/#!9/2fd94f2/1/0
Schema
CREATE TABLE `products` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`title` varchar(255) CHARACTER SET utf8 NOT NULL,
`price` double NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=5 DEFAULT CHARSET=utf8mb4;
CREATE TABLE `products_attributes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=utf8mb4;
CREATE TABLE `products_attributes_mapping` (
`product_id` int(11) NOT NULL,
`attribute_value_id` int(11) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
CREATE TABLE `products_attributes_values` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`attribute_id` int(11) NOT NULL,
`name` varchar(255) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=9 DEFAULT CHARSET=utf8mb4;
Data
INSERT INTO `products` VALUES
(1,'Example product 1, blue, size 1',10),
(2,'Example product 2, yellow, size 1',10),
(3,'Example product 3, black, size 2',15),
(4,'Example product 4, red, size 2',15);
INSERT INTO `products_attributes` VALUES
(1,'Color'),
(2,'Size');
INSERT INTO `products_attributes_mapping` VALUES
(1,2),
(1,5),
(2,4),
(2,5),
(3,3),
(3,6),
(4,1),
(4,6);
INSERT INTO `products_attributes_values` VALUES
(1,1,'red'),
(2,1,'blue'),
(3,1,'black'),
(4,1,'yellow'),
(5,2,'1'),
(6,2,'2'),
(7,2,'3'),
(8,2,'4');
Using aggregation could indeed be a solution. You can use a HAVING clause to ensure that a products has certain attribute values:
SELECT p.title, p.price
FROM products p
INNER JOIN products_attributes_mapping pm ON p.id = pm.product_id
GROUP BY p.id, p.title, p.price
HAVING
MAX(pm.attribute_value_id = 2) = 1
AND MAX(pm.attribute_value_id = 5) = 1
In your DB fiddle, this query returns:
title | price
---------------------------------|-------
Example product 1, blue, size 1 | 10
You can easily extend the expression by adding more AND MAX(...) = 1 conditions.
Another option would be to use a series of WHERE EXISTS conditions with correlated subqueries to search the attributes tables. This is just as good, but will expand as a longer query if you need to add many conditions.

Mysql sort alphanumeric data

I have two tables tt1 and tt2 both contains same fields. I want to sort data by no from both table.
Table : tt1
CREATE TABLE IF NOT EXISTS `tt1` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`no` varchar(5) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=4 ;
INSERT INTO `tt1` (`id`, `no`) VALUES
(1, '1A'),
(2, '3A'),
(3, '2A');
Table : tt2
CREATE TABLE IF NOT EXISTS `tt2` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`no` varchar(5) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=4 ;
INSERT INTO `tt2` (`id`, `no`) VALUES
(1, '2A'),
(2, '3A'),
(3, '1A');
Expected output
ID | No
========
1 | 1A
3 | 1A
1 | 2A
3 | 2A
2 | 3A
2 | 3A
I want to ascending order of no field from both table as given output how I can get.
SQLFiddle
In this case using order by after every select is wrong (won't return desired output), because that will order both row-sets separately and union them after that.
What you want here is to order already combined data, therefore you should be using order by only once, after mysql makes union of tables (i.e. combines), because once it combines tables it has all data together but unordered, so when mysql sees order by it orders whole data for you.
Sample:
select * from `tt1`
union all
select * from `tt2`
order by `no`
Note: I've noticed you have wrong syntax in your fiddle. You need to add parentheses:
(select * from tt1 order by no)
union
(select * from tt2 order by no)
Note 2: Thanks #AlmaDo's notice. You should not use * with union queries. Because modification of your tables columns will break query. Use column names that you actually need. E.g. query in sample becomes:
select `id`, `no` from `tt1`
union all
select `id`, `no` from `tt2`
order by `no`

mysql select from table 1 join and order by table 2 , optimization filesort

read mysql post but couldnt find the solution.
Any pointers would be appreciated.
I have 2 tables A,B
Table A has folderid mapped to multiple feedid (1 to many )
Table B has feedid and data associated with it.
The following query gives Using index; Using temporary; Using filesort
SELECT A.feed_id,B.feed_id,B.entry_id
FROM feed_folder A
LEFT JOIN feed_data B on A.feed_id=B.feed_id
WHERE A.folder_id=29
AND B.entry_id <= 123
AND B.entry_created <= '2012-11-01 21:38:54'
ORDER by B.entry_created desc limit 0,20;
Any ideas as to how the temporary filesort can be avoided.
Following is the table structure
CREATE TABLE `feed_folder` (
`folder_id` bigint(20) unsigned NOT NULL,
`feed_id` bigint(20) unsigned NOT NULL,
UNIQUE KEY `folder_id` (`folder_id`,`feed_id`),
KEY `feed_id` (`feed_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
CREATE TABLE `feed_data` (
`entry_id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`feed_id` bigint(20) unsigned NOT NULL,
`entry_created` datetime NOT NULL,
`entry_object` text,
`entry_permalink` varchar(150) NOT NULL,
`entry_orig_created` datetime NOT NULL,
PRIMARY KEY (`entry_id`),
UNIQUE KEY `feed_id_2` (`feed_id`,`entry_permalink`),
KEY `entry_created` (`entry_created`),
KEY `feed_id` (`feed_id`,`entry_created`),
KEY `feed_id_entry_id` (`feed_id`,`entry_id`),
KEY `entry_permalink` (`entry_permalink`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
Try this query:
SELECT * FROM(SELECT
A.feed_id,B.feed_id,B.entry_id
FROM feed_folder as A
INNER JOIN feed_data as B on A.feed_id=B.feed_id
WHERE A.folder_id=29
AND B.entry_id <= 123
AND B.entry_created <= '2012-11-01 21:38:54')joinAndSortTable
ORDER by B.entry_created desc limit 0,20;
The problem is in alias, you are forgot to give as A and as B
i'm also remind you about LEFT JOIN and RIGHT JOIN, INNER JOIN...
To join all records what you selected, Using INNER JOIN is a workaround.
Because INNER JOIN can join all columns with full accessed.
GOOD LUCK

Slow query with multiple where and order by clauses

I'm trying to find a way to speed up a slow (filesort) MySQL query.
Tables:
categories (id, lft, rgt)
questions (id, category_id, created_at, votes_up, votes_down)
Example query:
SELECT * FROM questions q
INNER JOIN categories c ON (c.id = q.category_id)
WHERE c.lft > 1 AND c.rgt < 100
ORDER BY q.created_at DESC, q.votes_up DESC, q.votes_down ASC
LIMIT 4000, 20
If I remove the ORDER BY clause, it's fast. I know MySQL doesn't like both DESC and ASC orders in the same clause, so I tried adding a composite (created_at, votes_up) index to the questions table and removed q.votes_down ASC from the ORDER BY clause. That didn't help and it seems that the WHERE clause gets in the way here because it filters by columns from another (categories) table. However, even if it worked, it wouldn't be quite right since I do need the q.votes_down ASC condition.
What are good strategies to improve performance in this case? I'd rather avoid restructuring the tables, if possible.
EDIT:
CREATE TABLE `categories` (
`id` int(11) NOT NULL auto_increment,
`lft` int(11) NOT NULL,
`rgt` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `lft_idx` (`lft`),
KEY `rgt_idx` (`rgt`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE `questions` (
`id` int(11) NOT NULL auto_increment,
`category_id` int(11) NOT NULL,
`votes_up` int(11) NOT NULL default '0',
`votes_down` int(11) NOT NULL default '0',
`created_at` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `questions_FI_1` (`category_id`),
KEY `votes_up_idx` (`votes_up`),
KEY `votes_down_idx` (`votes_down`),
KEY `created_at_idx` (`created_at`),
CONSTRAINT `questions_FK_1` FOREIGN KEY (`category_id`) REFERENCES `categories` (`id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE q ALL questions_FI_1 NULL NULL NULL 31774 Using filesort
1 SIMPLE c eq_ref PRIMARY,lft_idx,rgt_idx PRIMARY 4 ttt.q.category_id 1 Using where
Try a subquery to get the desired categories:
SELECT * FROM questions
WHERE category_id IN ( SELECT id FROM categories WHERE lft > 1 AND rgt < 100 )
ORDER BY created_at DESC, votes_up DESC, votes_down ASC
LIMIT 4000, 20
Try selecting only what you need in your query, instead of the SELECT *
Why not to use SELECT * ( ALL ) in MySQL
Try putting conditions, concerning joined tables into ON clauses:
SELECT * FROM questions q
INNER JOIN categories c ON (c.id = q.category_id AND c.lft > 1 AND c.rgt < 100)
ORDER BY q.created_at DESC, q.votes_up DESC, q.votes_down ASC
LIMIT 4000, 20

GROUP BY query optimization

Dears,
I need your help to optimize the below query. I have two tables, one for storing books data and the second table for mapping the books to tags related to it. I want to count how many books form a certain publisher in every category. This query do the job but I need to optimize it:
select count(book.id),publisher from book,tag where book.id=tag.book_id AND publisher ='Addison-Wesley Professional' AND tag.name='PHP' group by category
the result of explain is
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tag ref PRIMARY PRIMARY 92 const 1 Using where; Using index; Using temporary; Using f...
1 SIMPLE book eq_ref PRIMARY PRIMARY 4 test.tag.book_id 1 Using where
the tables are:
--
-- Table structure for table `book`
--
CREATE TABLE `book` (
`id` int(11) NOT NULL auto_increment,
`name` varchar(30) NOT NULL,
`ISBN` varchar(10) NOT NULL,
`category` varchar(30) NOT NULL,
`publisher` varchar(30) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=3 ;
--
-- Dumping data for table `book`
--
INSERT INTO `book` VALUES (1, 'PHP and MySQL Web Development', '9780672329', 'Web Development', 'Addison-Wesley Professional');
INSERT INTO `book` VALUES (2, 'JavaScript Patterns', '0596806752', 'Web Development', 'O''Reilly Media');
--
-- Table structure for table `tag`
--
CREATE TABLE `tag` (
`name` varchar(30) NOT NULL,
`book_id` int(11) NOT NULL,
PRIMARY KEY (`name`,`book_id`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
--
-- Dumping data for table `tag`
--
INSERT INTO `tag` VALUES ('MySQL', 1);
INSERT INTO `tag` VALUES ('PHP', 1);
You need a composite index on (publisher, category) in your book table, in that exact order, so the subquery doing grouping on category can restrict results fast to those having exact publisher and then use second part of the index to group on category.
ALTER TABLE book ADD INDEX publ_cat( publisher, category );