I have tables with following structure:
-- Table structure for table `temp_app`
--
CREATE TABLE IF NOT EXISTS `temp_app` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`vid` int(5) NOT NULL,
`num` varchar(64) NOT NULL,
`timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `vid` (`vid`),
KEY `num` (`num`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=69509;
-- Table structure for table `inv_flags`
--
CREATE TABLE IF NOT EXISTS `inv_flags` (
`num` varchar(64) NOT NULL,
`vid` int(11) NOT NULL,
`f_special` tinyint(1) NOT NULL, /*0 or 1*/
`f_inserted` tinyint(1) NOT NULL, /*0 or 1*/
`f_notinserted` tinyint(1) NOT NULL, /*0 or 1*/
`userID` int(11) NOT NULL,
`timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
KEY `num` (`num`),
KEY `userID` (`userID`),
KEY `vid` (`vid`),
KEY `timestamp` (`timestamp`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Execution time of the following query is 9 seconds to display 30 records. What is wrong?
SELECT date_format(ifs.`timestamp`,'%y/%m/%d') as `date`
,count(DISTINCT ta.num) as inserted /*Unique nums*/
,SUM(ifs.f_notinserted) as not_inserted
,SUM(ifs.f_special) as special
,count(ta.num) as links /*All nums*/
from inventory_flags ifs
LEFT JOIN temp_app ta ON ta.num = ifs.num AND ta.vid = ifs.vid
WHERE ifs.userID = 3
GROUP BY date(ifs.`timestamp`) DESC LIMIT 30
EXPLAIN RESULT
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ifs ref userID userID 4 const 12153 Using where
1 SIMPLE ta ref vid,num num 194 ifs.num 1
COUNT DISTINCT can sometimes cause rotten performance with MySql. Try this instead:
select count(*) from (select distinct...
as it can sometimes prevent MySql from writing the entire interim result to disk.
Here is the MySql bug info:
http://bugs.mysql.com/bug.php?id=21849
Related
I have the following query:
SELECT id, user_id, cookieId, text_date
FROM `_history`
WHERE text_date BETWEEN '2014-09-01' AND '2014-10-01' AND user_id = 1
GROUP BY cookieId
ORDER BY id DESC
My table schema:
CREATE TABLE `_history` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`cookieId` varchar(50) NOT NULL,
`text_from` varchar(50) NOT NULL,
`text_body` text NOT NULL,
`text_date` datetime NOT NULL,
`aName` varchar(50) NOT NULL,
`hasArrived` enum('0','1') NOT NULL COMMENT,
`agent_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `cookieId` (`cookieId`),
KEY `user_id` (`user_id`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
An EXPLAN yields this:
1 SIMPLE _history ref cookieId,user_id user_id 4 const 49837 Using where; Using temporary; Using filesort
Sometimes the query takes 2 seconds and sometimes its up to 5s.
Any ideas how to make this run faster?
The group by does nothing at the moment so drop it.
The user_id already has an index on it, so the query and sort on it are fine.
The text_date has no index on it, adding an index on it should speed up your query.
If this query occurs often, add a composite index on both user_id and text_date.
eg.
create index idx_text_date on `_history` (text_date);
Based on the comments, the query should look like this:
SELECT cookieId, max(id), max(user_id), max(text_date)
FROM `_history`
WHERE text_date BETWEEN '2014-09-01' AND '2014-10-01'
AND user_id = 1
GROUP BY cookieId
ORDER BY id DESC
And the index should look like this:
create index idx__history_text_date_cookieId on `_history` (text_date, cookieId);
Create a composite index on (user_id, cookieId).
CREATE TABLE `_history` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`cookieId` varchar(50) NOT NULL,
`text_from` varchar(50) NOT NULL,
`text_body` text NOT NULL,
`text_date` datetime NOT NULL,
`aName` varchar(50) NOT NULL,
`hasArrived` enum('0','1') NOT NULL COMMENT,
`agent_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `cookieId` (`cookieId`),
KEY `user_id_X_cookieId` (`user_id`, `cookieId`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
It will then be able to use the user_id index to find the rows, and use the cookieId suffix of that index to group them.
When you have this index, you don't need the user_id index, because a prefix of a composite index can be used as an index.
I have a database with the records near about 2.7 milion . I need to fetch records from that for that i am using the below query
for result
SELECT r3.original_image_title,r3.uuid,r3.original_image_URL FROM `image_attributes` AS r1 INNER JOIN `filenames` as r3 WHERE r1.`uuid` = r3.`uuid` and r3.`status` = 1 and r1.status=1 and (r1.`attribute_name` like "Quvenzhané Wallis%" or r3.original_image_URL like "Quvenzhané Wallis%") group by r3.`uuid` limit 0,20
for total count
SELECT count(DISTINCT(r1.`uuid`)) as count FROM `image_attributes` AS r1 INNER JOIN `filenames` as r3 WHERE r1.`uuid` = r3.`uuid` and r3.`status` = 1 and r1.status=1 and (r1.`attribute_name` like "Quvenzhané Wallis%" or r3.original_image_URL like "Quvenzhané Wallis%")
table structures are as below
CREATE TABLE IF NOT EXISTS `image_attributes` (
`index` int(11) NOT NULL AUTO_INCREMENT,
`attribute_name` text NOT NULL,
`attribute_type` varchar(255) NOT NULL,
`uuid` varchar(255) NOT NULL,
`status` tinyint(1) NOT NULL DEFAULT '1',
PRIMARY KEY (`index`),
KEY `attribute_type` (`attribute_type`),
KEY `uuid` (`uuid`),
KEY `status` (`status`),
KEY `attribute_name` (`attribute_name`(50))
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=2730431 ;
CREATE TABLE IF NOT EXISTS `filenames` (
`index` int(11) NOT NULL AUTO_INCREMENT,
`original_image_title` text NOT NULL,
`original_image_URL` text NOT NULL,
`uuid` varchar(255) NOT NULL,
`status` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`index`),
KEY `uuid` (`uuid`),
KEY `original_image_URL` (`original_image_URL`(50))
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=591967 ;
please suggest me how can i optimize the queries to make the search result faster
I would recommend to you a book called 'High Performance MySql'. There is a section called Optimize databases and queries, or something like that.
I have a table with about 700.000 rows:
CREATE TABLE IF NOT EXISTS `ext_log_entries` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`action` varchar(8) NOT NULL,
`logged_at` datetime NOT NULL,
`object_id` varchar(32) DEFAULT NULL,
`object_class` varchar(255) NOT NULL,
`version` int(11) NOT NULL,
`data` longtext COMMENT '(DC2Type:array)',
`username` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `log_date_lookup_idx` (`logged_at`),
KEY `log_user_lookup_idx` (`username`),
KEY `log_class_lookup_idx` (`object_class`),
KEY `log_version_lookup_idx` (`object_id`,`object_class`,`version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1219777 ;
I try to run the following query:
SELECT n0_.id AS id0, n0_.action AS action1, n0_.logged_at AS logged_at2, n0_.object_id AS object_id3, n0_.object_class AS object_class4, n0_.version AS version5, n0_.data AS data6, n0_.username AS username7
FROM ext_log_entries n0_
WHERE n0_.object_id =275634
AND n0_.object_class = 'My\\MyBundle\\Entity\\Field'
AND n0_.version <=1
ORDER BY n0_.version ASC
Here is the MySQL plan:
id 1
select_type SIMPLE
table n0_
type ref
possible_keys log_class_lookup_idx,log_version_lookup_idx
key log_class_lookup_idx
key_len 767
ref const
rows 641159
Extra Using where; Using filesort
My query need about 37 seconds to be executed for only 1 row in the result...
I tried to run the same query by deleting my indexes and it goes a little bit faster : about 31 seconds...
I don't understand why my query is taking so much time and why my indexes don't help the performance? Do you know how I can do to have good performance on this query?
Thanks in advance for your help !
EDIT
Here are the cardinalties of the indexes
log_date_lookup_idx BTREE logged_at 1221578 A
log_user_lookup_idx BTREE username 40 A YES
log_class_lookup_idx BTREE object_class 1010 A
log_version_lookup_idx BTREE object_id 1221578 A YES
object_class 1221578 A
version 1221578 A
I found a solution, not THE solution, but at least it works for me.
I think it could help anyway all people who are using gedmo loggable and who are lucky (like me) to have objects with only integers IDs.
I changes my column object_id to integer instead of varchar(255). My query now take 0.008 second ! It works for me because i'm sure i'll always have only integers, for people who have varchar, I'm sorry i tried many things but nothing worked....
CREATE TABLE IF NOT EXISTS `ext_log_entries` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`action` varchar(8) NOT NULL,
`logged_at` datetime NOT NULL,
`object_id` int(11) DEFAULT NULL,
`object_class` varchar(255) NOT NULL,
`version` int(11) NOT NULL,
`data` longtext COMMENT '(DC2Type:array)',
`username` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `log_date_lookup_idx` (`logged_at`),
KEY `log_user_lookup_idx` (`username`),
KEY `log_class_lookup_idx` (`object_class`),
KEY `log_version_lookup_idx` (`object_id`,`object_class`,`version`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 AUTO_INCREMENT=1219777 ;
I'm trying to optimize sql query which now takes about 20s to execute.
Here is the structure of my tables.
last_login
id | ip_address |when
1 2130706433 2012-05-04 12:00:36
and
country_by_ip
ip_from | ip_to | country
16843008 | 16843263 | CN
Here is the query I use:
SELECT
ll.ip_address,
ll.when,
cbi.country
FROM last_login ll
LEFT JOIN `country_by_ip` cbi on ll.ip_address BETWEEN cbi.ip_from AND cbi.ip_to
The fields ip_from and ip_to are indexed.
Can you recommend me how to speed up this query ?
//EDIT
CREATE TABLE `last_login` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`ip_address` int(11) unsigned NOT NULL,
`when` datetime NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=32 DEFAULT CHARSET=utf8
CREATE TABLE `country_by_ip` (
`ip_from` int(20) unsigned NOT NULL,
`ip_to` int(20) DEFAULT NULL,
`country` varchar(255) DEFAULT NULL,
KEY `ip_from` (`ip_from`),
KEY `ip_to` (`ip_to`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8
How about:
SELECT
ll.ip_address,
ll.when,
cbi.country
FROM last_login ll
LEFT JOIN `country_by_ip` cbi on ll.ip_address > cbi.ip_from
WHERE ll.ip_address < cbi.ip_to
However, I am totally agree with #Romain, change the DB schema to better design.
You're not doing yourself any favours by splitting the country_by_ip range across 2 sperate indexes - change these to KEY ip_range (ip_from,ip_to).
I have this query showing up in MySql slow query log. (It is not slow, but it is not using indexes right). I need some help on how to set up the index right.
SELECT tbladded.amount*SUM(tbladdeditem.amount)
FROM tbladded
INNER JOIN tbladdeditem ON tbladded.addedid = tbladdeditem.addedid AND tbladdeditem.deleted='False'
WHERE tbladded.userid=100
AND tbladded.date='2012-01-01'
AND tbladded.deleted='False'
GROUP BY tbladded.addedid
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE tbladded ref PRIMARY,userid_date userid_date 8 const,const 1 Using where
1 SIMPLE tbladdeditem ref addedid addedid 5 tbladded.addedid 1 Using where
This is how the tables look like:
CREATE TABLE `tbladded` (
`addedid` int(11) NOT NULL AUTO_INCREMENT,
`amount` double DEFAULT NULL,
`date` date DEFAULT NULL,
`userid` mediumint(9) DEFAULT NULL,
`deleted` enum('False','True') CHARACTER SET latin1 DEFAULT 'False',
PRIMARY KEY (`addedid`),
KEY `userid_date` (`userid`,`date`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
CREATE TABLE `tbladdeditem` (
`addeditemid` int(11) NOT NULL AUTO_INCREMENT,
`amount` double DEFAULT NULL,
`addedid` int(11) DEFAULT NULL,
`userid` mediumint(9) DEFAULT NULL,
`deleted` enum('False','True') CHARACTER SET latin1 DEFAULT 'False',
PRIMARY KEY (`addeditemid`),
KEY `addedid` (`addedid`),
KEY `userid` (`userid`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
try this:
ALTER TABLE `tbladded` ADD INDEX
`tbladdedIndex` (`userid`, `date`, `deleted`);