I have a query like
SELECT `table1`.*
FROM `table1`
WHERE `table1`.`table2_id` IN (1,2,6,12,53,666)
and it works more than 20 seconds
Explain looks like:
+----+-------------+--------------------------+------------+-------+-------------------------------------------------------------------------------+----------------------------------+---------+------+-------+----------+-----------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+--------------------------+------------+-------+-------------------------------------------------------------------------------+----------------------------------+---------+------+-------+----------+-----------------------+
| 1 | SIMPLE | table1 | NULL | range | table2_id | table2_id | 4 | NULL | 74778 | 100.00 | Using index condition |
+----+-------------+--------------------------+------------+-------+-------------------------------------------------------------------------------+----------------------------------+---------+------+-------+----------+-----------------------+
Table looks like
CREATE TABLE `table1` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`table2_id` int(11) NOT NULL,
`table3_id` int(11) NOT NULL,
`field1` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`field2` int(11) NOT NULL DEFAULT '0',
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
`field3` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `name_of_index_id` (`table3_id`),
KEY `other_name_of_index` (`field2`),
KEY `table2_id` (`table2_id`)
) ENGINE=InnoDB AUTO_INCREMENT=86623178 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
You can use index to fetch the result quickly. But, Indexing will take more space to generate and store in the database. So, if you are fine to utilize speed compare to space, Indexing will be used with following SQL statement.
ALTER TABLE `table1` ADD INDEX(`table2_id`);
Related
I was migrating from my old MariaDB 10.0 database to new Google Cloud Sql with Mysql 5.7 with mysqldump method. After migrating, i got some very slow query regarding with WHERE EXISTS statement.
I tried to EXPLAIN my query on both my old DB and new DB and it explained different result. Since it using dump, i am assuming that no changes with the table indexes. This is the query that i wanted to run
SELECT * FROM detitem
where exists (select 1 from detlayanan
where detitem.iddetlayanan = detlayanan.id
and detlayanan.layanan_idlayanan='LYN15176176101503')
the EXPLAIN from old DB
+------+-------------+------------+------+------------------------------------+----------------------------+---------+--------------------------------+-------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+------+-------------+------------+------+------------------------------------+----------------------------+---------+--------------------------------+-------+--------------------------+
| 1 | PRIMARY | detlayanan | ref | PRIMARY,fk_detlayanan_layanan1_idx | fk_detlayanan_layanan1_idx | 22 | const | 11030 | Using where; Using index |
| 1 | PRIMARY | detitem | ref | FK_detitem_detlayanan | FK_detitem_detlayanan | 52 | citridia_sinadme.detlayanan.id | 1 | |
+------+-------------+------------+------+------------------------------------+----------------------------+---------+--------------------------------+-------+--------------------------+
and the EXPLAIN from new DB
+----+--------------------+------------+------------+--------+------------------------------------+---------+---------+---------------------------------------+---------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+--------------------+------------+------------+--------+------------------------------------+---------+---------+---------------------------------------+---------+----------+-------------+
| 1 | PRIMARY | detitem | NULL | ALL | NULL | NULL | NULL | NULL | 2079094 | 100.00 | Using where |
| 2 | DEPENDENT SUBQUERY | detlayanan | NULL | eq_ref | PRIMARY,fk_detlayanan_layanan1_idx | PRIMARY | 52 | citridia_sinadme.detitem.iddetlayanan | 1 | 5.00 | Using where |
+----+--------------------+------------+------------+--------+------------------------------------+---------+---------+---------------------------------------+---------+----------+-------------+
The new one is doing Full-table scan even there is index exist. Am i missing something here?
Here is the "detlayanan" table
CREATE TABLE `detlayanan` (
`transaksi_idtransaksi` varchar(40) NOT NULL,
`layanan_idlayanan` varchar(20) NOT NULL,
`nama_layanan` varchar(255) DEFAULT NULL,
`jumlah_beli` float DEFAULT NULL,
`id` varchar(50) NOT NULL,
`harga` decimal(20,2) DEFAULT '0.00',
`hargatotal` decimal(20,2) DEFAULT '0.00',
`luas_p` double(255,2) DEFAULT '0.00',
`luas_l` double(255,2) DEFAULT '0.00',
`luas_q` double(255,2) DEFAULT '0.00',
`keterangan` varchar(255) DEFAULT '',
`iddeposit` varchar(255) DEFAULT NULL,
`posisi` tinyint(4) DEFAULT '1',
`idworkshop` varchar(60) DEFAULT NULL,
`is_wsot` tinyint(4) DEFAULT '0',
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
`total_bersih` varchar(20) DEFAULT '0',
`total_min_order` decimal(20,2) DEFAULT '0.00',
`kondisi_barang` text,
PRIMARY KEY (`id`),
KEY `fk_detlayanan_layanan1_idx` (`layanan_idlayanan`),
KEY `fk_detlayanan_deposit` (`iddeposit`),
KEY `transaksi_idtransaksi` (`transaksi_idtransaksi`),
CONSTRAINT `detlayanan_ibfk_1` FOREIGN KEY (`transaksi_idtransaksi`) REFERENCES `transaksi` (`idtransaksi`),
CONSTRAINT `fk_detlayanan_layanan1` FOREIGN KEY (`layanan_idlayanan`) REFERENCES `layanan` (`idlayanan`) ON DELETE NO ACTION ON UPDATE NO ACTION
)
And here is the "detitem" table
CREATE TABLE `detitem` (
`item_iditem` varchar(20) NOT NULL,
`layanan_idlayanan` varchar(255) NOT NULL,
`jumlah_item` int(255) DEFAULT NULL,
`transaksi_idtransaksi` varchar(255) NOT NULL,
`id` varchar(50) NOT NULL,
`iddetlayanan` varchar(50) NOT NULL,
`created_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_at` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
`hapus` tinyint(4) DEFAULT '0',
PRIMARY KEY (`id`),
KEY `FK_detitem_item` (`item_iditem`),
KEY `FK_detitem_layanan` (`layanan_idlayanan`),
KEY `FK_detitem_transaksi` (`transaksi_idtransaksi`),
KEY `FK_detitem_detlayanan` (`iddetlayanan`),
CONSTRAINT `FK_detitem_detlayanan` FOREIGN KEY (`iddetlayanan`) REFERENCES `detlayanan` (`id`),
CONSTRAINT `FK_detitem_item` FOREIGN KEY (`item_iditem`) REFERENCES `item` (`iditem`),
CONSTRAINT `FK_detitem_layanan` FOREIGN KEY (`layanan_idlayanan`) REFERENCES `layanan` (`idlayanan`),
CONSTRAINT `FK_detitem_transaksi` FOREIGN KEY (`transaksi_idtransaksi`) REFERENCES `transaksi` (`idtransaksi`)
)
I expect the "rows" on explain stay small, in the old DB, "rows" stays at very low even in another table. but in the new DB it can shows up to million.
UPDATE
After some research, actually i must explicitly add some indexed column in the where statement to accompany EXISTS statement. so the query would be like this
SELECT * FROM detitem WHERE
<indexed column> in (<some id's>)
AND EXISTS ( SELECT 1 FROM detlayanan WHERE detitem.iddetlayanan =
detlayanan.id AND detlayanan.layanan_idlayanan = 'LYN15176176101503' )
apparently MySQL doing full scan on detitem table to check the subquery values is exists or not, when some indexed column are declared, sql does not need to did that. Also this case were found in MySQL.
some references:
https://mariadb.com/kb/en/library/exists-to-in-optimization/
This looks like a case where MariaDB's Optimizer is a step or two ahead of MySQL's.
See if this works well:
SELECT i.*
FROM ( SELECT id
FROM detlayanan
WHERE layanan_idlayanan = 'LYN15176176101503'
) AS x
JOIN detitem AS i ON x.id = i.iddetlayanan
I think it will work well in both servers.
Simpler yet:
SELECT i.*
FROM detlayanan AS lay
JOIN detitem AS i ON lay.id = i.iddetlayanan
WHERE lay.layanan_idlayanan = 'LYN15176176101503'
Please refer the table strcuture below.
CREATE TABLE `oarc` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`zID` int(11) NOT NULL,
`cID` int(11) NOT NULL,
`bID` int(11) NOT NULL,
`rtype` char(1) COLLATE utf8_unicode_ci NOT NULL,
`created` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1821039 ;
Other than the PRIMARY KEY, I have not set any index on this, and when I run the following query
select COUNT(oarc.ID) as total
from `oarc` where`oarc`.`rtype` = 'v'
group
by `oarc`.`zID`
I am getting the result in less than 1 second. But if I add index to zID it is taking more than 5 seconds.
Please see below explain result :
id | select_type | table | type | possible_keys | key | key_len | ref | row | Extra
--------------------------------------------------------------------------------------------------------
1 | SIMPLE | oarc | index | NULL | zone_ID | 4 | NULL | 1909387 | Using where
Currently the table have more than 1821039 records in it and it will increase on a hourly basis. What are the things I need to do in order to reduce the query execution time. I am expecting only something at the table and query level, nothing on my.cnf or server side because I can not do anything there.
Thanks in advance.
Is this better?
CREATE TABLE `oarc` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT,
`zID` int(11) NOT NULL,
`cID` int(11) NOT NULL,
`bID` int(11) NOT NULL,
`rtype` char(1) COLLATE utf8_unicode_ci NOT NULL,
`created` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
PRIMARY KEY (`ID`),
KEY(rtype,zid)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci AUTO_INCREMENT=1821039 ;
explain
select COUNT(oarc.ID) as total
from `oarc` where`oarc`.`rtype` = 'v'
group
by `oarc`.`zID`
+----+-------------+-------+------+---------------+-------+---------+-------+------+--------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+-------+---------+-------+------+--------------------------+
| 1 | SIMPLE | oarc | ref | rtype | rtype | 3 | const | 1 | Using where; Using index |
+----+-------------+-------+------+---------------+-------+---------+-------+------+--------------------------+
i have one MySQL issue. I have to optimize some queries on my website. One of them i have already done, but there are still some which i cannot resolve without your help.
I have a table called "news":
CREATE TABLE IF NOT EXISTS `news` (
`id` int(10) NOT NULL auto_increment,
`edited` smallint(1) NOT NULL default '0',
`site` varchar(30) default NULL,
`foreign_id` varchar(25) default NULL,
`title` varchar(255) NOT NULL,
`text` text NOT NULL,
`image` varchar(255) default NULL,
`horizontal` smallint(1) NOT NULL,
`image_author` varchar(255) default NULL,
`text_author` varchar(255) default NULL,
`lang` varchar(3) NOT NULL,
`link` varchar(255) NOT NULL,
`date` date NOT NULL,
`redirect` smallint(1) NOT NULL,
`parent` int(10) NOT NULL,
`views` int(5) NOT NULL,
`status` smallint(1) NOT NULL,
PRIMARY KEY (`id`),
KEY `lang` (`lang`,`status`),
KEY `date` (`date`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 AUTO_INCREMENT=47122 ;
as you can see i have two indexes: "lang" and "date"
I have tried some combinations of different indexes and this one has produced me the best results ... unfortunately only on my local computer. On the server i still have bad results. I want to say that the database is the same.
query:
SELECT id FROM news WHERE lang = 'en' AND STATUS =1 ORDER BY DATE DESC LIMIT 0, 10
localhost explain:
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
| 1 | SIMPLE | news | index | lang | date | 3 | NULL | 23 | Using where |
+----+-------------+-------+-------+---------------+------+---------+------+------+-------------+
server explain:
+----+-------------+-------+------+---------------+--------+---------+-------------+-------+-----------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------+--------+---------+-------------+-------+-----------------------------+
| 1 | SIMPLE | news | ref | status | status | 13 | const,const | 15840 | Using where; Using filesort |
+----+-------------+-------+------+---------------+--------+---------+-------------+-------+-----------------------------+
I have looked a lot of other similar topics, but unfortunately i cannot find any solution to work on my server. I will be very glad to here from you some solution with some explanation for that so i can optimize my other queries.
Thanks !
This is your query:
SELECT id
FROM news
WHERE lang = 'en' AND STATUS =1
ORDER BY DATE DESC
LIMIT 0, 10
The best index is one that contains all the fields used in the query (four fields in all). The ordering in the index is by equality conditions in the where clause followed by the order by clause followed by other columns in the select clause.
So, try this index: ndws(leng, status, date, id).
Why does lean_users show NULL in the ref column? This causes my query to use a temporary table and a filesort later (when I've added more joins)...
14:45:21 (60) > EXPLAIN select * from users u inner join lean_users lu on u.id = lu.user_id;
+----+-------------+-------+--------+---------------+---------+---------+----------------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+--------+---------------+---------+---------+----------------+------+-------------+
| 1 | SIMPLE | lu | index | PRIMARY | PRIMARY | 4 | NULL | 358 | Using index |
| 1 | SIMPLE | u | eq_ref | PRIMARY | PRIMARY | 4 | nwa.lu.user_id | 1 | |
+----+-------------+-------+--------+---------------+---------+---------+----------------+------+-------------+
users table
14:45:24 (61) > show create table users;
+-------+-----------------------------------------------------------------------------+
| Table | Create Table |
+-------+-----------------------------------------------------------------------------+
| users | CREATE TABLE `users` (
`id` int(11) NOT NULL auto_increment,
`email` varchar(255) default NULL,
`first_name` varchar(50) NOT NULL,
`last_name` varchar(50) NOT NULL,
`address1` varchar(255) NOT NULL,
`address2` varchar(255) default NULL,
`city` varchar(25) NOT NULL,
`state` mediumint(9) default NULL,
`zip` varchar(10) NOT NULL,
`phone` varchar(20) default NULL,
`country` smallint(6) NOT NULL,
`username` varchar(10) NOT NULL,
`password` varchar(50) default NULL,
`cdate` datetime NOT NULL,
`last_used` timestamp NOT NULL default CURRENT_TIMESTAMP on update CURRENT_TIMESTAMP,
`level` varchar(25) default 'user',
PRIMARY KEY (`id`),
) ENGINE=InnoDB AUTO_INCREMENT=38076 DEFAULT CHARSET=utf8 |
+-------+-----------------------------------------------------------------------------+
lean_users table
14:45:40 (62) > show create table lean_users;
+-------------+-----------------------------------------------------------------------------+
| Table | Create Table |
+-------------+-----------------------------------------------------------------------------+
| lean_users | CREATE TABLE `lean_users` (
`user_id` int(11) NOT NULL,
PRIMARY KEY (`user_id`),
CONSTRAINT `lean_users_ibfk_1` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
+-------------+-----------------------------------------------------------------------------+
Why does lean_users show NULL in the ref column?
Because this table is leading in the join and you don't filter on any indexed fields.
This means that each record should be read and evaluated.
I use django on webfaction, and I've got a "MySql query taking too long" message,
the sql is
SELECT (1) AS `a` FROM `main_userprofile` WHERE `main_userprofile`.`id` = 98
This is a rather simple sql, why the query taken too long?
here it is the 'create table':
main_userprofile | CREATE TABLE `main_userprofile` (
`id` int(11) NOT NULL auto_increment,
`user_id` int(11) NOT NULL,
`sex` smallint(6) NOT NULL,
`active_number` varchar(64) NOT NULL,
`phone_number` varchar(32) NOT NULL,
`work_number` varchar(32) NOT NULL,
...
...
PRIMARY KEY (`id`),
UNIQUE KEY `user_id` (`user_id`)
) ENGINE=MyISAM AUTO_INCREMENT=1652 DEFAULT CHARSET=utf8 |
the id is the primary key
the explain:
explain SELECT (1) AS `a` FROM `main_userprofile` WHERE `main_userprofile`.`id` = 98;
+----+-------------+------------------+-------+---------------+---------+---------+-------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+------------------+-------+---------------+---------+---------+-------+------+-------------+
| 1 | SIMPLE | main_userprofile | const | PRIMARY | PRIMARY | 4 | const | 1 | Using index |
+----+-------------+------------------+-------+---------------+---------+---------+-------+------+-------------+
Seems like id is not indexed, isn't it. Is id the primary key of your table? Could you post the result of SHOW CREATE TABLE main_userprofile;?