TO show indexes we use the following query,
show indexes from student;
+---------+------------+-----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+-----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| student | 0 | PRIMARY | 1 | ROLL_NO | A | 6 | NULL | NULL | | BTREE | | |
| student | 1 | stu_roll_no_age_index | 1 | ROLL_NO | A | 6 | NULL | NULL | | BTREE | | |
| student | 1 | stu_roll_no_age_index | 2 | AGE | A | 6 | NULL | NULL | YES | BTREE | | |
+---------+------------+-----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+--------------+
Now I am trying to select only Table from the above as follows,
select Table, Key_name, Column_name,Index_type from (show indexes from student);
I think I might be wrong but I am not getting how to reason it? Please tell me.
And also , how do I get only specific columns from the result of such query.
You can use table STATISTICS from db information_schema, like this:
select TABLE_NAME, INDEX_NAME, COLUMN_NAME, INDEX_TYPE
from information_schema.STATISTICS WHERE TABLE_NAME = 'student';
Related
I am trying to figure out why a query in our production environment is taking so much time (~20 sec). I have tried creating the database locally and generated the same data and am getting under half a second run time with the exact same queries.
Would anyone know what could cause such a slowdown?
My best guess right now is a possible memory limitation that causes the index being used to swap to disk
See tables and Explain the query below:
(.15 sec) SELECT COUNT(*) FROM user;
(.32 sec) SELECT COUNT(*) FROM profile;
(20.80 sec) SELECT COUNT(*) FROM user INNER JOIN profile ON user.id = profile.user_id;
user table:
| id | guid | status | username | email |
user index:
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-------+------------+----------------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| user | 0 | PRIMARY | 1 | id | A | 355448 | NULL | NULL | | BTREE | | |
| user | 0 | unique_email | 1 | email | A | 2 | NULL | NULL | YES | BTREE | | |
| user | 0 | unique_username | 1 | username | A | 2 | NULL | NULL | YES | BTREE | | |
profile table:
| user_id | firstname | lastname | full_name |
profile index:
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+---------+------------+-----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| profile | 0 | PRIMARY | 1 | user_id | A | 330496 | NULL | NULL | | BTREE | | |
| profile | 1 | idx_profile_full_name | 1 | full_name | A | 2 | NULL | NULL | YES | BTREE | | |
+---------+------------+-----------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
EXPLAIN of query:
mysql> EXPLAIN SELECT COUNT(*) FROM profile JOIN user ON user.id = profile.user_id;
+----+-------------+---------+--------+---------------+-----------------------+---------+------------------------+--------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+--------+---------------+-----------------------+---------+------------------------+--------+-------------+
| 1 | SIMPLE | profile | index | PRIMARY | idx_profile_full_name | 258 | NULL | 330496 | Using index |
| 1 | SIMPLE | user | eq_ref | PRIMARY | PRIMARY | 4 | humhub.profile.user_id | 1 | Using index |
+----+-------------+---------+--------+---------------+-----------------------+---------+------------------------+--------+-------------+
Most likely a MySQL 5.7/8.0 query planner sudden-brain-death heisenbug.
Run ANALYZE TABLE user; ANALYZE TABLE profile; and see if the query planner gets un-confused.
The only way to avoid recurrence is to use an index hint:
SELECT COUNT(*) FROM profile USE INDEX (PRIMARY) JOIN user ON user.id = profile.user_id;
I'm using Rails 2.2.2 with the Mysql gem v 2.7. If I launch the MySql client from my terminal, and do a query, I get a table of results like this:
mysql> show indexes from lesson_units;
+--------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| lesson_units | 0 | PRIMARY | 1 | id | A | 2 | NULL | NULL | | BTREE | | |
| lesson_units | 1 | index_lesson_units_on_user_id | 1 | user_id | A | 2 | NULL | NULL | YES | BTREE | | |
+--------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
2 rows in set (0.00 sec)
The interface methods in Rails generally do the most useful thing, eg returning the above data as an array of hashes, such as
#in Rails console
> ActiveRecord::Base.connection.select_all("show indexes from lesson_units")
=> [{"Index_comment"=>"", "Sub_part"=>nil, "Key_name"=>"PRIMARY", "Comment"=>"", "Collation"=>"A", "Table"=>"lesson_units", "Packed"=>nil, "Seq_in_index"=>"1", "Index_type"=>"BTREE", "Null"=>"", "Cardinality"=>"2", "Non_unique"=>"0", "Column_name"=>"id"}, {"Index_comment"=>"", "Sub_part"=>nil, "Key_name"=>"index_lesson_units_on_user_id", "Comment"=>"", "Collation"=>"A", "Table"=>"lesson_units", "Packed"=>nil, "Seq_in_index"=>"1", "Index_type"=>"BTREE", "Null"=>"YES", "Cardinality"=>"2", "Non_unique"=>"1", "Column_name"=>"user_id"}]
Now, in terms of actually doing stuff with the data, this is great. But, sometimes I'd like to just have a mysql-style view - eg, to get that first table back as text, which I can just print out. Is this possible, via Rails? ie, to call a function which gives me back this string?
> ActiveRecord::Base.connection.methodname_here("show indexes from lesson_units")
=> "+--------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+\n | Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |\n +--------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+\n | lesson_units | 0 | PRIMARY | 1 | id | A | 2 | NULL | NULL | | BTREE | | |\n | lesson_units | 1 | index_lesson_units_on_user_id | 1 | user_id | A | 2 | NULL | NULL | YES | BTREE | | |\n +--------------+------------+-------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+\n 2 rows in set (0.00 sec)"
I've used table_print before which is pretty handy in development, so you can try something like below:
# Gemfile
group :development, :test do
gem 'table_print'
end
rails c
result = ActiveRecord::Base.connection.exec_query('show indexes from lesson_units;')
tp result.to_a
TABLE | NON_UNIQUE | KEY_NAME | SEQ_IN_INDEX | COLUMN_NAME | COLLATION | CARDINALITY | SUB_PART | PACKED | NULL | INDEX_TYPE | COMMENT | INDEX_COMMENT
-------------|------------|-------------------------------|--------------|-------------|-----------|-------------|----------|--------|------|------------|---------|--------------
lesson_units | 0 | PRIMARY | 1 | id | A | 0 | | | | BTREE | |
lesson_units | 1 | index_lesson_units_on_comment | 1 | comment | A | 0 | | | YES | BTREE | |
I have two tables CUSTOMER_ORDER_PUBLIC and LINEITEM_PUBLIC which have the following indices:
+-----------------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| CUSTOMER_ORDER_PUBLIC | 1 | O_ORDERKEY | 1 | O_ORDERKEY | A | 2633457 | NULL | NULL | YES | BTREE | | |
| CUSTOMER_ORDER_PUBLIC | 1 | O_ORDERDATE | 1 | O_ORDERDATE | A | 2350 | NULL | NULL | YES | BTREE | | |
| CUSTOMER_ORDER_PUBLIC | 1 | PUB_C_CUSTKEY | 1 | PUB_C_CUSTKEY | A | 273000 | NULL | NULL | | BTREE | | |
+-----------------------+------------+---------------+--------------+---------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
and:
+-----------------+------------+----------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+-----------------+------------+----------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| LINEITEM_PUBLIC | 0 | PRIMARY | 1 | PUB_L_ORDERKEY | A | 16488602 | NULL | NULL | | BTREE | | |
| LINEITEM_PUBLIC | 0 | PRIMARY | 2 | PUB_L_LINENUMBER | A | 44146904 | NULL | NULL | | BTREE | | |
| LINEITEM_PUBLIC | 1 | LINEITEM_PRIVATE_FK2 | 1 | PUB_L_PARTKEY | A | 2083757 | NULL | NULL | | BTREE | | |
| LINEITEM_PUBLIC | 1 | LINEITEM_PRIVATE_FK3 | 1 | PUB_L_SUPPKEY | A | 85599 | NULL | NULL | | BTREE | | |
+-----------------+------------+----------------------+--------------+------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
Each time I run an Explain of a specific query I get the following:
mysql> EXPLAIN SELECT *
FROM CUSTOMER_ORDER_PUBLIC
LEFT OUTER JOIN LINEITEM_PUBLIC ON O_ORDERKEY= PUB_L_ORDERKEY;
+----+-------------+-----------------------+------------+------+---------------+---------+---------+---------------------------------------+---------+----------+-------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-----------------------+------------+------+---------------+---------+---------+---------------------------------------+---------+----------+-------+
| 1 | SIMPLE | CUSTOMER_ORDER_PUBLIC | NULL | ALL | NULL | NULL | NULL | NULL | 2900769 | 100.00 | NULL |
| 1 | SIMPLE | LINEITEM_PUBLIC | NULL | ref | PRIMARY | PRIMARY | 4 | TPCH.CUSTOMER_ORDER_PUBLIC.O_ORDERKEY | 2 | 100.00 | NULL |
+----+-------------+-----------------------+------------+------+---------------+---------+---------+---------------------------------------+---------+----------+-------+
For some reason the query optimizer is not using the index (O_ORDERKEY) even if I use a FORCE INDEX. I know a lot of people posted similar questions but I tried everything and nothing seems to help!
Any other suggestions would be greatly appreciated!
Edit:
The query used is the following:
SELECT * FROM CUSTOMER_ORDER_PUBLIC
LEFT OUTER JOIN LINEITEM_PUBLIC ON O_ORDERKEY= PUB_L_ORDERKEY;
For this query:
SELECT *
FROM CUSTOMER_ORDER_PUBLIC cop LEFT OUTER JOIN
LINEITEM_PUBLIC lp
ON cop.O_ORDERKEY = lp.PUB_L_ORDERKEY;
For this query, you want an index on LINEITEM_PUBLIC(PUB_L_ORDERKEY). Of course, you already have this index because this is the first key in the primary key.
There is no reason to use an index on CUSTOMER_ORDER_PUBLIC, because all rows in the table are going to the result set.
The FORCE INDEX hint tells the optimizer that a full scan of the table is very expensive.
The most likely explanation for the observed behavior is that the optimizer thinks it needs to access every row in the table, and the index suggested in the hint is not a covering index for the query.
Based on the EXPLAIN output, we only see evidence of a single predicate on the JOIN operation. And it looks like the optimizer is choosing CUSTOMER_ORDER_PUBLIC as the driving table for the join, and using an index on the LINEITEM_PUBLIC table.
I'm not sure any of that answers the question you asked. (I'm not sure that there was a question asked.) Absent an actual SQL statement, we are just making guesses.
I have a question: Aside from the FORCE INDEX hint, why would we expect the optimizer to use a particular index? And why would that be a reasonable expectation?
I am not sure why this query isn't using an index on the table world_cities. The indexes are the same and in the same order. The data types are of the same type and same length. The only difference is the key_name.
So why does it not use the key?
if I run the query I get: 318824 rows in set (2 min 51.30 sec)
Here is what I have done:
explain select * from zip_codes zc
inner join world_cities wc on
zc.city = wc.city
and zc.country = wc.country
and zc.region = wc.region;
+----+-------------+-------+------+---------------------+---------------------+---------+---------------------------------------------------------------+---------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------+------+---------------------+---------------------+---------+---------------------------------------------------------------+---------+-------------+
| 1 | SIMPLE | wc | ALL | city_country_region | NULL | NULL | NULL | 3173958 | |
| 1 | SIMPLE | zc | ref | country_region_city | country_region_city | 165 | largedbapi.wc.city,largedbapi.wc.country,largedbapi.wc.region | 1 | Using where |
+----+-------------+-------+------+---------------------+---------------------+---------+---------------------------------------------------------------+---------+-------------+
mysql> show indexes from world_cities where Key_name like '%city%';
+--------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| world_cities | 1 | city_country_region | 1 | city | A | NULL | NULL | NULL | YES | BTREE | |
| world_cities | 1 | city_country_region | 2 | country | A | NULL | NULL | NULL | YES | BTREE | |
| world_cities | 1 | city_country_region | 3 | region | A | NULL | NULL | NULL | YES | BTREE | |
+--------------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
mysql> show indexes from zip_codes where Key_name like '%city%';
+-----------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-----------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| zip_codes | 1 | country_region_city | 1 | city | A | 218424 | NULL | NULL | YES | BTREE | |
| zip_codes | 1 | country_region_city | 2 | country | A | 218424 | NULL | NULL | YES | BTREE | |
| zip_codes | 1 | country_region_city | 3 | region | A | 436849 | NULL | NULL | YES | BTREE | |
+-----------+------------+---------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
Your city_country_region key on world_cities has a NULL cardinality, while a non-NULL one is required for it to be use. Running analyze on it should fix it.
ANALYZE TABLE world_cities
Also have a look at this post for more info on NULL cardinality.
sorry but after browsing nearly every posts and questions about it, I still can't manage to get rid of "Using temporary" and "Using filesort" in a simple query. I know this is a problem of keys but I can't find the right combination...
I also don't know if the order of the join defined by the optimizer is ok, I tested other orders using STRAIGHT_JOIN but nothing better... The query is pretty slow using ORDER BY, but really fast without it and of course without "Using temporary" and "Using filesort"! (there is something like 100.000 rows in points table)
The query :
SELECT points.id,
points.id_owner,
points.point_title,
points.point_desc,
users.user_id,
users.username
FROM points,
JOIN users ON points.id_owner = users.user_id
JOIN follows ON follows.id_followed = points.id_owner
WHERE points.deleted = 0
AND follows.id_follower = 22
ORDER BY points.id DESC
LIMIT 10
the explain :
+----+-------------+---------+--------+---------------+------------+---------+---------------------+------+----------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+---------+--------+---------------+------------+---------+---------------------+------+----------------------------------------------+
| 1 | SIMPLE | follows | ref | FOLLOW_DUO | FOLLOW_DUO | 4 | const | 2 | Using index; Using temporary; Using filesort |
| 1 | SIMPLE | users | eq_ref | PRIMARY | PRIMARY | 4 | follows.id_followed | 1 | |
| 1 | SIMPLE | points | ref | GETPOINT1 | GETPOINT1 | 5 | users.user_id,const | 460 | Using where |
+----+-------------+---------+--------+---------------+------------+---------+---------------------+------+----------------------------------------------+
And here is the SHOW INDEX from the three tables :
SHOW INDEX FROM points
+--------+------------+--------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+--------+------------+--------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
| points | 0 | PRIMARY | 1 | id | A | 91987 | NULL | NULL | | BTREE | |
| points | 0 | GETPOINT1 | 1 | id_owner | A | NULL | NULL | NULL | | BTREE | |
| points | 0 | GETPOINT1 | 2 | deleted | A | NULL | NULL | NULL | | BTREE | |
| points | 0 | GETPOINT1 | 3 | id | A | 91987 | NULL | NULL | | BTREE | |
+--------+------------+--------------+--------------+-----------------+-----------+-------------+----------+--------+------+------------+---------+
SHOW INDEX FROM users
+-------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+-------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| users | 0 | PRIMARY | 1 | user_id | A | 4 | NULL | NULL | | BTREE | |
+-------+------------+------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
SHOW INDEX FROM follows
+---------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment |
+---------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
| follows | 0 | PRIMARY | 1 | id | A | 5 | NULL | NULL | | BTREE | |
| follows | 0 | FOLLOW_DUO | 1 | id_follower | A | NULL | NULL | NULL | | BTREE | |
| follows | 0 | FOLLOW_DUO | 2 | id_followed | A | 5 | NULL | NULL | | BTREE | |
| follows | 1 | id_follower | 1 | id_follower | A | NULL | NULL | NULL | | BTREE | |
| follows | 1 | id_followed | 1 | id_followed | A | NULL | NULL | NULL | | BTREE | |
+---------+------------+-------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+
From now I don't know what to test to try to avoid the "Using temporary" and "Using filesort"... So if you have an idea for me... Thank you in advance for your help !
Looks like so many rows are being examined from points table. I had tried following trick to avoid temporary table usage in my project. Please do as follows and give it an explain to see any improvement:
Delete all indexes called 'GETPOINT1' except Primary Key Index form points table.
Add covering index on columns (deleted, id_owner). Please keep the order of columns as mentioned.
If you still don't see any improvement, remove above index and add index again in order (id, deleted, id_owner) and (deleted, id_owner, id) columns and try again
In addition you may remove follows.id_follower = 22 from where clause and put it in join condition like JOIN follows ON follows.id_followed = points.id_owner AND follows.id_follower = 22
Please also add index in order as (id_follower, id_owner) in follows table.
I do not guarantee but above should be able to give you improvements.