MySQL not using index WHERE IN - mysql

I have a simple SELECT query that's using WHERE IN across 5 integer values to get 50 results in a table of roughly 20 million rows. Here's the query:
select `prf_profiles_id` as `id`, `first`, `last`, `suffix` from `prf_names`
where prf_names`.`src_sources_id` in (9, 10, 11, 34, 37) limit 50 offset 0;
And the explain values for the query:
select_type: SIMPLE
table: prf_names
type: ALL
possible_keys: prf_names_src_sources_id_index
key: NULL
ref: NULL
rows: 20012960
Extra: Using where
As can be seen, the query knows the index is possible, but chooses not to use it and as a result the query takes about 4.5 seconds.
src_sources_id is indexed, and contains about 20 distinct values applied to all 20 million rows.
I know that the index would help immensely because when I run the same query using FORCE INDEX:
select `prf_profiles_id` as `id`, `first`, `last`, `suffix` from `prf_names`
force index (prf_names_src_sources_id_index)
where `prf_names`.`src_sources_id` in (9, 10, 11, 34, 37) limit 50 offset 0;
The query takes under 0.0 seconds.
I'd like to avoid forcing MySQL to use the index as the query is being called through an ORM and forcing it would defeat some of the purpose of having the ORM in the first place.
What can I do to ensure MySQL uses this index going forward?
EDIT:
Here's the create table statement:
CREATE TABLE `prf_names` (
`prf_profiles_id` int(10) unsigned NOT NULL,
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`created` datetime NOT NULL,
`verified` datetime NOT NULL,
`title` varchar(20) COLLATE utf8_unicode_ci DEFAULT NULL,
`first` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
`middle` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
`last` varchar(50) COLLATE utf8_unicode_ci DEFAULT NULL,
`suffix` varchar(10) COLLATE utf8_unicode_ci DEFAULT NULL,
`sanitized` enum('yes','no') COLLATE utf8_unicode_ci NOT NULL DEFAULT 'no',
`src_sources_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`prf_profiles_id`,`id`),
UNIQUE KEY `prf_names_id_unique` (`id`),
KEY `prf_names_first_index` (`first`),
KEY `prf_names_middle_index` (`middle`),
KEY `prf_names_last_index` (`last`),
KEY `prf_names_src_sources_id_index` (`src_sources_id`)
) ENGINE=InnoDB AUTO_INCREMENT=31633081 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci

Related

Mysql slow query on large table

With Mysql 5.6.10, I have a table like this:
CREATE TABLE `es_user_action` (
`id` bigint(11) unsigned NOT NULL AUTO_INCREMENT,
`user_id` bigint(11) unsigned NOT NULL,
`company_id` bigint(11) unsigned NOT NULL,
`work_id` bigint(11) unsigned NOT NULL,
`action` tinyint(2) NOT NULL COMMENT '10, 20, 30, 40',
`action_id` bigint(11) unsigned DEFAULT NULL,
`apply_id` bigint(11) unsigned DEFAULT '0',
`apply_display_id` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`action_time` datetime NOT NULL,
`scout_time` datetime NOT NULL,
`register_datetime_sorting` datetime DEFAULT NULL,
`score` tinyint(2) DEFAULT '0',
`is_pending` tinyint(2) DEFAULT '0',
`apply_status` tinyint(2) DEFAULT '2' COMMENT '1: paid, 2: free',
`has_response` tinyint(2) DEFAULT '0',
`response_time` datetime DEFAULT NULL,
`is_shown` tinyint(2) DEFAULT '1',
`source` varchar(15) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `user_company_work` (`user_id`,`work_id`,`company_id`,`apply_id`,`source`),
KEY `IDX_2` (`company_id`,`is_shown`,`apply_status`,`apply_id`,`work_id`,`action_time`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=436896779 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
The full query I want to perform on the table would be the following:
SELECT
`id` AS `seqNo`,
`company_id` AS `companyId`,
`work_id` AS `workId`,
`action` AS `userActionType`,
`action_time` AS `userActionDatetime`,
`is_pending`,
`apply_status`,
`action_id`,
`source`,
`user_id` AS `userId`,
`apply_display_id`
FROM
`es_user_action`
WHERE
`company_id` = 449664
AND `is_shown` = 1
AND `apply_status` = 1
AND `apply_id` = 0
AND `action_time` >= '2021-01-05 15:56:14'
ORDER BY
is_pending ASC,
action ASC,
score DESC,
CASE
source
WHEN "entenshoku" THEN
action_time
END DESC,
CASE
WHEN source <> "entenshoku" THEN
action_time
END ASC
LIMIT 100 OFFSET 0;
The table has around 15 million rows and the following query takes around 15 seconds. The query becomes very slow.
Can anyone help out? Thanks in advance.
UPDATED:
This is the explain query result:
KEY `IDX_2` (`company_id`,`is_shown`,`apply_status`,`apply_id`,`work_id`,`action_time`)
Work_id is in the way of that index being effective for that SELECT. Remove it. That should speed up the query some. Still the are too many columns and expressions involved in the WHER and ORDER BY to do much more to help.
Changing BIGINTs to smaller INT types will help some (by shrinking the table).
I'm pretty sure that what you have cannot work.
You have is a dynamically changing ORDER BY, depending on the value of source. But which row's source?
If you could decide before issuing the query whether to scan action_time ascending or descending, which value of source would you pick?
Delete this Question, rewrite the query, then open another Question if you still have an issue.

MySQL Composite Index Question - Performance issue

I have done some research on the index before posting the question here. So far I believe I have done this correctly, but for some reason, the performance of a query that returns around 2400 records has not been good.
Here is the table schema
CREATE TABLE `tblCheck` (
`id` VARCHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`token` VARCHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`domainId` VARCHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`time` DATETIME NULL DEFAULT NULL,
`responseCode` INT(11) NULL DEFAULT NULL,
`totalTime` DECIMAL(10,2) NULL DEFAULT NULL,
`namelookupTime` INT(11) NULL DEFAULT NULL,
`connectTime` INT(11) NULL DEFAULT NULL,
`pretransferTime` INT(11) NULL DEFAULT NULL,
`startTransferTime` INT(11) NULL DEFAULT NULL,
`redirectTime` INT(11) NULL DEFAULT NULL,
`appconnectTime` INT(11) NULL DEFAULT NULL,
`responseText` TEXT(65535) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`agentId` VARCHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`isHealthy` CHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
`ftp_connect_time` DECIMAL(10,4) NULL DEFAULT NULL,
`ftp_login_time` DECIMAL(10,4) NULL DEFAULT NULL,
`ftp_change_mode_time` DECIMAL(10,4) NULL DEFAULT NULL,
`ftp_list_time` DECIMAL(10,4) NULL DEFAULT NULL,
`syntheticToken` VARCHAR(50) NULL DEFAULT NULL COLLATE 'latin1_swedish_ci',
UNIQUE INDEX `id` (`id`) USING BTREE,
INDEX `domainId` (`domainId`) USING BTREE,
INDEX `deleteTime` (`time`) USING BTREE,
INDEX `SearchIndex` (`domainId`, `time`, `agentId`) USING BTREE
)
COLLATE='latin1_swedish_ci'
ENGINE=InnoDB
ROW_FORMAT=COMPACT
;
The Query
SELECT *
FROM `tblCheck`
WHERE (`time` BETWEEN '2020-05-04 22:15:04' AND '2020-05-05 22:15:04')
AND `domainId` = '03d4c1ce-8b13-11ea-abf5-124e96b5f417'
AND `agentId` != '145a-f6bb-11e8-983f-1231322cbdb6'
ORDER BY `time` DESC
;
/* Affected rows: 0 Found rows: 2,418 Warnings: 0 Duration for 1 query: 0.109 sec. (+ 10.360 sec. network) */
It returned 2418 rows but took almost 10s.
Running it with EXPLAIN
EXPLAIN SELECT *
FROM `tblCheck`
WHERE (`time` BETWEEN '2020-05-04 22:15:04' AND '2020-05-05 22:15:04')
AND `domainId` = '03d4c1ce-8b13-11ea-abf5-124e96b5f417'
AND `agentId` != '145a-f6bb-11e8-983f-1231322cbdb6'
ORDER BY `time` DESC
Returns this
This looks like it is using the index "SearchIndex". However, I can't figure out why it would take to process 10s for 2k rows
For this query:
SELECT *
FROM `tblCheck`
WHERE
`time` BETWEEN '2020-05-04 22:15:04' AND '2020-05-05 22:15:04'
AND `domainId` = '03d4c1ce-8b13-11ea-abf5-124e96b5f417'
AND `agentId` != '145a-f6bb-11e8-983f-1231322cbdb6'
ORDER BY `time` DESC
The right index would be: (domainId, agentId, time), or (domainId, time, agentId). You have an the second index in place, and the query plan shows that MySQL happily uses it.
Looking at the explain summary, you can see:
Duration for 1 query: 0.109 sec. (+ 10.360 sec. network)
The query runs fast in the database. The bottleneck is the network, that is the time taken to return the 2000+ rows from the database to the client. Not much can be done from database perspective. Speed up your network, or switch to a local database if you can.
As a side note: select * is a not good for performance; you should try and reduce the number of columns that the query returns (this might also reduce the amount that needs to transmitted over the network).
You don't have the primary key.
You only have a UNIQUE INDEX.
If you con't nee that bulky responseText column in the results, don't include it. This may dramatically speed up the query.
(This is because large columns are stored "off-record", thereby taking an extra disk read if the table is huge.)

Mysql JOIN query apparently slow

I have 2 tables. The first, called stazioni, where I store live weather data from some weather station, and the second called archivio2, where are stored archived day data. The two tables have in common the ID station data (ID on stazioni, IDStazione on archvio2).
stazioni (1,743 rows)
CREATE TABLE `stazioni` (
`ID` int(10) NOT NULL,
`user` varchar(100) NOT NULL,
`nome` varchar(100) NOT NULL,
`email` varchar(50) NOT NULL,
`localita` varchar(100) NOT NULL,
`provincia` varchar(50) NOT NULL,
`regione` varchar(50) NOT NULL,
`altitudine` int(10) NOT NULL,
`stazione` varchar(100) NOT NULL,
`schermo` varchar(50) NOT NULL,
`installazione` varchar(50) NOT NULL,
`ubicazione` varchar(50) NOT NULL,
`immagine` varchar(100) NOT NULL,
`lat` double NOT NULL,
`longi` double NOT NULL,
`file` varchar(255) NOT NULL,
`url` varchar(255) NOT NULL,
`temperatura` decimal(10,1) DEFAULT NULL,
`umidita` decimal(10,1) DEFAULT NULL,
`pressione` decimal(10,1) DEFAULT NULL,
`vento` decimal(10,1) DEFAULT NULL,
`vento_direzione` decimal(10,1) DEFAULT NULL,
`raffica` decimal(10,1) DEFAULT NULL,
`pioggia` decimal(10,1) DEFAULT NULL,
`rate` decimal(10,1) DEFAULT NULL,
`minima` decimal(10,1) DEFAULT NULL,
`massima` decimal(10,1) DEFAULT NULL,
`orario` varchar(16) DEFAULT NULL,
`online` int(1) NOT NULL DEFAULT '0',
`tipo` int(1) NOT NULL DEFAULT '0',
`webcam` varchar(255) DEFAULT NULL,
`webcam2` varchar(255) DEFAULT NULL,
`condizioni` varchar(255) DEFAULT NULL,
`Data2` datetime DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
archivio2 (2,127,347 rows)
CREATE TABLE `archivio2` (
`ID` int(10) NOT NULL,
`IDStazione` int(4) NOT NULL DEFAULT '0',
`localita` varchar(100) NOT NULL,
`temp_media` decimal(10,1) DEFAULT NULL,
`temp_minima` decimal(10,1) DEFAULT NULL,
`temp_massima` decimal(10,1) DEFAULT NULL,
`pioggia` decimal(10,1) DEFAULT NULL,
`pressione` decimal(10,1) DEFAULT NULL,
`vento` decimal(10,1) DEFAULT NULL,
`raffica` decimal(10,1) DEFAULT NULL,
`records` int(10) DEFAULT NULL,
`Data2` datetime DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
The indexes that I set
-- Indexes for table `archivio2`
--
ALTER TABLE `archivio2`
ADD PRIMARY KEY (`ID`),
ADD KEY `IDStazione` (`IDStazione`),
ADD KEY `Data2` (`Data2`);
-- Indexes for table `stazioni`
--
ALTER TABLE `stazioni`
ADD PRIMARY KEY (`ID`),
ADD KEY `Tipo` (`Tipo`);
ALTER TABLE `stazioni` ADD FULLTEXT KEY `localita` (`localita`);
On a map, I call by a calendar the date to search data on archive2 table, by this INNER JOIN query (I put an example date):
SELECT *, c.pioggia AS rain, c.raffica AS raff, c.vento AS wind, c.pressione AS press
FROM stazioni as o
INNER JOIN archivio2 as c ON o.ID = c.IDStazione
WHERE c.Data2 LIKE '2019-01-01%'
All works fine, but the time needed to show result are really slow (4/5 seconds), even if the query execution time seems to be ok (about 0.5s/1.0s).
I tried to execute the query on PHPMyadmin, and the results are the same. Execution time quickly, but time to show result extremely slow.
EXPLAIN query result
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE o ALL PRIMARY,ID NULL NULL NULL 1743 NULL
1 SIMPLE c ref IDStazione,Data2 IDStazione 4 sccavzuq_rete.o.ID 1141 Using where
UPDATE: the query goes fine if I remove the index from 'IDStazione'. But in this way I lost all advantages and speed on other queries... why only that query become slow if I put index on that field?
In your WHERE clause
WHERE c.Data2 LIKE '2019-01-01%'
the value of Data2 must be casted to a string. No index can be used for that condition.
Change it to
WHERE c.Data2 >= '2019-01-01' AND c.Data2 < '2019-01-01' + INTERVAL 1 DAY
This way the engine should be able to use the index on (Data2).
Now check the EXPLAIN result. I would expect, that the table order is swapped and the key column will show Data2 (for c) and ID (for o).
(Fixing the DATE is the main performance solution; here is a less critical issue.)
The tables are much bigger than necessary. Size impacts disk space and, to some extent, speed.
You have 1743 stations, yet the datatype is a 32-bit (4-byte) number (INT). SMALLINT UNSIGNED would allow for 64K stations and use only 2 bytes.
Does it get really, really, hot there? Like 999999999.9 degrees? DECIMAL(10.1) takes 5 bytes; DECIMAL(4,1) takes only 3 and allows up to 999.9 degrees. DECIMAL(3,1) has a max of 99.9 and takes only 2 bytes.
What is "localita varchar(100)" doing in the big table? Seems like you could JOIN to the stations table when you need it? Removing that might cut the table size in half.

MySQL - Any addtional indexes would speed up this query?

I see that my query does full table scan and takes a lot of time. I heard that making indexes would speed this up and I have added some to the tables. Is there any other indexes I should create to make this query faster?
My query is:
SELECT p.id, n.people_type_id, n.full_name, n.post, p.nick,
p.key_name, p.email, p.internal_user_id FROM email_routing e
JOIN people_emails p ON p.id=e.receiver_email_id
JOIN people n ON n.id = p.people_id
WHERE e.message_id = 897360 AND e.basket=1
Here is the EXPLAIN result:
EXPLAIN SELECT p.id, n.people_type_id, n.full_name, n.post, p.nick,
p.key_name, p.email, p.internal_user_id FROM email_routing e
JOIN people_emails p ON p.id=e.receiver_email_id
JOIN people n ON n.id = p.people_id
WHERE e.message_id = 897360 AND e.basket=1
id select_type table partitions type possible_keys key key_len ref rows filtered Extra
1 SIMPLE n NULL ALL PRIMARY NULL NULL NULL 1 100.00 NULL
1 SIMPLE p NULL ALL PRIMARY NULL NULL NULL 3178 10.00 Using where; Using join buffer (Block Nested Loop)
1 SIMPLE e NULL ref bk1 bk1 4 server.p.id 440 1.00 Using where; Using
And here are the tables strucutre:
SHOW CREATE TABLE people_emails;
CREATE TABLE `people_emails` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`nick` varchar(255) NOT NULL,
`email` varchar(255) NOT NULL,
`key_name` varchar(255) NOT NULL,
`people_id` int(11) NOT NULL,
`status` int(11) NOT NULL DEFAULT '0',
`activity` int(11) NOT NULL,
`internal_user_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
FULLTEXT KEY `email` (`email`)
) ENGINE=MyISAM AUTO_INCREMENT=22114 DEFAULT CHARSET=utf8
SHOW CREATE TABLE email_routing;
CREATE TABLE `email_routing` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`message_id` int(11) NOT NULL,
`sender_email_id` int(11) NOT NULL,
`receiver_email_id` int(11) NOT NULL,
`basket` int(11) NOT NULL,
`status` int(11) NOT NULL,
`popup` int(11) NOT NULL DEFAULT '0',
`tm` int(11) NOT NULL DEFAULT '0',
KEY `id` (`id`),
KEY `bk1` (`receiver_email_id`,`status`,`sender_email_id`,`message_id`,`basket`),
KEY `bk2` (`sender_email_id`,`tm`)
) ENGINE=InnoDB AUTO_INCREMENT=1054618 DEFAULT CHARSET=utf8
SHOW CREATE TABLE people;
CREATE TABLE `people` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`fname` varchar(255) CHARACTER SET cp1251 NOT NULL,
`lname` varchar(255) CHARACTER SET cp1251 NOT NULL,
`patronymic` varchar(255) CHARACTER SET cp1251 NOT NULL,
`gender` tinyint(1) NOT NULL,
`full_name` varchar(255) NOT NULL DEFAULT ' ',
`category` int(11) NOT NULL,
`people_type_id` int(255) DEFAULT NULL,
`tags` varchar(255) CHARACTER SET cp1251 NOT NULL,
`job` varchar(255) CHARACTER SET cp1251 NOT NULL,
`post` varchar(255) CHARACTER SET cp1251 NOT NULL,
`profession` varchar(255) CHARACTER SET cp1251 DEFAULT NULL,
`zip` varchar(16) CHARACTER SET cp1251 NOT NULL,
`country` int(11) DEFAULT NULL,
`region` varchar(10) NOT NULL,
`city` varchar(255) CHARACTER SET cp1251 NOT NULL,
`address` varchar(255) CHARACTER SET cp1251 NOT NULL,
`address_date` date DEFAULT NULL,
`inner` tinyint(4) NOT NULL,
`contact_through` varchar(255) DEFAULT '',
`next_call` date NOT NULL,
`additional` text CHARACTER SET cp1251 NOT NULL,
`user_id` int(11) NOT NULL,
`changed` datetime NOT NULL,
`status` int(11) DEFAULT NULL,
`nick` varchar(255) DEFAULT NULL,
`birthday` date DEFAULT NULL,
`last_update_ts` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`area` text NOT NULL,
`reviewed_` tinyint(4) NOT NULL,
`phones_old` text NOT NULL,
`post_sticker` text NOT NULL,
`permissions` int(120) NOT NULL DEFAULT '0',
`internal_user_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `most_used` (`category`,`status`,`city`,`lname`,`next_call`),
KEY `registrars` (`category`,`status`,`contact_through`,`next_call`),
FULLTEXT KEY `lname` (`lname`),
FULLTEXT KEY `fname` (`fname`),
FULLTEXT KEY `mname` (`patronymic`),
FULLTEXT KEY `Full Name` (`full_name`)
) ENGINE=MyISAM AUTO_INCREMENT=415009 DEFAULT CHARSET=utf8
How to choose columns for building indexes, should I pick text columnts too or that only will work with numer columnts
The table email_routing seem to have 1054618 rows .
And you try to find one row , by message_id .
e.message_id = 897360
BUT message_id must be indexed to speed-up the query .
message_id is part of the index bk1 , but this is not enough because message_id is not the first columns of the index .
email_routing needs
INDEX ( message_id, basket, -- first, in either order
receiver_email_id ) -- for "covering"
Your bk1 starts with receiver_email_id; this is not nearly as good.
Include column(s) in WHERE that are tested with =.
Include other columns from WHERE, GROUP BY, and ORDER BY (none in your case); the order is important, but beyond the scope of this discussion.
Include any other columns of the same table used anywhere in the query -- this is to make it a "covering" index. But don't bother if this would lead to more than, say, 5 columns or would involve TEXT, which cannot be in an index.
Then move on to the other tables. In both JOINs, it seems that they would be hit by their PRIMARY KEYs (JOIN x ON x.id = ...)
More discussion: Cookbook for creating indexes
On other issues...
You really should move to InnoDB. As of 5.6, it includes FULLTEXT, but there are some differences. In particular, you may need more fulltext indexes. For example, MATCH(lname, fname) requires FULLTEXT(lname, fname).
Do you really want to stick to cp1251? It limits your internalization mostly to English, Russian, Bulgarian, Serbian and Macedonian. And it is unclear how well FULLTEXT (MyISAM or InnoDB) will work with those non-English languages.
INTs are always 4 bytes; consider using smaller versions.
Is there really only one people? The Optimizer decided that was the best table to start with, but it wasn't. I'm hoping my improved index on email_routing will trick it into starting with that table -- which will definitely be optimal.

query slows down if add field to where condition

I have a table Mysql fiddle with about 500k records.
CREATE TABLE IF NOT EXISTS `p_transactions` (
`transaction_id` bigint(10) unsigned NOT NULL,
`amount` decimal(19,2) NOT NULL,
`dt` bigint(1) NOT NULL,
`transaction_status` int(1) NOT NULL,
`transaction_type` varchar(15) NOT NULL,
`payment_method` varchar(25) NOT NULL,
`notes` text NOT NULL,
`member_id` int(10) unsigned NOT NULL,
`new_amount` decimal(19,2) NOT NULL,
`paid_amount` decimal(19,2) NOT NULL,
`secret_code` char(40) NOT NULL,
`internal_status` varchar(40) NOT NULL,
`ip_addr` varchar(15) NOT NULL,
`description` text NOT NULL,
`seller_transaction_id` varchar(50) DEFAULT NULL,
`return_url` varchar(255) DEFAULT NULL,
`fail_url` varchar(255) DEFAULT NULL,
`success_url` varchar(255) DEFAULT NULL,
`result_url` varchar(255) DEFAULT NULL,
`user_fee` decimal(19,3) DEFAULT '0.000',
`currency` char(255) DEFAULT 'USD',
`gateway_transaction_id` char(255) DEFAULT NULL,
`load_amount` decimal(19,2) NOT NULL,
`transaction_mode` varchar(1) NOT NULL DEFAULT '',
`p_fee` decimal(19,2) NOT NULL,
`country` varchar(2) NOT NULL,
`email` varchar(255) NOT NULL,
`vat` decimal(19,2) NOT NULL DEFAULT '0.00',
`name` varchar(255) NOT NULL,
`bdate` varchar(255) NOT NULL,
`child_method` varchar(255) NOT NULL,
`processing_fee` decimal(19,2) NOT NULL DEFAULT '0.00',
`flat_fee` varchar(1) NOT NULL DEFAULT 'n',
`user_fee_sum` decimal(19,2) NOT NULL DEFAULT '0.00',
`p_fee_sum` decimal(19,2) NOT NULL DEFAULT '0.00',
`dt_open` bigint(1) NOT NULL DEFAULT '0',
`user_fee_type` varchar(1) NOT NULL DEFAULT 'r',
`custom_gateway_fee` decimal(19,2) NOT NULL DEFAULT '0.00',
`paid_currency` varchar(3) NOT NULL DEFAULT 'USD',
`paid_microtime` bigint(10) unsigned NOT NULL,
`check_ballance` varchar(1) NOT NULL DEFAULT 'n',
PRIMARY KEY (`transaction_id`),
KEY `member_id` (`member_id`),
KEY `payment_method` (`payment_method`),
KEY `child_method` (`child_method`),
KEY `check_ballance` (`check_ballance`),
KEY `dt` (`dt`),
KEY `transaction_type` (`transaction_type`),
KEY `paid_microtime` (`paid_microtime`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
When I execute query
SELECT *
FROM `p_transactions`
WHERE dt >= 1517443200
AND dt <= 1523404799
AND member_id = 2051
ORDER BY `paid_microtime` DESC
LIMIT 50;
it runs 0,000 sec. (+ 0,016 sec. network)
but if I add to query this condition AND transaction_status = 7
SELECT *
FROM `p_transactions`
WHERE dt >= 1517443200
AND dt <= 1523404799
AND member_id = 2051
AND transaction_status = 7
ORDER BY `paid_microtime` DESC
LIMIT 50
query run 12,938 sec. (+ 0,062 sec. network)
Please help me to find out the reason of such behavior
PS. There was index on transaction_status and it increased execution time even more.
Add a suitable index, such as:
ON payzoff_transactions (member_id, dt)
or
ON payzoff_transactions (member_id, dt, transaction_status)
We want member_id column as the leading column in the index, because of the equality comparison, and we expect the result to be a substantially smaller subset of the entire table. We want dt column after that, because of the "range scan" on that.
Including additional columns in the index may allow MySQL to check that condition using values from the index, without a visit/lookup of the row in the underlying table pages.
Either of these indexes would be suitable for both of the queries shown in the question.
Use EXPLAIN to see the execution plan... which index is being used.
There's really no getting around the "Using filesort" operation, since we're pulling a small subset of the entire table.
(If we were pulling the entire table (or a huge subset), we might be able to avoid an expensive sort operation with an access plan that pulls rows in reverse index order, with that has an index with leading column of paid_microtime.)
For the original query have these
INDEX(member_id, dt)
INDEX(member_id, paid_microtime)
For the secondary query, have
INDEX(transaction_status, member_id, dt)
INDEX(transaction_status, member_id, paid_microtime)
Without getting into the details of the distribution of the data values, we cannot explain why one query so much slower; however, my 4 indexes should make both queries run faster most of the time.
More discussion of how I came up with those indexes (and why (member_id, dt, transaction_status) is not so good): http://mysql.rjweb.org/doc.php/index_cookbook_mysql