Why using inner join itself is more faster than where - mysql

Here is my table:
CREATE TABLE `cerp_oms_order` (
`id` bigint NOT NULL,
`company_id` bigint NOT NULL,
`order_no` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`source_type` int NOT NULL,
`shop_id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`outer_shop` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`origin_status` int NOT NULL,
`system_status` int NOT NULL,
`created_time` datetime DEFAULT NULL,
`paid_time` datetime DEFAULT NULL,
`sent_time` datetime DEFAULT NULL,
`end_time` datetime DEFAULT NULL,
`modify_time` datetime DEFAULT NULL,
`delivery_deadline_time` datetime DEFAULT NULL,
`amount` text CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL ,
`spu_kind` int NOT NULL,
`sku_kind` int NOT NULL,
`total_quantity` decimal(16,4) NOT NULL,
`buyer_id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`outer_buyer_identifier` varchar(64) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL DEFAULT '',
`tax_info` varchar(256) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`warehouse_owner` int DEFAULT NULL,
`warehouse_id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`logistics_type` int NOT NULL,
`logistics_outer_info` varchar(256) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`delivery_id` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`delivery_no` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`wave_no` varchar(64) DEFAULT '',
`is_deleted` tinyint NOT NULL DEFAULT '0',
`backend_processing_type` tinyint NOT NULL,
`create_type` tinyint NOT NULL,
`is_hang_up` tinyint NOT NULL,
`hang_up_case_type` smallint DEFAULT NULL,
`hang_up_case_id` bigint DEFAULT NULL,
`rc_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`rm_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`vat` varchar(128) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`weight` decimal(16,4) NOT NULL DEFAULT '0.0000',
`volume` decimal(16,4) NOT NULL DEFAULT '0.0000',
`is_abnormal` tinyint NOT NULL DEFAULT '0',
`estimate_profit` decimal(16,4) NOT NULL DEFAULT '0.0000',
`business_man_id` varchar(256) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`business_man` varchar(256) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`currency` varchar(32) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`net_amount_summary` decimal(19,4) NOT NULL,
`domestic_amount` decimal(19,4) NOT NULL,
`secret_key` varchar(64) DEFAULT '',
`secretKey` varchar(255) DEFAULT NULL,
`sale_id` bigint DEFAULT NULL,
`total_refund_include_tax` decimal(16,4) NOT NULL DEFAULT '0.0000',
`total_refund_money` decimal(16,4) NOT NULL DEFAULT '0.0000',
`total_refund_tax` decimal(16,4) NOT NULL DEFAULT '0.0000',
`total_return_goods` decimal(16,2) NOT NULL DEFAULT '0.00',
PRIMARY KEY (`id`),
UNIQUE KEY `key_order_no` (`order_no`),
KEY `idx_order_company_id` (`company_id`,`created_time`),
KEY `IDX_RM_TIME` (`rm_time`),
KEY `IDX_IS_ABNORMAL` (`is_abnormal`),
KEY `cerp_oms_order_company_id_index` (`company_id`),
KEY `idx_order_company_status_deleted` (`company_id`,`is_deleted`,`system_status`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_0900_ai_ci COMMENT='system order table'
explain select *
from cerp_oms_order
inner join (select id
from cerp_oms_order
where source_type = 43
order by created_time) as tmp using (id);
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
cerp_oms_order
null
ALL
PRIMARY
null
null
null
60787
10
Using where
1
SIMPLE
cerp_oms_order
null
eq_ref
PRIMARY
PRIMARY
8
cerp_oms_1.cerp_oms_order.id
1
100
null
using inner join execution: 7 ms, fetching: 109 ms
VS
explain
select *
from cerp_oms_order
where source_type = 43
order by created_time;
id
select_type
table
partitions
type
possible_keys
key
key_len
ref
rows
filtered
Extra
1
SIMPLE
cerp_oms_order
null
ALL
null
null
null
null
60787
10
Using where; Using filesort
using simple where clause execution: 80 ms, fetching: 138 ms
I don't understand why use inner join can speed up my sql?

If you had
INDEX(source_type, created_time)
both formulations would run faster. And, I think, the second would be faster.
As for "why". Look at what the second does:
Scan the entire table ("table scan") looking for rows with source_type = 43.
For each matching row, gather all the columns (*) from the row, into a temp table.
Sort that temp table. Note that this is bulkier than with the subquery that hauls around only id.
Deliver the results.
With the INDEX I suggest:
Look in the index's B+Tree for all 'rows' with source_type = 43.
Since those rows are sorted by created_time, not "sort" pass will be needed now or later.
Each of those index rows will contain id. (That's the way InnoDB gets to the rest of the columns), so
Look up the desired rows in the data's B+Tree and pull out all (*) the columns.
Deliver them.
Note that the JOIN version works essentially the same, but, technically speaking, the ORDER BY may get lost. If it does get lost, you would need to add a second ORDER BY, thereby forcing a sort.
(Unrelated) Your "KEY cerp_oms_order_company_id_index (company_id)," can be dropped since there are two other indexes starting with company_id.
More on optimizing indexes: Index Cookbook

Related

MySQL COUNT(Joining table) is taking too long to fetch the data inside large tables

Description:
We have two tables as below:
table_1 ("question" main table)
table_2 ("question_attempted" joining table)
Cases:
In "table_2" we have a column that has a column "is_correct" (holds 1,0) for right or wrong answers.
In "table_1" we have 1 m records and in "table_2" we have 10m records
We want to sort our listing data by below columns/values:
Total number of times questions were attempted
Total number of times questions were answered correctly
The percentages questions were answered correctly (based on above two values)
Issue:
As soon as we join the table_1 and table_2 to get the count of total_questions_attempted, total_questiones_give_correct_answer, perntage_corrected_given_answers. The query starts taking around 8-10 minutes to run. Table structures are given below. Thanks in advance.
Table structures:
CREATE TABLE IF NOT EXISTS `question` (
`id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`category` bigint(20) NOT NULL DEFAULT 0,
`parent` bigint(20) UNSIGNED NOT NULL DEFAULT 0,
`name` text COLLATE utf8mb4_unicode_ci NOT NULL,
`questiontext` text COLLATE utf8mb4_unicode_ci NOT NULL,
`questiontextformat` tinyint(4) NOT NULL DEFAULT 0,
`generalfeedback` text COLLATE utf8mb4_unicode_ci NOT NULL,
`generalfeedbackformat` tinyint(4) NOT NULL DEFAULT 0,
`defaultmark` decimal(12,7) NOT NULL DEFAULT 1.0000000,
`penalty` decimal(12,7) NOT NULL DEFAULT 0.3333333,
`qtype` varchar(20) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '' COMMENT '''1''',
`length` bigint(20) UNSIGNED NOT NULL DEFAULT 1,
`stamp` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`version` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`hidden` tinyint(3) UNSIGNED NOT NULL DEFAULT 0,
`timecreated` bigint(20) UNSIGNED NOT NULL DEFAULT 0,
`timemodified` bigint(20) UNSIGNED NOT NULL DEFAULT 0,
`createdby` bigint(20) UNSIGNED DEFAULT NULL,
`modifiedby` bigint(20) UNSIGNED DEFAULT NULL,
`type_data_id` bigint(20) NOT NULL,
`img_id` bigint(20) DEFAULT NULL,
`qimg_gallary_text` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`qrimg_gallary_text` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`qimg_gallary_ids` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`qrimg_gallary_ids` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`case_id` bigint(20) NOT NULL DEFAULT 0,
`ques_type_id` bigint(20) DEFAULT NULL,
`year` bigint(20) DEFAULT NULL,
`spec` bigint(20) DEFAULT NULL,
`sub_speciality_id` int(11) DEFAULT NULL,
`sub_sub_speciality_id` int(11) DEFAULT NULL,
`spec_level` bigint(20) DEFAULT 1,
`is_deleted` int(11) NOT NULL DEFAULT 0,
`sequence` int(11) NOT NULL DEFAULT 0,
`sort_order` bigint(20) NOT NULL DEFAULT 0 COMMENT 'Question order in list',
`idnumber` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`addendum` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`text_for_search` longtext COLLATE utf8mb4_unicode_ci DEFAULT NULL COMMENT 'this is for the text based searching, this will store the text of the question without html tags',
`text_for_search_ans` longtext COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `type_data_id` (`type_data_id`),
UNIQUE KEY `mdl_ques_catidn_uix` (`category`,`idnumber`),
KEY `mdl_ques_cat_ix` (`category`),
KEY `mdl_ques_par_ix` (`parent`),
KEY `mdl_ques_cre_ix` (`createdby`),
KEY `mdl_ques_mod_ix` (`modifiedby`),
KEY `id` (`id`),
KEY `mq_spec_ix` (`spec`),
KEY `sort_order` (`sort_order`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='The questions themselves';
CREATE TABLE IF NOT EXISTS `question_attempted` (
`id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`questionusageid` bigint(20) UNSIGNED NOT NULL,
`slot` bigint(20) UNSIGNED NOT NULL,
`behaviour` varchar(32) COLLATE utf8mb4_unicode_ci NOT NULL DEFAULT '',
`questionid` bigint(20) UNSIGNED NOT NULL,
`variant` bigint(20) UNSIGNED NOT NULL DEFAULT 1,
`maxmark` decimal(12,7) NOT NULL,
`minfraction` decimal(12,7) NOT NULL,
`flagged` tinyint(3) UNSIGNED NOT NULL DEFAULT 2,
`questionsummary` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`rightanswer` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`responsesummary` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`timemodified` bigint(20) UNSIGNED NOT NULL,
`maxfraction` decimal(12,7) DEFAULT 1.0000000,
`in_remind_state` int(11) NOT NULL DEFAULT 0,
`is_correct` tinyint(1) DEFAULT 1,
PRIMARY KEY (`id`),
UNIQUE KEY `mdl_quesatte_queslo_uix` (`questionusageid`,`slot`),
KEY `mdl_quesatte_que_ix` (`questionid`),
KEY `mdl_quesatte_que2_ix` (`questionusageid`),
KEY `mdl_quesatte_beh_ix` (`behaviour`),
KEY `questionid` (`questionid`),
KEY `is_correct` (`is_correct`)
) ENGINE=InnoDB AUTO_INCREMENT=151176 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci COMMENT='Each row here corresponds to an attempt at one question, as ';
I tried with the below query:
SELECT mq.id, mq.name, COUNT(is_correct)
FROM mdl_question_attempts as mqa
LEFT JOIN mdl_question mq on mq.id = mqa.questionid where mq.id IS NOT NULL and mq.is_deleted = '0'
GROUP by mqa.questionid
ORDER by mq.sort_order desc, mq.id DESC
LIMIT 50
https://i.stack.imgur.com/mHK6W.png
The correct query is
SELECT mq.id, mq.name, COUNT(mqa.questionid)
FROM mdl_question mq
LEFT JOIN mdl_question_attempts mqa ON mq.id = mqa.questionid AND mqa.is_correct
WHERE NOT mq.is_deleted
GROUP by mq.id
ORDER by mq.sort_order DESC, mq.id DESC
LIMIT 50;
Now let's see, how fast this can get. There is just one criteria on the question table (WHERE NOT mq.is_deleted). We can probably assume that very many if not most questions are not deleted, so using an index here makes no sense on first glance; reading the full table seems quicker.
Then we outer join the answers on the question ID and the is_correct flag. This means we should at least have an index on the ID, better even on the ID and the flag:
CREATE INDEX idx1 ON mdl_question_attempts (questionid, is_correct);
Now we must order all rows by the question's sort_order and ID to get the first 50 rows. It would be great to have an index that is already sorted, so we could just take the first 50 entries from there. But then, we are only looking at rows matching NOT mq.is_deleted, so the index must include that flag:
CREATE INDEX idx2 ON mdl_question (is_deleted, sort_order DESC, id DESC);
We could even include the name, so all data is available from the index and the table must not be read anymore (covering index).
CREATE INDEX idx2 ON mdl_question (is_deleted, sort_order DESC, id DESC, name);
It is still up to the DBMS to use these indexes or not. We are just providing them to give the DBMS the option. With this query it depends on how well MySQL's optimizer works. Does it see that it can just read the 50 first entries from the question index and then use the answer index for the simple counting?
add index in your table
CREATE INDEX index_name ON table_name (column_name);
references : https://www.w3schools.com/sql/sql_create_index.asp

MySQL does not recognize possible_keys in join

I tried to query by join 2 tables giao_vien and lop_mon.
The following is my tables structure:
CREATE TABLE `lop_mon` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`key_index` varchar(250) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`khoi_key_index` varchar(250) DEFAULT NULL,
`lop_key_index` varchar(250) DEFAULT NULL,
`gv_key_index` varchar(250) DEFAULT NULL,
`mon_key_index` varchar(250) DEFAULT NULL,
`ma_so` varchar(50) DEFAULT NULL,
`ma_phong` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`nam_hoc` int(11) NOT NULL,
`ma_truong` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`ma_khoi` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL,
`ma_lop` varchar(50) NOT NULL,
`ma_giao_vien` varchar(50) NOT NULL,
`ma_mon_hoc` varchar(50) DEFAULT NULL,
`hoc_ky` tinyint(4) NOT NULL,
`guid` varchar(250) DEFAULT NULL,
`processid_monhoc` int(11) DEFAULT NULL,
`processid_giaovien` int(11) DEFAULT NULL,
`processid_canbo` int(11) DEFAULT NULL,
`source_data` varchar(50) DEFAULT NULL,
`ten_gv_chat_ph` varchar(255) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`ma_giao_vien_bgd` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`ma_lop_bgd` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`ma_mon_hoc_bgd` varchar(50) CHARACTER SET utf8 COLLATE utf8_general_ci DEFAULT NULL,
`gv_guid` char(36) DEFAULT NULL,
`truong_key_index` varchar(255) DEFAULT NULL,
`create_date` datetime DEFAULT NULL,
`updatedAt` datetime DEFAULT NULL,
PRIMARY KEY (`id`) USING BTREE,
UNIQUE KEY `key_UNIQUE` (`key_index`),
UNIQUE KEY `unique_lop_mon_gv_keyindex_hoc_ky` (`lop_key_index`,`gv_key_index`,`mon_key_index`,`hoc_ky`) USING BTREE,
KEY `lopmon_keyindex` (`mon_key_index`),
KEY `lopmon_idx_mamonhoc` (`ma_mon_hoc`),
KEY `idx_gv_key_index` (`gv_key_index`),
KEY `lopmon_idx_lop_key_index` (`lop_key_index`),
KEY `lopmon_idx_key_index` (`khoi_key_index`),
KEY `lopmon_idx_ma_truong` (`ma_truong`),
KEY `lopmon_idx_nam_hoc` (`nam_hoc`),
KEY `lopmon_idx_ma_khoi` (`ma_khoi`),
KEY `lopmon_idx_hoc_ky` (`hoc_ky`),
KEY `lopmon_idx_key_lop_mon` (`lop_key_index`,`mon_key_index`),
KEY `lopmon_idx_key_lop_gv` (`lop_key_index`,`gv_key_index`),
KEY `lopmon_idx_key_mon_gv` (`mon_key_index`,`gv_key_index`),
KEY `lopmon_idx_key_lop_mon_gv` (`lop_key_index`,`mon_key_index`,`gv_key_index`),
KEY `lopmon_idx_key_truong_lop_mon_gv` (`ma_truong`,`lop_key_index`,`mon_key_index`,`gv_key_index`),
KEY `lop_mon__index_gv_guid` (`gv_guid`),
KEY `lopmon_idx_truong_key_index` (`truong_key_index`)
) ENGINE=MyISAM AUTO_INCREMENT=23142297 DEFAULT CHARSET=latin1
My Query:
select * from giao_vien inner join lop_mon on lop_mon.gv_key_index = giao_vien.key_index
where giao_vien.key_index = 'shcm_2019_01645344018'
When I explain it:
MySQL does not recognize the key:
KEY `idx_gv_key_index` (`gv_key_index`)
I check another query:
SELECT * FROM enetviet.lop_mon where gv_key_index = 'shcm_2019_01645344018';
The key idx_gv_key_index is still work. I don't know what is the problem.

mysql optimizer confusion explain plan

I am hoping someone can help decipher an explain plan and information in the optimizer trace.
I have the following query:
select doc_id,
group_concat(distinct concat(a.event_time, id mod 10) order by id) as times,
count(distinct id) as count_of_items
FROM doc_audit a,
rwfd1213310051e2821b404f29bc453ce7386f764e b
WHERE b.rptid = a.doc_id
AND a.property = 'status'
group by 1;
And get this execution plan.
*************************** 1. row ***************************
id: 1
select_type: SIMPLE
table: b
partitions: NULL
type: index
possible_keys: idx_tmp_rptid
key: idx_tmp_rptid
key_len: 8
ref: NULL
rows: 7153
filtered: 100.00
Extra: Using index; Using temporary; Using filesort
*************************** 2. row ***************************
id: 1
select_type: SIMPLE
table: a
partitions: NULL
type: ref
possible_keys: DOC_ID_PROPERTY_IDX,event_time,event_time2
key: DOC_ID_PROPERTY_IDX
key_len: 776
ref: database.b.rptid,const
rows: 2
filtered: 43.75
Extra: Using where
My limited understanding is the order in the explain plan is the order the plan is executed. If this is the case I don't understand where the "Using filsort" comes from on table b? If I enable the optimzer profile trace it suggests the filsort is related to table a. I reached this conclusion because doc_id is in table a. Furthermore I believe filsort simply means "Sort"? doc_id is indexed so I don't understand why it needs sorting further.
"filesort_information": [
{
"direction": "asc",
"table": "intermediate_tmp_table",
"field": "**doc_id**"
}
],
"filesort_priority_queue_optimization": {
"usable": false,
"cause": "not applicable (no LIMIT)"
},
"filesort_execution": [
],
"filesort_summary": {
"rows": 17620,
"examined_rows": 17620,
"number_of_tmp_files": 2,
"sort_buffer_size": 262128,
"sort_mode": "<sort_key, rowid>"
I would really appreciate any help on this. Apologies if my understanding is way off.
doc_audit
PRIMARY KEY (id),
KEY DOC_ID_PROPERTY_IDX (doc_id,property),
KEY event (instance_id,event,event_time),
KEY event_time (doc_id,event,event_time),
KEY event_user (instance_id,user_id,event,event_time),
KEY event_time2 (event_time,doc_id,event),
KEY user (user_id,event_time,instance_id),
KEY event_time_user_event (event_time,user_id,event(80))
Adding the table defs:
CREATE TABLE doc_audit (
id bigint(20) NOT NULL AUTO_INCREMENT,
doc_id bigint(20) NOT NULL,
event varchar(255) ,
event_time datetime NOT NULL,
instance_id bigint(20) NOT NULL,
major_version int(11) NOT NULL,
minor_version int(11) NOT NULL,
new_value varchar(1500) DEFAULT NULL,
old_value varchar(1500) DEFAULT NULL,
property varchar(255) DEFAULT NULL,
signature_meaning varchar(255) DEFAULT NULL,
task_name varchar(255) DEFAULT NULL,
user_id bigint(20) NOT NULL,
version int(11) DEFAULT NULL,
workflow_name varchar(255) DEFAULT NULL,
user_on_behalf_of bigint(20) DEFAULT NULL,
xml longtext,
user_name varchar(255) DEFAULT NULL,
user_display_name varchar(255) DEFAULT NULL,
document_number varchar(255) DEFAULT NULL,
property_public_key varchar(100) DEFAULT NULL,
property_label varchar(500) DEFAULT NULL,
old_value_public_key varchar(100) DEFAULT NULL,
old_value_label varchar(1500) DEFAULT NULL,
new_value_public_key varchar(100) DEFAULT NULL,
new_value_label varchar(1500) DEFAULT NULL,
workflow_label varchar(500) DEFAULT NULL,
task_label varchar(500) DEFAULT NULL,
precalculated bit(1) DEFAULT b'0',
PRIMARY KEY (id),
KEY DOC_ID_PROPERTY_IDX (doc_id,property),
KEY event (instance_id,event,event_time),
KEY event_time (doc_id,event,event_time),
KEY event_user (instance_id,user_id,event,event_time),
KEY event_time2 (event_time,doc_id,event),
KEY user (user_id,event_time,instance_id),
KEY event_time_user_event (event_time,user_id,event(80))
) ENGINE=InnoDB AUTO_INCREMENT=1209095 DEFAULT CHARSET=utf8
and..
CREATE TEMPORARY TABLE rwfd1213310051e2821b404f29bc453ce7386f764e (
wfrptid bigint(20) NOT NULL DEFAULT '0',
workflow_status__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
workflow_id__v bigint(20) DEFAULT NULL,
task_capacity_key__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
workflow_process_version__v bigint(20) DEFAULT NULL,
workflow_name__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
workflow_initiator_name__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
task_delegate__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
workflow_completionDate__v datetime DEFAULT NULL,
workflow_initiator__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
workflow_cancelationDate__v datetime DEFAULT NULL,
workflow_document_id__v bigint(20) DEFAULT NULL,
task_verdict_key__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
workflow_lifecycle__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
task_assignee_name__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
task_meta_data_id__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
task_completionDate__v datetime DEFAULT NULL,
workflow_startDate__v datetime DEFAULT NULL,
task_iteration_count__v bigint(20) DEFAULT NULL,
rptid bigint(20) NOT NULL DEFAULT '0',
major_version_number__v bigint(20) DEFAULT NULL,
minor_version_number__v bigint(20) DEFAULT NULL,
study__v varchar(1500) COLLATE utf8_bin DEFAULT NULL,
document_number__v varchar(255) COLLATE utf8_bin DEFAULT NULL,
status__v varchar(1500) COLLATE utf8_bin DEFAULT NULL,
name__v varchar(100) COLLATE utf8_bin DEFAULT NULL,
days_in_qc__c bigint(20) DEFAULT NULL,
durationInValue15f07c5893cf4ba98a3441aab045c460
varchar(255) COLLATE utf8_bin DEFAULT NULL,
0PERIOD int(6) DEFAULT NULL,
KEY idx_tmp_rptid (rptid)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_bin
Two exceptions to the order of tables in EXPLAIN:
Subqueries tend to be at the end of the EXPLAIN.
Using temporary and Using filesort tend to be on the first line of the EXPLAIN, giving you no clue of when they are actually used. Also, there may be more than one sort, again without a clue.
The EXPLAIN FORMAT=JSON is better at saying what part needed the sort. Multiple filesorts show up clearly.
"filesort" is a generic term. It might involve disk (slowest), it might be a 'quicksort' in RAM, or it might be "filesort_priority_queue_optimization" (handy with LIMIT). There may be other options.
(Please provide SHOW CREATE TABLE.)
event(80) -- Index prefixes are rarely useful. Was this a TEXT column?
The optimal index for this query for a is
INDEX(property, doc_id, event_time, id)

Mysql make 3 queries in 1 and improve performance

I'm trying to create a report and running 4 queries, but performance is so terrible.
I'm using 2 tables
This one has 2500 items in it
CREATE TABLE `bolt_accounts` (
`id` int(11) NOT NULL,
`slug` varchar(128) COLLATE utf8_unicode_ci NOT NULL,
`datecreated` datetime NOT NULL,
`datechanged` datetime NOT NULL,
`datepublish` datetime DEFAULT NULL,
`datedepublish` datetime DEFAULT NULL,
`username` varchar(32) COLLATE utf8_unicode_ci DEFAULT '',
`ownerid` int(11) DEFAULT NULL,
`status` varchar(32) COLLATE utf8_unicode_ci NOT NULL,
`templatefields` longtext COLLATE utf8_unicode_ci COMMENT '(DC2Type:json_array)',
`managerid` varchar(128) COLLATE utf8_unicode_ci DEFAULT '',
`parentid` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`name` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`qualify` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`regdate` date DEFAULT NULL,
`city` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`phone` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`passhash` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`cookie` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`resettoken` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`block` tinyint(1) NOT NULL DEFAULT '0',
`blocksms` tinyint(1) NOT NULL DEFAULT '0',
`birthday` date DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
ALTER TABLE `bolt_accounts`
ADD PRIMARY KEY (`id`),
ADD KEY `IDX_9C703491989D9B62` (`slug`),
ADD KEY `IDX_9C703491AFBA6FD8` (`datecreated`),
ADD KEY `IDX_9C703491BE74E59A` (`datechanged`),
ADD KEY `IDX_9C703491A5131421` (`datepublish`),
ADD KEY `IDX_9C703491B7805520` (`datedepublish`),
ADD KEY `IDX_9C7034917B00651C` (`status`),
ADD KEY `IDX_9C703491C13A5CC2` (`managerid`),
ADD KEY `IDX_9C703491856A684C` (`parentid`(255)),
ADD KEY `IDX_9C7034911E6AC3AE` (`regdate`),
ADD KEY `IDX_9C7034914709B432` (`birthday`);
and another one with all statistics, it has more than 1 400 000 items in it
CREATE TABLE `bolt_statistics` (
`id` int(11) NOT NULL,
`slug` varchar(128) COLLATE utf8_unicode_ci NOT NULL,
`datecreated` datetime NOT NULL,
`datechanged` datetime NOT NULL,
`datepublish` datetime DEFAULT NULL,
`datedepublish` datetime DEFAULT NULL,
`username` varchar(32) COLLATE utf8_unicode_ci DEFAULT '',
`ownerid` int(11) DEFAULT NULL,
`status` varchar(32) COLLATE utf8_unicode_ci NOT NULL,
`templatefields` longtext COLLATE utf8_unicode_ci COMMENT '(DC2Type:json_array)',
`managerid` varchar(256) COLLATE utf8_unicode_ci DEFAULT '',
`statdate` datetime DEFAULT NULL,
`lopv` double NOT NULL DEFAULT '0',
`gope` double NOT NULL DEFAULT '0',
`gopv` double NOT NULL DEFAULT '0'
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
ALTER TABLE `bolt_statistics`
ADD PRIMARY KEY (`id`),
ADD KEY `IDX_BE38DFD2989D9B62` (`slug`),
ADD KEY `IDX_BE38DFD2AFBA6FD8` (`datecreated`),
ADD KEY `IDX_BE38DFD2BE74E59A` (`datechanged`),
ADD KEY `IDX_BE38DFD2A5131421` (`datepublish`),
ADD KEY `IDX_BE38DFD2B7805520` (`datedepublish`),
ADD KEY `IDX_BE38DFD27B00651C` (`status`),
ADD KEY `IDX_BE38DFD2C13A5CC2` (`managerid`(255));
So the problem is, when I join this tables together, performance become low...
SELECT ba.managerid,name,replace(phone,'+','') as phone,passhash, date_format(ba.datepublish,'%d.%m.%Y %H:%i') as datepublish, max(bs.lopv) as lopv, max(bs.gopv) as gopv
FROM bolt_accounts ba
LEFT JOIN bolt_statistics bs ON ba.managerid=bs.managerid
WHERE (parentid='007-645930')
AND (date(ba.datechanged)=('2018-01-06'))
AND (date(bs.datecreated)=('2018-01-06'))
GROUP BY ba.managerid
ORDER BY gopv desc
this query will run for 360-450ms ~0,3 of a sec..
It will return all managerids that has parentid=007-645930
some thing like that:
managerid
007-663360
007-677590
007-697191
007-1526400
007-1155884
007-1842169
077-1564660
007-1883072
007-777143
007-1865946
007-1875083
007-1753407
007-1322124
007-1100631
007-1603795
007-1171656
007-1890892
007-1166247
007-1564611
007-1882959
007-1145375
007-1878383
007-1128857
007-1762655
007-1346877
007-1714252
007-1709538
007-1319044
007-1698517
007-1316756
007-1679094
007-1298984
007-1905146
007-1675451
007-1287166
007-1899632
007-1629224
007-1190862
007-1894824
007-1616741
007-1171665
007-1894330
Than I take 1 id from that list, and run 3 queries
SELECT max(s.lopv) as lopv, max(s.gopv) as gopv
FROM bolt_statistics s WHERE (managerid='007-663360')
AND (datecreated between DATE_FORMAT('2018-01-06' - INTERVAL 1 MONTH,'%Y-%m-28 23:00:00') and DATE_FORMAT(LAST_DAY('2018-01-06' - INTERVAL 1 MONTH),'%Y-%m-%d 23:59:59'))
execution time 20-25ms
SELECT max(s.lopv) as lopv, max(s.gopv) as gopv
FROM bolt_statistics s
WHERE (managerid='007-663360')
AND (date(datecreated) = date('2018-01-06' -INTERVAL 1 day))
execution time 15-20ms
SELECT max(s.lopv) as lopv, max(s.gopv) as gopv
FROM bolt_statistics s
WHERE (managerid='007-663360')
AND (date(datecreated) = date('2018-01-06' -INTERVAL 2 day))
execution time 15-20ms
When all executions are over, it took 1,5 sec (1500ms) to render the php report.
I know, that I'm not quite good at mysql querying ;)) but I wonder, how can I improve performance on that queries?
Will it be much faster if I union all this queries in 1?
Do those fields really need a full 256 characters? Change them to a reasonable number, then get rid of the prefixing on ADD KEY IDX_BE38DFD2C13A5CC2 (managerid(255)), etc. (Prefix indexes are often useless.)
Don't hid columns inside functions (date(ba.datechanged)). Instead:
AND ba.datechanged >= '2018-01-06' - INTERVAL 2 DAY
AND ba.datechanged < '2018-01-06' - INTERVAL 3 DAY
Note: The above pattern works fine regardless of what datatype datechanged is -- DATE, DATETIME, DATETIME(6), TIMESTAMP. And the Optimizer can make use of an index such as ...
After that, have the composite INDEX(managerid, datecreated) for significant performance improvement.
Use a derived table instead of LEFT JOIN plus GROUP BY. This is likely to improve speed a bunch.
What is status? Why VARCHAR(32)? If it is a simple, low-cardinality, value, don't index it by itself; the index won't be used.
(There may be more tips, but this should get you started.)

Select query taking too long in mysql

I have table with following create statement
CREATE TABLE `table1` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`workname` varchar(25) COLLATE utf8_bin DEFAULT NULL,
`setup_dt` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`setup_new_dt` timestamp NULL DEFAULT CURRENT_TIMESTAMP,
`check_type` varchar(20) COLLATE utf8_bin DEFAULT NULL,
`work_id` int(21) DEFAULT NULL,
`workstat` varchar(25) COLLATE utf8_bin DEFAULT NULL,
`data` longtext COLLATE utf8_bin,
`a_id` int(20) DEFAULT NULL,
`workstat2` varchar(20) COLLATE utf8_bin DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1031 DEFAULT CHARSET=utf8 COLLATE=utf8_bin
My select query is select * from table
My select query is taking quite a long time to execute.
What changes (such as indexing) can i make in the table to make sure that select queries are no more slow.
Is it taking long because of ``datalongtext COLLATE utf8_bin
Can I do some indexing to resolve this slowness?
When I do explain in front of query I get following output
# id, select_type, table, type, possible_keys, key, key_len, ref, rows, Extra
'1', 'SIMPLE', 'table1', 'ALL', NULL, NULL, NULL, NULL, '8756', NULL