Related
looking for help to speed up this query:
SELECT tickets.ticketid, datenew, products.name
FROM tickets
INNER JOIN ticketlines ON tickets.id = ticketlines.ticket
INNER JOIN products ON ticketlines.product = products.id
INNER JOIN receipts ON receipts.id = tickets.id
WHERE (category !='feb765ef-c8a8-4fa2-969c-90f67fe6b3be' AND category!='888f4893-f300-43b5-9933-d549ade744e0' AND category !='8f2031e8-64a4-4abf-8175-3d2bedd9f950' AND category !='ca370ced-1c3b-434c-905e-ec1bc709543b' AND category !='f92ff0ac-fa11-4a5f-a3dd-e0d9ed9c171a' AND category !='445e8605-1cd9-4714-b3fd-7389ac29c206' and category !='05143c54-8a7e-4ce2-97cc-f84f9cf41395' AND category !='8c78afea-b9e2-44cf-b497-c384045b3202' AND category !='95919f7f-ff2e-4aa1-8110-ef63c022c01b' AND category !='f4f88b05-38a1-4956-9182-4c04a0808df7') AND datedone IS NULL
ORDER BY ticketid
This is from a cash register database that I am using to pull data for order display. The structure is the receipt has the primary id, the timein and timedone(DATEDONE) timestamps. The ticket table has ticket id which is the same as receipt id. Ticketlines table is the line items on the receipt/ticket. And then Products table has the human readable definitions of the products.
The query is to pull all items that aren't completed, i.e. DATEDONE is null, and display the items that were ordered.
Ticket and Receipt tables have 15K rows, Ticketlines has ~20K rows, Products has 1.5k.
Pretty small data. But this query takes over 20 seconds. I think since I am using primary key IDs for everything, I don't need to index anything, but I am a total noob, so I'm happy to be told I'm wrong. I'll appreciate any help and can provide any further details. Thanks!
EDIT:
per comments, I am showing table structures. I'm sorry for the formatting nightmare, I'm unfamiliar with this platform and unsure how to make it more readable :-(
| receipts | CREATE TABLE `receipts` (
`ID` varchar(255) NOT NULL,
`MONEY` varchar(255) NOT NULL,
`DATENEW` datetime NOT NULL,
`ATTRIBUTES` mediumblob,
`PERSON` varchar(255) DEFAULT NULL,
`DATEDONE` datetime DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `RECEIPTS_FK_MONEY` (`MONEY`),
KEY `RECEIPTS_INX_1` (`DATENEW`),
CONSTRAINT `RECEIPTS_FK_MONEY` FOREIGN KEY (`MONEY`) REFERENCES `closedcash` (`MONEY`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| tickets | CREATE TABLE `tickets` (
`ID` varchar(255) NOT NULL,
`TICKETTYPE` int(11) NOT NULL DEFAULT '0',
`TICKETID` int(11) NOT NULL,
`PERSON` varchar(255) NOT NULL,
`CUSTOMER` varchar(255) DEFAULT NULL,
`STATUS` int(11) NOT NULL DEFAULT '0',
`DONE` varchar(255) DEFAULT NULL,
PRIMARY KEY (`ID`),
KEY `TICKETS_FK_2` (`PERSON`),
KEY `TICKETS_CUSTOMERS_FK` (`CUSTOMER`),
KEY `TICKETS_TICKETID` (`TICKETTYPE`,`TICKETID`),
CONSTRAINT `TICKETS_CUSTOMERS_FK` FOREIGN KEY (`CUSTOMER`) REFERENCES `customers` (`ID`),
CONSTRAINT `TICKETS_FK_2` FOREIGN KEY (`PERSON`) REFERENCES `people` (`ID`),
CONSTRAINT `TICKETS_FK_ID` FOREIGN KEY (`ID`) REFERENCES `receipts` (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| ticketlines | CREATE TABLE `ticketlines` (
`TICKET` varchar(255) NOT NULL,
`LINE` int(11) NOT NULL,
`PRODUCT` varchar(255) DEFAULT NULL,
`ATTRIBUTESETINSTANCE_ID` varchar(255) DEFAULT NULL,
`UNITS` double NOT NULL,
`PRICE` double NOT NULL,
`TAXID` varchar(255) NOT NULL,
`ATTRIBUTES` mediumblob,
PRIMARY KEY (`TICKET`,`LINE`),
KEY `TICKETLINES_FK_2` (`PRODUCT`),
KEY `TICKETLINES_ATTSETINST` (`ATTRIBUTESETINSTANCE_ID`),
KEY `TICKETLINES_FK_3` (`TAXID`),
CONSTRAINT `TICKETLINES_ATTSETINST` FOREIGN KEY (`ATTRIBUTESETINSTANCE_ID`) REFERENCES `attributesetinstance` (`ID`),
CONSTRAINT `TICKETLINES_FK_2` FOREIGN KEY (`PRODUCT`) REFERENCES `products` (`ID`),
CONSTRAINT `TICKETLINES_FK_3` FOREIGN KEY (`TAXID`) REFERENCES `taxes` (`ID`),
CONSTRAINT `TICKETLINES_FK_TICKET` FOREIGN KEY (`TICKET`) REFERENCES `tickets` (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| products | CREATE TABLE `products` (
`ID` varchar(255) NOT NULL,
`REFERENCE` varchar(255) NOT NULL,
`CODE` varchar(255) NOT NULL,
`CODETYPE` varchar(255) DEFAULT NULL,
`NAME` varchar(255) NOT NULL,
`PRICEBUY` double NOT NULL,
`PRICESELL` double NOT NULL,
`CATEGORY` varchar(255) NOT NULL,
`TAXCAT` varchar(255) NOT NULL,
`ATTRIBUTESET_ID` varchar(255) DEFAULT NULL,
`STOCKCOST` double DEFAULT NULL,
`STOCKVOLUME` double DEFAULT NULL,
`IMAGE` mediumblob,
`ISCOM` bit(1) NOT NULL DEFAULT b'0',
`ISSCALE` bit(1) NOT NULL DEFAULT b'0',
`ISKITCHEN` bit(1) NOT NULL DEFAULT b'0',
`PRINTKB` bit(1) NOT NULL DEFAULT b'0',
`SENDSTATUS` bit(1) NOT NULL DEFAULT b'0',
`ISSERVICE` bit(1) NOT NULL DEFAULT b'0',
`ATTRIBUTES` mediumblob,
`DISPLAY` varchar(255) DEFAULT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `PRODUCTS_INX_0` (`REFERENCE`),
UNIQUE KEY `PRODUCTS_INX_1` (`CODE`),
UNIQUE KEY `PRODUCTS_NAME_INX` (`NAME`),
KEY `PRODUCTS_FK_1` (`CATEGORY`),
KEY `PRODUCTS_TAXCAT_FK` (`TAXCAT`),
KEY `PRODUCTS_ATTRSET_FK` (`ATTRIBUTESET_ID`),
CONSTRAINT `PRODUCTS_ATTRSET_FK` FOREIGN KEY (`ATTRIBUTESET_ID`) REFERENCES `attributeset` (`ID`),
CONSTRAINT `PRODUCTS_FK_1` FOREIGN KEY (`CATEGORY`) REFERENCES `categories` (`ID`),
CONSTRAINT `PRODUCTS_TAXCAT_FK` FOREIGN KEY (`TAXCAT`) REFERENCES `taxcategories` (`ID`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
Also, here is EXPLAIN output:
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------------+------------+--------+--------------------------+---------+---------+------------------------------------+-------+----------+----------------------------------------------+
| 1 | SIMPLE | receipts | NULL | ALL | PRIMARY | NULL | NULL | NULL | 14624 | 10.00 | Using where; Using temporary; Using filesort |
| 1 | SIMPLE | tickets | NULL | eq_ref | PRIMARY | PRIMARY | 767 | receipts.ID | 1 | 100.00 | NULL |
| 1 | SIMPLE | ticketlines | NULL | ref | PRIMARY,TICKETLINES_FK_2 | PRIMARY | 767 | receipts.ID | 1 | 100.00 | Using where |
| 1 | SIMPLE | products | NULL | eq_ref | PRIMARY,PRODUCTS_FK_1 | PRIMARY | 767 | ticketlines.PRODUCT | 1 | 97.97 | Using where |
+----+-------------+-------------+------------+--------+--------------------------+---------+---------+------------------------------------+-------+----------+----------------------------------------------+
4 rows in set, 1 warning (0.04 sec)
Try changing data types of all columns to the minimal required ones.
Add indexes on columns in WHERE.
Add index on ticketlines.ticket.
I ended up deleted 10K rows from every linked table and it queries running in .2 seconds now. I guess absent any more advice, i will be doing annual cleanup of old tickets.
Here's my db tables:
#1 User
Create Table: CREATE TABLE `User` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`password` varchar(128) NOT NULL,
`last_login` datetime(6) DEFAULT NULL,
`is_superuser` tinyint(1) NOT NULL,
`username` varchar(150) DEFAULT NULL,
`first_name` varchar(30) NOT NULL,
`last_name` varchar(150) NOT NULL,
`email` varchar(254) NOT NULL,
`is_staff` tinyint(1) NOT NULL,
`is_active` tinyint(1) NOT NULL,
`date_joined` datetime(6) NOT NULL,
`user_name` varchar(150) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=133 DEFAULT CHARSET=utf8
# settlement
Create Table: CREATE TABLE `settlement` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`title` varchar(255) NOT NULL,
`short_title` varchar(25) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `title` (`title`),
UNIQUE KEY `short_title` (`short_title`)
) ENGINE=InnoDB AUTO_INCREMENT=3553 DEFAULT CHARSET=utf8
# sign
Create Table: CREATE TABLE `sign` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`our_side_entity` smallint(6) NOT NULL,
`contact` varchar(63) DEFAULT NULL,
`address` varchar(255) NOT NULL,
`emails` varchar(1023) NOT NULL,
`settlement_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `sign_our_side_entity_settlement_id_6ccd6800_uniq` (`our_side_entity`,`settlement_id`),
KEY `sign_settlement_id_07f0b7bb_fk_settlement_id` (`settlement_id`),
CONSTRAINT `sign_ibfk_1` FOREIGN KEY (`settlement_id`) REFERENCES `settlement` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3684 DEFAULT CHARSET=utf8
# sale
Create Table: CREATE TABLE `sale` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`is_leader` tinyint(1) NOT NULL,
`default_leader_id` int(11) NOT NULL,
`sale_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `default_leader_id` (`default_leader_id`),
KEY `sale_id` (`sale_id`),
CONSTRAINT `sale_ibfk_1` FOREIGN KEY (`default_leader_id`) REFERENCES `User` (`id`),
CONSTRAINT `sale_ibfk_2` FOREIGN KEY (`sale_id`) REFERENCES `User` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=47 DEFAULT CHARSET=utf8
# account_token
Create Table: CREATE TABLE `account_token` (
`token_id` varchar(32) NOT NULL,
`token_type` smallint(6) NOT NULL,
`id` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=133 DEFAULT CHARSET=utf8
# account
Create Table: CREATE TABLE `account` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`account_id` varchar(45) NOT NULL,
`entity` varchar(255) NOT NULL,
`is_pc` tinyint(1) DEFAULT NULL,
`type` smallint(6) NOT NULL,
`settlement_type` smallint(6) NOT NULL,
`medium` smallint(6) NOT NULL,
`ae_id` int(11) NOT NULL,
`sale_id` int(11) DEFAULT NULL,
`sign_id` int(11) NOT NULL,
`sale_manage_id` int(11) DEFAULT NULL,
`is_new_customer` tinyint(1) DEFAULT '0',
`agency` tinyint(1) DEFAULT '0',
`related_entity` varchar(255) DEFAULT NULL,
`token_id` varchar(45) DEFAULT NULL,
`account_token_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `account_account_id_medium_b46819c8_uniq` (`account_id`) USING BTREE,
KEY `account_ae_id_4d0b721c_fk_auth_user_id` (`ae_id`),
KEY `account_sale_id_7770cbad_fk_auth_user_id` (`sale_id`),
KEY `account_sign_id_17d08191_fk_sign_id` (`sign_id`),
KEY `sale_manage_id` (`sale_manage_id`),
KEY `account_token` (`account_token_id`),
CONSTRAINT `account_ibfk_1` FOREIGN KEY (`ae_id`) REFERENCES `User` (`id`),
CONSTRAINT `account_ibfk_2` FOREIGN KEY (`sale_id`) REFERENCES `User` (`id`),
CONSTRAINT `account_ibfk_3` FOREIGN KEY (`sign_id`) REFERENCES `sign` (`id`),
CONSTRAINT `account_ibfk_4` FOREIGN KEY (`sale_manage_id`) REFERENCES `sale` (`id`),
CONSTRAINT `account_ibfk_5` FOREIGN KEY (`account_token_id`) REFERENCES `account_token` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=119532 DEFAULT CHARSET=utf8
# sale_leader
Create Table: CREATE TABLE `sale_leader` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`year` smallint(6) NOT NULL,
`month` smallint(6) NOT NULL,
`leader_id` int(11) DEFAULT NULL,
`sale_manage_id` int(11) NOT NULL,
`date` date DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `leader_id` (`leader_id`),
KEY `sale_manage_id` (`sale_manage_id`),
CONSTRAINT `sale_leader_ibfk_1` FOREIGN KEY (`leader_id`) REFERENCES `User` (`id`),
CONSTRAINT `sale_leader_ibfk_2` FOREIGN KEY (`sale_manage_id`) REFERENCES `sale` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=985 DEFAULT CHARSET=utf8
# Table: spend_daily_level
Create Table: CREATE TABLE `spend_daily_level` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`date` date NOT NULL,
`system_value` decimal(16,2) NOT NULL,
`checked_value` decimal(16,2) NOT NULL,
`account_id` int(11) NOT NULL,
`sale_leader_id` int(11) DEFAULT NULL,
`account_status` tinyint(3) DEFAULT '1',
PRIMARY KEY (`id`),
UNIQUE KEY `spend_daily_level_date_account_id_f38b1186_uniq` (`date`,`account_id`),
KEY `spend_daily_level_account_id_f6df4f99_fk_account_id` (`account_id`),
KEY `sale_leader_id` (`sale_leader_id`),
KEY `date_active` (`active`,`date`),
CONSTRAINT `spend_daily_level_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `account` (`id`),
CONSTRAINT `spend_daily_level_ibfk_2` FOREIGN KEY (`sale_leader_id`) REFERENCES `sale_leader` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3682983 DEFAULT CHARSET=utf8
The final query made by ORM is:
SELECT `User`.`user_name`, `account`.`medium`, `account`.`type`, `settlement`.`title`,
`spend_daily_level`.`system_value`, T7.`user_name`, `spend_daily_level`.`date`
FROM `spend_daily_level` INNER JOIN `account` ON (`spend_daily_level`.`account_id` = `account`.`id`)
LEFT OUTER JOIN `User` ON (`account`.`sale_id` = `User`.`id`)
INNER JOIN `sign` ON (`account`.`sign_id` = `sign`.`id`)
INNER JOIN `settlement` ON (`sign`.`settlement_id` = `settlement`.`id`)
LEFT OUTER JOIN `sale_leader` ON (`spend_daily_level`.`sale_leader_id` = `sale_leader`.`id`)
LEFT OUTER JOIN `User` T7 ON (`sale_leader`.`leader_id` = T7.`id`)
WHERE (`spend_daily_level`.`date` >= '2020-05-13' AND `spend_daily_level`.`date` <= '2020-10-28');
Explain SQL ABOVE:
+----+-------------+-------------------+--------+-----------------------------------------------------------------------------------------------------+-----------------------------------------------------+---------+----------------------------------------------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+--------+-----------------------------------------------------------------------------------------------------+-----------------------------------------------------+---------+----------------------------------------------+------+-------------+
| 1 | SIMPLE | settlement | index | PRIMARY | title | 767 | NULL | 3563 | Using index |
| 1 | SIMPLE | sign | ref | PRIMARY,sign_settlement_id_07f0b7bb_fk_settlement_id | sign_settlement_id_07f0b7bb_fk_settlement_id | 4 | project.settlement.id | 1 | Using index |
| 1 | SIMPLE | account | ref | PRIMARY,account_sign_id_17d08191_fk_sign_id | account_sign_id_17d08191_fk_sign_id | 4 | project.sign.id | 28 | NULL |
| 1 | SIMPLE | User | eq_ref | PRIMARY | PRIMARY | 4 | project.account.sale_id | 1 | NULL |
| 1 | SIMPLE | spend_daily_level | ref | spend_daily_level_date_account_id_f38b1186_uniq,spend_daily_level_account_id_f6df4f99_fk_account_id | spend_daily_level_account_id_f6df4f99_fk_account_id | 4 | project.account.id | 13 | Using where |
| 1 | SIMPLE | sale_leader | eq_ref | PRIMARY | PRIMARY | 4 | project.spend_daily_level.sale_leader_id | 1 | NULL |
| 1 | SIMPLE | T7 | eq_ref | PRIMARY | PRIMARY | 4 | project.sale_leader.leader_id | 1 | NULL |
+----+-------------+-------------------+--------+-----------------------------------------------------------------------------------------------------+-----------------------------------------------------+---------+----------------------------------------------+------+-------------+
And it cost me about 8s with total 145k rows data. How can I speed up this query?
Great Thanks.
Updated with adding index suggested
# User count(110)
Create Table: CREATE TABLE `User` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`password` varchar(128) NOT NULL,
`last_login` datetime(6) DEFAULT NULL,
`is_superuser` tinyint(1) NOT NULL,
`username` varchar(150) DEFAULT NULL,
`first_name` varchar(30) NOT NULL,
`last_name` varchar(150) NOT NULL,
`email` varchar(254) NOT NULL,
`is_staff` tinyint(1) NOT NULL,
`is_active` tinyint(1) NOT NULL,
`date_joined` datetime(6) NOT NULL,
`user_name` varchar(150) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`) USING BTREE,
KEY `user_idx` (`id`,`user_name`)
) ENGINE=InnoDB AUTO_INCREMENT=136 DEFAULT CHARSET=utf8
# settlement count(4151)
Create Table: CREATE TABLE `settlement` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`title` varchar(255) NOT NULL,
`short_title` varchar(25) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `title` (`title`),
UNIQUE KEY `short_title` (`short_title`),
KEY `settle_idx` (`id`,`title`)
) ENGINE=InnoDB AUTO_INCREMENT=4162 DEFAULT CHARSET=utf8
# sign count(4306)
Create Table: CREATE TABLE `sign` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`our_side_entity` smallint(6) NOT NULL,
`contact` varchar(63) DEFAULT NULL,
`address` varchar(255) NOT NULL,
`emails` varchar(1023) NOT NULL,
`settlement_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `sign_our_side_entity_settlement_id_6ccd6800_uniq` (`our_side_entity`,`settlement_id`),
KEY `sign_settlement_id_07f0b7bb_fk_settlement_id` (`settlement_id`),
KEY `sign_idx` (`id`,`settlement_id`),
CONSTRAINT `sign_ibfk_1` FOREIGN KEY (`settlement_id`) REFERENCES `settlement` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4314 DEFAULT CHARSET=utf8
# sale (46)
Create Table: CREATE TABLE `sale` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`is_leader` tinyint(1) NOT NULL,
`default_leader_id` int(11) NOT NULL,
`sale_id` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `default_leader_id` (`default_leader_id`),
KEY `sale_id` (`sale_id`),
CONSTRAINT `sale_ibfk_1` FOREIGN KEY (`default_leader_id`) REFERENCES `User` (`id`),
CONSTRAINT `sale_ibfk_2` FOREIGN KEY (`sale_id`) REFERENCES `User` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=48 DEFAULT CHARSET=utf8
# account_token (125)
Create Table: CREATE TABLE `account_token` (
`token_id` varchar(32) NOT NULL,
`token_type` smallint(6) NOT NULL,
`id` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=133 DEFAULT CHARSET=utf8
# account (134663)
Create Table: CREATE TABLE `account` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`account_id` varchar(45) NOT NULL,
`entity` varchar(255) NOT NULL,
`is_pc` tinyint(1) DEFAULT NULL,
`type` smallint(6) NOT NULL,
`settlement_type` smallint(6) NOT NULL,
`medium` smallint(6) NOT NULL,
`ae_id` int(11) NOT NULL,
`sale_id` int(11) DEFAULT NULL,
`sign_id` int(11) NOT NULL,
`sale_manage_id` int(11) DEFAULT NULL,
`is_new_customer` tinyint(1) DEFAULT '0',
`agency` tinyint(1) DEFAULT '0',
`related_entity` varchar(255) DEFAULT NULL,
`account_token_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `account_account_id_medium_b46819c8_uniq` (`account_id`) USING BTREE,
KEY `account_ae_id_4d0b721c_fk_auth_user_id` (`ae_id`),
KEY `account_sale_id_7770cbad_fk_auth_user_id` (`sale_id`),
KEY `account_sign_id_17d08191_fk_sign_id` (`sign_id`),
KEY `sale_manage_id` (`sale_manage_id`),
KEY `account_token_id` (`account_token_id`),
KEY `account_idx` (`id`,`sale_id`,`sign_id`,`medium`,`type`),
CONSTRAINT `account_ibfk_1` FOREIGN KEY (`ae_id`) REFERENCES `User` (`id`),
CONSTRAINT `account_ibfk_2` FOREIGN KEY (`sale_id`) REFERENCES `User` (`id`),
CONSTRAINT `account_ibfk_3` FOREIGN KEY (`sign_id`) REFERENCES `sign` (`id`),
CONSTRAINT `account_ibfk_4` FOREIGN KEY (`sale_manage_id`) REFERENCES `sale` (`id`),
CONSTRAINT `account_ibfk_5` FOREIGN KEY (`account_token_id`) REFERENCES `account_token` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=134910 DEFAULT CHARSET=utf8
# sale_leader (1003)
Create Table: CREATE TABLE `sale_leader` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`year` smallint(6) NOT NULL,
`month` smallint(6) NOT NULL,
`leader_id` int(11) DEFAULT NULL,
`sale_manage_id` int(11) NOT NULL,
`date` date DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `leader_id` (`leader_id`),
KEY `sale_manage_id` (`sale_manage_id`),
CONSTRAINT `sale_leader_ibfk_1` FOREIGN KEY (`leader_id`) REFERENCES `User` (`id`),
CONSTRAINT `sale_leader_ibfk_2` FOREIGN KEY (`sale_manage_id`) REFERENCES `sale` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1007 DEFAULT CHARSET=utf8
# spend_daily_level (2,177,793)
Create Table: CREATE TABLE `spend_daily_level` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) NOT NULL,
`created_time` datetime(6) NOT NULL,
`updated_time` datetime(6) NOT NULL,
`date` date NOT NULL,
`system_value` decimal(16,2) NOT NULL,
`checked_value` decimal(16,2) NOT NULL,
`account_id` int(11) NOT NULL,
`sale_leader_id` int(11) DEFAULT NULL,
`account_status` tinyint(3) DEFAULT '1',
PRIMARY KEY (`id`),
UNIQUE KEY `spend_daily_level_date_account_id_f38b1186_uniq` (`date`,`account_id`),
KEY `spend_daily_level_account_id_f6df4f99_fk_account_id` (`account_id`),
KEY `sale_leader_id` (`sale_leader_id`),
KEY `date_active` (`active`,`date`),
KEY `spend_idx` (`date`,`account_id`,`system_value`),
CONSTRAINT `spend_daily_level_ibfk_1` FOREIGN KEY (`account_id`) REFERENCES `account` (`id`),
CONSTRAINT `spend_daily_level_ibfk_2` FOREIGN KEY (`sale_leader_id`) REFERENCES `sale_leader` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=4181988 DEFAULT CHARSET=utf8
EXPLAIN SELECT `User`.`user_name`, `account`.`medium`, `account`.`type`, `settlement`.`title`, `spend_daily_level`.`system_value`, T7.`user_name`, `spend_daily_level`.`date` FROM `spend_daily_level` INNER JOIN `account` ON (`spend_daily_level`.`account_id` = `account`.`id`) LEFT OUTER JOIN `User` ON (`account`.`sale_id` = `User`.`id`) INNER JOIN `sign` ON (`account`.`sign_id` = `sign`.`id`) INNER JOIN `settlement` ON (`sign`.`settlement_id` = `settlement`.`id`) LEFT OUTER JOIN `sale_leader` ON (`spend_daily_level`.`sale_leader_id` = `sale_leader`.`id`) LEFT OUTER JOIN `User` T7 ON (`sale_leader`.`leader_id` = T7.`id`) WHERE (`spend_daily_level`.`date` >= '2020-05-13' AND `spend_daily_level`.`date` <= '2020-10-28');
+----+-------------+-------------------+--------+---------------------------------------------------------------------------------------------------------------+-----------------------------------------------------+---------+----------------------------------------------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-------------------+--------+---------------------------------------------------------------------------------------------------------------+-----------------------------------------------------+---------+----------------------------------------------+------+-------------+
| 1 | SIMPLE | settlement | index | PRIMARY,settle_idx | title | 767 | NULL | 4214 | Using index |
| 1 | SIMPLE | sign | ref | PRIMARY,sign_settlement_id_07f0b7bb_fk_settlement_id,sign_idx | sign_settlement_id_07f0b7bb_fk_settlement_id | 4 | bv_crm_1029.settlement.id | 1 | Using index |
| 1 | SIMPLE | account | ref | PRIMARY,account_sign_id_17d08191_fk_sign_id,account_idx | account_sign_id_17d08191_fk_sign_id | 4 | bv_crm_1029.sign.id | 20 | NULL |
| 1 | SIMPLE | User | eq_ref | PRIMARY,user_idx | PRIMARY | 4 | bv_crm_1029.account.sale_id | 1 | NULL |
| 1 | SIMPLE | spend_daily_level | ref | spend_daily_level_date_account_id_f38b1186_uniq,spend_daily_level_account_id_f6df4f99_fk_account_id,spend_idx | spend_daily_level_account_id_f6df4f99_fk_account_id | 4 | bv_crm_1029.account.id | 20 | Using where |
| 1 | SIMPLE | sale_leader | eq_ref | PRIMARY | PRIMARY | 4 | bv_crm_1029.spend_daily_level.sale_leader_id | 1 | NULL |
| 1 | SIMPLE | T7 | eq_ref | PRIMARY,user_idx | PRIMARY | 4 | bv_crm_1029.sale_leader.leader_id | 1 | NULL |
+----+-------------+-------------------+--------+---------------------------------------------------------------------------------------------------------------+-----------------------------------------------------+---------+----------------------------------------------+------+-------------+
It seems that the index added not used.
You may try adding the following indices:
CREATE INDEX spend_idx ON spend_daily_level (date, account_id, system_value);
CREATE INDEX account_idx ON account (id, sale_id, sign_id, medium, type);
CREATE INDEX user_idx ON User (id, user_name);
CREATE INDEX sign_idx ON sign (id, settlement_id);
CREATE INDEX settle_idx ON settlement (id, title);
These first index spend_idx covers the entire WHERE clause. It, along with the other indices, cover the joins and the select clause.
The query below returns a set of User and for each row a number of relations from the user perpective who is searching (id = 4)
SELECT `users`.`firstname` AS firstname,
`users`.`lastname` AS lastname,
COUNT(`trusted_users`.`id`) AS number_of_friend_in_common,
CASE ... AS friend,
CASE ... AS facebook_invitable,
CASE ... AS address_book_invitable,
CASE ... AS virtual_user,
FROM `users`
LEFT OUTER JOIN `trusted_users`
ON `trusted_users`.`user_id` = 4 AND `trusted_users`.`trust_user_id` = `users`.`id`
LEFT OUTER JOIN `facebook_friends`
ON (`facebook_friends`.`user_id` = 4 AND `facebook_friends`.`friend_user_id` = `users`.`id`
OR `facebook_friends`.`user_id` = `users`.`id` AND `facebook_friends`.`friend_user_id` = 4)
LEFT OUTER JOIN `address_book_contacts`
ON `address_book_contacts`.`owner_id` = 4 AND `address_book_contacts`.`email_digest` = `users`.`email_digest`
LEFT OUTER JOIN `friends`
ON (`friends`.`me_id` = `users`.`id` AND `friends`.`him_id` = 4
OR `friends`.`me_id` = 4 AND `friends`.`him_id` = `users`.`id`)
WHERE `users`.`id` NOT IN
(SELECT CASE
WHEN `friends`.`me_id` = 4 THEN `friends`.`him_id`
ELSE `friends`.`me_id`
END
FROM `friends`
WHERE (`friends`.`status` = 0
AND `friends`.`him_id` = 4
AND `friends`.`him_status` = 7
OR `friends`.`status` = 0
AND `friends`.`me_id` = 4
AND `friends`.`me_status` = 7))
AND (`users`.`firstname` LIKE '%a%' OR `users`.`lastname` LIKE '%a%')
GROUP BY `users`.`id`
ORDER BY friend DESC,
facebook_invitable DESC,
address_book_invitable DESC,
number_of_friend_in_common DESC,
virtual_user DESC,
firstname,
lastname LIMIT 0, 20
Number of row for each table:
trusted_users: 255k
facebook_friends: 1k
address_book_contacts: 1.5M
friends: 70k
users: 32k
All fields of join are indexed. The query takes 1.1s which is not acceptable for the amount of data we have.
What am I doing wrong? Should I split in multiple query and paginate my self?
Edit 1: EXPLAIN result
+----+-------------+-----------------------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+---------+----------------------------------+-------+----------------------------------------------------------------------------------------------------------------------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+-------------+-----------------------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+---------+----------------------------------+-------+----------------------------------------------------------------------------------------------------------------------------+
| 1 | PRIMARY | users | ALL | PRIMARY,index_users_on_chat_id,index_users_login_facebook_id,index_users_on_login,index_users_on_parent_id,index_users_account_type,index_users_email_digest,index_users_id | NULL | NULL | NULL | 31847 | Using where; Using temporary; Using filesort |
| 1 | PRIMARY | trusted_users | ref | index_trusted_users_user,index_trusted_users_trust_user | index_trusted_users_trust_user | 5 | messenger_dev.users.id | 6 | Using where |
| 1 | PRIMARY | facebook_friends | index_merge | index_facebook_friends_user,index_facebook_friends_friend | index_facebook_friends_user,index_facebook_friends_friend | 5,5 | NULL | 2 | Using union(index_facebook_friends_user,index_facebook_friends_friend); Using where; Using join buffer (Block Nested Loop) |
| 1 | PRIMARY | address_book_contacts | ref | index_address_book_contacts_owner_id,index_address_book_contacts_email | index_address_book_contacts_email | 767 | messenger_dev.users.email_digest | 1 | Using where |
| 1 | PRIMARY | friends | index_merge | index_friends_me_him,index_friends_me,index_friends_him | index_friends_him,index_friends_me | 5,5 | NULL | 18 | Using union(index_friends_him,index_friends_me); Using where; Using join buffer (Block Nested Loop) |
| 2 | SUBQUERY | friends | index_merge | index_friends_me_him,index_friends_me,index_friends_him | index_friends_him,index_friends_me | 5,5 | NULL | 18 | Using union(index_friends_him,index_friends_me); Using where |
+----+-------------+-----------------------+-------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------------------------------------------------------+---------+----------------------------------+-------+----------------------------------------------------------------------------------------------------------------------------+
6 rows in set (0,00 sec)
Edit 2: Table structure
CREATE TABLE `address_book_contacts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`email_digest` varchar(191) DEFAULT NULL,
`code` varchar(191) DEFAULT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
`owner_id` int(11) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_address_book_contacts_owner_id` (`owner_id`),
KEY `index_address_book_contacts_email` (`email_digest`)
) ENGINE=InnoDB AUTO_INCREMENT=1598109 DEFAULT CHARSET=utf8mb4;
CREATE TABLE `trusted_users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`friend_id` int(11) DEFAULT NULL,
`user_id` int(11) DEFAULT NULL,
`trust_user_id` int(11) DEFAULT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `index_trusted_users_on_friend_id` (`friend_id`),
KEY `index_trusted_users_user` (`user_id`),
KEY `index_trusted_users_trust_user` (`trust_user_id`),
CONSTRAINT `fk_rails_007c31c802` FOREIGN KEY (`trust_user_id`) REFERENCES `users` (`id`),
CONSTRAINT `fk_rails_ca24cb4e23` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=275576 DEFAULT CHARSET=utf8mb4;
CREATE TABLE `facebook_friends` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) DEFAULT NULL,
`friend_user_id` int(11) DEFAULT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
`code` varchar(191) DEFAULT NULL,
PRIMARY KEY (`id`),
KEY `index_facebook_friends_user` (`user_id`),
KEY `index_facebook_friends_friend` (`friend_user_id`),
KEY `index_facebook_friends_code` (`code`(5)),
CONSTRAINT `fk_rails_78285a074e` FOREIGN KEY (`friend_user_id`) REFERENCES `users` (`id`),
CONSTRAINT `fk_rails_aa3ac53a81` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=1149 DEFAULT CHARSET=utf8mb4;
CREATE TABLE `friends` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`me_id` int(11) DEFAULT NULL,
`him_id` int(11) DEFAULT NULL,
`owner_id` int(11) DEFAULT NULL,
`status` int(11) DEFAULT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
`me_status` int(11) DEFAULT '0',
`him_status` int(11) DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `index_friends_me_him` (`me_id`,`him_id`),
KEY `index_friends_me` (`me_id`),
KEY `index_friends_him` (`him_id`),
KEY `index_friends_owner` (`owner_id`),
CONSTRAINT `fk_rails_9fa3474d31` FOREIGN KEY (`owner_id`) REFERENCES `users` (`id`),
CONSTRAINT `fk_rails_d3ebb6657f` FOREIGN KEY (`him_id`) REFERENCES `users` (`id`),
CONSTRAINT `fk_rails_fccfd1b821` FOREIGN KEY (`me_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=95724 DEFAULT CHARSET=utf8mb4;
CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`account_type` varchar(191) NOT NULL,
`firstname` varchar(191) DEFAULT NULL,
`lastname` varchar(191) DEFAULT NULL,
`login` varchar(191) DEFAULT NULL,
`avatar` varchar(191) DEFAULT NULL,
`gender` varchar(1) DEFAULT NULL,
`locale` varchar(191) DEFAULT NULL,
`birthdate` date DEFAULT NULL,
`password_digest` varchar(191) DEFAULT NULL,
`email_digest` varchar(191) DEFAULT NULL,
`created_at` datetime NOT NULL,
`updated_at` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `index_users_on_login` (`login`),
KEY `index_users_account_type` (`account_type`),
KEY `index_users_email_digest` (`email_digest`),
KEY `index_uses_firstname` (`firstname`),
KEY `index_users_lastname` (`lastname`)
) ENGINE=InnoDB AUTO_INCREMENT=32516 DEFAULT CHARSET=utf8mb4;
It looks like MySQL is not choosing any of the available indicies for the users table.
First, run ANALYZE TABLE users;, then re-run the EXPLAIN command. Is the value of the first rows cell now substantially lower than 31847? If so, your problem should be solved!
If not, run OPTIMIZE TABLE users;, then re-run the EXPLAIN command. Is the value of the first rows cell now substantially lower than 31847? If so, your problem should be solved!
If neither of those steps help, try adding USE INDEX (PRIMARY) or USE INDEX (users_id) immediately after the FROM users portion of your query.
Hope this helps!
I want to create a table so that I have an ID number that is based on the date, AND an ID as a unique, primary key.
Ie:
2015-2-1-1
2015-2-1-2
but, if I create:
2015-2-2-1
the counter should restart at 1.
I've tried using the following:
CREATE TABLE `invoices` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`date` date NOT NULL,
`po_id` int(11) unsigned DEFAULT NULL,
`description` varchar(256) NOT NULL,
`client_id` int(11) unsigned DEFAULT NULL,
`status` enum('unpaid','paid','partial') DEFAULT 'unpaid',
PRIMARY KEY (`id`,`date`),
UNIQUE KEY `id_UNIQUE` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=latin1;
But it doesn't work like I want.
If you want a two column primary key (date, id) and id that should auto increment starting from 1 each date. Just set date, for your primary column key, and id in second position in your key definition.
CREATE TABLE `invoices` (
`date` date NOT NULL,
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`po_id` int(11) unsigned DEFAULT NULL,
`description` varchar(256) NOT NULL,
`client_id` int(11) unsigned DEFAULT NULL,
`status` enum('unpaid','paid','partial') DEFAULT 'unpaid',
PRIMARY KEY (`date`, `id`)
) ENGINE=MyIsam DEFAULT CHARSET=latin1;
The internal key will be set as your example, but you will see two columns:
+----+------------+
| id | date |
+----+------------+
| 1 | 2012-01-01 |
| 2 | 2012-01-01 |
| 1 | 2012-01-02 |
| 2 | 2012-01-02 |
+----+------------+
EDIT: You have to use MyIsam format
I am still new to SQL and I am trying to improve the performance of my query. I have been searching around and have come to the conclusion that using JOINS instead of so many WHERE INS would help improve my performance, but I am unsure of how I would convert my statement. This is my current statement.
SELECT stop_id, stop_name FROM stops WHERE stop_id IN (
SELECT DISTINCT stop_id FROM stop_times WHERE trip_id IN (
SELECT trip_id from trips WHERE route_id = <routeid> ));
It takes anywhere from 5-25 seconds to return the results which is unacceptable. I was hoping to get it below 1 second. If anyone was wondering the data is from a GTFS feed. The stops and trips tables have about ~10,000 rows each, while the stop_times table has ~900,000. I have created indexes at each of the columns I am using. Here is the output of EXPLAIN, and also what was used to create each table.
Thanks for any help and if you need any more info let me know!
+----+--------------------+------------+-----------------+------------------+---------+---------+------+------+-------------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | Extra |
+----+--------------------+------------+-----------------+------------------+---------+---------+------+------+-------------+
| 1 | PRIMARY | stops | ALL | NULL | NULL | NULL | NULL | 6481 | Using where |
| 2 | DEPENDENT SUBQUERY | stop_times | index_subquery | stop_id | stop_id | 63 | func | 63 | Using where |
| 3 | DEPENDENT SUBQUERY | trips | unique_subquery | PRIMARY,route_id | PRIMARY | 62 | func | 1 | Using where |
+----+--------------------+------------+-----------------+------------------+---------+---------+------+------+-------------+
| stops | CREATE TABLE `stops` (
`stop_id` varchar(20) NOT NULL,
`stop_code` varchar(50) DEFAULT NULL,
`stop_name` varchar(255) DEFAULT NULL,
`stop_desc` varchar(255) DEFAULT NULL,
`stop_lat` decimal(8,6) DEFAULT NULL,
`stop_lon` decimal(8,6) DEFAULT NULL,
`zone_id` int(11) DEFAULT NULL,
`stop_url` varchar(255) DEFAULT NULL,
`location_type` int(2) DEFAULT NULL,
`parent_station` int(11) DEFAULT NULL,
`wheelchair_boarding` int(2) DEFAULT NULL,
PRIMARY KEY (`stop_id`),
KEY `zone_id` (`zone_id`),
KEY `stop_lat` (`stop_lat`),
KEY `stop_lon` (`stop_lon`),
KEY `location_type` (`location_type`),
KEY `parent_station` (`parent_station`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| stop_times | CREATE TABLE `stop_times` (
`trip_id` varchar(20) DEFAULT NULL,
`arrival_time` varchar(8) DEFAULT NULL,
`arrival_time_seconds` int(11) DEFAULT NULL,
`departure_time` varchar(8) DEFAULT NULL,
`departure_time_seconds` int(11) DEFAULT NULL,
`stop_id` varchar(20) DEFAULT NULL,
`stop_sequence` int(11) DEFAULT NULL,
`stop_headsign` varchar(50) DEFAULT NULL,
`pickup_type` int(2) DEFAULT NULL,
`drop_off_type` int(2) DEFAULT NULL,
`shape_dist_traveled` varchar(50) DEFAULT NULL,
KEY `trip_id` (`trip_id`),
KEY `arrival_time_seconds` (`arrival_time_seconds`),
KEY `departure_time_seconds` (`departure_time_seconds`),
KEY `stop_id` (`stop_id`),
KEY `stop_sequence` (`stop_sequence`),
KEY `pickup_type` (`pickup_type`),
KEY `drop_off_type` (`drop_off_type`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
| trips | CREATE TABLE `trips` (
`route_id` varchar(20) DEFAULT NULL,
`service_id` varchar(20) DEFAULT NULL,
`trip_id` varchar(20) NOT NULL,
`trip_headsign` varchar(255) DEFAULT NULL,
`trip_short_name` varchar(255) DEFAULT NULL,
`direction_id` tinyint(1) DEFAULT NULL,
`block_id` int(11) DEFAULT NULL,
`shape_id` varchar(50) DEFAULT NULL,
PRIMARY KEY (`trip_id`),
KEY `route_id` (`route_id`),
KEY `service_id` (`service_id`),
KEY `direction_id` (`direction_id`),
KEY `block_id` (`block_id`),
KEY `shape_id` (`shape_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
You're right in thinking that JOINS are usually faster than WHERE IN subqueries.
Try this:
SELECT T3.stop_id, T3.stop_name
FROM trips AS T1
JOIN
stop_times AS T2
ON T1.trip_id=T2.trip_id AND route_id = <routeid>
JOIN stops AS T3
ON T2.stop_id=T3.stop_id
GROUP BY T3.stop_id, T3.stop_name