I am using MySQL 8.0.20 and I have a huge table with some data in it.
CREATE TABLE `db_table` (
`utcDateTime` datetime DEFAULT NULL,
`dateTime` datetime DEFAULT NULL,
`col1` varchar(250) DEFAULT NULL,
`col2` varchar(10) DEFAULT NULL,
`col3` varchar(5) NOT NULL DEFAULT '1',
`col4` varchar(5) NOT NULL DEFAULT '1',
`id` int NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`),
UNIQUE KEY `idx_dateIns` (`utcDateTime`,`col2`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8'
Below is the query I am trying to optimise,
SELECT col3,col4
from db_table
WHERE col2='123456'
AND (dateTime BETWEEN '2020-11-25 00:01' AND '2020-11-26 00:00')
group by utcDateTime
order by utcDateTime asc limit 1;
This query takes about 12 seconds to execute. But if I change the sorting order to desc, it takes about 0.015 seconds.
Can someone please help optimise the query or the database table?
Thank you
Related
With Mysql 5.6.10, I have a table like this:
CREATE TABLE `es_user_action` (
`id` bigint(11) unsigned NOT NULL AUTO_INCREMENT,
`user_id` bigint(11) unsigned NOT NULL,
`company_id` bigint(11) unsigned NOT NULL,
`work_id` bigint(11) unsigned NOT NULL,
`action` tinyint(2) NOT NULL COMMENT '10, 20, 30, 40',
`action_id` bigint(11) unsigned DEFAULT NULL,
`apply_id` bigint(11) unsigned DEFAULT '0',
`apply_display_id` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`action_time` datetime NOT NULL,
`scout_time` datetime NOT NULL,
`register_datetime_sorting` datetime DEFAULT NULL,
`score` tinyint(2) DEFAULT '0',
`is_pending` tinyint(2) DEFAULT '0',
`apply_status` tinyint(2) DEFAULT '2' COMMENT '1: paid, 2: free',
`has_response` tinyint(2) DEFAULT '0',
`response_time` datetime DEFAULT NULL,
`is_shown` tinyint(2) DEFAULT '1',
`source` varchar(15) COLLATE utf8mb4_unicode_ci NOT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `user_company_work` (`user_id`,`work_id`,`company_id`,`apply_id`,`source`),
KEY `IDX_2` (`company_id`,`is_shown`,`apply_status`,`apply_id`,`work_id`,`action_time`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=436896779 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
The full query I want to perform on the table would be the following:
SELECT
`id` AS `seqNo`,
`company_id` AS `companyId`,
`work_id` AS `workId`,
`action` AS `userActionType`,
`action_time` AS `userActionDatetime`,
`is_pending`,
`apply_status`,
`action_id`,
`source`,
`user_id` AS `userId`,
`apply_display_id`
FROM
`es_user_action`
WHERE
`company_id` = 449664
AND `is_shown` = 1
AND `apply_status` = 1
AND `apply_id` = 0
AND `action_time` >= '2021-01-05 15:56:14'
ORDER BY
is_pending ASC,
action ASC,
score DESC,
CASE
source
WHEN "entenshoku" THEN
action_time
END DESC,
CASE
WHEN source <> "entenshoku" THEN
action_time
END ASC
LIMIT 100 OFFSET 0;
The table has around 15 million rows and the following query takes around 15 seconds. The query becomes very slow.
Can anyone help out? Thanks in advance.
UPDATED:
This is the explain query result:
KEY `IDX_2` (`company_id`,`is_shown`,`apply_status`,`apply_id`,`work_id`,`action_time`)
Work_id is in the way of that index being effective for that SELECT. Remove it. That should speed up the query some. Still the are too many columns and expressions involved in the WHER and ORDER BY to do much more to help.
Changing BIGINTs to smaller INT types will help some (by shrinking the table).
I'm pretty sure that what you have cannot work.
You have is a dynamically changing ORDER BY, depending on the value of source. But which row's source?
If you could decide before issuing the query whether to scan action_time ascending or descending, which value of source would you pick?
Delete this Question, rewrite the query, then open another Question if you still have an issue.
I have a query like that:
SELECT * , (
( 1584392725 ) - ( suprayts.time )
) AS timeDiff
FROM (
`suprayts`
)
WHERE `suprayts`.`is_deleted` = '0'
AND `suprayts`.`is_approved` =1
AND `suprayts`.`username` != 'rayben1'
AND `suprayts`.`time` >1584306325
ORDER BY `suprayts`.`is_boosted_by_user` DESC , `suprayts`.`id` ASC
LIMIT 10
This query runs very slow (avg 0.2 seconds), if i delete the following line:
AND `suprayts`.`username` != 'rayben1'
It runs 10x faster. (avg 0.02 secs) How can i speed up this query?
My indexes:
Explain:
My table:
CREATE TABLE IF NOT EXISTS `suprayts` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`username` varchar(15) CHARACTER SET utf8 NOT NULL,
`question` varchar(70) COLLATE utf8mb4_unicode_ci NOT NULL,
`suprayt_photo` varchar(50) CHARACTER SET utf8 NOT NULL,
`time` int(11) NOT NULL,
`endTime` int(11) NOT NULL,
`datetext` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`like_count` int(10) unsigned NOT NULL DEFAULT '0',
`dislike_count` int(10) unsigned NOT NULL DEFAULT '0',
`is_approved` bit(1) NOT NULL DEFAULT b'0',
`is_deleted` enum('1','0') CHARACTER SET utf8 NOT NULL DEFAULT '0',
`is_end_notification_sent` bit(1) NOT NULL DEFAULT b'0',
`open_vote` enum('0','1') CHARACTER SET utf8 NOT NULL DEFAULT '1',
`boost` int(11) NOT NULL DEFAULT '0',
`is_boosted_by_user` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`id`),
UNIQUE KEY `username_3` (`username`,`suprayt_photo`),
KEY `id` (`id`,`time`,`is_approved`,`is_deleted`),
KEY `username` (`username`,`is_deleted`),
KEY `username_2` (`username`,`datetext`),
KEY `id_2` (`id`,`username`,`time`,`is_approved`,`is_deleted`),
KEY `username_4` (`username`,`time`,`is_approved`,`is_deleted`),
KEY `ix1` (`id`,`time`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci AUTO_INCREMENT=130789 ;
Depending on the SQL pre-compiler if one is being used, or the SQL itself, possible it is sensitive to the order of the query "where" clause not being in DB key order and it is not using the index and instead doing a sequential scan? Just to eliminate the possibility, try putting the WHERE items in the DB index key order.
USE NOT EXISTS
SELECT * , (
( 1584392725 ) - ( suprayts.time )
) AS timeDiff
FROM (
`suprayts`
)
WHERE `suprayts`.`is_deleted` = '0'
AND `suprayts`.`is_approved` =1
AND NOT EXISTS (
SELECT x.no FROM (SELECT 1 AS no) x WHERE `suprayts`.`username` = 'rayben1'
)
AND `suprayts`.`time` >1584306325
ORDER BY `suprayts`.`is_boosted_by_user` DESC , `suprayts`.`id` ASC
LIMIT 10
My problem is a slow search query with a one-to-many relationship between the tables. My tables look like this.
Table Assignment
CREATE TABLE `Assignment` (
`Id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`ProjectId` int(10) unsigned NOT NULL,
`AssignmentTypeId` smallint(5) unsigned NOT NULL,
`AssignmentNumber` varchar(30) NOT NULL,
`AssignmentNumberExternal` varchar(50) DEFAULT NULL,
`DateStart` datetime DEFAULT NULL,
`DateEnd` datetime DEFAULT NULL,
`DateDeadline` datetime DEFAULT NULL,
`DateCreated` datetime DEFAULT NULL,
`Deleted` datetime DEFAULT NULL,
`Lat` double DEFAULT NULL,
`Lon` double DEFAULT NULL,
PRIMARY KEY (`Id`),
KEY `idx_assignment_assignment_type_id` (`AssignmentTypeId`),
KEY `idx_assignment_assignment_number` (`AssignmentNumber`),
KEY `idx_assignment_assignment_number_external`
(`AssignmentNumberExternal`)
) ENGINE=InnoDB AUTO_INCREMENT=5280 DEFAULT CHARSET=utf8;
Table ExtraFields
CREATE TABLE `ExtraFields` (
`assignment_id` int(10) unsigned NOT NULL,
`name` varchar(30) NOT NULL,
`value` text,
PRIMARY KEY (`assignment_id`,`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
My search query
SELECT
`Assignment`.`Id`, COL_5_72, COL_5_73, COL_5_74, COL_5_75, COL_5_76,
COL_5_77 FROM (
SELECT
`Assignment`.`Id`,
`Assignment`.`AssignmentNumber` AS COL_5_72,
`Assignment`.`AssignmentNumberExternal` AS COL_5_73 ,
`AssignmentType`.`Name` AS COL_5_74,
`Assignment`.`DateStart` AS COL_5_75,
`Assignment`.`DateEnd` AS COL_5_76,
`Assignment`.`DateDeadline` AS COL_5_77 FROM `Assignment`
CASE WHEN `ExtraField`.`Name` = "WorkDistrict" THEN
`ExtraField`.`Value` end as COL_5_78 FROM `Assignment`
LEFT JOIN `ExtraFields` as `ExtraField` on
`ExtraField`.`assignment_id` = `Assignment`.`Id`
WHERE `Assignment`.`Deleted` IS NULL -- Assignment should not be removed.
AND (1=1) -- Add assignment filters.
) AS q1
GROUP BY `Assignment`.`Id`
HAVING 1 = 1
AND COL_5_78 LIKE '%Amsterdam East%'
ORDER BY COL_5_72 ASC, COL_5_73 ASC;
When the table is only around 3500 records my query takes a couple of seconds to execute and return the results.
What is a better way to search in the related data? Should I just add a JSON field to the Assignment table and use the MySQL 5.7 Json query features? Or did I made a mistake in designing my database?
You are using select from subquery that forces MySQL to create unindexed temp table for each execution. Remove subquery (you really don't need it here) and it will be much faster.
I need to create mysql table with default value on column CURRENT_DATE()
I try
DROP TABLE IF EXISTS `visitors`;
CREATE TABLE `visitors` (
`id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`ip` VARCHAR(32) NOT NULL,
`browser` VARCHAR(500) NOT NULL,
`version` VARCHAR(500) NOT NULL,
`platform` ENUM('w','l','m') NOT NULL,
`date` TIMESTAMP NOT NULL DEFAULT CURRENT_DATE(),
PRIMARY KEY (`id`),
UNIQUE KEY `person` (`ip`,`date`)
) ENGINE=INNODB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
but it is writting an error
Query: CREATE TABLE `visitors` ( `id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT, `ip` VARCHAR(32) NOT NULL, `browser` VARCHAR(500) NO...
Error Code: 1064
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'CURRENT_DATE() NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `person` (`ip`,`date`)
) ENGINE=INNODB AUTO_INCREMENT=1' at line 7
Execution Time : 0 sec
Transfer Time : 0 sec
Total Time : 0.063 sec
what is the problem?
I need to uniques only date
not with full time info...
can you help me?
Use CURRENT_TIMESTAMP function instead of CURRENT_DATE() function
Try this:
DROP TABLE IF EXISTS `visitors`;
CREATE TABLE `visitors` (
`id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`ip` VARCHAR(32) NOT NULL,
`browser` VARCHAR(500) NOT NULL,
`version` VARCHAR(500) NOT NULL,
`platform` ENUM('w','l','m') NOT NULL,
`date` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
UNIQUE KEY `person` (`ip`,`date`)
) ENGINE=INNODB AUTO_INCREMENT=1 DEFAULT CHARSET=utf8;
you just have to replace CURRENT_DATE() by NOW() in your query.
I tried it and it's look ok.
This may be a little late but I think it might be helpful to someone else.
My approach has been to use getdate() like this:
[TimeStamp] [datetime] NOT NULL CONSTRAINT [DF_myTable_TimeStamp] DEFAULT (getdate()).
Where [TimeStamp] is the column in question.
The result is for example: 2017-11-02 11:58:34.203
To trim this, I use the following
declare #mydate datetime
set #mydate = '2017-11-02 11:58:34.203'
SELECT try_convert(nvarchar(20), #mydate, 120)
This final result is: 2017-11-02 11:58:34
You can actually set this in MSSQL Management Studio
I would like to optimise an update query of this table as it currently takes a couple of seconds to execute. I am using MySQL 5.6
UPDATE skus1 SET `available` = '1', `locked` = NULL WHERE `locked` IS NOT NULL AND `locked` < (NOW() - INTERVAL 30 MINUTE)
I would like to optimise this query also
UPDATE skus1 SET `available` = '2', `locked` = NOW() WHERE sku = '111111'
I have a SELECT query that runs fast now I've added my indexes
Here is my table structure
CREATE TABLE `skus1` (
`id` int(10) NOT NULL AUTO_INCREMENT,
`sku` varchar(9) NOT NULL,
`available` tinyint(1) NOT NULL DEFAULT '1',
`locked` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `sku_2` (`sku`,`locked`),
KEY `available` (`available`),
KEY `locked` (`locked`),
KEY `sku` (`sku`)
) ENGINE=MyISAM AUTO_INCREMENT=500001 DEFAULT CHARSET=utf8