Have the following table structure:
CREATE TABLE `PARSER_U_R_L` (
`PARSER_ID` varchar(20) COLLATE latin1_general_cs NOT NULL,
`URL_MD5` varchar(255) COLLATE latin1_general_cs NOT NULL,
`ENTRY_POINT_ID` varchar(20) COLLATE latin1_general_cs DEFAULT NULL,
`TYPE_ID` varchar(20) COLLATE latin1_general_cs DEFAULT NULL,
`STATUS_ID` varchar(20) COLLATE latin1_general_cs DEFAULT NULL,
`INDEXED_TIME` datetime DEFAULT NULL,
PRIMARY KEY (`PARSER_ID`,`URL_MD5`),
KEY `PURL_PARSER` (`PARSER_ID`))
ENGINE=InnoDB DEFAULT CHARSET=latin1 COLLATE=latin1_general_cs;
As you see PRIMARY KEY in the Parser_id and URL_MD5.
When I try a simple select
EXPLAIN
SELECT *
FROM `PARSER_U_R_L`
WHERE `URL_MD5` IN ( ids )
In EXPLAIN of this select I have possible keys = Null.
What can be problem ?
The problem is that the URL_MD5 is not the first column in your primary key. Since you have an index with multiple columns, the optimizer won't use that index unless you supply a value for the first column as well.
If you supply a value for just the first column, the optimizer will use it,
so try reversing the columns in the index.
Related
I'm running a count query which is very slow, how can improve this?
I've got the following query, but it takes around 1.33 seconds:
select
count(*) as aggregate
from
`tickets`
inner join `orders` on `orders`.`id` = `tickets`.`order_id`
where
`orders`.`status` = 'paid' and
`tickets`.`created_at` > '2023-01-01 00:00:00'
The tickets table has around 650000 rows and the order table has around 320000 rows.
This is the result of SHOW CREATE TABLE tickets:
CREATE TABLE `tickets` (
`id` bigint unsigned NOT NULL AUTO_INCREMENT,
`tickettype_id` int unsigned NOT NULL,
`order_id` int unsigned NOT NULL,
`variant_id` bigint unsigned DEFAULT NULL,
`seat_id` bigint unsigned DEFAULT NULL,
`barcode` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`first_name` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`last_name` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`email` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`telephone` varchar(191) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`birthday` date DEFAULT NULL,
`age` int unsigned DEFAULT NULL,
`gender` enum('m','f') CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`price` double(10,2) DEFAULT NULL,
`extra_info` varchar(255) CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `tickets_barcode_unique` (`barcode`),
KEY `tickets_tickettype_id_foreign` (`tickettype_id`),
KEY `tickets_order_id_foreign` (`order_id`),
KEY `tickets_order_id_index` (`order_id`),
KEY `tickets_tickettype_id_index` (`tickettype_id`),
KEY `tickets_seat_id_foreign` (`seat_id`),
KEY `tickets_variant_id_foreign` (`variant_id`),
CONSTRAINT `tickets_ibfk_1` FOREIGN KEY (`order_id`) REFERENCES `orders` (`id`) ON DELETE CASCADE,
CONSTRAINT `tickets_seat_id_foreign` FOREIGN KEY (`seat_id`) REFERENCES `seatplan_seats` (`id`) ON DELETE SET NULL,
CONSTRAINT `tickets_tickettype_id_foreign` FOREIGN KEY (`tickettype_id`) REFERENCES `tickets_types` (`id`) ON DELETE CASCADE,
CONSTRAINT `tickets_variant_id_foreign` FOREIGN KEY (`variant_id`) REFERENCES `ticket_variants` (`id`) ON DELETE SET NULL
) ENGINE=InnoDB AUTO_INCREMENT=2945088 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
How can I improve the speed?
The performance of your query depends on several factors. Such as,
The table size
Performance of your machine
Indexing etc.
If you dont have indices created for status, order_id and created_at, better create them. Which can significantly improve the query performance.
CREATE INDEX order_id_index ON tickets(order_id);
CREATE INDEX status_index ON orders(status);
CREATE INDEX created_at_index ON tickets(created_at);
Additionally, if you are using PostgreSQL, try running VACUUM on your tables which removes the dead tuples and improves performance.
First of all
You need to add two indexes :
CREATE INDEX order_id_idx ON tickets(order_id);
// composite index since you are using both columns in where
CREATE INDEX status_created_at_idx ON tickets(status, created_at);
The query optimizer uses the composite indexes for queries that test all columns in the index, or queries that test the first columns, the first two columns, and so on.
More informations regarding composite can be found here
Here is my posts table:
CREATE TABLE `posts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`date` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`img` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`vid` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`title` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`subtitle` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`auth` varchar(54) COLLATE utf8_croatian_ci NOT NULL,
`story` longtext COLLATE utf8_croatian_ci NOT NULL,
`tags` varchar(255) COLLATE utf8_croatian_ci NOT NULL,
`status` varchar(100) COLLATE utf8_croatian_ci NOT NULL,
`moder` varchar(50) COLLATE utf8_croatian_ci NOT NULL,
`rec` varchar(50) COLLATE utf8_croatian_ci NOT NULL,
`pos` varchar(50) COLLATE utf8_croatian_ci NOT NULL,
`inde` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=117 DEFAULT CHARSET=utf8 COLLATE=utf8_croatian_ci
I want to make two partitions in order to improve query performances.
First partition should contain all non-archive rows.
Second partition - all archive rows.
ALTER TABLE posts
PARTITION BY LIST COLUMNS (status)
(
PARTITION P1 VALUES IN ('admin', 'moder', 'public', 'rec'),
PARTITION P2 VALUES IN ('archive')
);
phpmyadmin error:
Static analysis:
1 errors were found during analysis.
Unrecognized alter operation. (near "" at position 0)
MySQL said:
#1503 - A PRIMARY KEY must include all columns in the table's partitioning function
Any help?
What queries are you trying to speed up? Since the only index you currently have, WHERE id=... or WHERE id BETWEEN ... AND ... are the only queries that will be fast. And the partitioning you suggest will not help much for other queries.
You seem to have only dozens of rows; don't consider partitioning unless you expect to have at least a million rows.
status has only 5 values? Then make it ENUM('archive', 'admin', 'moder', 'public', 'rec') NOT NULL. That will take 1 byte instead of lots.
If you will be querying on date and/or status and/or auth, then let's talk about indexes, especially 'composite' indexes on such. And, to achieve the "archive" split you envision, put status as the first column in the index.
Using MySQL 5.6 and the following table structure:
CREATE TABLE `dataitem` (
`AI` int(11) unsigned NOT NULL AUTO_INCREMENT,
`ID` binary(16) NOT NULL,
`OwnerID` binary(16) NOT NULL,
`DataItemTimeUtc` datetime NOT NULL,
`DataItemTimeLocal` datetime NOT NULL,
`DataItemTimeMicroSeconds` int(11) NOT NULL,
`DataItemArrivalTimeUtc` datetime NOT NULL DEFAULT '0001-01-01 00:00:00',
`DataItemTimeTimeZoneID` binary(16) NOT NULL,
`QuestionID` binary(16) NOT NULL,
`QuestionHistoryID` binary(16) DEFAULT NULL,
`QuestionAbsolutePositionID` varchar(1000) COLLATE utf8_unicode_ci DEFAULT NULL,
`GroupSessionIDString` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
`DataItemType` int(11) NOT NULL,
`DataEntryDevice` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
`DataEntryDeviceCradle` varchar(250) COLLATE utf8_unicode_ci DEFAULT NULL,
`DataItemXml` longtext COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`AI`),
UNIQUE KEY `dataitem_ID_UQ_Idx` (`ID`),
KEY `dataitem_OwnerID_Idx` (`OwnerID`),
KEY `dataitem_DataItemTimeUtc_Idx` (`DataItemTimeUtc`),
KEY `dataitem_QuestionID_Idx` (`QuestionID`),
KEY `dataitem_QuestionHistoryID_Idx` (`QuestionHistoryID`),
KEY `dataitem_QuestionAbsolutePositionID_Idx` (`QuestionAbsolutePositionID`(255)),
KEY `dataitem_DataItemType_Idx` (`DataItemType`)
) ENGINE=InnoDB AUTO_INCREMENT=23467 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
I am experiencing something that I am struggling to understand. The following query causes a fatal error because it is taking forever to execute:
Guid patientid = new Guid("cfed2acf-acbd-4ab2-8c23-7ab0b3a8cfa3");
var latestRecord = (from f in QueryHelper.GetEntityTable<DataItem>()
where
f.OwnerID == patientid
&& f.QuestionAbsolutePositionID == "5867FF5EC08B9C0422EFD1359B2802B29A8E167952D381EC70AE53CE6D4C9318"
orderby f.DataItemTimeUtc descending
select f.ID).FirstOrDefault();
However if I change .FirstOrDefault() to .ToArray() the query runs like a flash and returs 2 results. Can someone explain this?
SQL Query generated from .ToArray():
SELECT t0.`ID`
FROM `DataItem` AS t0
WHERE ((t0.`OwnerID` = #p0) AND (t0.`QuestionAbsolutePositionID` = #p1))
ORDER BY t0.`DataItemTimeUtc` DESC
-- p0 = [cfed2acf-acbd-4ab2-8c23-7ab0b3a8cfa3]
-- p1 = [5867FF5EC08B9C0422EFD1359B2802B29A8E167952D381EC70AE53CE6D4C9318]
SQL query generated from .FirstOrDefault():
SELECT t0.`ID`
FROM `DataItem` AS t0
WHERE ((t0.`OwnerID` = #p0) AND (t0.`QuestionAbsolutePositionID` = #p1))
ORDER BY t0.`DataItemTimeUtc` DESC
LIMIT 0, 1
-- p0 = [cfed2acf-acbd-4ab2-8c23-7ab0b3a8cfa3]
-- p1 = [5867FF5EC08B9C0422EFD1359B2802B29A8E167952D381EC70AE53CE6D4C9318]
First, figure out why QuestionAbsolutePositionID needs to be 1000 characters long. It it can be less than 256, make it so. If no, then ask yourself whether it can be changed to CHARACTER SET ascii. It looks like hex, which works fine with ascii. (Rarely do "ids" include accented letters, Cyrillic, Japanese, etc.) If neither of those 'fixes' are possible, can you upgrade to MySQL 5.7?
Once you have fixed the problem of index size (above), add this 'composite' (and 'covering') index; it should speed up the query:
INDEX(OwnerID, QuestionAbsolutePositionID, DataItemTimeUtc, ID)
(The first two columns can be in either order.)
If it does not help, then we need to discuss the #variables.
I have following table - 'element'
CREATE TABLE `element` (
`eid` bigint(22) NOT NULL AUTO_INCREMENT,
`tag_name` varchar(45) COLLATE utf8_bin DEFAULT NULL,
`text` text COLLATE utf8_bin,
`depth` tinyint(2) DEFAULT NULL,
`classes` tinytext COLLATE utf8_bin,
`webarchiver_uniqueid` int(11) DEFAULT NULL,
`created` datetime DEFAULT NULL,
`updated` datetime DEFAULT NULL,
`rowstatus` char(1) COLLATE utf8_bin DEFAULT 'A',
PRIMARY KEY (`eid`)
) ENGINE=InnoDB AUTO_INCREMENT=12090 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
Column details and current index details are given above. Almost 90% of queries on this table are like:
select * from element
where tag_name = 'XXX'
and text = 'YYYY'
and depth = 20
and classes = 'ZZZZZ'
and rowstatus = 'A'
What would be the most optimal way to create index on this table? The table has around 60k rows.
Change classes from TINYTEXT to VARCHAR(255) (or some more reasonable size), then have
INDEX(tag_name, depth, classes)
with the columns in any order. I left out rowstatus because it smells like a column that is likely to change. (Anyway, a flag does not add much to an index.)
You can't include TEXT or BLOB columns in an index. And it is not worth it to do a 'prefix' index.
Since a PRIMARY KEY is a UNIQUE key, DROP INDEX eid_UNIQUE.
Is there some reason for picking "binary" / "utf8_bin" for all the character fields?
So i get an error when i try and use
SELECT views, keywords, title, url, thumbnail,
MATCH(keywords,title) AGAINST ('%$search_value%') AS relevance
FROM straight
WHERE MATCH (keywords,title) AGAINST ('%$search_value%')
ORDER BY relevance DESC
This is due to me not having FULLtext search enabled, but i cant seem to enable it. when i run the sql below:
ALTER TABLE straight ADD FULLTEXT(keywords, title)
i get this response:
MySQL returned an empty result set (i.e. zero rows). (Query took 3.8022 sec)
Then when trying to run the first query again i get the failed
#1191 - Can't find FULLTEXT index matching the column list
I can't tell why it's not registering. Any help would be great.
Thanks!
Edit:
My tabel:
CREATE TABLE `straight` (
`url` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`title` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`keywords` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`production` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`categories` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`views` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`likes` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`length` varchar(80) COLLATE utf8_unicode_ci DEFAULT NULL,
`thumbnail` varchar(200) COLLATE utf8_unicode_ci DEFAULT NULL,
`date` varchar(12) COLLATE utf8_unicode_ci DEFAULT NULL,
UNIQUE KEY `url` (`url`),
FULLTEXT KEY `url_2` (`url`,`title`,`keywords`,`production`,
`categories`,`views`,`likes`,`length`,`thumbnail`,`date`
), ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
You need a FULLTEXT index that matches, exactly, the columns upon which you want to search. The FULLTEXT index you have has more columns than you need.
Try adding the one you mentioned.
ALTER TABLE straight ADD FULLTEXT(keywords, title)
Then look at the table definition and make sure it's there.