My slow query log is showing 5,6+ second runs randomly of a seemingly normal query. Running the same query on my local clone of the site gives a 0.05s query run time. Here is the EXPLAIN of both:
I'm a bit dense when it comes to MySQL optimization but, I don't understand how this can give such different results? The tables are roughly the same, the live site is more updated but otherwise very similar.
Edit:
Instead of comparing to my local installation, I instead am comparing to my "old" live server. I just moved servers this morning.
The OLD server DDL of the table:
CREATE TABLE `wp_bp_activity` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) NOT NULL,
`component` varchar(75) COLLATE utf8mb4_unicode_ci NOT NULL,
`type` varchar(75) COLLATE utf8mb4_unicode_ci NOT NULL,
`action` text COLLATE utf8mb4_unicode_ci NOT NULL,
`content` longtext COLLATE utf8mb4_unicode_ci NOT NULL,
`primary_link` text COLLATE utf8mb4_unicode_ci NOT NULL,
`item_id` bigint(20) NOT NULL,
`secondary_item_id` bigint(20) DEFAULT NULL,
`date_recorded` datetime NOT NULL,
`hide_sitewide` tinyint(1) DEFAULT 0,
`mptt_left` int(11) NOT NULL DEFAULT 0,
`mptt_right` int(11) NOT NULL DEFAULT 0,
`is_spam` tinyint(1) NOT NULL DEFAULT 0,
PRIMARY KEY (`id`),
KEY `date_recorded` (`date_recorded`),
KEY `user_id` (`user_id`),
KEY `item_id` (`item_id`),
KEY `secondary_item_id` (`secondary_item_id`),
KEY `component` (`component`),
KEY `type` (`type`),
KEY `mptt_left` (`mptt_left`),
KEY `mptt_right` (`mptt_right`),
KEY `hide_sitewide` (`hide_sitewide`),
KEY `is_spam` (`is_spam`),
FULLTEXT KEY `content` (`content`)
) ENGINE=InnoDB AUTO_INCREMENT=1060622 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
The NEW server DDL:
CREATE TABLE `wp_bp_activity` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`user_id` bigint(20) NOT NULL,
`component` varchar(75) COLLATE utf8mb4_unicode_ci NOT NULL,
`type` varchar(75) COLLATE utf8mb4_unicode_ci NOT NULL,
`action` text COLLATE utf8mb4_unicode_ci NOT NULL,
`content` longtext COLLATE utf8mb4_unicode_ci NOT NULL,
`primary_link` text COLLATE utf8mb4_unicode_ci NOT NULL,
`item_id` bigint(20) NOT NULL,
`secondary_item_id` bigint(20) DEFAULT NULL,
`date_recorded` datetime NOT NULL,
`hide_sitewide` tinyint(1) DEFAULT 0,
`mptt_left` int(11) NOT NULL DEFAULT 0,
`mptt_right` int(11) NOT NULL DEFAULT 0,
`is_spam` tinyint(1) NOT NULL DEFAULT 0,
PRIMARY KEY (`id`),
KEY `date_recorded` (`date_recorded`),
KEY `user_id` (`user_id`),
KEY `item_id` (`item_id`),
KEY `secondary_item_id` (`secondary_item_id`),
KEY `component` (`component`),
KEY `type` (`type`),
KEY `mptt_left` (`mptt_left`),
KEY `mptt_right` (`mptt_right`),
KEY `hide_sitewide` (`hide_sitewide`),
KEY `is_spam` (`is_spam`),
FULLTEXT KEY `content` (`content`)
) ENGINE=InnoDB AUTO_INCREMENT=1060840 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci
They are the same, but as you can see at the bottom of each the number of rows (by the AUTO_INCREMENT). They are very similar. As I said this OLD server was active this morning, so there are many recent datetimes here as well.
Running the query pictured above on both servers however, results in the same difference in "EXPLAIN" output, the NEW server is examining an insane amount of rows in 10x the time.
Use a "composite" index (in this order):
INDEX(is_spam, hide_sitewide, type)
Tip: When EXPLAIN says "Intersect", you need a composite index.
Get rid of a.type NOT IN ... since that test is already handled by a.type IN ...
Get rid of DISTINCT, it may be causing an extra pass to de-dup.
Use EXPLAIN FORMAT=JSON SELECT ... to get more detailed info.
Use text, not images, in this forum.
There may be a difference in version between the two servers -- and this may explain the different optimization technique used (or mis-used).
Related
query is simple, as below:
select count(1) from ec_account a join ec_card b on a.id = b.AccountId
there are 2.5 million rows in either ec_account and ec_card.(InnoDB)
here is the execution plan:
execution plan
as you see,
it already added index and used it, but the query still costed almost 60 seconds, is there any way could optimize it except changing database(mariadb has no such choke point as far as i know).
here is table DDL,ec_ccount:
CREATE TABLE `ec_account` (
`Id` varchar(64) NOT NULL,
`AccountType` varchar(32) NOT NULL,
`Name` varchar(32) NOT NULL,
`Status` tinyint(3) unsigned NOT NULL,
`IDCardType` varchar(32) DEFAULT NULL,
`IDCardNo` varchar(64) DEFAULT NULL,
`Password` varchar(256) DEFAULT NULL,
`PasswordHalt` varchar(128) DEFAULT NULL,
`Sex` varchar(8) DEFAULT NULL,
`BirthDay` datetime NOT NULL,
`Mobile` varchar(16) DEFAULT NULL,
`Address` varchar(64) DEFAULT NULL,
`Linkman` varchar(32) DEFAULT NULL,
`LinkmanRelation` varchar(16) DEFAULT NULL,
`LinkmanTel` varchar(16) DEFAULT NULL,
`Remark` varchar(128) DEFAULT NULL,
`Nationality` varchar(32) DEFAULT NULL,
`Nation` varchar(32) DEFAULT NULL,
`MaritalStatus` varchar(8) DEFAULT NULL,
`NativePlace` varchar(64) DEFAULT NULL,
`Occupation` varchar(32) DEFAULT NULL,
`BloodType` varchar(8) DEFAULT NULL,
`Education` varchar(8) DEFAULT NULL,
`LinkmanAddress` varchar(64) DEFAULT NULL,
`HomeAddress` varchar(128) DEFAULT NULL,
`Email` varchar(64) DEFAULT NULL,
`CompanyName` varchar(64) DEFAULT NULL,
`CompanyAddress` varchar(128) DEFAULT NULL,
`CompanyTel` varchar(16) DEFAULT NULL,
`Creator` char(36) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`CreateTime` datetime NOT NULL,
`LastModifier` char(36) CHARACTER SET utf8 COLLATE utf8_bin DEFAULT NULL,
`LastModifyTime` datetime DEFAULT NULL,
`Avatar` longblob,
PRIMARY KEY (`Id`),
KEY `IX_Name` (`Name`) USING HASH,
KEY `Idx_IDCard_Account` (`IDCardType`,`IDCardNo`) USING HASH,
KEY `Idx_Mobile` (`Mobile`) USING HASH,
KEY `Idx_CreateTime` (`CreateTime`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
and ec_card :
CREATE TABLE `ec_card` (
`Id` char(36) CHARACTER SET utf8 COLLATE utf8_bin NOT NULL DEFAULT '',
`AccountId` varchar(64) NOT NULL,
`CardType` varchar(32) NOT NULL,
`CardNo` varchar(32) NOT NULL,
`Status` tinyint(3) unsigned NOT NULL,
`IsPasswordAuth` tinyint(1) NOT NULL,
PRIMARY KEY (`Id`),
UNIQUE KEY `Idx_Unique_AccountId_CardType` (`AccountId`,`CardType`) USING HASH,
UNIQUE KEY `Idx_Unique_CardType_CardNo` (`CardType`,`CardNo`) USING HASH,
KEY `Idx_Uniques_AccountId` (`AccountId`) USING BTREE,
CONSTRAINT `FK_ec_card_ec_account_AccountId` FOREIGN KEY (`AccountId`) REFERENCES `ec_account` (`Id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Not without fundamentally changing the query.
There are no conditions on your query! It selects all 2.5 million rows from ec_card, as well as every matching row from ec_account. Reading all this data from disk and sending it over the network is the bottleneck; there is no way to change that without changing what the query does.
Here is a workaround for you. I think it would run much faster, and get the same result.
Calculate the total count of ec_account:
SELECT count(1) AS total_count FROM ec_account;
Calculate the amount of records those existed in ec_account but not existed in ec_card:
SELECT count(1) AS missing_count
FROM ec_account a LEFT JOIN ec_card b on a.id = b.AccountId
WHERE b.AccountId IS NULL;
Matched count = total_count - missing_count
The core problem here is that you combined two large table together, it requires a lot of memory and it apparently needs a lot of time to finish.
try it using correlated subquery. This might help:
select count(1) from ec_account a where exists (select * from ec_card b
where b.AccountId=a.id)
Also, other than indexing following strategies generally help:
- Denormalization
- Caching results
- Using a NoSQL database
I have the following schema:
CREATE TABLE `news` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`news_category_id` int(10) unsigned NOT NULL,
`news_type_id` int(10) unsigned NOT NULL,
`news_pictures_main_id` int(10) unsigned DEFAULT NULL,
`title` tinytext COLLATE latin1_general_ci,
`body` text COLLATE latin1_general_ci,
`tmstp` timestamp NULL DEFAULT NULL,
`subcategory` varchar(64) COLLATE latin1_general_ci DEFAULT NULL,
`source` varchar(128) COLLATE latin1_general_ci DEFAULT NULL,
`old_id` int(10) unsigned DEFAULT NULL,
`tags` text COLLATE latin1_general_ci,
PRIMARY KEY (`id`),
KEY `news_time_idx` (`tmstp`),
KEY `fk_news_news_pictures1` (`news_pictures_main_id`),
KEY `fk_news_news_category1` (`news_category_id`),
KEY `fk_news_news_type1` (`news_type_id`),
CONSTRAINT `fk_news_news_category1` FOREIGN KEY (`news_category_id`) REFERENCES `news_category` (`id`) ON UPDATE CASCADE,
CONSTRAINT `fk_news_news_pictures1` FOREIGN KEY (`news_pictures_main_id`) REFERENCES `news_pictures` (`id`) ON DELETE SET NULL ON UPDATE CASCADE,
CONSTRAINT `fk_news_news_type1` FOREIGN KEY (`news_type_id`) REFERENCES `news_type` (`id`) ON UPDATE CASCADE
) ENGINE=InnoDB
CREATE TABLE `news_pictures` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`path` text COLLATE latin1_general_ci,
`description` text COLLATE latin1_general_ci,
`author` varchar(45) COLLATE latin1_general_ci DEFAULT NULL,
`news_id` int(10) unsigned DEFAULT NULL,
`temp_id` varchar(40) COLLATE latin1_general_ci DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `filename_old_id_unq` (`path`(20),`temp_id`(6)),
KEY `fk_news_pictures_news1` (`news_id`),
KEY `temp_id_idx` (`temp_id`(8)),
CONSTRAINT `fk_news_pictures_news1` FOREIGN KEY (`news_id`) REFERENCES `news` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB
CREATE TABLE `news_category` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(45) COLLATE latin1_general_ci DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB
CREATE TABLE `news_type` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`name` varchar(45) COLLATE latin1_general_ci DEFAULT NULL,
`slug` varchar(45) COLLATE latin1_general_ci DEFAULT NULL,
PRIMARY KEY (`id`)
KEY `news_type_slug_idx` (`slug`)
) ENGINE=InnoDB
From that, there is derived the following view:
CREATE OR REPLACE VIEW `news_full` AS select `n`.`id` AS `id`,
`n`.`title` AS `title`,
`n`.`body` AS `body`,
`n`.`tmstp` AS `tmstp`,
`n`.`subcategory` AS `subcategory`,
`n`.`source` AS `source`,
`n`.`old_id` AS `old_id`,
`n`.`news_type_id` AS `news_type_id`,
`n`.`tags` AS `tags`,
`nt`.`name` AS `news_type_name`,
`nt`.`slug` AS `news_type_slug`,
`n`.`news_pictures_main_id` AS `news_pictures_main_id`,
`np`.`path` AS `news_pictures_main_path`,
`np`.`description` AS `news_pictures_main_description`,
`np`.`author` AS `news_pictures_main_author`,
`np`.`temp_id` AS `news_pictures_main_temp_id`,
`n`.`news_category_id` AS `news_category_id`,
`nc`.`name` AS `news_category_name`
from (((`news` `n`
left join `news_pictures` `np` on((`n`.`news_pictures_main_id` = `np`.`id`)))
join `news_category` `nc` on((`n`.`news_category_id` = `nc`.`id`)))
join `news_type` `nt` on((`n`.`news_type_id` = `nt`.`id`)));
However, if I try to run the following query:
select * from news_full order by tmstp limit 100
I get the following execution plan (please click on the image to expand it):
Notice the Using temporary; Using filesort field in the first step. But this is weird, because tmstp field is indexed on the base table.
First I thought this was due the left join on the view, but I've changed it to inner join and I got the same results.
Edit
As #Michael-sqlbot cleverly noticed, the query optimizer is inverting the order of the base tables, putting news_category (nc) first.
If I change the query that creates the view to use only LEFT JOINs it seems to work:
The execution times, as expected, as blatantly different:
Not satisfied, I created another view with the original query, adding the STRAIGHT_JOIN statement. So, the query plan comes as follows:
So, it's not using the index.
However, if I run the plan for the base query adding the same ORDER BY and LIMIT clauses, it does uses the index:
(Not an answer, but some other issues to bring up...)
UNIQUE KEY `filename_old_id_unq` (`path`(20),`temp_id`(6))
That constrains the first 20 characters of path, together with the first 6 characters of temp_id to be unique across the table. Did you really want that?
I suspect the optimizer will never use both columns of that index. (In general, prefixing is useless.)
And...
`title` tinytext COLLATE latin1_general_ci
Change to VARCHAR(255). There are disadvantages of TINYTEXT and perhaps no advantages.
this is my simple inner join:
SELECT
SUM(ASSNZ.assenzeDidattiche) AS TotaleAssenze,
SUM(ASSNZ.ore) AS totale_parziale,
FLOOR(((SUM(ASSNZ.assenzeDidattiche) / SUM(ASSNZ.ore)) * 100)) AS andamento,
MAX(ASSNZ.dataLezione) AS ultima_lezione,
ASSNZ.idServizio,
ASSNZ.idUtente
FROM
ciac_corsi_assenze AS ASSNZ
INNER JOIN
ciac_serviziAcquistati_ITA AS ACQ
ON ACQ.idContatto = ASSNZ.idUtente
AND ACQ.idServizio = ASSNZ.idServizio
AND ACQ.stato_allievo <> 'ritirato'
GROUP BY
ASSNZ.idServizio,
ASSNZ.idUtente
table "ASSNZ" has 213886 rows with index "idUtente", "idServizio"
table "ACQ" has 8950 rows with index "idContatto", "idServizio"
ASSNZ table:
CREATE TABLE `ciac_corsi_assenze` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`idUtente` int(11) DEFAULT NULL,
`idServizio` int(11) DEFAULT NULL,
`idCorso` int(11) DEFAULT NULL,
`idCalendario` int(11) DEFAULT NULL,
`modalita` varchar(255) DEFAULT NULL,
`ore` int(11) DEFAULT NULL,
`assenzeDidattiche` float DEFAULT NULL,
`assenzeAmministrative` float DEFAULT NULL,
`dataLezione` date DEFAULT NULL,
`ora_inizio` varchar(8) DEFAULT NULL,
`ora_fine` varchar(8) DEFAULT NULL,
`dataFineStage` date DEFAULT NULL,
`giustificata` varchar(128) DEFAULT NULL,
`motivazione` longtext,
`grouped` int(11) NOT NULL,
PRIMARY KEY (`id`),
KEY `idUtente` (`idUtente`) USING BTREE,
KEY `idServizio` (`idServizio`) USING BTREE,
KEY `dataLezione` (`dataLezione`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=574582 DEFAULT CHARSET=utf8;
ACQ table:
CREATE TABLE `ciac_serviziacquistati_ita` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`idServizio` int(11) NOT NULL,
`idContatto` int(11) NOT NULL,
`idAzienda` int(11) NOT NULL,
`idSede` int(11) NOT NULL,
`tipoPersona` int(11) NOT NULL,
`num_registro` int(11) NOT NULL,
`codice` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
`dal` date NOT NULL,
`al` date NOT NULL,
`ore` int(11) NOT NULL,
`costoOrario` decimal(10,0) NOT NULL,
`annoFormativo` varchar(128) CHARACTER SET latin1 NOT NULL,
`stato_attuale` int(11) NOT NULL,
`datore_attuale` int(11) NOT NULL,
`stato_allievo` varchar(64) CHARACTER SET latin1 NOT NULL DEFAULT 'corsista',
`data_ritiro` date DEFAULT NULL,
`crediti_formativi` int(11) NOT NULL,
`note` longtext CHARACTER SET latin1 NOT NULL,
`valore_economico` decimal(10,2) NOT NULL,
`dataInserimento` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `idServizio` (`idServizio`) USING BTREE,
KEY `idAzienda` (`idAzienda`) USING BTREE,
KEY `idContatto` (`idContatto`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=9542 DEFAULT CHARSET=utf8;
this is my EXPLAIN of the select
Now because the query is slow, during 1.5s / 2.0s??
Something wrong?
UPDATE
added new index (with the John Bollinger's answer) to the table ciac_corsi_assenze:
PRIMARY KEY (`id`),
KEY `dataLezione` (`dataLezione`) USING BTREE,
KEY `test` (`idUtente`,`idServizio`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=574582 DEFAULT CHARSET=utf8;
added new index to the table ciac_serviziAcquistati_ITA:
PRIMARY KEY (`id`),
KEY `idAzienda` (`idAzienda`) USING BTREE,
KEY `test2` (`idContatto`,`idServizio`) USING BTREE
) ENGINE=InnoDB AUTO_INCREMENT=9542 DEFAULT CHARSET=utf8;
New EXPLAIN:
But it's always slow :(
Your tables have separate indexes on various columns of interest, but MySQL will use at most one index per table to perform your query. This particular query would probably be sped by table ciac_corsi_assenze having an index on (idUtente, idServizio) (and such an index would supersede the existing one on (idUtente) alone). That should allow MySQL to avoid sorting the result rows to perform the grouping, and it will help more in performing the join than any of the existing indexes do.
The query would probably be sped further by table ciac_serviziAcquistati_ITA having an index on (idContatto, idServizio), or even on (idContatto, idServizio, ritirato). Either of those would supersede the existing index on just (idContatto).
John went the right direction. However the order of columns in the composite index needs changing.
For the GROUP BY, this order is needed (on ASSNZ):
INDEX(idServizio, idUtente)
(and that should replace KEY(idServizio), but not KEY(idUtente))
Then ACQ needs, in this order:
INDEX(idContatto, idServizio, stato_allievo)
replacing only KEY(idContatto).
This is my SQL code which links users to items based on tutorials:
CREATE TABLE IF NOT EXISTS `users` (
`user_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`user_name` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`user_password_hash` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`user_email` varchar(254) COLLATE utf8_unicode_ci NOT NULL,
`user_access_level` tinyint(3) unsigned NOT NULL DEFAULT '0',
`user_active` tinyint(1) NOT NULL DEFAULT '0',
`user_activation_hash` varchar(40) COLLATE utf8_unicode_ci DEFAULT NULL,
`user_password_reset_hash` char(40) COLLATE utf8_unicode_ci DEFAULT NULL,
`user_password_reset_timestamp` bigint(20) DEFAULT NULL,
`user_failed_logins` tinyint(1) NOT NULL DEFAULT '0',
`user_last_failed_login` int(10) DEFAULT NULL,
`user_registration_datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`user_registration_ip` varchar(39) COLLATE utf8_unicode_ci NOT NULL DEFAULT '0.0.0.0',
PRIMARY KEY (`user_id`),
UNIQUE KEY `user_name` (`user_name`),
UNIQUE KEY `user_email` (`user_email`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE IF NOT EXISTS `item` (
`item_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`item_title` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`item_location` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`item_description` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`item_datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`item_status` int(1) unsigned NOT NULL,
PRIMARY KEY (`item_id`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
http://sqlfiddle.com/#!9/fd011
I'm getting a bit confused about how to link the items to the user. It seems like I need something called a foreign key on the items, a bit like this in my item table:
FOREIGN KEY (user_id) REFERENCES user(ID)
I can't seem to get it to compile and query successfully. Can anyone please show me the right way to associate the items with the user.
You need to add the user_id column to the items table along with the constraint:
CREATE TABLE IF NOT EXISTS `users` (
`user_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`user_name` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`user_password_hash` varchar(255) COLLATE utf8_unicode_ci NOT NULL,
`user_email` varchar(254) COLLATE utf8_unicode_ci NOT NULL,
`user_access_level` tinyint(3) unsigned NOT NULL DEFAULT '0',
`user_active` tinyint(1) NOT NULL DEFAULT '0',
`user_activation_hash` varchar(40) COLLATE utf8_unicode_ci DEFAULT NULL,
`user_password_reset_hash` char(40) COLLATE utf8_unicode_ci DEFAULT NULL,
`user_password_reset_timestamp` bigint(20) DEFAULT NULL,
`user_failed_logins` tinyint(1) NOT NULL DEFAULT '0',
`user_last_failed_login` int(10) DEFAULT NULL,
`user_registration_datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`user_registration_ip` varchar(39) COLLATE utf8_unicode_ci NOT NULL DEFAULT '0.0.0.0',
PRIMARY KEY (`user_id`),
UNIQUE KEY `user_name` (`user_name`),
UNIQUE KEY `user_email` (`user_email`)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
CREATE TABLE IF NOT EXISTS `item` (
`item_id` int(11) unsigned NOT NULL AUTO_INCREMENT,
user_id int unsigned,
`item_title` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`item_location` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`item_description` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`item_datetime` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`item_status` int(1) unsigned NOT NULL,
PRIMARY KEY (`item_id`),
FOREIGN KEY (user_id) REFERENCES users(user_id)
) ENGINE=MyISAM AUTO_INCREMENT=1 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
The SQL Fiddle is here.
I should point out a few other things:
Pay attention to the database engine you are using. MyISAM doesn't actually enforce the relationship.
Having weird dates as the default value is probably less useful than just using NULL.
I'm not sure if there is a value to having explicit collations for every character definition, unless your database is going to be supporting a wide variety of collations.
Don't use single quotes for numeric constants. So, if a value is declared as a tinyint, set the default ot 0 not '0' (this doesn't affect performance in a CREATE TABLE statement; it is just misleading).
#GordonLindoff's solution is one method, but that assumes that each item belongs to exactly one user, and an item cannot be referenced by multiple users. If you have a many-to-many relationship, where a user can have multiple items and an item can be referenced by multiple users, then you need a third table that links them together:
CREATE TABLE IF NOT EXISTS `user_item` (
`user_id` int(11) unsigned NOT NULL,
`item_id` int(11) unsigned NOT NULL,
PRIMARY KEY (`user_item`,`item_id`),
FOREIGN KEY (user_id) REFERENCES users(user_id),
FOREIGN KEY (item_id) REFERENCES items(item_id)
)ENGINE=Innodb DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
The Foreign Key constraints enforce that for every row in user_item that the user_id exists in users, and the item_id exists in items. And as was mentioned in a previous comment, that you will need Innodb to have the foreign key constraints enforced.
Tables:
nodes
data_texts
data_profiles
data_locations
data_profiles
data_media
data_products
data_metas
categories
tags
categories_nodes
tags_nodes
This question is a generalized question and is on the back of another question
Explanation:
Each of the "data" tables has a node_id that refers back to the id of the nodes table (hasMany/belongsTo association).
A "Node" can be anything - a TV Show, a Movie, a Person, an Article...etc (all generated via a CMS, so the user can control what type of "Nodes" they want).
When pulling data, I want to be able to query against certain fields. For example if they do a search, I want to be able to pull nodes that have data_texts.title = '%george%' or order by the datetime field in data_locations.
The problem is, when I do a join on all seven data tables (or more), the query has to hit so many combined rows that it just times out (even with a nearly empty database.... total 200 rows across the entire database).
I realize I can determine IF I need a join depending on what I'm doing - but even with five or six joins (once the database gets to 10k+ records), it's going to be horribly slow, if it works at all. Per this question, the query I'm using just doing a join on these tables times out completely.
Each node can have multiple rows of each data type (for multi-language reasons among others).
I'm completely defeated - I'm at the point where I think I need to restructure the entire thing, but don't have the time for that. I've thought about combining all into one table, but aren't sure how....etc
nodes
CREATE TABLE `nodes` (
`id` CHAR(36) NOT NULL,
`name` VARCHAR(100) NOT NULL,
`slug` VARCHAR(100) NOT NULL,
`node_type_id` CHAR(36) NOT NULL,
`site_id` CHAR(36) NOT NULL,
`created` DATETIME NOT NULL,
`modified` DATETIME NOT NULL,
PRIMARY KEY (`id`),
INDEX `nodeTypeId` (`node_type_id`),
INDEX `slug` (`slug`),
INDEX `nodeId` (`id`)
)
COLLATE='latin1_swedish_ci'
ENGINE=MyISAM;
data_texts:
CREATE TABLE `data_texts` (
`id` CHAR(36) NOT NULL,
`title` VARCHAR(250) NULL DEFAULT NULL,
`subtitle` VARCHAR(500) NULL DEFAULT NULL,
`content` LONGTEXT NULL,
`byline` VARCHAR(250) NULL DEFAULT NULL,
`language_id` CHAR(36) NULL DEFAULT NULL,
`foreign_key` CHAR(36) NULL DEFAULT NULL,
`model` VARCHAR(40) NULL DEFAULT NULL,
`node_id` CHAR(36) NULL DEFAULT NULL,
`created` DATETIME NOT NULL,
`modified` DATETIME NOT NULL,
PRIMARY KEY (`id`),
INDEX `nodeId` (`node_id`),
INDEX `languageId_nodeId` (`language_id`, `node_id`),
INDEX `foreignKey_model` (`foreign_key`, `model`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM;
data_profiles
CREATE TABLE `data_profiles` (
`id` CHAR(36) NOT NULL,
`name` VARCHAR(80) NULL DEFAULT NULL,
`email_personal` VARCHAR(100) NULL DEFAULT NULL,
`email_business` VARCHAR(100) NULL DEFAULT NULL,
`email_other` VARCHAR(100) NULL DEFAULT NULL,
`title` VARCHAR(100) NULL DEFAULT NULL,
`description` LONGTEXT NULL,
`prefix` VARCHAR(40) NULL DEFAULT NULL,
`phone_home` VARCHAR(40) NULL DEFAULT NULL,
`phone_business` VARCHAR(40) NULL DEFAULT NULL,
`phone_mobile` VARCHAR(40) NULL DEFAULT NULL,
`phone_other` VARCHAR(40) NULL DEFAULT NULL,
`foreign_key` CHAR(36) NULL DEFAULT NULL,
`model` VARCHAR(40) NULL DEFAULT NULL,
`node_id` CHAR(36) NULL DEFAULT NULL,
`language_id` CHAR(36) NULL DEFAULT NULL,
`created` DATETIME NOT NULL,
`modified` DATETIME NOT NULL,
`user_id` CHAR(36) NULL DEFAULT NULL,
PRIMARY KEY (`id`),
INDEX `nodeId` (`node_id`),
INDEX `languageId_nodeId` (`node_id`, `language_id`),
INDEX `foreignKey_model` (`foreign_key`, `model`)
)
COLLATE='latin1_swedish_ci'
ENGINE=MyISAM;
categories
CREATE TABLE `categories` (
`id` CHAR(36) NOT NULL,
`name` VARCHAR(100) NOT NULL,
`node_type_id` CHAR(36) NOT NULL,
`site_id` CHAR(36) NOT NULL,
`slug` VARCHAR(100) NOT NULL,
`created` DATETIME NOT NULL,
`modified` DATETIME NOT NULL,
PRIMARY KEY (`id`),
INDEX `nodeTypeId` (`node_type_id`),
INDEX `slug` (`slug`)
)
COMMENT='Used to categorize nodes'
COLLATE='utf8_general_ci'
ENGINE=MyISAM;
categories_nodes
CREATE TABLE `categories_nodes` (
`id` CHAR(36) NOT NULL,
`category_id` CHAR(36) NOT NULL,
`node_id` CHAR(36) NOT NULL,
PRIMARY KEY (`id`),
INDEX `categoryId_nodeId` (`category_id`, `node_id`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM;
node_tags
CREATE TABLE `node_tags` (
`id` CHAR(36) NOT NULL,
`name` VARCHAR(40) NOT NULL,
`site_id` CHAR(36) NOT NULL,
`created` DATETIME NOT NULL,
`modified` DATETIME NOT NULL,
PRIMARY KEY (`id`),
INDEX `siteId` (`site_id`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM;
nodes_node_tags
CREATE TABLE `nodes_node_tags` (
`id` CHAR(36) NOT NULL,
`node_id` CHAR(36) NOT NULL,
`node_tag_id` CHAR(36) NOT NULL,
PRIMARY KEY (`id`),
INDEX `node_id_node_tag_id` (`node_id`, `node_tag_id`)
)
COLLATE='utf8_general_ci'
ENGINE=MyISAM;
MySQL:
SELECT `Node`.`id`, `Node`.`name`, `Node`.`slug`, `Node`.`node_type_id`, `Node`.`site_id`, `Node`.`created`, `Node`.`modified`
FROM `mysite`.`nodes` AS `Node`
LEFT JOIN `mysite`.`data_date_times` AS `DataDateTime` ON (`DataDateTime`.`node_id` = `Node`.`id`)
LEFT JOIN `mysite`.`data_locations` AS `DataLocation` ON (`DataLocation`.`node_id` = `Node`.`id`)
LEFT JOIN `mysite`.`data_media` AS `DataMedia` ON (`DataMedia`.`node_id` = `Node`.`id`)
LEFT JOIN `mysite`.`data_metas` AS `DataMeta` ON (`DataMeta`.`node_id` = `Node`.`id`)
LEFT JOIN `mysite`.`data_profiles` AS `DataProfile` ON (`DataProfile`.`node_id` = `Node`.`id`)
LEFT JOIN `mysite`.`data_products` AS `DataProduct` ON (`DataProduct`.`node_id` = `Node`.`id`)
LEFT JOIN `mysite`.`data_texts` AS `DataText` ON (`DataText`.`node_id` = `Node`.`id`)
WHERE 1=1
GROUP BY `Node`.`id`
Firstly, try InnoDB, not MyISAM.
Secondly, remove the group by, see how well it runs then, and how many rows are involved. Shouldn't be that many, but it's interesting.
You don't need the 'nodeId' index on node (as you already have it as a primary key). Again, shouldn't make any difference.
The where clause is irrelevant. You can remove it with no effect one way or another.
Thirdly, well, something is seriously broken.
Have a quick look on how to start profiling (e.g. http://dev.mysql.com/doc/refman/5.0/en/show-profile.html) , and run a profile command to see where all the time is going. Post it here if it doesn't immediately show that something is broken.
I'm unfortunately not in a position where I can do any tests right now. I'll just throw out some ideas. I might be able to do some tests later.
Be suspicious of different collations.
Some of your ids are useless. For example, you should drop the column categories_nodes.id, and put a primary key constraint on {category_id, node_id} instead.
Be suspicious of any design that requires joining all the tables at run time. There are better ways.
Use innodb and foreign key constraints.