mysql InnoDB: FOREIGN KEY constraint performance - mysql

I have the following InnoDB tables:
CREATE TABLE `vehicle` (
`ID` mediumint(8) unsigned NOT NULL AUTO_INCREMENT,
`Name` varchar(50) DEFAULT NULL,
`Model` varchar(100) DEFAULT NULL,
`Engine_Type` varchar(70) DEFAULT NULL,
`Construction_From` date DEFAULT NULL,
`Construction_To` date DEFAULT NULL,
`Engine_Power_KW` mediumint(8) unsigned DEFAULT NULL,
`Engine_Power_HP` mediumint(8) unsigned DEFAULT NULL,
`CC` mediumint(8) unsigned DEFAULT NULL,
`TTC_TYP_ID` int(11) unsigned DEFAULT NULL,
`Vehicle_Type` tinyint(1) DEFAULT NULL,
`ID_Body_Type` tinyint(3) unsigned DEFAULT NULL,
PRIMARY KEY (`ID`)
) ENGINE=InnoDB AUTO_INCREMENT=49407 DEFAULT CHARSET=utf8;
CREATE TABLE `part` (
`ID` int(11) unsigned NOT NULL AUTO_INCREMENT,
`ID_Brand` smallint(5) unsigned DEFAULT NULL,
`Code_Full` varchar(50) DEFAULT NULL,
`Code_Condensed` varchar(50) DEFAULT NULL,
`Ean` varchar(50) DEFAULT NULL COMMENT 'The part barcode.',
`TTC_ART_ID` int(11) unsigned DEFAULT NULL COMMENT 'TecDoc ID.',
`ID_Product_Status` tinyint(3) unsigned DEFAULT NULL,
PRIMARY KEY (`ID`),
UNIQUE KEY `TTC_ART_ID_UNIQUE` (`TTC_ART_ID`),
UNIQUE KEY `ID_Brand_Code_Full_UNIQUE` (`ID_Brand`,`Code_Full`)
) ENGINE=InnoDB AUTO_INCREMENT=3732260 DEFAULT CHARSET=utf8;
CREATE TABLE `vehicle_part` (
`ID_Vehicle` mediumint(8) unsigned NOT NULL,
`ID_Part` int(11) unsigned NOT NULL,
PRIMARY KEY (`ID_Vehicle`,`ID_Part`),
KEY `fk_vehicle_part_vehicle_id_vehicle_idx` (`ID_Vehicle`),
KEY `fk_vehicle_part_part_id_part_idx` (`ID_Part`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Table vehicle has about 45.000 records, table part has about 3.500.000 records and table vehicle_part has approximately 100.000.000 records.
Creating the secondary indexes for vehicle_part did not take too long, about 30 min for both.
What I cannot do though is create the foreign key constraints: for example
ALTER TABLE `vehicle_part`
ADD CONSTRAINT `fk_vehicle_part_vehicle_id_vehicle`
FOREIGN KEY (`ID_Vehicle`)
REFERENCES `vehicle` (`ID`)
ON DELETE NO ACTION
ON UPDATE NO ACTION;
takes ages to complete. I understand the table is rebuilt since it consumes a lot of disk space. What can I do to improve the performance?
If I create the table with the fk constraints and then add the records the insert process in vehicle_part also takes ages (about 3 days).
I am using a laptop with 4GB RAM.
EDIT 12/01/2016
The answer given by Drew helped a lot in improving the performance dramatically. I changed every script using SELECT ... INTO outfile and then LOAD DATA INFILE from the exported csv file. Also sometimes before LOAD DATA INFILE dropping the indexes and recreating them after the load proccess saves even more time. There is no need to drop the fk constraints just the secondary indexes.

If you know your data is pristine from an FK perspective, then establish your structure without secondary indexes as suggested in comments, but with the FK in the schema yet with FK checks temporarily disabled.
Load your data. If external data, certainly do it with LOAD DATA INFILE.
After your data is loaded, turn on FK checks. And establish secondary indexes with Alter Table.
Again, going with the assumption that your data is clean. There are other ways of proving that after-the-fact for the risk-adverse.
create table student
( id int auto_increment primary key,
sName varchar(100) not null
-- secondary indexes to be added later
);
create table booksAssigned
( id int auto_increment primary key,
studentId int not null,
isbn varchar(20) not null,
constraint foreign key `fk_b_s` (studentId) references student(id)
-- secondary indexes to be added later
);
insert booksAssigned(studentId,isbn) values (1,'asdf'); -- Error 1452 as expected
set FOREIGN_KEY_CHECKS=0; -- turn FK checks of temporarily
insert booksAssigned(studentId,isbn) values (1,'asdf'); -- Error 1452 as expected
set FOREIGN_KEY_CHECKS=1; -- succeeds despite faulty data
insert booksAssigned(studentId,isbn) values (2,'38383-asdf'); -- Error 1452 as expected
As per op comments, how to drop auto-generated index in referencing table after initial schema creation:
mysql> show create table booksAssigned;
| booksAssigned | CREATE TABLE `booksassigned` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`studentId` int(11) NOT NULL,
`isbn` varchar(20) NOT NULL,
PRIMARY KEY (`id`),
KEY `fk_b_s` (`studentId`),
CONSTRAINT `booksassigned_ibfk_1` FOREIGN KEY (`studentId`) REFERENCES `student` (`id`)
) ENGINE=InnoDB |
mysql> set FOREIGN_KEY_CHECKS=0;
Query OK, 0 rows affected (0.00 sec)
mysql> drop index `fk_b_s` on booksAssigned;
Query OK, 0 rows affected (0.49 sec)
Records: 0 Duplicates: 0 Warnings: 0
mysql> show create table booksAssigned;
| booksAssigned | CREATE TABLE `booksassigned` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`studentId` int(11) NOT NULL,
`isbn` varchar(20) NOT NULL,
PRIMARY KEY (`id`),
CONSTRAINT `booksassigned_ibfk_1` FOREIGN KEY (`studentId`) REFERENCES `student` (`id`)
) ENGINE=InnoDB |
Further links
Temporarily disable foreign keys
A Rolando Answer

Related

MySQL InnoDB FOREIGN KEY ERROR

I have the followiing tables:
CREATE TABLE `Atletica` (
`Universidade` varchar(100) NOT NULL,
`Nome` varchar(100) NOT NULL,
`Logo` varchar(100) NOT NULL,
`GritoDeGuerra` varchar(100) NOT NULL,
`EnderecoCEP` int(11) NOT NULL,
`EnderecoNumero` int(11) NOT NULL,
`MedalhaOuro` int(6) NOT NULL,
`MedalhaPrata` int(6) NOT NULL,
`MedalhaBronze` int(6) NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
-- --------------------------------------------------------
--
-- Table structure for table `Endereco`
--
CREATE TABLE `Endereco` (
`Rua` varchar(50) NOT NULL,
`Numero` int(11) NOT NULL,
`Bairro` varchar(50) DEFAULT NULL,
`CEP` int(11) NOT NULL,
`Cidade` varchar(50) NOT NULL,
`Estado` varchar(50) NOT NULL,
`Complemento` varchar(50) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
--
-- Indexes for dumped tables
--
--
-- Indexes for table `Atletica`
--
ALTER TABLE `Atletica`
ADD PRIMARY KEY (`Universidade`,`Nome`),
ADD KEY `EnderecoCEP` (`EnderecoCEP`),
ADD KEY `EnderecoNumero` (`EnderecoNumero`);
--
-- Indexes for table `Endereco`
--
ALTER TABLE `Endereco`
ADD PRIMARY KEY (`Numero`,`CEP`);
And I keep getting the error:
Error creating foreign key on EnderecoCEP, EnderecoNumero (check data types)
when I try to execute the following command:
ALTER TABLE `Atletica`
ADD FOREIGN KEY (`EnderecoCEP`, `EnderecoNumero`)
REFERENCES `proj3`.`Endereco`(`CEP`, `Numero`)
ON DELETE RESTRICT ON UPDATE RESTRICT;
Ive read tons of similar questions here but all of them the error was an obvious data type mismatch. I only have those two table on my database. Please help.
Thank you very much for your time.
As it turns out, when you create a foreign key with multiple columns, it should be in the same order ar the primary key in the referencing table.

What is the cause for MySQL: Errorno 150

Can someone please explain the cause for the following error, 'Can't create table 'Activities' (errno: 150)'
I'm under the understading that the data types and lengths have to be the same, does is have anything to do with the auto increment?
Create Table `LinkMemberActivity` (
`LinkID` int(11) unsigned NOT NULL AUTO_INCREMENT,
`MID` int(11) unsigned NOT NULL,
`AID` int(11) unsigned NOT NULL,
PRIMARY KEY (`LinkID`),
FOREIGN KEY (`MID`) REFERENCES Members(`MID`)) ENGINE=InnoDB DEFAULT CHARSET=latin1;
)
CREATE TABLE `Activities` (
`AID` int(11) unsigned NOT NULL AUTO_INCREMENT,
`Name` varchar(25) DEFAULT NULL,
`MaxCapacity` int(25) DEFAULT NULL,
`StartTime` time DEFAULT NULL,
`EndTime` time DEFAULT NULL,
PRIMARY KEY (`AID`),
FOREIGN KEY (`AID`) REFERENCES LinkMemberActivity(`AID`))
ENGINE=InnoDB DEFAULT CHARSET=latin1 );
You are trying to make a primary key column a foreign key dependent field. This is not only unusual but makes no sense in a datamodel, unless it is part of a composite key. Common practice has a column foreign key dependent on another tables primary key. Not sure what reasons you have for the way you designed your datamodel this way, but you can fix this problem by creating a not null autoincrement column named ID and make this column the primary key. Next remove autoincrement from aid.

MySQL Query Optimization for Analytics

I've about 30 tables in MySQL Data Warehouse database for analytical data. For now its around 2 Millions of data rows, but I'm sure in future it will become billions very soon. Challenge is, the queries should return the data in a fast way. I've following simple query which is taking over 60 seconds to process those 2 Million rows:
SELECT count(distinct fact.dim_pageview_id)
FROM `datawarehouse_schema_alpha`.`fact_master` fact
left join dim_visit visit on visit.dim_visit_id = fact.dim_visit_id
left join dim_datetime datetim on datetim.dim_datetime_id = fact.dim_datetime_id
where fact.dim_site_id = 552
Explain query result:
1 SIMPLE fact ref fk_fact_bb_pageview_dim_bb_site,fk_fact_master_dim_site fk_fact_bb_pageview_dim_bb_site 5 const 17490 Using where
1 SIMPLE visit eq_ref PRIMARY PRIMARY 4 datawarehouse_schema_alpha.fact.dim_visit_id 1 Using index
1 SIMPLE datetim eq_ref PRIMARY PRIMARY 4 datawarehouse_schema_alpha.fact.dim_datetime_id 1 Using index
I've following sample database structure:
--
-- Table structure for table `dim_datetime`
--
CREATE TABLE IF NOT EXISTS `dim_datetime` (
`dim_datetime_id` int(11) NOT NULL AUTO_INCREMENT,
`datetime_date` varchar(45) CHARACTER SET latin1 DEFAULT NULL,
`datetime_year` varchar(45) CHARACTER SET latin1 DEFAULT NULL,
`datetime_full` varchar(45) CHARACTER SET latin1 DEFAULT NULL,
PRIMARY KEY (`dim_datetime_id`)
) ENGINE=InnoDB DEFAULT CHARSET=big5 AUTO_INCREMENT=4568326 ;
-- --------------------------------------------------------
--
-- Table structure for table `dim_visit`
--
CREATE TABLE IF NOT EXISTS `dim_visit` (
`dim_visit_id` int(11) NOT NULL AUTO_INCREMENT,
`visit_start_time` datetime DEFAULT NULL,
`visit_end_time` datetime DEFAULT NULL,
`visit_duration` varchar(45) DEFAULT NULL,
PRIMARY KEY (`dim_visit_id`)
) ENGINE=InnoDB DEFAULT CHARSET=big5 AUTO_INCREMENT=1295102 ;
-- --------------------------------------------------------
--
-- Table structure for table `dim_site`
--
CREATE TABLE IF NOT EXISTS `dim_site` (
`dim_site_id` int(11) NOT NULL AUTO_INCREMENT,
`site_name` varchar(255) CHARACTER SET latin1 DEFAULT NULL,
`site_url` text CHARACTER SET latin1,
`site_key` text CHARACTER SET latin1,
PRIMARY KEY (`dim_site_id`)
) ENGINE=InnoDB DEFAULT CHARSET=big5 AUTO_INCREMENT=870 ;
--
-- Table structure for table `fact_master`
--
CREATE TABLE IF NOT EXISTS `fact_master` (
`fact_master_id` int(11) NOT NULL AUTO_INCREMENT,
`dim_pageview_id` int(11) DEFAULT NULL,
`dim_visit_id` int(11) DEFAULT NULL,
`dim_site_id` int(11) DEFAULT NULL,
`dim_datetime_id` int(11) DEFAULT NULL,
`master_ip` varchar(255) DEFAULT NULL,
`master_spent_time` varchar(255) DEFAULT NULL,
`master_datetime` datetime DEFAULT NULL,
PRIMARY KEY (`fact_master_id`),
KEY `fk_fact_bb_pageview_dim_bb_visit` (`dim_visit_id`),
KEY `fk_fact_bb_pageview_dim_bb_datetime` (`dim_datetime_id`),
KEY `fk_fact_bb_pageview_dim_bb_pageview` (`dim_pageview_id`),
KEY `fk_fact_master_dim_pageview` (`dim_pageview_id`),
KEY `fk_fact_master_dim_visit` (`dim_visit_id`),
KEY `fk_fact_master_dim_datetime` (`dim_datetime_id`),
KEY `fk_fact_master_dim_site` (`dim_site_id`),
) ENGINE=InnoDB DEFAULT CHARSET=big5 AUTO_INCREMENT=1 ;
--
-- Constraints for dumped tables
--
--
-- Constraints for table `fact_master`
--
ALTER TABLE `fact_master`
ADD CONSTRAINT `fk_fact_master_dim_datetime` FOREIGN KEY (`dim_datetime_id`) REFERENCES `dim_datetime` (`dim_datetime_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
ADD CONSTRAINT `fk_fact_master_dim_pageview` FOREIGN KEY (`dim_pageview_id`) REFERENCES `dim_pageview` (`dim_pageview_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
ADD CONSTRAINT `fk_fact_master_dim_site` FOREIGN KEY (`dim_site_id`) REFERENCES `dim_site` (`dim_site_id`) ON DELETE NO ACTION ON UPDATE NO ACTION,
ADD CONSTRAINT `fk_fact_master_dim_visit` FOREIGN KEY (`dim_visit_id`) REFERENCES `dim_visit` (`dim_visit_id`) ON DELETE NO ACTION ON UPDATE NO ACTION;
Please let me know what problem could there be? How can I use Indexes and Views to get the data in a really really quick way? Any other suggestions than using Indexes or Views, to optimize the speed is also welcome. Thanks!

mysql won't allow foreign key

Many people had this problem already, but there was no fitting solution in other posts.
I have two tables, one named "sales", the other named "host_flags". I would like to have a foreign key for host_flags.sales_id to sales.id, but mysql won't let me! I have primary indexes defined in each table, so I wonder why...
The host_flags table already has a foreign key on the column host_id, but even when I tried and created the foreign key for the sales id first, it wouldn't let me.
The tables look like:
CREATE TABLE `sales` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`email` varchar(255) DEFAULT NULL,
`password` varchar(255) DEFAULT NULL,
`creation` datetime DEFAULT NULL,
`lastupdate` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=3 DEFAULT CHARSET=latin1;
CREATE TABLE `host_flags` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`host_id` int(11) DEFAULT NULL,
`sales_id` int(11) DEFAULT NULL,
`creation` datetime DEFAULT NULL,
`lastupdate` timestamp NULL DEFAULT NULL ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`),
KEY `host_id6` (`host_id`),
CONSTRAINT `host_id6` FOREIGN KEY (`host_id`) REFERENCES `hosts` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE `hosts` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`creation` datetime NOT NULL,
`lastupdate` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=32225 DEFAULT CHARSET=utf8 COLLATE=utf8_bin;
I get this error message:
MySQL said: Can't create table 'primarydata.#sql-191_1' (errno: 150)
Thanks!
Charles
SOLUTION FOUND
All ints of the primary indexes have to be either signed or unsigned - not mixed.
Typically:
I like to declare the FK constraints outside of the table definition after all tables have been constructed.
ALTER TABLE `tbl`
ADD CONSTRAINT `constr`
FOREIGN KEY `fk_id` REFERENCES `ftbl`(`id`)
ON UPDATE CASCADE
ON DELETE CASCADE;
This way I can make sure the problem isn't something like the datatype of tbl.fk_id not being the same as the one of ftbl.id (including UNSIGNED as #Devart said). Or not having declared ftbl.id as unique. Regardless of the order of declaration of the tables.
After i do this i can integrate the constraint back into the table definition and take into account the order in which the tables need to be created to allow the constraint to be added.
You problem:
-- creating the sales table
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
-- creating the host_flags table
`sales_id` int(11) DEFAULT NULL,
-- the sales.id is declared as unsigned
-- the host_flags.sales_id is declared signed
Additonally to Recursed's answer:
There is a requirement that foreign keys contstraints' names must be unique in database scope. Maybe changing the name will work?
There is also a huge thread on MySQL community forums about the problem containing several solutions for some specific situations.
Possible two errors:
Reference and referenced columns must have the same type - int(11) unsigned
Unknown referenced table hosts.

MySQL problem to add FOREIGN KEY?

I've got 2 innodb tables, here it is with SHOW CREATE TABLE query:
| top_menu | CREATE TABLE `top_menu` (
`t_id` mediumint(8) unsigned NOT NULL AUTO_INCREMENT,
`menu_photo` char(128) NOT NULL,
`title` char(64) NOT NULL,
`atdc_id` int(10) unsigned NOT NULL,
`menu_order` mediumint(8) unsigned NOT NULL,
PRIMARY KEY (`t_id`)
) ENGINE=InnoDB AUTO_INCREMENT=8 DEFAULT CHARSET=utf8 |
| attendance | CREATE TABLE `attendance` (
`atdc_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`title` char(128) NOT NULL,
`content` text,
`price` double NOT NULL,
`sale_percent` tinyint(3) unsigned NOT NULL,
`atdc_order` int(10) unsigned NOT NULL,
`s_id` int(10) unsigned NOT NULL,
PRIMARY KEY (`atdc_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 |
The result of doing, normal adding FOREIGN KEY is an error.
Query: ALTER TABLE top_menu ADD FOREIGN KEY (atdc_id) REFERENCES attendance(atdc_id);
Error: ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (database., CONSTRAINT #sql-a36a_5c109d2_ibfk_1 FOREIGN KEY (atdc_id) REFERENCES attendance (atdc_id))
What should I do with this? It always worked for me well.
It seems that some top_menu rows have the atdc_id which does not exist in attendance table.
in complement to #Alexey Smirnoff answer:
When are you adding the foreign key? If it is from a restore script ensure you have the attendance table content loaded before you perform this query. Or delay foreign key checks to the end of the process.
If you want to find which rows are giving you this problem run this query:
Select a.*
from top_menu a
left join attendance b
on a.atdc_id = b.atdc_id
where b.atdc_id IS NULL;