I am developing a Yii2 framework application and during my data modelling phase I decided to let DB engine handle simple updates and maintain my counter tables (which are used to return the number of unread messages and other stuff for the user).
I have designed a trigger on message table to increase and decrease number of unread messages for the user.
User can be either client or architect.
All of my id columns in the DB are UNSIGNED INTs (these are also the PK for respective tables).
I have a problem when forward engineering from Workbench. I have made a script with test data for integrity testing and initial population. The script executed fine, before I added the AFTER_INSERT trigger for the message table.
Here is the trigger code:
CREATE DEFINER = CURRENT_USER TRIGGER `visavis`.`message_AFTER_INSERT` AFTER INSERT ON `message` FOR EACH ROW
BEGIN
DECLARE archid INT UNSIGNED;
# Cheks if recevier is an architect
SELECT IFNULL(`visavis`.`architect`.`id`,0)
INTO archid
FROM `visavis`.`architect`
WHERE `visavis`.`architect`.`user` = NEW.`to`;
# Checks if the new message is set to sent (and not read)
IF NEW.status = 1
THEN
IF archid = 0 -- if the receiver is client
THEN
# Checks if the user receiving exists in the user_counter table
IF NOT EXISTS (SELECT 1 FROM `visavis`.`user_counter` WHERE `visavis`.`user_counter`.`user` = NEW.`to`)
THEN
# Insert new row into user_counter table
INSERT INTO `visavis`.`user_counter` (`user`,`messages`) VALUES (NEW.`to`,1);
ELSE
# Add one to the user followings counter
UPDATE `visavis`.`user_counter`
SET `visavis`.`user_counter`.`messages` = `visavis`.`user_counter`.`messages` + 1
WHERE `visavis`.`user_counter`.`user` = NEW.`to`;
END IF; -- if user_counter
ELSE
# Extra check if archid is null
#IF ISNULL(archid)
#THEN
# SET archid = 1; -- Testing value
#END IF;
# Checks if the architect receiving exists in the architect_counter table
IF NOT EXISTS (SELECT 1 FROM `visavis`.`architect_counter` WHERE `visavis`.`architect_counter`.`architect` = archid)
THEN
# Insert new row into architect_counter table
INSERT INTO `visavis`.`architect_counter` (`architect`,`messages`) VALUES (archid,1);
ELSE
# Add one to the user followings counter
UPDATE `visavis`.`architect_counter`
SET `visavis`.`architect_counter`.`messages` = `visavis`.`architect_counter`.`messages` + 1
WHERE `visavis`.`architect_counter`.`architect` = archid;
END IF; -- if architect_counter
END IF; -- if receiver is client
END IF; -- if message is sent
END
The problem is that I get this error:
ERROR: Error 1048: Column 'architect' cannot be null
In the code above in the ELSE branch of the client or architect check I inserted the extra check code to assign value to the variable (it is commented out). With this code script passes fine, but all unread messages end up with architect with id=1.
I am also adding my tables' DDLs:
CREATE TABLE IF NOT EXISTS `architect` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'A PK for the table.',
`user` INT UNSIGNED NOT NULL COMMENT 'Link to the user that is the architect.',
`office` INT UNSIGNED NULL COMMENT 'Link to the architect\'s office, if any.',
`short_description` INT UNSIGNED NOT NULL COMMENT 'Link to the text of short description.',
`description` INT UNSIGNED NOT NULL COMMENT 'Link to the text of description.',
`specialty_1` INT UNSIGNED NULL COMMENT 'Link to the specialty.',
`specialty_2` INT UNSIGNED NULL COMMENT 'Link to the specialty.',
`specialty_3` INT UNSIGNED NULL COMMENT 'Link to the specialty.',
`order` INT UNSIGNED NOT NULL,
PRIMARY KEY (`id`),
CONSTRAINT `fk_architect_specialty_1`
FOREIGN KEY (`specialty_1`)
REFERENCES `specialty` (`id`)
ON DELETE SET NULL
ON UPDATE NO ACTION,
CONSTRAINT `fk_architect_specialty_2`
FOREIGN KEY (`specialty_2`)
REFERENCES `specialty` (`id`)
ON DELETE SET NULL
ON UPDATE NO ACTION,
CONSTRAINT `fk_architect_specialty_3`
FOREIGN KEY (`specialty_3`)
REFERENCES `specialty` (`id`)
ON DELETE SET NULL
ON UPDATE NO ACTION,
CONSTRAINT `fk_architect_short_description`
FOREIGN KEY (`short_description`)
REFERENCES `text` (`id`)
ON DELETE RESTRICT
ON UPDATE NO ACTION,
CONSTRAINT `fk_architect_description`
FOREIGN KEY (`description`)
REFERENCES `text` (`id`)
ON DELETE RESTRICT
ON UPDATE NO ACTION,
CONSTRAINT `fk_architect_office`
FOREIGN KEY (`office`)
REFERENCES `office` (`id`)
ON DELETE SET NULL
ON UPDATE NO ACTION,
CONSTRAINT `fk_architect_user`
FOREIGN KEY (`user`)
REFERENCES `user` (`id`)
ON DELETE NO ACTION
ON UPDATE NO ACTION)
ENGINE = InnoDB
COMMENT = 'The info about the architect.';
CREATE TABLE IF NOT EXISTS `visavis`.`message` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'A PK of the table.',
`from` INT UNSIGNED NOT NULL COMMENT 'User that sent the message.',
`to` INT UNSIGNED NOT NULL COMMENT 'User that recieves the message.',
`text` VARCHAR(2000) NOT NULL COMMENT 'Text of the message. Length constrained in the frontend.',
`status` INT UNSIGNED NOT NULL DEFAULT 6 COMMENT 'Status of the message.',
`created_at` DATETIME NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT 'Date and time when the message was created. Automaticaly recieves status 6 (draft).',
`viewed_at` DATETIME NULL COMMENT 'Date and time when the message was viewed by the reciever. Set when the status changes to 1.',
`sent_at` DATETIME NULL COMMENT 'Date and time when the message was sent. Set when the status changes to 2.',
`replied_at` DATETIME NULL COMMENT 'Date and time when the message was replied, if any. Set when the status changes to 3.',
`shared_at` DATETIME NULL COMMENT 'Date and time when the message was shared to external board. Set when the status changes to 4.',
`deleted_at` DATETIME NULL COMMENT 'Date and time of message deletion (from the view). Set when the status changes to 5.',
`message_type` INT UNSIGNED NOT NULL COMMENT 'Link to the type of the message.',
`attachment` INT UNSIGNED NULL COMMENT 'Link to the attachment.',
`template` INT UNSIGNED NULL COMMENT 'Link to the template the message implements.',
PRIMARY KEY (`id`),
INDEX `fk_user_from_idx` (`from` ASC),
INDEX `fk_user_to_idx` (`to` ASC),
INDEX `fk_message_type_type_idx` (`message_type` ASC),
INDEX `fk_message_status_status_idx` (`status` ASC),
INDEX `fk_message_attachment_idx` (`attachment` ASC),
INDEX `fk_message_template_idx` (`template` ASC),
CONSTRAINT `fk_user_from`
FOREIGN KEY (`from`)
REFERENCES `visavis`.`user` (`id`)
ON DELETE CASCADE
ON UPDATE NO ACTION,
CONSTRAINT `fk_user_to`
FOREIGN KEY (`to`)
REFERENCES `visavis`.`user` (`id`)
ON DELETE CASCADE
ON UPDATE NO ACTION,
CONSTRAINT `fk_message_type_type`
FOREIGN KEY (`message_type`)
REFERENCES `visavis`.`message_type` (`id`)
ON DELETE RESTRICT
ON UPDATE NO ACTION,
CONSTRAINT `fk_message_status_status`
FOREIGN KEY (`status`)
REFERENCES `visavis`.`message_status` (`id`)
ON DELETE RESTRICT
ON UPDATE NO ACTION,
CONSTRAINT `fk_message_attachment`
FOREIGN KEY (`attachment`)
REFERENCES `visavis`.`attachment` (`id`)
ON DELETE SET NULL
ON UPDATE NO ACTION,
CONSTRAINT `fk_message_template`
FOREIGN KEY (`template`)
REFERENCES `visavis`.`message_template` (`id`)
ON DELETE SET NULL
ON UPDATE NO ACTION)
ENGINE = InnoDB
COMMENT = 'Internal messaging system.'
CREATE TABLE IF NOT EXISTS `visavis`.`architect_counter` (
`architect` INT UNSIGNED NOT NULL COMMENT 'A PK of the table.',
`houses` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of houses in the system.',
`followers` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of followers.',
`liked_houses` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of houses that the users liked.',
`sold` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of purchased items of the architect.',
`messages` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of unread messages.',
`customizings` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of customize request an architect has received.',
`workings` INT UNSIGNED NOT NULL DEFAULT 0 COMMENT 'The number of customizing work an archotect has begun work on. (accepted and assigned quotes)',
PRIMARY KEY (`architect`),
CONSTRAINT `fk_architect_counter_architect`
FOREIGN KEY (`architect`)
REFERENCES `visavis`.`architect` (`id`)
ON DELETE CASCADE
ON UPDATE NO ACTION)
ENGINE = InnoDB
COMMENT = 'Counter table for summed values, of the logged in achitect, ' /* comment truncated */ /*needed for the front end.*/
I also tried COALESCE(), CASE, read that IFNULL() returns SIGNED values so I CAST() the entire IFNULL clause.
I don't have any problem with delimiters (Workbench handles them)
THIS IS THE FIX:
IF EXISTS (SELECT 1 FROM `visavis`.`architect` WHERE `visavis`.`architect`.`user` = NEW.`to`)
THEN
SELECT `visavis`.`architect`.`id`
INTO archid
FROM `visavis`.`architect`
WHERE `visavis`.`architect`.`user` = NEW.`to`;
ELSE
SET archid = 0;
END IF;
I may be off track here, but this query
SELECT IFNULL(`visavis`.`architect`.`id`,0)
INTO archid
FROM `visavis`.`architect`
WHERE `visavis`.`architect`.`user` = NEW.`to`;
will set archid to the value of visavis.architect.id in the result row of the select statement or 0 if that value in the result row is null - i.e. there is a null value for that id in visavis.architect.
So what happens if the select statement doesn't match any rows? Well there is no result row, so the IFNULL statement is never run and archid remains at it's initial value of null.
I would remove the ifnull and let archid be set to null. Then just chnage your client/architect check to test if archid is null:
SELECT `visavis`.`architect`.`id`
INTO archid
FROM `visavis`.`architect`
WHERE `visavis`.`architect`.`user` = NEW.`to`;
IF NEW.status = 1
THEN
IF ISNULL(archid) THEN -- if the receiver is client
...
Related
I am creating an inventory management app in node.js that uses MySQL as a database. I have a weak entity “rental_item” that holds the items in a particualr rental. The issue is that the rental may not come back all at once so I need a way of marking the “rental_returned” boolean in the rental table true only when all of the “item_returned” entires are true.
Here is my table structure:
CREATE TABLE `rental` (
`rental_id` int NOT NULL AUTO_INCREMENT,
`renter_id` int NOT NULL,
`date_in` date NOT NULL,
`date_out` date NOT NULL,
`sig_path` varchar(50) NOT NULL,
`doc_path` varchar(50) NOT NULL,
`col_name` varchar(50) NOT NULL,
`col_path` varchar(50) NOT NULL,
`cost` decimal(15,2) NOT NULL,
`rental_returned` tinyint(1) NOT NULL DEFAULT '0',
PRIMARY KEY (`rental_id`),
UNIQUE KEY `doc_path` (`doc_path`),
UNIQUE KEY `col_path` (`col_path`),
UNIQUE KEY `sig_path` (`sig_path`),
KEY `renter_key` (`renter_id`),
CONSTRAINT `renter_key` FOREIGN KEY (`renter_id`) REFERENCES `renter` (`renter_id`)
)
CREATE TABLE `rental_item` (
`rental_id` int NOT NULL,
`i_ID` varchar(20) NOT NULL,
`item_returned` tinyint(1) NOT NULL DEFAULT '0',
KEY `rental_key` (`rental_id`),
KEY `rental_item_key` (`i_ID`),
CONSTRAINT `rental_item_key` FOREIGN KEY (`i_ID`) REFERENCES `item` (`i_ID`),
CONSTRAINT `rental_key` FOREIGN KEY (`rental_id`) REFERENCES `rental` (`rental_id`) ON DELETE CASCADE
)
I am currently doing this through the mysql2 node.js module and just checking for all the values of a given rental_id. I then found out about triggers and thought this way could be better. I fiddled round with things like this Trigger with table join, but couldn’t wrap my head around how to get the rental_id of the entry that was updated from rental_item, then check that all entires in rental_item with that id have item_returned = 1, and finally update the rental table to show that all the items/the complete rental has been returned.
I understand that this sould be an update after trigger on rental_item but dont know how to handle the conditionals or loops needed.
Use NEW.rental_id to get the ID of the row that was updated.
CREATE TRIGGER rental_returned AFTER UPDATE ON rental_item
FOR EACH ROW
UPDATE rental
SET rental_returned = (
NOT EXISTS (
SELECT *
FROM rental_item
WHERE rental_id = NEW.rental_id
AND item_returned = 0))
WHERE rental_id = NEW.rental_id
I have 3 tables:
create table users (
id int auto_increment primary key,
first_name varchar(40) not null,
last_name varchar(40) not null,
email varchar(40) not null,
password varchar(40) not null,
deleted tinyint(1) not null default 0
) engine = InnoDB;
create table comments (
id int auto_increment primary key,
task_id int not null,
foreign key fk_comment_task(task_id) references tasks(id),
user int not null,
foreign key fk_comment_user(user) references users(id),
comment varchar(1000) not null,
comment_date date not null,
viewed tinyint(1) not null default 0,
deleted tinyint(1) not null default 0
) engine = InnoDB;
create table viewed_comments (
id int auto_increment primary key,
comment int not null,
foreign key fk_viewed_comment(comment) references comments(id),
viewer int not null,
foreign key fk_viewed_viewer(viewer) references users(id),
unread tinyint(1) not null default 0,
deleted tinyint(1) not null default 0
) engine = InnoDB;
I made a trigger that creates rows in viewed_comments for every user in the users table that is not the user that submitted the comment:
delimiter |
create trigger ins_views after insert on comments
for each row
begin
DECLARE finished INT DEFAULT 0;
DECLARE id INT DEFAULT 0;
DECLARE currentId CURSOR FOR SELECT id FROM users WHERE id != NEW.user;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET finished = 1;
OPEN currentId;
update_views_loop: LOOP
FETCH currentId INTO id;
IF finished = 1 THEN LEAVE update_views_loop; END IF;
INSERT INTO viewed_comments (comment, viewer) VALUES (NEW.id, id);
END LOOP update_views_loop;
CLOSE currentId;
END;
|
delimiter ;
The problem is that when I try to insert a comment such as
insert into comments (task_id, user, comment, comment_date) values (24, 6, 'test', '2018-3-5');
It fails saying the 'Cannot add or update a child row: a foreign key constraint fails (viewed_comments, CONSTRAINT fk_viewed_viewer FOREIGN KEY (viewer) REFERENCES users (id)). When I change the trigger on the insert line to the following it works to a degree:
INSERT INTO viewed_comments (comment, viewer) VALUES (NEW.id, 6);
On this sample set I have 3 users, and on the viewed_comments table I get 3 new rows, instead of 2. I'm not sure how to troubleshoot this in MySQL or how to print the variables to see the values as it happens. It seems on the cursor line New keyword is disregarded resulting in 3 rows instead of 2. But, then it has a value on the insert statement, but the cursor doesn't have any values. I am using MariaDB 10.4.11 How do I fix this?
I have a MySQL DB. Acquired data are stored in raw_data_headers, raw_data_rows and raw_data_row_details table.
raw_data_row_details has a foreign key that reference raw_data_rows.ID, the same for raw_data_rows and raw_data_headers.
In raw_data_headers are stored data headers, in raw_data_rows are stored every stage of acquisition program and in raw_data_row_details are stored details for each stage of acquisition program.
This is the query:
SELECT
q1.ProcessTypeID,
q1.TestTypeID,
q1.ComponentID,
q1.TestResultID,
COUNT(*) AS Counter
FROM (
SELECT
raw_data_headers.batch_id AS BatchID,
raw_data_test_outputs.test_output_type_id AS TestOutputTypeID,
raw_data_test_types.process_type_id AS ProcessTypeID,
raw_data_test_types.ID AS TestTypeID,
raw_data_row_details.component_id AS ComponentID,
raw_data_test_results.ID AS TestResultID
FROM raw_data_row_details
INNER JOIN raw_data_rows ON raw_data_rows.ID = raw_data_row_details.row_id
INNER JOIN raw_data_headers ON raw_data_headers.ID = raw_data_rows.header_id
INNER JOIN raw_data_test_results ON raw_data_test_results.ID = raw_data_row_details.Value
INNER JOIN raw_data_test_outputs ON raw_data_test_outputs.ID = raw_data_row_details.test_output_id
INNER JOIN raw_data_test_types ON raw_data_test_types.ID = raw_data_test_outputs.test_type_id
HAVING TestOutputTypeID = 2 AND BatchID = 1
) AS q1
GROUP BY q1.ProcessTypeID, q1.TestTypeID, q1.ComponentID, q1.TestResultID
raw_data_headers has 989'180 entries, row_data_rows has 2'967'540 entries and raw_data_row_details has 13'848'520 entries.
The subquery q1 take about 3 minutes, but final query takes about 25 minutes. I think that the point is in the GROUP BY.
How can I improve performance?
EDIT 1:
SELECT
gnuhmi.raw_data_test_types.process_type_id AS ProcessTypeID,
gnuhmi.raw_data_test_types.ID AS TestTypeID,
gnuhmi.raw_data_row_details.component_id AS ComponentID,
gnuhmi.raw_data_test_results.ID AS TestResultID,
COUNT(*) AS Counter
FROM gnuhmi.raw_data_row_details
INNER JOIN gnuhmi.raw_data_rows ON gnuhmi.raw_data_rows.ID = gnuhmi.raw_data_row_details.row_id
INNER JOIN gnuhmi.raw_data_headers ON gnuhmi.raw_data_headers.ID = gnuhmi.raw_data_rows.header_id
INNER JOIN gnuhmi.raw_data_test_results ON gnuhmi.raw_data_test_results.ID = gnuhmi.raw_data_row_details.Value
INNER JOIN gnuhmi.raw_data_test_outputs ON gnuhmi.raw_data_test_outputs.ID = gnuhmi.raw_data_row_details.test_output_id
INNER JOIN gnuhmi.raw_data_test_types ON gnuhmi.raw_data_test_types.ID = gnuhmi.raw_data_test_outputs.test_type_id
WHERE gnuhmi.raw_data_test_outputs.test_output_type_id = 2 AND gnuhmi.raw_data_headers.batch_id = 1
GROUP BY
gnuhmi.raw_data_test_results.ID,
gnuhmi.raw_data_row_details.component_id,
gnuhmi.raw_data_test_types.ID,
gnuhmi.raw_data_test_types.process_type_id
This is the new query, without subquery and WHERE. This increased performance (thanks #Yogesh Sharma).
this is raw_data_headers structure:
CREATE TABLE `raw_data_headers` (
`ID` int(11) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Univocal record key',
`ProductID` int(11) NOT NULL COMMENT 'Product numeric ID',
`Datetime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'Univocal record creation date',
`batch_id` int(11) DEFAULT NULL COMMENT 'Univocal batch key',
`RecipeName` varchar(80) DEFAULT NULL COMMENT 'Used recipe name',
`RecipeVersion` smallint(6) DEFAULT NULL COMMENT 'Used recipe version',
`process_result_id` smallint(6) DEFAULT NULL COMMENT 'Process result key',
`invalidated` tinyint(1) NOT NULL DEFAULT '0' COMMENT 'invalidation after counters reset',
PRIMARY KEY (`ID`),
KEY `FK_raw_data_headers_batches_ID` (`batch_id`),
KEY `FK_raw_data_headers_process_re` (`process_result_id`),
CONSTRAINT `FK_raw_data_headers_batches_ID` FOREIGN KEY (`batch_id`) REFERENCES `batches` (`ID`) ON UPDATE CASCADE,
CONSTRAINT `FK_raw_data_headers_process_re` FOREIGN KEY (`process_result_id`) REFERENCES `process_result` (`ID`) ON DELETE NO ACTION ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COMMENT='Stores raw data headers'
This the raw_dato_rows:
CREATE TABLE `raw_data_rows` (
`ID` int(11) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Univocal record key',
`Datetime` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP COMMENT 'Univocal record creation date',
`header_id` int(11) unsigned NOT NULL COMMENT 'Univocal raw data header key',
`process_type_id` smallint(6) NOT NULL COMMENT 'Univocal process type key',
`process_result_id` smallint(6) NOT NULL COMMENT 'Univocal process result key',
PRIMARY KEY (`ID`),
KEY `FK_raw_data_rows_header_id` (`header_id`),
KEY `FK_raw_data_rows_process_resu2` (`process_result_id`),
KEY `FK_raw_data_rows_process_resul` (`process_type_id`),
CONSTRAINT `FK_raw_data_rows_header_id` FOREIGN KEY (`header_id`) REFERENCES `raw_data_headers` (`ID`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `FK_raw_data_rows_process_resu2` FOREIGN KEY (`process_result_id`) REFERENCES `process_result` (`ID`) ON DELETE NO ACTION ON UPDATE CASCADE,
CONSTRAINT `FK_raw_data_rows_process_resul` FOREIGN KEY (`process_type_id`) REFERENCES `process_types` (`ID`) ON DELETE NO ACTION ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=2967541 DEFAULT CHARSET=utf8 COMMENT='Stores row data rows'
and finally this is the raw_data_row_details one:
CREATE TABLE `raw_data_row_details` (
`ID` bigint(20) NOT NULL AUTO_INCREMENT COMMENT 'Univocal row detail key',
`row_id` int(11) unsigned NOT NULL COMMENT 'Univocal row key',
`test_output_id` int(11) NOT NULL COMMENT 'Univocal test output key',
`component_id` int(11) NOT NULL COMMENT 'The component that take the measurement',
`Value` double NOT NULL COMMENT 'Output value',
PRIMARY KEY (`ID`),
KEY `FK_raw_data_row_details_row_id` (`row_id`),
KEY `FK_raw_data_rows_raw_data_test_outputs_ID` (`test_output_id`),
KEY `raw_data_row_details_components_FK` (`component_id`),
CONSTRAINT `FK_raw_data_row_details_row_id` FOREIGN KEY (`row_id`) REFERENCES `raw_data_rows` (`ID`) ON DELETE CASCADE ON UPDATE CASCADE,
CONSTRAINT `FK_raw_data_rows_raw_data_test_outputs_ID` FOREIGN KEY (`test_output_id`) REFERENCES `raw_data_test_outputs` (`ID`) ON UPDATE CASCADE,
CONSTRAINT `raw_data_row_details_components_FK` FOREIGN KEY (`component_id`) REFERENCES `components` (`ID`) ON UPDATE CASCADE
) ENGINE=InnoDB AUTO_INCREMENT=13848521 DEFAULT CHARSET=utf8 COMMENT='Stores raw data rows details'
You don't need to use subquery, just use where clause with group by :
SELECT raw_data_test_types.process_type_id AS ProcessTypeID,
raw_data_test_types.ID AS TestTypeID,
raw_data_row_details.component_id AS ComponentID,
raw_data_test_results.ID AS TestResultID, COUNT(*) AS Counter
FROM raw_data_row_details INNER JOIN
raw_data_rows
ON raw_data_rows.ID = raw_data_row_details.row_id INNER JOIN
raw_data_headers
ON raw_data_headers.ID = raw_data_rows.header_id INNER JOIN
raw_data_test_results
ON raw_data_test_results.ID = raw_data_row_details.Value INNER JOIN
raw_data_test_outputs
ON raw_data_test_outputs.ID = raw_data_row_details.test_output_id INNER JOIN
raw_data_test_types
ON raw_data_test_types.ID = raw_data_test_outputs.test_type_id
WHERE raw_data_headers.batch_id = 1 AND raw_data_test_outputs.test_output_type = 2
GROUP BY raw_data_test_types.process_type_id, raw_data_test_types.ID,
raw_data_row_details.component_id, raw_data_test_results.ID;
Add indexes. TestOutputTypeID and BatchID need to be covered and probably are not.
To see what's currently going on, use EXPLAIN in the MySQL console. You will probably see an indication that a full table scan is happening i.e. the join type is marked as ALL.
It's often the case that the query optimiser will use the same execution plan for different queries e.g. by expanding the subquery as if you hadn't used it. Only EXPLAIN will show you what's what.
Here's the docs on how to interpret the EXPLAIN output: https://dev.mysql.com/doc/refman/8.0/en/explain-output.html
HAVING TestOutputTypeID = 2 AND BatchID = 1
Change that from HAVING to WHERE, and have indexes in each of those columns.
Also have these indexes:
raw_data_row_details: (row_id)
raw_data_rows: (header_id)
raw_data_row_details: (test_output_id)
raw_data_test_outputs: (test_type_id)
Get rid of raw_data_ from the table names; it just clutters the queries.
If those do not help enough, please provide EXPLAIN SELECT ... and SHOW CREATE TABLE.
To preface, I'm not very experienced with database design. I have a table of hashes and ids. When a group of new hashes are added, each row in the group gets the same id. If any hash within the new group already exists in the database, all hashes in the new group and existing group(s) get a new, shared id (effectively merging ids when hashes are repeated):
INSERT INTO hashes
(id, hash)
VALUES
($new_id, ...), ($new_id, ...)
ON DUPLICATE KEY UPDATE
repeat_count = repeat_count + 1;
INSERT INTO hashes_lookup SELECT DISTINCT id FROM hashes WHERE hash IN (...);
UPDATE hashes JOIN hashes_lookup USING (id) SET id = '$new_id';
TRUNCATE TABLE hashes_lookup;
Other tables reference these ids, so that if an id changes, foreign key constraints take care of updating the ids across tables. The issue here, however, is that I can't enforce uniqueness across any of the child tables. If I do, my queries fail with:
Foreign key constraint for table '...', record '...' would lead to a duplicate entry in table '...'
This error makes sense, given the following test case where id and value are a composite unique key:
id | value
---+-------
a | 1
b | 2
c | 1
Then a gets changed to c:
id | value
---+-------
c | 1
b | 2
c | 1
But c,1 already exists.
It would be ideal if there was an ON UPDATE IGNORE CASCADE option, so that if a duplicate row exists, any duplicating inserts are ignored. However, I'm pretty sure the real issue here is my database design, so I am open to any and all suggestions. My current solution is to not enforce uniqueness across child tables, which leads to a lot of redundant rows.
Edit:
CREATE TABLE `hashes` (
`hash` char(64) NOT NULL,
`id` varchar(128) NOT NULL,
`repeat_count` int(11) NOT NULL DEFAULT '0',
`insert_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`update_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
UNIQUE KEY `hash` (`hash`) USING BTREE,
KEY `id` (`id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=latin1
CREATE TABLE `emails` (
`id` varchar(128) NOT NULL,
`group_id` char(5) NOT NULL,
`email` varchar(500) NOT NULL,
KEY `index` (`id`) USING BTREE,
UNIQUE KEY `id` (`id`,`group_id`,`email`(255)) USING BTREE,
CONSTRAINT `emails_ibfk_1` FOREIGN KEY (`id`) REFERENCES `hashes` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1
I think will be good to create table hash_group to store id of hash group:
CREATE TABLE `hash_group` (
`id` BIGINT AUTO_INCREMENT NOT NULL,
`group_name` varchar(128) NOT NULL,
`insert_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`update_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
UNIQUE KEY `group_name` (`group_name`) USING BTREE,
PRIMARY KEY (`id`) USING BTREE
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
And change structure of existing tables:
CREATE TABLE `hashes` (
`hash` char(64) NOT NULL,
`hash_group_id` BIGINT NOT NULL,
`repeat_count` int(11) NOT NULL DEFAULT '0',
`insert_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`update_timestamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
UNIQUE KEY `hash` (`hash`) USING BTREE,
KEY `hashes_hash_group_id_index` (`hash_group_id`) USING BTREE,
CONSTRAINT `hashes_hash_group_id_fk` FOREIGN KEY (`hash_group_id`) REFERENCES `hash_group` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
CREATE TABLE `emails` (
`hash_group_id` BIGINT NOT NULL,
`group_id` char(5) NOT NULL,
`email` varchar(500) NOT NULL,
KEY `emails_hash_group_id_index` (`hash_group_id`) USING BTREE,
UNIQUE KEY `emails_unique` (`hash_group_id`,`group_id`,`email`(255)) USING BTREE,
CONSTRAINT `emails_ibfk_1` FOREIGN KEY (`hash_group_id`) REFERENCES `hash_group` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Also create trigger to update hash group if you need to do it:
DELIMITER $$
CREATE TRIGGER `update_hash_group_name` AFTER UPDATE ON `hashes`
FOR EACH ROW
BEGIN
UPDATE `hash_group`
SET `group_name` = md5(now()) -- replace to you hash formula
WHERE id = NEW.hash_group_id;
END;$$
DELIMITER ;
And create function for getting actual group id:
DROP FUNCTION IF EXISTS get_hash_group;
DELIMITER $$
CREATE FUNCTION get_hash_group(id INT) RETURNS INT
BEGIN
IF (id IS NULL) THEN
INSERT INTO `hash_group` (`group_name`)
VALUES (md5(now())); -- replace to you hash
RETURN LAST_INSERT_ID();
END IF;
RETURN id;
END;$$
DELIMITER ;
Scenario:
Initial fill:
INSERT INTO `hash_group` (id, group_name) VALUES
(1, 'test1'),
(2, 'test2'),
(3, 'test3');
INSERT INTO `hashes` (hash, hash_group_id) VALUES
('hash11', 1),
('hash12', 1),
('hash13', 1),
('hash2', 2),
('hash3', 3);
INSERT INTO `emails` (hash_group_id, group_id, email)
VALUES
(1, 'g1', 'example1#'),
(2, 'g1', 'example2#'),
(3, 'g1', 'example2#');
Updating of hash_group scenario:
START TRANSACTION;
-- Get #min_group_id - minimum group id (we will leave this id and delete other)
SELECT MIN(hash_group_id) INTO #min_group_id
FROM hashes
WHERE hash IN ('hash11', 'hash12', 'hash2', 'hash15');
-- Replace other group ids in email table to #min_group_id
UPDATE `emails`
SET `hash_group_id` = #min_group_id
WHERE `hash_group_id` IN (
SELECT hash_group_id
FROM hashes
WHERE #min_group_id IS NOT NULL
AND hash IN ('hash11', 'hash12', 'hash2', 'hash15')
-- Update only if we are gluy several hash_groups
AND `hash_group_id` > #min_group_id
);
-- Delete other hash_groups and leave only group with #min_group_id
DELETE FROM `hash_group` WHERE `id` IN (
SELECT hash_group_id
FROM hashes
WHERE #min_group_id IS NOT NULL
AND hash IN ('hash11', 'hash12', 'hash2', 'hash15')
-- Delete only if we are gluy several hash_groups
AND `hash_group_id` > #min_group_id
);
-- #group_id = existing hash_group.id or create new if #min_group_id is null (all inserted hashes are new)
SELECT get_hash_group(#min_group_id) INTO #group_id;
-- Now we can insert new hashes.
INSERT INTO `hashes` (hash, hash_group_id) VALUES
('hash11', #group_id),
('hash12', #group_id),
('hash2', #group_id),
('hash15', #group_id)
ON DUPLICATE KEY
UPDATE repeat_count = repeat_count + 1;
COMMIT;
I maybe wrong but I think you mis-named the id field in hashes.
I think you should rename the id field in hashes to something like group_id, then have a AUTO_INCREMENT field called id that should also be PRIMARY in hashes that the id in emails refers to this field instead. When you want to update and relate all the hashes together, you update the group_id field instead of id, and id remains unique across the table.
This way you can avoid the cascade problem, also you will always know the original hash that the email was referring to.
Sure, if you want to fetch all the hashes related to an email (old and the new) you must exectue and extra query, but I think it solves all your problems.
Edit:
you can use a trigger to do this
The trigger goes like this
DELIMITER $$
CREATE TRIGGER `update_hash_id` AFTER UPDATE ON `hashes`
FOR EACH ROW
BEGIN
UPDATE `emails` SET `id` = NEW.id WHERE `id` = OLD.id;
END;$$
DELIMITER ;
and you must remove the foreign key relation too.
The solution, which we have arrived in chat chat:
/* Tables */
CREATE TABLE `emails` (
`group_id` bigint(20) NOT NULL,
`email` varchar(500) NOT NULL,
UNIQUE KEY `group_id` (`group_id`,`email`) USING BTREE,
CONSTRAINT `emails_ibfk_1` FOREIGN KEY (`group_id`) REFERENCES `entities` (`group_id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1
CREATE TABLE `hashes` (
`group_id` bigint(20) NOT NULL,
`hash` varchar(128) NOT NULL,
`repeat_count` int(11) NOT NULL DEFAULT '0',
UNIQUE KEY `hash` (`hash`),
KEY `group_id` (`group_id`),
CONSTRAINT `hashes_ibfk_1` FOREIGN KEY (`group_id`) REFERENCES `entities` (`group_id`) ON DELETE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=latin1
CREATE TABLE `entities` (
`group_id` bigint(20) NOT NULL,
`entity_id` bigint(20) NOT NULL,
PRIMARY KEY (`group_id`),
KEY `entity_id` (`entity_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
CREATE TABLE `entity_lookup` (
`entity_id` bigint(20) NOT NULL,
PRIMARY KEY (`entity_id`) USING HASH
) ENGINE=MyISAM DEFAULT CHARSET=latin1
/* Inserting */
START TRANSACTION;
/* Determine next group ID */
SET #next_group_id = (SELECT MAX(group_id) + 1 FROM entities);
/* Determine next entity ID */
SET #next_entity_id = (SELECT MAX(entity_id) + 1 FROM entities);
/* Merge any entity ids */
INSERT IGNORE INTO entity_lookup SELECT entity_id FROM entities JOIN hashes USING(group_id) WHERE HASH IN(...);
UPDATE entities JOIN entity_lookup USING(entity_id) SET entity_id = #next_entity_id;
TRUNCATE TABLE entity_lookup;
/* Add the new group ID to entity_id */
INSERT INTO entities(group_id, entity_id) VALUES(#next_group_id, #next_entity_id);
/* Add new values into hashes */
INSERT INTO hashes (group_id, HASH) VALUES
(#next_group_id, ...)
ON DUPLICATE KEY UPDATE
repeat_count = repeat_count + 1;
/* Add other new values */
INSERT IGNORE INTO emails (group_id, email) VALUES
(#next_group_id, "email1");
COMMIT;
Adding an extra integer column to each of the child tables would avoid this problem altogether by using it as a primary key. The key never changes because it isn't a reference to anything else.
Using composite keys as primary keys is generally something that you want to avoid. And considering that this key combination is not always unique, I would definitely say you need a dedicated primary key in all of your child tables with this problem.
You can even auto increment it so you aren't manually assigning it every time. For example..
Create Table exampleTable
(
trueID int NOT NULL AUTO_INCREMENT,
col1 int NOT NULL,
col2 varChar(50)
PRIMARY KEY(trueID)
)
Then, when two of the rows in a child table are set with identical values (for whatever reason), the primary key stays unique, preventing any conflicts in the Database that could arise.
Help me please!
I am having this error.
Error: Cannot add or update a child row: a foreign key constraint fails (world.alarmes, CONSTRAINT fk_alarmes_registos1 FOREIGN KEY (idRegisto) REFERENCES registos (idRegisto) ON DELETE NO ACTION ON UPDATE NO ACTION)
I have these tables.
CREATE TABLE `registos` (
`data_registo` char(10) NOT NULL,
`hora_registo` time NOT NULL,
`idSensor` varchar(8) NOT NULL,
`Temperatura` char(6) DEFAULT NULL,
`Humidade` char(6) DEFAULT NULL,
`pt_orvalho` char(6) DEFAULT NULL,
`idRegisto` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`idRegisto`,`idSensor`,`data_registo`,`hora_registo`),
KEY `fk_registos_sensores1_idx` (`idSensor`),
CONSTRAINT `fk_registos_sensores1` FOREIGN KEY (`idSensor`) REFERENCES `sensores` (`idSensor`) ON DELETE NO ACTION ON UPDATE NO ACTION
)
CREATE TABLE `alarmes` (
`idAlarme` int(11) NOT NULL AUTO_INCREMENT,
`descricao_alarme` varchar(45) DEFAULT NULL,
`data_criacao` datetime DEFAULT CURRENT_TIMESTAMP,
`idRegisto` int(11) NOT NULL DEFAULT ''0'',
PRIMARY KEY (`idAlarme`,`idRegisto`),
KEY `fk_alarmes_registos1_idx` (`idRegisto`),
CONSTRAINT `fk_alarmes_registos1` FOREIGN KEY (`idRegisto`) REFERENCES `registos` (`idRegisto`) ON DELETE NO ACTION ON UPDATE NO ACTION
)
When I do an insert into the table records the error pops up.
insert into registos values ('2014-03-31', '14:03:32', 'BrgTH032', '22.3', '45.3', '9.9', '32');
If I do this:
SET FOREIGN_KEY_CHECKS=0
the next insertion already accepted, but when I try again. back to give the same error.
I've been researching and fails because the registos table references a foreign key from the sensores table. You can't directly insert into a relational table without there being a corresponding entry in the table that is being referenced.
But I don't know how to resolve this.
Help me please.
-------EDIT( I used a trigger to populate the table Alarmes)------------------------
DELIMITER $$
create TRIGGER alerta
BEFORE INSERT ON registos
FOR EACH ROW
begin
Set #tempmax=0;
Set #tempmin=0;
select lim_inf_temp, lim_sup_temp into #tempmin, #tempmax from sensores where idSensor=NEW.idSensor;
Set #maxidAlarme=0;
if (CAST(NEW.Temperatura AS UNSIGNED)<#tempmin) then
SELECT MAX(idAlarme) into #maxidAlarme FROM alarmes;
SET #maxidAlarme=#maxidAlarme+1;
INSERT INTO alarmes(idAlarme,descricao_alarme, idRegisto) VALUES (#maxidAlarme,"temperatura inserida inferior ao normal",New.idRegisto);
end if;
if (CAST(NEW.Temperatura AS UNSIGNED)>#tempmax) then
SELECT MAX(idAlarme) into #maxidAlarme FROM alarmes;
SET #maxidAlarme=#maxidAlarme+1;
INSERT INTO alarmes(idAlarme,descricao_alarme, idRegisto) VALUES (#maxidAlarme,"temperatura inserida superior ao normal",New.idRegisto);
end if;
end $$;
DELIMITER ;
You are trying to insert more values into the table than allowed 7 (seven) but 6 (six) expected.
Please, always include the columns that you are inserting to in an 'insert' query.
There are seven columns in this table but one is an 'auto increment' column so there should be 6 (six) values in the insert query.
CREATE TABLE `registos` (
`data_registo` char(10) NOT NULL,
`hora_registo` time NOT NULL,
`idSensor` varchar(8) NOT NULL,
`Temperatura` char(6) DEFAULT NULL,
`Humidade` char(6) DEFAULT NULL,
`pt_orvalho` char(6) DEFAULT NULL,
`idRegisto` int(11) NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`idRegisto`,`idSensor`,`data_registo`,`hora_registo`),
KEY `fk_registos_sensores1_idx` (`idSensor`),
CONSTRAINT `fk_registos_sensores1
Here is the 'insert' query:
insert into registos values ('2014-03-31', '14:03:32', 'BrgTH032', '22.3', '45.3', '9.9', '32');
There are seven values but you would expect the query to look like (columns added):
insert into registos (data_registo, hora_registo, idSensor, Temperatura, Humidade, pt_orvalho)
values ('2014-03-31', '14:03:32', 'BrgTH032', '22.3', '45.3', '9.9', '32');
I suggest that the query should be:
insert into registos (data_registo, hora_registo, idSensor, Temperatura, Humidade, pt_orvalho)
values ('2014-03-31', '14:03:32', 'BrgTH032', '22.3', '45.3', '9.9');
The trigger on 'registos' must be an after insert trigger to pick up the NEW 'idRegisto' value.
create TRIGGER alerta
AFTER INSERT ON registos
FOR EACH ROW
begin