Martin Kleppmann in his book "Designing Data-Intensive Applications" is showcasing the following problem:
Claiming a username
On a website where each user has a unique username, two users may try to create
accounts with the same username at the same time. You may use a transaction to
check whether a name is taken and, if not, create an account with that name.
However, like in the previous examples, that is not safe under snapshot isolation.
Fortunately, a unique constraint is a simple solution here (the second transaction
that tries to register the username will be aborted due to violating the constraint).
I have a very similar use case, where 2 transactions are trying to claim the name of the entity.
At the beginning of each transaction, I run a select to see if such name was already taken. If it wasn't - create or update, depending on the operation requested by the user. This logic crumbles under concurrent attempts to claim/modify the name.
I am trying to see if there is a mechanism that allows implementing correct behavior under the Repeatable Read isolation level. Unique constraint violation thrown by the DB is not acceptable in my case, neither is a downgrade to Serializable execution.
Can I employ Select For ... Update here? Obviously, I won't be locking the concrete rows, but rather an entire table (correct me if I am wrong in my assumption) as I will not have pk index columns in the WHERE subclause?
Table structure:
CREATE TABLE `application_domains` (
`id` varchar(255) NOT NULL,
`name` varchar(255) NOT NULL,
`description` varchar(10000) DEFAULT NULL,
`org_id` varchar(255) NOT NULL,
`created_time` bigint(20) NOT NULL,
`updated_time` bigint(20) NOT NULL,
`created_by` varchar(16) NOT NULL,
`changed_by` varchar(16) NOT NULL,
`revision_id` varchar(16) DEFAULT NULL,
`topic_domain` varchar(255) NOT NULL,
`enforce_unique_topic_names` tinyint(1) NOT NULL DEFAULT '1',
`sample_id` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `UK_orgId_name` (`org_id`,`name`),
UNIQUE KEY `UK_orgId_sampleId` (`org_id`,`sample_id`),
KEY `FK_references_application_domains_organization` (`org_id`),
KEY `FK_app_domain_samples_id_references_application_domains_tbl` (`sample_id`),
CONSTRAINT `FK_app_domain_samples_id_references_application_domains_tbl` FOREIGN KEY (`sample_id`) REFERENCES `application_domain_samples` (`id`) ON DELETE SET NULL ON UPDATE SET NULL,
CONSTRAINT `FK_references_application_domains_organization` FOREIGN KEY (`org_id`) REFERENCES `organizations` (`org_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
Related
I have 2 Database Tables "users" and "tasks".
The tasks table contains two foreign keys for the createdBy_user_id and updatedBy_user_id columns both referencing users.id. If I try to insert an entry to tasks (after making sure the user referenced by the foreign keys exists) like this:
INSERT INTO tasks (createdBy_user_id,updatedBy_user_id,noOfYear,createdAt,updatedAt,status,customer_id)
VALUES (1,1,1,NOW(),NOW(),"open",1)
The query fails with Error 1452:
ERROR 1452 (23000): Cannot add or update a child row: a foreign key constraint fails (`test`.`tasks`, CONSTRAINT `user_id_fk_constr` FOREIGN KEY (`createdBy_user_id`) REFERENCES `users` (`id`))
I don't know why this happens, because everything works fine if I remove the constraint. The same error does not happen for the "updatedBy_user_id" column making this such confusing.
The Tables have the following DDL:
Users table:
CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`active` tinyint(1) DEFAULT NULL,
`email` varchar(255) DEFAULT NULL,
`email_confirmed_at` datetime DEFAULT NULL,
`username` varchar(50) NOT NULL,
`password` varchar(255) NOT NULL,
`first_name` varchar(50) DEFAULT NULL,
`last_name` varchar(50) DEFAULT NULL,
`job` varchar(64) DEFAULT NULL,
`position` varchar(64) DEFAULT NULL,
`specialKnowledge` text,
`tasks` text,
PRIMARY KEY (`id`),
UNIQUE KEY `username` (`username`),
UNIQUE KEY `email` (`email`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4
Tasks table:
CREATE TABLE `tasks` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`createdAt` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updatedAt` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`noOfYear` int(11) NOT NULL,
`year` int(11) GENERATED ALWAYS AS (right(year(`createdAt`),2)) VIRTUAL NOT NULL,
`createdBy_user_id` int(11) NOT NULL,
`updatedBy_user_id` int(11) NOT NULL,
`status` enum('open','closed') NOT NULL,
`customer_id` int(11) NOT NULL,
`projectDescription` text,
PRIMARY KEY (`id`),
UNIQUE KEY `tasks_year_unique_constr` (`year`,`noOfYear`),
KEY `user_id_fk_constr` (`createdBy_user_id`),
KEY `customer_id_fk_constr` (`customer_id`),
KEY `user_up_id_fk_constr` (`updatedBy_user_id`),
CONSTRAINT `customer_id_fk_constr` FOREIGN KEY (`customer_id`) REFERENCES `Customer` (`id`),
CONSTRAINT `user_id_fk_constr` FOREIGN KEY (`createdBy_user_id`) REFERENCES `users` (`id`),
CONSTRAINT `user_up_id_fk_constr` FOREIGN KEY (`updatedBy_user_id`) REFERENCES `users` (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=utf8mb4
As you can see the datatypes match and both tables use the InnoDB Engine.
The users table contains one entry:
select id,username from users;
+----+----------+
| id | username |
+----+----------+
| 1 | admin |
+----+----------+
1 row in set (0.00 sec)
So there is no obvious reason, why the insertion should fail. Have you an idea what's wrong with my tables? Looks like a software bug.
This is a bug. MySQL 5.7 has had some troubles with generated columns and foreign keys, and I assume this is an unfixed variant of Bug #79772 Foreign key not allowed when a virtual index exists.
In your particular case, and probably depending on your exact version, any of the following modifications seems to prevent that bug from occurring:
do not use a virtual column, but make it stored
do not create a foreign key for a column directly following the virtual column, so e.g. change the column order to year, status, createdBy_user_id, updatedBy_user_id.
do not use a unique index on the virtual column, a normal index should be fine (at least in a version where the linked bug is fixed). You want a unique constraint, so this is not an option, but the fact that this fixes your problem emphasized the "bug" nature of the problem.
The second bullet point seems to be the underlying bug: I assume that some iterator doesn't count the virtual column properly, so the foreign key that shall check createdBy_user_id seems to mix up the columns and actually checks the value of year (in this case "20") against the users table. So if you have a user with id "20" in your users table, the foreign key will actually accept this, no matter what value for createdBy_user_id you are trying to insert, see the MySQL 5.7.29 fiddle.
Unless you have a specific reason to use a virtual column, using stored is probably the sane thing to do.
I have this Ticket table supposed to be used in a Queue system where I can forward a ticket from a Service queue to a different queue.
My Ticket table has fields such as number, ticketRequest (timestamp of when the ticket was created), expectedCallTimestamp (timestamp of predicted call to be attended), etc...
Here's the CREATE SCRIPT for the table Ticket:
CREATE TABLE `ticket` (
`ticket_id` int(11) NOT NULL AUTO_INCREMENT,
`wrong_ticket_id` int(11) DEFAULT NULL,
`number` int(3) NOT NULL,
`forwarded` tinyint(1) NOT NULL DEFAULT '0',
`answered` tinyint(1) DEFAULT NULL,
`ticketRequest` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`expectedCallTimestamp` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
`callTimestamp` timestamp NULL DEFAULT NULL,
`serviceDuration` time DEFAULT NULL,
`service_id` int(11) NOT NULL,
`organic_unit_id` int(11) NOT NULL,
`device_id` int(11) NOT NULL,
`user_id` int(11) DEFAULT NULL,
`counter_id` int(11) DEFAULT NULL,
PRIMARY KEY (`ticket_id`),
UNIQUE KEY `UQ_ticket_wrongTicketId` (`wrong_ticket_id`) USING BTREE,
KEY `FK_ticket_queue` (`service_id`,`organic_unit_id`),
KEY `FK_ticket_device` (`device_id`),
KEY `FK_ticket_workerbycounter` (`user_id`,`counter_id`),
CONSTRAINT `FK_ticket_device` FOREIGN KEY (`device_id`) REFERENCES `device` (`device_id`),
CONSTRAINT `FK_ticket_queue` FOREIGN KEY (`service_id`, `organic_unit_id`) REFERENCES `queue` (`service_id`, `organic_unit_id`),
CONSTRAINT `FK_ticket_workerbycounter` FOREIGN KEY (`user_id`, `counter_id`) REFERENCES `workerbycounter` (`user_id`, `counter_id`)
) ENGINE=InnoDB AUTO_INCREMENT=21 DEFAULT CHARSET=latin1
OK, so what I thought was - when a client picks a different service that he intended to pick (it happens sometimes) I, as the operator behind the counter, can create a new ticket with the same ticketRequest timestamp of the wrongServiceTicket and use that wrongServiceTicket's.ticket_id to reference it through a forwardedTicket's.wrong_ticket_id (the one that has the intended/correct service) for several purposes.
My approach to this is having a 1 to 0 or 1 recursive relationship on the table Ticket, with a nullable & unique field of wrong_ticket_id that is the same as the wrongServiceTicket.ticket_id when a forwardedTicket is created, as previously explained.
Here's a data model example of what I'm trying to build:
Is it possible to have this implementation or is there a better way to handle this problem? And how do I select the info of the wrongServiceTicket when, for example, calling the next ticket in line when having a ticket that's forwarded in the same queue as other tickets that aren't?
As I understood you intend to establish parent-child relationship between existing ticket (wrong one) and new ticket (corrected one).
Your suggested approach is viable except few minor corrections -
Keep wrong_ticket_id column as NULLABLE as it can be null in most of the tickets which were created with a valid service.
You can optionally put a check constraint on wrong_ticket_id. This column will be populated only when 'forwarded' = True.
I have defined 2 tables and a foriegn key constraint between them as follows:
| users | CREATE TABLE `users` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`account_master_id` int(11) NOT NULL,
`user_type_id` int(11) NOT NULL,
`user_group_id` int(11) NOT NULL,
`user_type_code` char(1) NOT NULL,
`membership_number` varchar(40) NOT NULL,
`password` varchar(60) NOT NULL,
`email` varchar(200) NOT NULL,
`isd` varchar(10) NOT NULL,
`mobile` varchar(20) NOT NULL,
`passenger_id` int(11) NOT NULL,
`added_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`updated_on` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`added_by` int(11) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `email` (`email`),
KEY `account_master_id` (`account_master_id`),
CONSTRAINT `acMaster_to_user` FOREIGN KEY (`account_master_id`) REFERENCES `account_master` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=189 DEFAULT CHARSET=utf8 |
user_oauth | CREATE TABLE `user_oauth` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`user_id` int(11) NOT NULL,
`service` varchar(30) NOT NULL,
`auth_id` varchar(100) NOT NULL,
`email` varchar(100) NOT NULL,
`auto_share` tinyint(4) NOT NULL,
`photo` varchar(255) NOT NULL,
`auth_token_short` varchar(255) DEFAULT NULL,
`auth_details` text NOT NULL,
`device_type` varchar(60) NOT NULL,
`login_date` datetime NOT NULL,
`login_ip` varchar(20) NOT NULL,
PRIMARY KEY (`id`),
KEY `user` (`user_id`),
CONSTRAINT `user_to_oauth` FOREIGN KEY (`user_id`) REFERENCES `users` (`id`) ON DELETE CASCADE ON UPDATE NO ACTION
) ENGINE=InnoDB AUTO_INCREMENT=109 DEFAULT CHARSET=latin1 |
on deleting a row from users table, deletes corresponding entries from user_oauth table in LOCAL and Staging environment. but however same thing is not working in PRODUCTION environment. I want to know how can i debug this.
update:
both tables are innodb
running mysql in strict mode
The reasons why foreign keys might not work are:
they are not supported (e.g. on MyISAM-tables)
they are disabled
MySQL allows you to disable foreign key contraints by setting the system variable foreign_key_checks to 0. This will allow you to violate all foreign key constraints (e.g. delete a parent or add parentless childs). But it will in turn also disable related features like cascades, which are there to automatically prevent specific constraint violations (by e.g. deleting the children) - which of course won't occur anymore if the option is disabled.
The idea is to help you with some administration tasks, e.g. importing data when the referenced data is not yet there, but should usually not be used during normal operation. If the setting reappears, you might want to check your apps if one is setting this option by accident, as it is enabled after every server start by default and has to be disabled explicitely.
You can check the current setting by e.g. using
select ##foreign_key_checks;
You can use
SET foreign_key_checks = 1;
to enable it again, but be aware that it will not check your current data:
Setting foreign_key_checks to 1 does not trigger a scan of the existing table data. Therefore, rows added to the table while foreign_key_checks=0 will not be verified for consistency.
So you will have to check and fix it yourself. You can do it either before or after you enabled the setting again, although it might be easier to do it before. To trigger a recheck, you can and should drop and recreate the foreign key, just to make sure everything is consistent now.
I have defined the following 2 tables:
record_status
SHOW CREATE TABLE record_status
CREATE TABLE `record_status` (
`record_status_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`status` char(6) NOT NULL,
`status_description` varchar(15) NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`record_status_id`,`status`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
user
SHOW CREATE TABLE user
CREATE TABLE `user` (
`user_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`handle` varchar(45) NOT NULL,
`email` varchar(255) NOT NULL,
`password` char(64) DEFAULT NULL,
`password_salt` binary(1) DEFAULT NULL,
`first_name` varchar(50) NOT NULL,
`last_name` varchar(50) NOT NULL,
`gender` char(1) DEFAULT NULL,
`birthday` date NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`user_status` char(6) DEFAULT NULL,
PRIMARY KEY (`user_id`),
KEY `usr_status_idx` (`user_status`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
and I tried adding the foreign key user_status of type CHAR using mysql Workbench as follows:
ALTER TABLE `mydatabase`.`user`
ADD CONSTRAINT `usr_status`
FOREIGN KEY (`user_status`)
REFERENCES `mydatabase`.`record_status` (`status`)
ON DELETE NO ACTION
ON UPDATE NO ACTION;
but I am getting the following error:
Error:
Executing SQL script in server
ERROR: Error 1005: Can't create table 'mydatabase.#sql-420_1b0' (errno: 150)
ALTER TABLE 'mydatabase'.'user'
ADD CONSTRAINT 'usr_status'
FOREIGN KEY ('user_status')
REFERENCES 'mydatabase'.'record_status'('status')
ON DELETE NO ACTION
ON UPDATE NO ACTION
SQL script execution finished: statements: 4 succeeded, 1 failed.
Question
My intention is to have the status column clearly show the current status for each user (ACTIVE, INACTV, DELETD) while still having the flexibility to join the record_status table with the user table using the record_status_id to find any rows with a given status for better performance.
I found a similar post here
Adding foreign key of type char in mysql
which suggests to change my primary key's collation but, how would that affect my user table?
Will I have to change the collation to the user_status field in my user table as well? The user table will be queried every time a user logs in and I am concerned about performance or any constraints this may cause.
I also intend to add a foreign key for the status to a few other tables as well. I would just like to know how this affects performance, or does it add any constraints?
Any input regarding my design will also be appreciated. Thank you for your help!
The issue you're facing isn't actually related to collation (though collation can be a cause of the error you're experiencing under different circumstances).
Your FOREIGN KEY constraint is failing because you don't have an index individually on record_status.status. You have that column as part of the composite PRIMARY KEY (record_status_id, status), but for successful foreign key constraint creation, both the referencing table and the referenced table must have indexes on exactly the columns used in the key relationship (in addition to the same data types).
Adding the FOREIGN KEY constraint implicitly creates the necessary index on the referencing table, but you must still ensure you have the corresponding index on the referenced table.
So given what you have now, if you added a single index on record_status.status, the constraint would correctly be created.
CREATE TABLE `record_status` (
`record_status_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`status` char(6) NOT NULL,
`status_description` varchar(15) NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (`record_status_id`,`status`),
-- This would make your relationship work...
KEY (`status`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
However, I don't think that's the best course of action. I don't see a need for the composite primary key on (record_status_id, status), chiefly because the record_status_id is itself AUTO_INCREMENT and guaranteed to be unique. That column alone could be the PRIMARY KEY, while still adding an additional UNIQUE KEY on status to satisfy the foreign key constraint's indexing requirement. After all, it is not the combination of record_status_id and status which uniquely identifies each row (making a primary key)
CREATE TABLE `record_status` (
`record_status_id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`status` char(6) NOT NULL,
`status_description` varchar(15) NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- Primary only on record_status_id
PRIMARY KEY (`record_status_id`),
-- Additional UNIQUE index on status
UNIQUE KEY (`status`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
About the design -- eliminating record_status_id...
Without knowing how the rest of your application currently uses record_status_id, I can't say for sure if it required by your application code. But, if you wish to make the actual status value easily available to other tables, and it is merely CHAR(6), it is possible that you actually have no need for record_status_id as an integer value. After all, if the status string is intended to be unique, then it is perfectly capable of serving as the PRIMARY KEY on its own, without any auto-increment integer key.
In that case, your record_status table would look like below, and your FOREIGN KEY constraint would correctly be added to users.
CREATE TABLE `record_status` (
-- Remove the auto_increment column!!
`status` char(6) NOT NULL,
`status_description` varchar(15) NOT NULL,
`created_at` datetime NOT NULL,
`updated_at` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
-- Status is unique, and therefore can be the PK on its own
PRIMARY KEY (`status`)
) ENGINE=InnoDB AUTO_INCREMENT=4 DEFAULT CHARSET=latin1
Given this setup, here's a sample showing the successful creation of the tables and addition of the FK constraint.
You asked about performance implications of adding a status FK to other tables as well. It's tough to speculate on that without knowing the purpose, but if other tables share the same status values, then it makes sense to create their FK constraints to link to it in the same say you're doing with users. And if that's the case, I would recommend doing it the same way, wherein the status column is CHAR(6) (or consider changing all of them to VARCHAR(6)). The value of record_status.status still makes sense as the true primary key, and can be used as the FK in as many related tables as necessary.
In all but the most gigantic scale, there should be no appreciable performance difference between using an INT value and a CHAR(6)/VARCHAR(6) value as the foreign key. And the storage size difference between them is equally tiny. It isn't worth worrying about unless you must scale this to positively enormous proportions.
I have to implement the tasks below:
Task:
At present, the database knows two types of messages:
Messages that a user posts and that are public for anyone and everyone to read
Messages that a user posts and that are non-public. These messages can only be read by users that the posting user has marked as friends.
In this step, you should add a third type of message. This third type of message should be readable by specified recipients only.
This means the database needs to provide the following:
A way of distinguishing between the three types of messages. This involves a change to the Message table.
A way of specifying who the recipients of a particular message are. This will probably require an additional table.
All this must again be achieved with minimal amount of storage, i.e., you must choose the appropriate data types from the MySQL manual. You may assume that the total number of messages that are added over time may reach 1,000,000,000.
Your job is to implement the necessary changes and additional table for this purpose and any keys and foreign key relationships required.
Here are my two tables first : User
CREATE TABLE IF NOT EXISTS `User` (
`user_id` int(10) unsigned NOT NULL auto_increment,
`given_name` varchar(60) default NULL,
`surname` varchar(60) default NULL,
`address` varchar(255) default NULL,
`city_id` int(10) unsigned NOT NULL,
`date_of_birth` datetime default NULL,
`email` varchar(80) default NULL,
PRIMARY KEY (`user_id`),
KEY `ix_user_surname` (`surname`),
KEY `ix_user_given_name` (`given_name`),
KEY `ix_user_name` (`given_name`,`surname`),
KEY `ix_user_date_of_birth` (`date_of_birth`),
KEY `ix_user_email` (`email`),
KEY `ix_user_city_id` (`city_id`)
) ENGINE=InnoDB
2nd table :Message
CREATE TABLE IF NOT EXISTS `Message` (
`message_id` int(10) unsigned NOT NULL auto_increment,
`owner_id` int(10) unsigned default NULL,
`subject` varchar(255) default NULL,
`body` text,
`posted` datetime default NULL,
`is_public` tinyint(4) default '0',
PRIMARY KEY (`message_id`),
KEY `ix_message_owner_id` (`owner_id`)
) ENGINE=InnoDB
MY SOLUTION: I was thinking of creating a new table called 'Message_level' and have columns 'message_level_id'(will refer to 1,2,3 as 1=public, 2=private, 3=specific) & 'message_level'(where it would state public,private and specific next to level). Then I can use the 'Message_level' as a foreign key into the 'Message' table and replace the 'is_public' column with 'message_level_id'.
Is my approach to this question right? is there another way I can do this to make it more efficient?
and how would I approach the second task of specifying who the recipients of a particular message are?
I would go like this:
User: user_id, given_name, ...
Message: message_id, owner_id (fk User), subject, body, posted, message_type_id (fk Message_type)...
Message_recipients: user_id (fk User), message_id (fk Message)
Message_type: message_type_id, description (1:public, 2:friends, 3:specific_recipients)