Performance concerns regarding ALTER TABLE ADD COLUMN - mysql

I have a big MySQL InnoDB table having 5 million rows. I need to add a column to the table which will have a default int value.
What is the best way to do it? The normal alter table command appears to take a lot of time. Is there any better way to do it? Basically I want to know if there is any faster way or efficient way of doing it.
And if the table has foreign key references, is there any way other than alter table to do this?
Any help appreciated.

I would not say this is a better way, but ... You could create a separate table for the new data and set it up as foreign key relationship to the existing table. That would be "fast", but if the data really belongs in the main table and every (or most) existing records will have a value, then you should just alter the table and add it.

Suppose the table looked like this:
CREATE TABLE mytable
(
id INT NOT NULL AUTO_INCREMENT,
name VARCHAR(25),
PRIMARY KEY (id),
KEY name (name)
);
and you want to add an age column with
ALTER TABLE mytable ADD COLUMN age INT NOT NULL DEFAULT 0;
You could perform the ALTER TABLE in stages as follows:
CREATE TABLE mytablenew LIKE mytable;
ALTER TABLE mytablenew ADD COLUMN age INT NOT NULL DEFAULT 0;
INSERT INTO mytablenew SELECT id,name FROM mytable;
ALTER TABLE mytable RENAME mytableold;
ALTER TABLE mytablenew RENAME mytable;
DROP TABLE mytableold;
If mytable uses the MyISAM Storage Engine and has nonunique indexes, add two more lines
CREATE TABLE mytablenew LIKE mytable;
ALTER TABLE mytablenew ADD COLUMN address VARCHAR(50);
ALTER TABLE mytablenew DISABLE KEYS;
INSERT INTO mytablenew SELECT id,name FROM mytable;
ALTER TABLE mytable RENAME mytableold;
ALTER TABLE mytablenew RENAME mytable;
DROP TABLE mytableold;
ALTER TABLE mytable ENABLE KEYS;
This will let you see how many seconds each stage takes. From here, you can decide whether or not a straightforward ALTER TABLE is better.
This technique gets a little gory if there are foreign key references.
Your steps would be
SET UNIQUE_CHECKS = 0;
SET FOREIGN_KEY_CHECKS = 0;
Drop the foreign key references in mytable.
Perform the ALTER TABLE in Stages
Create the foreign key references in mytable.
SET UNIQUE_CHECKS = 1;
SET FOREIGN_KEY_CHECKS = 1;
Give it a Try !!!

Related

Error 156: table already exists when adding a primary key

I'm using the following statement ALTER TABLE my_tbl ADD PRIMARY KEY (id ); to add a primary key to an existing MySQL table. In reply I'm getting the error:
Error 156 : Table 'db_name.my_tbl#1' already exists.
I checked and the table has no duplicate id entries, and if I do something like DROP TABLE my_tbl#1 then the original table (my_tbl) is deleted. It's perhaps interesting to note that my_tbl was created by Create Table my_tbl SELECT id, ... FROM tmp_tbl (where tmp_tbl is a temporary table).
Anyone has an idea what's going on here?
Update: there seems to be some kind of an orphaned table situation here. I tried the suggestions in the answers below, but in my case they did not resolve the problem. I finally used a workaround: I created a table with a different name (e.g. my_tbl_new) , copied the information to this table and added to it the primary key. I Then deleted the original table and renamed the new one back to my_tbl.
try something like this:-
ALTER TABLE my_tbl DROP PRIMARY KEY, ADD PRIMARY KEY(id,id);
or try this:-
IF NOT EXISTS (SELECT * FROM INFORMATION_SCHEMA.TABLE_CONSTRAINTS WHERE CONSTRAINT_TYPE = 'PRIMARY KEY'
AND TABLE_NAME = '[my_tbl]'
AND TABLE_SCHEMA ='dbo' )
BEGIN
ALTER TABLE [dbo].[my_tbl] ADD CONSTRAINT [PK_ID] PRIMARY KEY CLUSTERED ([ID])
END
or try to flush the table like this:-
DROP TABLE IF EXISTS `my_tbl` ;
FLUSH TABLES `my_tbl` ;
CREATE TABLE `my_tbl` ...
DROP TABLE IF EXISTS `mytable` ;
FLUSH TABLES `mytable` ;
CREATE TABLE `mytable` ...
Also it might be a permission issue.
I had the same problem while trying to alter indexes, through SQLyog, when my database name contained "-" chars. So I renamed the database to not have them and then it worked just fine.
(Since there's no direct way to rename a DB, I had to copy it to a new one, with correct name)

How to do a MySQL Insert if unique, but the columns are too long for unique index

I have found a great answer for when inserting a new record, ignore if the data already exists.
1) Create a UNIQUE INDEX on the columns.
2) INSERT IGNORE INTO ...
But my problem is that one of the columns is a VARCHAR(2000)**, and MySQL has a 1000-character limit to indexes.
The columns are: id (int), type (varchar 35), data (varchar 2000)
So is there a way to make sure I'm not adding the same data twice with a single query? Or do I need to do a select first to check for existence and if false, perform the insert?
Thanks.
** This is not design, I'm just moving data around so no chance of making this column smaller.
Given the table design you mentioned:
CREATE TABLE mydb.mytable
(
id int not null auto_increment,
type varchar(35),
data varchar(2000),
primary key (id)
);
Your best chance would be the following:
CREATE TABLE mydb.mytable_new LIKE mydb.mytable;
ALTER TABLE mydb.mytable_new ADD COLUMN data_hash CHAR(40);
ALTER TABLE mydb.mytable_new ADD UNIQUE INDEX (data_hash);
INSERT INTO mydb.mytable_new (id,type,data,data_hash)
SELECT id,type,data,UPPER(SHA(data)) FROM mydb.mytable;
ALTER TABLE mydb.mytable RENAME mydb.mytable_old;
ALTER TABLE mydb.mytable_new RENAME mydb.mytable;
DROP TABLE mydb.mytable_old;
Once you add this new column and index, table should now look like this:
CREATE TABLE mydb.mytable
(
id int not null auto_increment,
type varchar(35),
data varchar(2000),
data_hash char(40),
primary key (id),
unique key data_hash (data_hash)
);
Simply perform your operations as follows:
INSERTs
INSERT INTO mydb.mytable (type,data,data_hash)
VALUES ('somtype','newdata',UPPER(SHA('newdata')));
INSERTs should fail on data_hash is you attempt a duplicate key insertion
SELECTs
SELECT * FROM mydb.mytable
WHERE data_hash = UPPER(SHA('data_I_am_searching_for'));
Give it a Try !!!
Hashes have collisions so unless you don’t care about that, this is not a foolproof solution

How to delete a column from a table in MySQL

Given the table created using:
CREATE TABLE tbl_Country
(
CountryId INT NOT NULL AUTO_INCREMENT,
IsDeleted bit,
PRIMARY KEY (CountryId)
)
How can I delete the column IsDeleted?
ALTER TABLE tbl_Country DROP COLUMN IsDeleted;
Here's a working example.
Note that the COLUMN keyword is optional, as MySQL will accept just DROP IsDeleted. Also, to drop multiple columns, you have to separate them by commas and include the DROP for each one.
ALTER TABLE tbl_Country
DROP COLUMN IsDeleted,
DROP COLUMN CountryName;
This allows you to DROP, ADD and ALTER multiple columns on the same table in the one statement. From the MySQL reference manual:
You can issue multiple ADD, ALTER, DROP, and CHANGE clauses in a single ALTER TABLE statement, separated by commas. This is a MySQL extension to standard SQL, which permits only one of each clause per ALTER TABLE statement.
Use ALTER TABLE with DROP COLUMN to drop a column from a table, and CHANGE or MODIFY to change a column.
ALTER TABLE tbl_Country DROP COLUMN IsDeleted;
ALTER TABLE tbl_Country MODIFY IsDeleted tinyint(1) NOT NULL;
ALTER TABLE tbl_Country CHANGE IsDeleted IsDeleted tinyint(1) NOT NULL;
To delete a single column:
ALTER TABLE `table1` DROP `column1`;
To delete multiple columns:
ALTER TABLE `table1`
DROP `column1`,
DROP `column2`,
DROP `column3`;
You can use
alter table <tblname> drop column <colname>
ALTER TABLE `tablename` DROP `columnname`;
Or,
ALTER TABLE `tablename` DROP COLUMN `columnname`;
If you are running MySQL 5.6 onwards, you can make this operation online, allowing other sessions to read and write to your table while the operation is been performed:
ALTER TABLE tbl_Country DROP COLUMN IsDeleted, ALGORITHM=INPLACE, LOCK=NONE;
Use ALTER:
ALTER TABLE `tbl_Country` DROP COLUMN `column_name`;
ALTER TABLE tbl_Country DROP columnName;
It is worth mentioning that MySQL 8.0.23 and above supports Invisible Columns
CREATE TABLE tbl_Country(
CountryId INT NOT NULL AUTO_INCREMENT,
IsDeleted bit,
PRIMARY KEY (CountryId)
);
INSERT INTO tbl_Country VALUES (1, 1), (2,0);
ALTER TABLE tbl_Country ALTER COLUMN IsDeleted SET INVISIBLE;
SELECT * FROM tbl_Country;
CountryId
1
2
ALTER TABLE tbl_Country DROP COLUMN IsDeleted;
db<>fiddle demo
It may be useful in scenarios when there is need to "hide" a column for a time being before it could be safely dropped(like reworking corresponding application/reports etc.).

Resetting the primary key to 1 after deleting all the data

So i have MySql and i have a table user with a user_id column and it is the primary key and auto incremented. Now when i delete all my data from the table and add the new one, the user_id does not start from 1 but from the number it had before deletion. What if i want to reset it without dropping the whole table and creating it again.
ALTER TABLE some_table AUTO_INCREMENT=1
So some_table would be the table you want to alter.
You could also use:
TRUNCATE TABLE some_table
This will reset the Auto Increment on the table as well as deleting all records from that table.
The code below is best if you have some data in the database already but want to reset the user_id to 1 without deleting the data. Copy and run in SQL command
ALTER TABLE members DROP user_id;
ALTER TABLE members AUTO_INCREMENT = 1;
ALTER TABLE members ADD user_id int UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY FIRST;
you can use DBCC check identity to reset your Primary key.
here is the Sytax:
DBCC CHECKIDENT(TableName,RESEED,0)

How to add AUTO_INCREMENT to an existing column?

How do I add auto_increment to an existing column of a MySQL table?
I think you want to MODIFY the column as described for the ALTER TABLE command. It might be something like this:
ALTER TABLE users MODIFY id INTEGER NOT NULL AUTO_INCREMENT;
Before running above ensure that id column has a Primary index.
Method to add AUTO_INCREMENT to a table with data while avoiding “Duplicate entry” error:
Make a copy of the table with the data using INSERT SELECT:
CREATE TABLE backupTable LIKE originalTable;
INSERT backupTable SELECT * FROM originalTable;
Delete data from originalTable (to remove duplicate entries):
TRUNCATE TABLE originalTable;
To add AUTO_INCREMENT and PRIMARY KEY
ALTER TABLE originalTable ADD id INT PRIMARY KEY AUTO_INCREMENT;
Copy data back to originalTable (do not include the newly created column (id), since it will be automatically populated)
INSERT originalTable (col1, col2, col3)
SELECT col1, col2,col3
FROM backupTable;
Delete backupTable:
DROP TABLE backupTable;
More on the duplication of tables using CREATE LIKE:
Duplicating a MySQL table, indices, and data
Alter table table_name modify column_name datatype(length) AUTO_INCREMENT PRIMARY KEY
You should add primary key to auto increment, otherwise you got error in mysql.
Simply just add auto_increment Constraint In column or MODIFY COLUMN :-
ALTER TABLE `emp` MODIFY COLUMN `id` INT NOT NULL UNIQUE AUTO_INCREMENT FIRST;
Or add a column first then change column as -
1. Alter TABLE `emp` ADD COLUMN `id`;
2. ALTER TABLE `emp` CHANGE COLUMN `id` `Emp_id` INT NOT NULL UNIQUE AUTO_INCREMENT FIRST;
This worked for me in case you want to change the AUTO_INCREMENT-attribute for a not-empty-table:
1.)Exported the whole table as .sql file
2.)Deleted the table after export
2.)Did needed change in CREATE_TABLE command
3.)Executed the CREATE_TABLE and INSERT_INTO commands from the .sql-file
...et viola
I managed to do this with the following code:
ALTER TABLE `table_name`
CHANGE COLUMN `colum_name` `colum_name` INT(11) NOT NULL AUTO_INCREMENT FIRST;
This is the only way I could make a column auto increment.
INT(11) shows that the maximum int length is 11, you can skip it if you want.
Alter table table_name modify table_name.column_name data_type AUTO_INCREMENT;
eg:
Alter table avion modify avion.av int AUTO_INCREMENT;
if you have FK constraints and you don't want to remove the constraint from the table. use "index" instead of primary. then you will be able to alter it's type to auto increment
I had existing data in the first column and they were 0's.
First I made the first column nullable.
Then I set the data for the column to null.
Then I set the column as an index.
Then I made it a primary key with auto incrementing turned on. This is where I used another persons answer above:
ALTER TABLE `table_name` CHANGE COLUMN `colum_name` `colum_name` INT(11) NOT NULL AUTO_INCREMENT FIRST;
This Added numbers to all the rows of this table starting at one. If I ran the above code first it wasn't working because all the values were 0's. And making it an index was also required before making it auto incrementing.
Next I made the column a primary key.
This worked in my case , if you want to change the column attribute to auto-increment which is already having some data
1.GO to structure, select the column to want to change.
2.After selecting the column , choose primary key from the options below.
[1]: https://i.stack.imgur.com/r7w8f.png
3.Then change the column attribute to auto-increment using alter method
This is to alter the column adding PRIMARY key:
ALTER TABLE `schema_name`.`table_name`
CHANGE COLUMN `id` `id` INT(11) NOT NULL AUTO_INCREMENT ,
ADD UNIQUE INDEX `id_UNIQUE` (`id` ASC) VISIBLE,
ADD PRIMARY KEY (`id`);
I copied it from MySQL Workbench... I got curious to see if it was possible to do it all in one command. I'm a little rusty in SQL.
If you are working in an specific schema, you don't need to specify it.
The above statement will create the index, set the column as the PRIMARY KEY as well with just one query.
KEEP IN MIND: There could not be duplicated values in the same column, if there are, the statement will fail to commit.
ALTER TABLE Table name ADD column datatype AUTO_INCREMENT,ADD primary key(column);