Best way to add column to a big table with longblob - mysql

I have the following table:
CREATE TABLE IF NOT EXISTS PDF_STORAGE (
ID_PDF_STORAGE bigint(20) NOT NULL AUTO_INCREMENT,
DESC_FILE varchar(255) DEFAULT NULL,
PDF_FILE longblob,
LINK_FILE varchar(255) DEFAULT NULL,
VERSION int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (ID_PDF_STORAGE)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;
Where PDF_FILE is a file of 10 MB on average. Today it has about 50.000 rows. I need to add a new column to this table but it is taking a long time, more than 10 min, some times giving a 401 error in PhpMyAdmin, so I'd like to know what is the proper way to achieve this...
I already tried:
ALTER TABLE PDF_STORAGE ADD VERSION INT NOT NULL DEFAULT '0' AFTER LINK_FILE ;
and
SET FOREIGN_KEY_CHECKS=0;
CREATE TABLE PDF_STORAGE_new LIKE PDF_STORAGE;
ALTER TABLE PDF_STORAGE_new ADD VERSION INT NOT NULL DEFAULT '0' AFTER LINK_FILE ;
INSERT INTO PDF_STORAGE_new (PDF_STORAGE, DESC_FILE, ID_PDF_STORAGE, LINK_FILE) SELECT * FROM PDF_STORAGE;
RENAME TABLE PDF_STORAGE TO PDF_STORAGE_old, PDF_STORAGE_new TO PDF_STORAGE;
DROP TABLE PDF_STORAGE_old;
SET FOREIGN_KEY_CHECKS = 1;
but they are also slow.. is there a better way?
Thanks

What you are doing now ALTER TABLE is the best approach to my knowledge. You can try making this change when there is not much transaction (or) DB operation going on. I mean say, do the changes in idle time.
ALTER TABLE PDF_STORAGE ADD VERSION INT NOT NULL DEFAULT '0' AFTER LINK_FILE ;
You can as well create a new table same as this table schema along with the new column.
Insert all the records from this table to the newly created table.
Rename the new table to the old table name.
delete the old table.

Related

I need to put in a script an id with zero value

CREATE TABLE IF NOT EXISTS home (
id int(11) NOT NULL AUTO_INCREMENT,
description(200) CHARACTER SET latin1 COLLATE latin1_general_ci NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1 COLLATE=latin1_bin;
I need to insert a default value with id=0 but, I can't in this way. Anyone can help me?
You cannot have a DEFAULT value on an AUTOINCREMENT column. Also, as ID is your primary key, it would not really make sense to have a DEFAULT.
If you are just trying to insert an ID with a « 0 » value : mysql by default does not allow this on an AUTOINCREMENT field, but this can be tweaked with :
SET [GLOBAL|SESSION] sql_mode='NO_AUTO_VALUE_ON_ZERO'
However this is not recommended. See the mysql docs
I would be very cautious about trying to do such a thing. MySQL is so adamant about keeping zeros out that it changes the values of 0 to 1 when a column becomes an auto-increment column. Consider what happens with this code:
CREATE TABLE IF NOT EXISTS home (
id int(11) NOT NULL,
description varchar(200) CHARACTER SET latin1 COLLATE latin1_general_ci NOT NULL,
PRIMARY KEY (id)
) ENGINE=InnoDB AUTO_INCREMENT=1;
insert into home (id, description)
values (0, 'my home');
select *
from home;
alter table home modify column id int auto_increment;
select *
from home;
The table no longer has 0 after the alter table.
Presumably, this is because long ago MySQL decided that 0 is a signal for using the default value, so the value is baked in.
You can use any other value that you want. So you might consider -1 if you want a "default" values for your tables that use auto_increment.

I need to create a MyIsam table that has an iniital autoincrement value of 0

I am using mysql 5.7.18.
I tried:
CREATE TABLE t1 (
`UID` int(10) auto_increment,
day int(10),
PRIMARY KEY (`UID`)
) ENGINE=MyISAM AUTO_INCREMENT=0;
When i looked at the table info in workbench, it shows the autoinc value is 1.
The weird thing is that I managed to set it to 0 in two other tables!
I also tried:
alter table t1 auto_increment=0;
I did this before any data was written into the table.
I'm at a bit of a loss.
Any recommendations?
Update:
CREATE TABLE `sname`.`z` (
`Uid` INT NOT NULL DEFAULT -1,
PRIMARY KEY (`Uid`))
ENGINE = MyISAM
AUTO_INCREMENT = 0;
This creates the table with the autoincrement start at 0
I do not understand why the top example does not* do what I want it to and the bottom one does.
After even more research, it would appear that you cannot have 0 for an initial value in an autoincrement field.

mysql update a table with LOAD DATA

So I have someone working on a csv (generated from my table) where they will be updating some fields by hand. This may happen multiple times. I'd like to take the modified csv and update my existing table.
From my understanding, I will need to create a tmp table and then use that to update the existing table. So I can create the temporary table, but how can I iterate through that table and use it to update the existing table?
My sql querying skills are pretty basic. I think its possible, but I'm not sure where to start.
You don't need a temporary table. Just make sure the CSV file includes the primary key of the table. Then you can use the REPLACE modifier in LOAD DATA INFILE. From the documentation:
If you specify REPLACE, input rows replace existing rows. In other words, rows that have the same value for a primary key or unique index as an existing row.
In the CSV you are generating that gets edited, you must include a unique value that will allow you to match the edited record to the original record. Make sure the user doesn't change that column! Also, make sure you have a unique key on that column.
You can then import the edited data into a table with the same (or at least very similar) structure as the original table.
Once the data is imported, you can use an INSERT ... ON DUPLICATE KEY UPDATE ... statement to update the original table. Here's an example:
Main data table:
DROP TABLE IF EXISTS `my_table`;
CREATE TABLE IF NOT EXISTS `my_table` (
`id` INT(10) UNSIGNED NOT NULL AUTO_INCREMENT COMMENT 'Primary Key',
`fld1` VARCHAR(100) NULL,
`fld2` VARCHAR(100) NULL,
`fld3` VARCHAR(100) NULL,
PRIMARY KEY (`id`)
)
ENGINE=MyISAM
AUTO_INCREMENT=1
DEFAULT CHARSET=utf8
COLLATE=utf8_unicode_ci;
Temporary table for edited CSV import:
DROP TABLE IF EXISTS `import_table`;
CREATE TABLE IF NOT EXISTS `import_table` (
`n_id` INT(10) NOT NULL COMMENT 'Original Primary Key',
`n_fld1` VARCHAR(100) NULL,
`n_fld2` VARCHAR(100) NULL,
`n_fld3` VARCHAR(100) NULL,
PRIMARY KEY (`id`)
)
ENGINE=MyISAM
AUTO_INCREMENT=1
DEFAULT CHARSET=utf8
COLLATE=utf8_unicode_ci;
Simulated data before export and editing:
INSERT INTO `my_table`
(`fld1`,`fld2`,`fld3`)
VALUES
('John','Doe','Atlanta'),
('Jane','Smith','New York'),
('Bill','Howe','San Antonio'),
('Harry','Fields','Paris');
Simulate the imported, edited records:
INSERT INTO `import_table`
(`n_id`,`n_fld1`,`n_fld2`,`n_fld3`)
VALUES
(1,'John','Doe','Decatur, IL'),
(2,'Jane','Smithsonian','New York, NY'),
(3,'Bill','Bellweather','San Antonio, TX'),
(4,'Harry','Belefonte','Houston, TX');
Merge the imported, edited records bak into the main table:
INSERT INTO `my_table`
(`id`,`fld1`,`fld2`,`fld3`)
SELECT `n_id`,`n_fld1`,`n_fld2`,`n_fld3`
FROM `import_table`
ON DULPICATE KEY UPDATE
`fld1` = `n_fld1`,
`fld2` = `n_fld2`,
`fld3` = `n_fld3`;

MYSQL (Percona) Failed to read auto-increment value from storage engine

I am getting a "Failed to read auto-increment value from storage engine" error from MySQL (Percona), after trying to:
LOAD DATA LOCAL INFILE
I have been trying to find an answer on stackoverflow and google, but the answers did not solve the problem/error for my case.
The following shows Auto_increment as 1:
show table status like 'my_table_name';
Setting the auto increment value to the next value or to 1 did not solve it:
ALTER TABLE `my_table_name` AUTO_INCREMENT = 1;
Dropping the auto incrementing id column and recreating it did not solve it.
Dropping the table and recreating it did not solve it.
Dropping the database and recreating it did not solve it.
Destroying all data and rebuilding my docker development environment did not solve it.
Check table extended gave "OK":
CHECK TABLE distributor_users EXTENDED;
And it is the only auto incrementing value in the table.
This is what I use to create the table:
CREATE TABLE `my_table_name` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`x0` varchar(191) DEFAULT NULL,
`x1` varchar(4) DEFAULT NULL,
`x2` varchar(191) DEFAULT NULL,
`x3` varchar(191) DEFAULT NULL,
`x4` varchar(191) DEFAULT NULL,
`x5` varchar(191) DEFAULT NULL,
`x6` DATE DEFAULT NULL,
`x7` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `x0` (`x0`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
What could it be? Any lead to start looking at is highly welcomed!
I solved it myself by adding to my query:
SET id = NULL
The new query now looks like this:
LOAD DATA LOCAL INFILE '$csv_file_location' INTO TABLE $destination_table
FIELDS TERMINATED BY ','
ENCLOSED BY '\"'
IGNORE 1 LINES
(x0,x1,x2,x3,x4,x5,x6,x7)
SET id = NULL;
Although my query worked previously without it.
MySQL was trying to place the first column in the csv file into the first column of the database. Setting the column that has the auto_increment to NULL solved it.

Forcing an alter table on MySQL

I'm trying to add a column to a live database, that's constantly being accessed:-
ALTER TABLE `projects` ADD COLUMN `cTime` INT(30) NULL DEFAULT NULL AFTER `session`
It ALWAYS hangs. This is a MyISAM table and I'm guessing it's trying to lock the table. I've tried IGNORE, but is there anyway to force this?
You have a double null in you statement
ALTER TABLE `projects` ADD COLUMN `cTime` INT(30) DEFAULT NULL AFTER `session`