I have a table (cars) that has 26500 rows. Is it possible to delete from the row number 10001 through the end?
in InnoDB Tables
If you are deleting many rows from a large table, you may exceed the lock table size for an InnoDB table. To avoid this problem, or simply to minimize the time that the table remains locked, the following strategy (which does not use DELETE at all) might be helpful:
Step 1: Select the rows not to be deleted into an empty table that has the same structure as the original table:
INSERT INTO `cars_copy` SELECT * FROM `cars` LIMIT 10000 ;
Step 2: Use RENAME TABLE to atomically move the original table out of the way and rename the copy to the original name:
RENAME TABLE `cars` TO `cars_old`, `cars_copy` TO `cars` ;
Step 3: Drop the original table:
DROP TABLE `cars_old`;
No other sessions can access the tables involved while RENAME TABLE executes, so the rename operation is not subject to concurrency problems.
When your Rows are labelled with an ID you can just do this:
DELETE FROM cars WHERE ID > 10000
Related
I need to rename MySQL table and create a new MySQL table at the same time.
There is critical live table with large number of records. master_table is always inserted records from scripts.
Need to backup the master table and create a another master table with same name at the same time.
General SQL is is like this.
RENAME TABLE master_table TO backup_table;
Create table master_table (id,value) values ('1','5000');
Is there a possibility to record missing data during the execution of above queries?
Any way to avoid missing record? Lock the master table, etc...
What I do is the following. It results in no downtime, no data loss, and nearly instantaneous execution.
CREATE TABLE mytable_new LIKE mytable;
...possibly update the AUTO_INCREMENT of the new table...
RENAME TABLE mytable TO mytable_old, mytable_new TO mytable;
By renaming both tables in one statement, they are swapped atomically. There is no chance for any data to be written "in between" while there is no table to receive the write. If you don't do this atomically, some writes may fail.
RENAME TABLE is virtually instantaneous, no matter how large the table. You don't have to wait for data to be copied.
If the table has an auto-increment primary key, I like to make sure the new table starts with an id value greater than the current id in the old table. Do this before swapping the table names.
SELECT AUTO_INCREMENT FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA='mydatabase' AND TABLE_NAME='mytable';
I like to add some comfortable margin to that value. You want to make sure that the id values inserted to the old table won't exceed the value you queried from INFORMATION_SCHEMA.
Change the new table to use this new value for its next auto-increment:
ALTER TABLE mytable_new AUTO_INCREMENT=<increased value>;
Then promptly execute the RENAME TABLE to swap them. As soon as new rows are inserted to the new, empty table, it will use id values starting with the increased auto-increment value, which should still be greater than the last id inserted into the old table, if you did these steps promptly.
Instead of renaming the master_backup table and recreating it, you could
just create a backup_table with the data from the master_table for the first backup run.
CREATE TABLE backup_table AS
SELECT * FROM master_table;
If you must add a primary key to the backup table then run this just once, that is for the first backup:
ALTER TABLE backup_table ADD CONSTRAINT pk_backup_table PRIMARY KEY(id);
For future backups do:
INSERT INTO backup_table
SELECT * FROM master_table;
Then you can delete all the data in the backup_table found in the master_table like:
DELETE FROM master_table A JOIN
backup_table B ON A.id=B.id;
Then you can add data to the master_table with this query:
INSERT INTO master_table (`value`) VALUES ('5000'); -- I assume the id field is auto_incrementable
I think this should work perfectly even without locking the master table, and with no missing executions.
If a table in MySQL containing suppose 1 million record, how can I add a column at any position with no downtime expected.
MySQL's ALTER TABLE performance can become very frustrating with very large tables. ALTER statements makes a new temporary table, copies records from your existing table into the new table even if the data wouldn't strictly need to be copied, and then replaces the old table with the new table.
Suppose you have a table with one million records and if you try to add 3 columns in it, then it will certainly copy the table 3 times, which means coping 3 million records.
A faster way of adding columns is to create your own new table, then select all of the rows from the existing table into it. You can create the structure from the existing table, then modify the structure however you’d like, then select in the data. Make sure that you select the information into the new table in the same order as the fields are defined.
1. CREATE TABLE new_table LIKE table
2. INSERT INTO new_table SELECT * FROM table
3. RENAME TABLE table = old_table, table = new_table;
If you have foreign key constraints you can handle these foreign keys using
SET FOREIGN_KEY_CHECKS = 0;
I have a 6 million record table with an auto-increment ID PK. Due to various operations over the last several weeks, my starting ID is 2 million. Updates and other queries take a long time, and I'm wondering if having an iD range from 2mil to 8mil vs starting at 1 to 6 million could be responsible? I've noticed anecdotally that if I do selects/updates using a range of say ID>1000000 and ID<1001000 seems to be slower than ID>1 and ID<1000.
Is it worth it to remove the existing PK and add a new one starting at 1? I know I can do
ALTER TABLE tablename Auto-Increment=1
but I cannot do this here with 6 million existing records and Auto-Increment IDs already.
Clearly I can do try and test but for various reasons including the time it is going to take given the size of table, indexes, etc I'd prefer to ask before going to the time and effort if anyone knows the answer definitively.
Update:
For now I did the following:
CREATE TABLE table_new LIKE table;
To dupe the table with indexes and all
Then:
Alter Table table_new set Auto-Increment=1
So the empty duped table re-sets count to 1
Then I inserted from the original table to the duped table:
Insert into Table_New (FieldA,FieldB,FieldC)
Select FieldA,FIeldB,FieldC from Table
To insert all the records minus the ID field so that the Auto-Increment is added per inserted record, starting at 1 as the re-set specified and finally of course:
RENAME TABLE table TO table_old;
RENAME TABLE table_new TO table;
I have a DB schema composed of MYISAM tables, i am interested to delete old records from time to time from some of the tables.
I know that delete does not reclaim the memory space, but as i found in a description of DELETE command, inserts may reuse the space deleted
In MyISAM tables, deleted rows are maintained in a linked list and subsequent INSERT operations reuse old row positions.
I am interested if LOAD DATA command also reuses the deleted space?
UPDATE
I am also interested how the index space reclaimed?
UPDATE 2012-12-03 23:11
some more info supplied based on the answer received from #RolandoMySQLDBA
after executing the following suggested query i got different results for different tables for which space need to be reused or reclaimed:
SELECT row_format FROM information_schema.tables
WHERE table_schema='mydb' AND table_name='mytable1';
> Dynamic
SELECT row_format FROM information_schema.tables
WHERE table_schema='mydb' AND table_name='mytable2';
> Fixed
UPDATE 2012-12-09 08:06
LOAD DATA do reuses previously deleted space (i have checked it by running a short script) if and only if the row format is fixed or (the row format is dynamic and there is a deleted row with exactly the same size).
it seems that if the row_format is dynamic, full look-up over the deleted list is made for each record , and if the exact row size is not found , the deleted record is not used, and the table memory usage will raise, additionally LOAD DATA will take much more time to import records.
I will except the answer given here , since it describes all the process perfectly.
For a MySQL table called mydb.mytable just run the following:
OPTIMIZE TABLE mydb.mytable;
You could also do this in stages:
CREATE TABLE mydb.mytable_new LIKE mydb.mytable;
ALTER TABLE mydb.mytable_new DISABLE KEYS;
INSERT INTO mydb.mytable_new SELECT * FROM mydb.mytable;
ALTER TABLE mydb.mytable_new ENABLE KEYS;
ALTER TABLE mydb.mytable RENAME mydb.mytable_old;
ALTER TABLE mydb.mytable_new RENAME mydb.mytable;
ALTER TABLE mydb.mytable_old;
ANALYZE TABLE mydb.mytable;
In either case, the table ends up with no fragmentation.
Give it a Try !!!
UPDATE 2012-12-03 12:50 EDT
If you are concerned whether or not rows are reused upon bulk INSERTs via LOAD DATA INFILE, please note the following:
When you created the MyISAM table, I assumed the default row format would be dynamic. You can check what it is with either
SHOW CREATE TABLE mydb.mytable\G
or
SELECT row_format FROM information_schema.tables
WHERE table_schema='mydb' AND table_name='mytable';
Since the row format of your table is Dynamic, the fragmented rows are of various sizes. The MyISAM storage engine would have keep checking for the row length of each deleted to see if the next set of data being insert will fit. If the incoming data cannot fit in any of the deleted rows, then the new row data is appended.
The presence of such rows can make myisamchk struggle.
This is why I recommended running OPTIMIZE TABLE. That way, data would be appended quicker.
UPDATE 2012-12-03 12:58 EDT
Here is something interesting you can also do: Try setting concurrent_insert to 2. That way, you are always appending to a MyISAM table without checking for gaps in the table. This will speed up INSERTs dramatically but leave all known gaps alone.
You could still defragment your table at your earliest convenience using OPTIMIZE TABLE.
UPDATE 2012-12-03 13:40 EDT
Why don't run the my second sugesstion
CREATE TABLE mydb.mytable_new LIKE mydb.mytable;
ALTER TABLE mydb.mytable_new DISABLE KEYS;
INSERT INTO mydb.mytable_new SELECT * FROM mydb.mytable;
ALTER TABLE mydb.mytable_new ENABLE KEYS;
ALTER TABLE mydb.mytable RENAME mydb.mytable_old;
ALTER TABLE mydb.mytable_new RENAME mydb.mytable;
ANALYZE TABLE mydb.mytable;
This will give you an idea
How long OPTIMIZE TABLE would take to run
How much smaller the .MYD and .MYI would be after running OPTIMIZE TABLE
After you run my second suggestion, you can compare them with
SELECT
A.mydsize,B.mydsize,A.mydsize - B.mydsize myd_diff,
A.midsize,B.myisize,A.myisize - B.myisize myi_diff
FROM
(
SELECT data_length mydsize,index_length myisize
FROM information_schema.tables
WHERE table_schema='mydb' AND table_name='mytable'
) A,
(
SELECT data_length mydsize,index_length myisize
FROM information_schema.tables
WHERE table_schema='mydb' AND table_name='mytable_new'
) B;
UPDATE 2012-12-03 16:42 EDT
Any table whose ROW_FORMAT is set to fixed has the luxury of allocating the same length row every time. If MyISAM tables maintain a list of deleted rows, the very first row in the list should always be selected as the next row to insert data. There would be no need to traverse a whole list until a suitable row gaps with sufficient length is found. Each deleted row is quickly appended after a DELETE. Each INSERT would pick the first row of the deleted rows.
We can assume these things because MyISAM tables can do concurrent inserts. In order for this feature to be available via the concurrent_insert option, INSERTs into a MyISAM table must be able to detect one of three(3) things:
The presence of a list of deleted rows, thus choosing from the list
Row_Format=Dynamic : list of deleted rows with each row with a different length
Row_Format=Fixed : list of deleted rows with all rows the same length
The absence of a list of deleted rows, thus appending
Bypass checking for the presence of a list of deleted rows (set concurrent_insert to 2)
For detection #1 to be the fastest possible, a MyISAM table's row_format must be Fixed. If it is Dynamic, it is very possible that a list traversal is necessary.
Which one will delete all data from the table, and which one will remove the table from the database?
What is the right thing to do if I don't want this table in my database any more?
Should I drop or delete?
DROP command deleting the table and its structure from the data base.
DROP TABLE tbl_user;
DELETE command used for deleting the records from the table,and it removing the table space which is allocated by the data base, and returns number of rows deleted.
DELETE FROM tbl_user WHERE id = 1;
TRUNCATE command is also delete the records but it doesn't delete the table space which is created by the data base, and does not return number of deleted rows.
TRUNCATE TABLE tbl_user;
DROP is used to remove tables (and databases).
DELETE is used to delete rows from tables.
Maybe are you talking about TRUNCATE and DELETE ?
TRUNCATE TABLE users;
is equivalent (logically) to
DELETE FROM users;
This will erase all data in table users. If you want to delete the whole table structure you should write:
DROP TABLE users;
But, DELETE is DML command while TRUNCATE and DROP are DDL commands. There is also some other differences in different RDBMS. More info - here
And another useful link: Difference between TRUNCATE, DELETE and DROP commands
drop removes the contents and the table (but not user permissions on the table). This is what you want if you want to completely remove the table from your schema.
delete selectively (or not) removes rows from a table. It does not alter the table structure.
There's no such thing as DELETE TABLE. You should use DROP TABLE to delete your table.
The DROP TABLE statement is used to delete a table.
DROP TABLE table_name
The DELETE statement is used to delete data(records) in a table.
DELETE FROM table_name WHERE some_column=some_value;
So, as you want to delete table from database then you will go for DROP Command
DELETE is used to delete one or several rows from the table. DROP TABLE would remove the entire table from the database, so if you want to remove the table, you should use DROP TABLE.
DROP TABLE reference
Delete will update the log, while drop does not.
Delete is used to remove the rows, while drop is used to remove the tables and DB.
delete
- removes the rows from the table - where clause can be used to delete specific rows.
- If where is not specified, then all the rows in the table will be removed
- rollback can be done.
without using where condition
DELETE * FROM empl;
using where condition
DELETE FROM empl WHERE job = 'Manager';
truncate
- removes all the rows from the table.
- rollback cannot be done.
SQL> TRUNCATE TABLE empl;
drop
- removes a table from the database.
- rows, indexes and privileges will also be removed.
- rollback cannot be done.
SQL> DROP TABLE empl;
Delete remove all data from a specific table and drop remove whole database and also remove specific table from database.
If you don't want the table in database any more then you should drop the table.