how to reindex mysql table - mysql

I have a table with many rows but they are out of order. Im using the field "id" as the primary key. I also have a "date" field which is a datetime field.
How could i reindex the table so that the entries are id'd in chronological order according to the date field

How about something like a simple query using a variable:
set #ROW = 0;
UPDATE `tbl_example` SET `id` = #ROW := #ROW+1 ORDER BY `fld_date` ASC;
This will order your rows like: 0,1,2,4,5...etc by your date.

the way i would do it is to create a new table with auto increment index and just select all your old table into it ordering by date. you can then remove your old table.

Why do you want the sequence of IDs to correlate with the dates? It sounds like you want to do ORDER BY id and have the rows come back in date order. If you want rows in date order, just use ORDER BY date instead.
Values in an autoincrement ID column should be treated as arbitrary. Relying on your IDs being in date order is a bad idea.

The following SQL snippet should do what you want.
ALTER TABLE test_table ADD COLUMN id2 int unsigned not null;
SET #a:=0;
UPDATE test_table SET id2=#a:=#a+1 ORDER BY `date`;
ALTER TABLE test_table DROP id;
ALTER TABLE test_table CHANGE id2 id int UNSIGNED NOT NULL AUTO_INCREMENT,
ADD PRIMARY KEY (id);
Keep in mind that you can never guarantee the order of an auto-incremented column once you start inserting and removing data, so you shouldn't be relying on any order except that which you specify using ORDER BY in your queries. This is an expensive operation that you are doing, as it requires indexes to be completely re-created, so I wouldn't suggest doing it often.

You can use ALTER TABLE t ORDER BY col;
The allowed syntax of ORDER BY is as in SELECT statements.

I had to do something similar. The best way to do it was the following (you can run it in one SQL Query if you want, but bare in mind that this is a slow and very resource consuming operation):
BE SURE TO MAKE A BACKUP OF YOUR TABLE, INCLUDING STRUCTURE AND DATA BEFORE STARTING THIS QUERY!
ALTER TABLE your_table ADD COLUMN temp_id INT UNSIGNED NOT NULL;
SET #a:=0;
UPDATE your_table SET temp_id=#a:=#a+1 ORDER BY `date` ASC;
ALTER TABLE your_table DROP id;
ALTER TABLE your_table CHANGE temp_id id INT UNSIGNED NOT NULL AUTO_INCREMENT, ADD PRIMARY KEY (id);
ALTER TABLE your_table CHANGE COLUMN id id INT(10) FIRST;
Just don't forget to change "your_table" with the name of your table, and the ORDER BY columns.
Here I explain you what you're doing this way step by step:
First you add a new column named "temp_id" (make sure it's not a name you're using already);
Next you add a temp variable equal to 0 (or to whatever you want for your ID to start from);
Then you update your table, row by row by the set ORDER logic, setting a value for your new column "temp_id" equal to the variable you've set, then increment this variable by 1 (you can do something funky here, for example if you want your ID's to be always even, the you can set #a+2);
Next step you drop (remove) your old column ID;
Then you change the name of your temp_id column back to ID and it as a positive integer with auto increment which is the primary key of your table.
Because ID now is the newly added temp_id column, it's located at the end of your table structure. To move it again as first column, you run the last query, to make sure it's the first column.

If you are using something like phpmysql this could be achieved by:
going to the table (left side list of db's and tables), then
from the options in the upper bar select 'SQL'. Follow the advice by #Ryun, then go to 'Operations' (from the upper bar),
look for 'TABLE OPTIONS', leave everything except 'AUTO_INCREMENT' unchanged,
set the 'AUTO_INCREMENT' value to 1 and press go at the bottom of the form.
What will this do, in all?
It will set the id columns in each from 1 to {count}.
Then it will reset the index of the table so that your next inserted row will equal +1 the number of columns (and not +1 the old index).
#Wyzard made reference to just ordering the columns by date when you retrieve them from the table (and not re-indexing). Since, indeed, the Primary Key should be arbitrary (except to any foreign keys and perhaps the consuming platform (but that is another matter)).

Related

Delete date column with varchar(255)

I want to delete data which are past 2years. filed name is Date and type is varchar(255)
delete from <table_name> where <Filed> like '%2022';
running very longtime but no deletion of data
I have check and tried the query, you can try with
DELETE From <datatable> WHERE <date> LIKE '%2022';
DELETE From post WHERE date LIKE '%2022'; #Example
May you provide the database or screenshot? I have tried the query and no issue https://www.db-fiddle.com/f/syhtgVyEcSPcHRXBXHLtor/0
If primary key(probably id) and the date column are correlated, meaning bigger id will result the later dates(in this case, it is a of type varchar, and thanks to P.Salmon for pointing this out),then
I think you can delete using primary key(normally it is column id), for example:
select id from table where date > '2020' order by id asc limit 1;
// assume this id = 123456789, and delete rows that created before this id was created
delete from table where id < 123456789;
if there is not correlation, I have some ideas like below:
create a new column called created_at of type year/date/datetime/timestamp(probably date or year will do), it will store the actual year or date or datetime, use it to replace the date column of type varchar, probably create an index on created_at, and delete with the new column
If there is a index on date(varchar), since the % sign in like clause will cause the server not using index, so it is a full table scan for sure, and can you like enumerate all date like '01-01-2020', '01-02-2020', and delete rows one date by one date, with a script, I think in this way at least you get to use the index
if there are too many rows, like 10 years or even more, is it possible just migrate data within 2 years to a new table, and just remove the old table?
write a script, fetch 10000 row each time from beginning of primary key, and delete those that are over 2 years, and fetch next 10000
last_id = 0
select * from table where id > last_id order by id asc limit 10000;
last_id = [last id of the query]
delete from table where id in (xxx);

Setting AUTO_INCREMENT attr [duplicate]

I am trying to set up a script to generate a particular set of test data into my database, at the beginning of which I want to clear the tables concerned without dropping constraints (because the test data is not the appropriate place to be rebuilding constraints) and reset the AUTO_INCREMENT for each table since setting up the test data is much, much simpler if I can hard-code many of the IDs.
For example, I have two statements like this (there's a pair for nearly every table):
DELETE FROM AppointmentAttr
ALTER TABLE AppointmentAttr AUTO_INCREMENT = 1
and while the records are deleted, the auto-increment value is not reverting to 1, even though all the documentation and SO answers I can find indicate that this should work.
If I do the same statement in MySQL Workbench it also does not revert it.
This is on an INNODB database.
What am I missing?
(Note: I cannot use TRUNCATE due to the presence of constraints).
MySQL does not permit you to decrease the AUTO_INCREMENT value, as specified here:
http://dev.mysql.com/doc/refman/5.6/en/alter-table.html
You cannot reset the counter to a value less than or equal to the value that is currently in use. For both InnoDB and MyISAM, if the value is less than or equal to the maximum value currently in the AUTO_INCREMENT column, the value is reset to the current maximum AUTO_INCREMENT column value plus one.
Even with your constraints, I would try one of the following:
Explicitly insert your identities for your test data. MySQL doesn't have problems with this, unlike some other database engines
Delete and recreate your identity column (or just change it from being an identity), if the constraints aren't on it itself.
Not use an Identity column and use another method (such as a procedure or outside code) to control your Identity. This is really a last resort and I wouldn't generally recommend it...
Note from OP: It was (1) that was what I needed.
From what I can see about the alter table statement.
You can reset auto increment value by using the ALTER TABLE statement. The syntax of the ALTER TABLE statement to reset auto increment value is as follows:
ALTER TABLE table_name AUTO_INCREMENT = value;
You specify the table name after the ALTER TABLE clause and the value which we want to reset to in the expression AUTO_INCREMENT = value.
Notice that the value must be greater than or equal to the current maximum value of the auto-increment column.
Which is where your problem lies I suspect. So basically you are left with a couple of options as follows:
TRUNCATE TABLE: which according to our discussion is not a option
DROP and RECREATE the table: A long and painful experience
ALTER auto number column: I have not tried this but you could theoretically alter the primary key column from auto number to a int and then make it a auto number again. Something like:
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT NOT NULL;
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT AUTONUMBER NOT NULL;
Hope these help a little.
Can you not drop the relevant, auto increment column and recreate it? Example follows:
;;assuming your column is called id and your table is tbl
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD COLUMN id BIGINT UNSIGNED DEFAULT 1 PRIMARY KEY FIRST;
This should work, but I don't use MySQL, just going off the docs. If you need further help, leave a comment and I'll do my best to help out.
I'm sure this has been long answered but when i need to truncate and can't I just do a set foreign_key_checks = 0 then run my truncate and then set foreign_key_checks = 1.
I've run into this problem when I've deleted middle rows from my table.
My answer would be to INSERT NEW DATA TO NOT EXISTING ID.
I expect that my answer still be usefull even if it's PHP not MYSQL.
First fetch your data.
if found not existing row Insert values and exit;
else if not found in whole loop then do insertion for default value;
$rez = mysql_query("SELECT * FROM users");
$exists = 1;
while($row = mysql_fetch_array($rez)){
if ( $exists != $row{'id'} ){
echo "found not existing id: ".$exists;
INSERT INTO users VALUES($exists,.. ,..);
exit;
} $exists += 1;
}
INSERT INTO users VALUES(NULL,.. ,..); ##auto_inc column converts NULL to latest
I HOPE it will help somehow.
In non-problematic circumstances you can do
ALTER TABLE tbl AUTO_INCREMENT = 0;
which brings auto_increment value to the lowest allowed at the time.
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD id INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (id); in your phpMyAdmin
ALTER TABLE table_name AUTO_INCREMENT = value;
This worked for me, I had to set it to the last record in my database while going through the operations panel never worked for me.
This worked for me hope it helps.
SET #autoid = 0; UPDATE users set id = #autoid := (#autoid+1); ALTER TABLE users AUTO_INCREMENT = 1;

Auto-increment is not resetting in MySQL

I am trying to set up a script to generate a particular set of test data into my database, at the beginning of which I want to clear the tables concerned without dropping constraints (because the test data is not the appropriate place to be rebuilding constraints) and reset the AUTO_INCREMENT for each table since setting up the test data is much, much simpler if I can hard-code many of the IDs.
For example, I have two statements like this (there's a pair for nearly every table):
DELETE FROM AppointmentAttr
ALTER TABLE AppointmentAttr AUTO_INCREMENT = 1
and while the records are deleted, the auto-increment value is not reverting to 1, even though all the documentation and SO answers I can find indicate that this should work.
If I do the same statement in MySQL Workbench it also does not revert it.
This is on an INNODB database.
What am I missing?
(Note: I cannot use TRUNCATE due to the presence of constraints).
MySQL does not permit you to decrease the AUTO_INCREMENT value, as specified here:
http://dev.mysql.com/doc/refman/5.6/en/alter-table.html
You cannot reset the counter to a value less than or equal to the value that is currently in use. For both InnoDB and MyISAM, if the value is less than or equal to the maximum value currently in the AUTO_INCREMENT column, the value is reset to the current maximum AUTO_INCREMENT column value plus one.
Even with your constraints, I would try one of the following:
Explicitly insert your identities for your test data. MySQL doesn't have problems with this, unlike some other database engines
Delete and recreate your identity column (or just change it from being an identity), if the constraints aren't on it itself.
Not use an Identity column and use another method (such as a procedure or outside code) to control your Identity. This is really a last resort and I wouldn't generally recommend it...
Note from OP: It was (1) that was what I needed.
From what I can see about the alter table statement.
You can reset auto increment value by using the ALTER TABLE statement. The syntax of the ALTER TABLE statement to reset auto increment value is as follows:
ALTER TABLE table_name AUTO_INCREMENT = value;
You specify the table name after the ALTER TABLE clause and the value which we want to reset to in the expression AUTO_INCREMENT = value.
Notice that the value must be greater than or equal to the current maximum value of the auto-increment column.
Which is where your problem lies I suspect. So basically you are left with a couple of options as follows:
TRUNCATE TABLE: which according to our discussion is not a option
DROP and RECREATE the table: A long and painful experience
ALTER auto number column: I have not tried this but you could theoretically alter the primary key column from auto number to a int and then make it a auto number again. Something like:
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT NOT NULL;
ALTER TABLE tblName MODIFY COLUMN pkKeyColumn BIGINT AUTONUMBER NOT NULL;
Hope these help a little.
Can you not drop the relevant, auto increment column and recreate it? Example follows:
;;assuming your column is called id and your table is tbl
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD COLUMN id BIGINT UNSIGNED DEFAULT 1 PRIMARY KEY FIRST;
This should work, but I don't use MySQL, just going off the docs. If you need further help, leave a comment and I'll do my best to help out.
I'm sure this has been long answered but when i need to truncate and can't I just do a set foreign_key_checks = 0 then run my truncate and then set foreign_key_checks = 1.
I've run into this problem when I've deleted middle rows from my table.
My answer would be to INSERT NEW DATA TO NOT EXISTING ID.
I expect that my answer still be usefull even if it's PHP not MYSQL.
First fetch your data.
if found not existing row Insert values and exit;
else if not found in whole loop then do insertion for default value;
$rez = mysql_query("SELECT * FROM users");
$exists = 1;
while($row = mysql_fetch_array($rez)){
if ( $exists != $row{'id'} ){
echo "found not existing id: ".$exists;
INSERT INTO users VALUES($exists,.. ,..);
exit;
} $exists += 1;
}
INSERT INTO users VALUES(NULL,.. ,..); ##auto_inc column converts NULL to latest
I HOPE it will help somehow.
In non-problematic circumstances you can do
ALTER TABLE tbl AUTO_INCREMENT = 0;
which brings auto_increment value to the lowest allowed at the time.
ALTER TABLE tbl DROP COLUMN id;
ALTER TABLE tbl ADD id INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (id); in your phpMyAdmin
ALTER TABLE table_name AUTO_INCREMENT = value;
This worked for me, I had to set it to the last record in my database while going through the operations panel never worked for me.
This worked for me hope it helps.
SET #autoid = 0; UPDATE users set id = #autoid := (#autoid+1); ALTER TABLE users AUTO_INCREMENT = 1;

Reset/compact mysql AI ID field?

I have a fairly large table with about 250k rows. It has an auto incremented ID column that is really sort of useless. I can't just get rid of the column without rewriting too much of the app, but the ID is never used as a foreign key or anything else (except simply as an identifier when you want to delete a row, I guess).
The majority of the data gets deleted and rewritten at least a few times a day (don't ask! it's not important, though I realize it's poor design!), though the total count of the rows stays fairly uniform. What this means is that each day to AI # increases by a quarter million or so.
My question is this: in several years' time, the ID column will get too large for the INT value. Is there a way to "reset" the ID, like an OPTIMIZE or something, or should I just plan on doing a SELECT INTO a temp table and truncating the original table, resetting the ID to 0?
Thanks
If you have the id as integer you can have 2^32 / 2 (2.147.483.647) rows, if is unsigned integer duplicate to 4.294.967.295, no worry 250.000 in nothing, if you want more, use unsigned bigint (18.446.744.073.709.551.615) :P
For reset the auto_numeric position:
ALTER TABLE table AUTO_INCREMENT = 1
Either change the datatype of ID to BIGINT and adjust your program accordingly, or if you're clearing everything out when you delete data you can use TRUNCATE TABLE TABLENAME which will reset the sequence.
Easiest and fastest :) Just drop the index, set autoincrement=1, and add it back :)
ALTER TABLE yourtable DROP id_field;
ALTER TABLE yourtable AUTO_INCREMENT=1;
ALTER TABLE yourtable ADD id_field INT NOT NULL AUTO_INCREMENT FIRST, ADD PRIMARY KEY (id_field);

MySQl field type

I have a table in my db where I store records of user actions. Currently the column that contains user IDs is set to int(11), however i am making some changes to my code where I will be adding temporary user IDs.
To differentiate the temporary IDs from the regular ones, I prepend 0 to the id.
Example: 4 -- regular user; 023 -- temporary
However when I populate this ID into ym table the zero gets discarded. What field type do I need to change it to to keep all IDs in tact?
You could change it to an varchar if you want to prefix the id's with a 0
But you might want to try this.
Add a new column:
ALTER TABLE `your_table` ADD COLUMN `temp_id` INT(11) NULL AFTER `original_id`;
Then migrate your id's
UPDATE `your_table` SET temp_id = `original_id`;
I think you'll have to go with a varchar field but note that this will eliminate your auto_increment if you have one.
The user ID is an int and ints are binary numbers. A leading zero is the SAME as the number without a leading zero.
I would suggest negating the number to indicate a temporary id.
You can't add a 0 before an int ( (01 == 1) -- mostly but I'm not going to get into the vagaries of that).
Just add a type column. You can always drop the column later.