I have this situation:
Until recently I had a table in my db which got updated through a python script (running as a cronjob).
I've handled this the following way (commands in mysqldb-python):
DROP TABLE tmp IF EXISTS
CREATE TABLE tmp LIKE original
INSERT INTO tmp...
DROP TABLE original
ALTER TABLE tmp RENAME TO original
Now I have 2 sources of data for the same table. The table is going to be updated from the 2 sources in different timeintervalls (source 1 each 'x minutes', source 2 each 'y minutes'). I can't create a tmp table and insert the data there because the data from the other source isn't going to be included. I thought about adding a field Sourceto the table and then in the scripts:
DELETE FROM original WHERE Source = 'currentsource'
INSERT INTO original ...
Problem with this method is that the ID AUTO_INCREMENT doesn't reset. I don't particularly want to check for the current max ID and set the increment if there's a better solution.
Which solution would you suggest?
Related
I need to alter the huge table around 200GB (INNODB).
As i seen many blogs, there is suggestion for creating a new structure, copying the table content to new table and rename the tables.
As given below
CREATE TABLE WorkingTableNew LIKE WorkingTable;
ALTER TABLE WorkingTableNew MODIFY BigColumn VARCHAR(50);
INSERT INTO WorkingTableNew SELECT SQL_NO_CACHE * FROM WorkingTable;
ALTER TABLE WorkingTable RENAME WorkingTableOld;
ALTER TABLE WorkingTableNew RENAME WorkingTable;
DROP TABLE WorkingTableOld;
But if do above steps, How could we handle the newly inserted data into original table. Since many process updating and inserting the records into original table.
I have one master and 8 slaves
If a table in MySQL containing suppose 1 million record, how can I add a column at any position with no downtime expected.
MySQL's ALTER TABLE performance can become very frustrating with very large tables. ALTER statements makes a new temporary table, copies records from your existing table into the new table even if the data wouldn't strictly need to be copied, and then replaces the old table with the new table.
Suppose you have a table with one million records and if you try to add 3 columns in it, then it will certainly copy the table 3 times, which means coping 3 million records.
A faster way of adding columns is to create your own new table, then select all of the rows from the existing table into it. You can create the structure from the existing table, then modify the structure however you’d like, then select in the data. Make sure that you select the information into the new table in the same order as the fields are defined.
1. CREATE TABLE new_table LIKE table
2. INSERT INTO new_table SELECT * FROM table
3. RENAME TABLE table = old_table, table = new_table;
If you have foreign key constraints you can handle these foreign keys using
SET FOREIGN_KEY_CHECKS = 0;
I can "copy" a table using:
CREATE TABLE copy LIKE original_table
and
CREATE TABLE copy as select * from original_table
In the latter case only the data are copied but not e.g primary keys etc.
So I was wondering when would I prefer using a select as?
These do different things. CREATE TABLE LIKE creates an empty table with the same structure as the original table.
CREATE TABLE AS SELECT inserts the data into the new table. The resulting table is not empty. In addition, CREATE TABLE AS SELECT is often used with more complicated queries, to generate temporary tables. There is no "original" table in this case. The results of the query are just captured as a table.
EDIT:
The "standard" way to do backup is to use . . . . backup at the database level. This backs up all objects in the database. Backing up multiple tables is important, for instance, to maintain relational integrity among the objects.
If you just want a real copy of a table, first do a create table like and then insert into. However, this can pose a challenge with auto_increment fields. You will probably want to drop the auto_increment property on the column so you can populate such columns.
The second form is often used when the new table is not an exact copy of the old table, but contains only selected columns or columns that result from a join.
"Create Table as Select..." are most likely used when you have complex select
e.g:
create table t2 as select * from t1 where x1=7 and y1 <>2 from t1;
Now, apparently you should use Create Like if you don't need such complex selects. You can change the PI in this syntax also.
The case:
Have huge table that every 24 hours need to delete all data and get new data instead.
How can I truncate this table and insert new data without block users to read data from this table?
What I thinking to do: (please write which option better or suggest more proper option or standard solution)
Option1:
1. insert new data to temp table
2. drop old table
3. rename temp table to the table name
Option2(problem: user cant access the table data in middle):
1. truncate the table
2. insert new data to table
1.insert new data to temp table
2.CREATE TABLE newtable select * from oldtable where 1=2
3.drop old table
4.INSERT INTO newtable select * from temp
If you have any old data to keep:
Add a id column to your table. Fill it with a hash value computed by the value of all columns in a row. Load the new data into a temporary table. Update the old data with the changed and new rows.
Otherwise your first solution is the way to go, since the renaming of a table does not take mch time.
i have a shop and the products are stored in a csv file. The csv file are imported by a cronjob in a temp table. my problem is to synchronize the temp table with the normal table (productive). some rows must be updated, added or deleted. I can't import the CSV file in the productive table because the intervall must be every 30 minutes.
Does anybody knows a program for this problem? For windows and mysql 5.0.x?
greetings!
You could just replace the original table with the temporary table:
DROP TABLE `ORIGINAL_TABLE`;
RENAME TABLE `TEMPORARY_TABLE` TO `ORIGINAL_TABLE`;
Or even safer...back up the original table first...
DROP TABLE IF EXISTS `BACKUP_TABLE`;
CREATE TABLE `BACKUP_TABLE` LIKE `ORIGINAL_TABLE`;
INSERT INTO `BACKUP_TABLE` SELECT * FROM `ORIGINAL_TABLE`;
DROP TABLE `ORIGINAL_TABLE`;
RENAME TABLE `TEMPORARY_TABLE` TO `ORIGINAL_TABLE`;