How long does it take to create a large table in MYSQL? - mysql

When I imported a table with 30 million rows from a text file to a MYSQL table it only took 1 minute. However, I realized that I missed a column and that I needed to add it to the table. From the MYSQL command line, I wrote the following command:
create tableC as(tableA.T1, tableB.ZID from tableA, table B where A.ZID = B.ZID)
It's been over one hour and the command has not terminated. Does anyone know the reason why? TableB was already in the MYSQL server.

Not 100% sure but you might be better off altering the table first and adding the column, then doing an update to populate that column
ALTER TABLE tableB ADD COLUMN colA yourColumnDefinition;
UPDATE tableB SET colA = <however you do it>;
I'm not totally sure which table you are adding the column to or how you are computing it. Based on the query you posted it looks like you are creating a mapping table to map two table's IDs to each other. If that is the case you would probably be better off putting a foreign key in one of the tables.
Again, this might not be exactly what you are looking for but if you just want to add a column to a table you already created this might be a better approach.

Related

"Filtering" huge MariaDB/Mysql table based on different table

Struggling with a large dataset in my mariaDB database. I have two tables, where table A contains 57 million rows and table B contains around 500. Table B is a subset of ids related to a column in table A. I want to delete all rows from A which do not have a corresponding ID in table B.
Example table A:
classification_id
Name
20
Mercedes
30
Kawasaki
80
Leitz
70
HP
Example table B:
classification_id
Type
20
car
30
bike
40
bus
50
boat
So in this example the last two rows from table A would be deleted (or a mirror table would be made containing only the first two rows, thats also fine).
I tried to do the second one using an inner join but this query took a few minutes before giving an out of memory exception.
Any suggestions on how to tackle this?
try this:
delete from "table A" where classification_id not in (select classification_id from "table B");
Since you say that the filter table contains a relatively small number of rows, your best bet would be creating a separate table that contains the same columns as the original table A and the rows that match your criteria, then replace the original table and drop it. Also, with this number of IDs you probably want to use WHERE IN () instead of joins - as long as the field you're using there is indexed, it will usually be way faster. Bringing it all together:
CREATE TABLE new_A AS
SELECT A.* FROM A
WHERE classification_id IN (SELECT classification_id FROM B);
RENAME TABLE A TO old_A, new_A to A;
DROP TABLE old_A;
Things to be aware of:
Backup your data! And test the queries thoroughly before running that DROP TABLE. You don't want to lose 57M rows of data because of a random answer at StackOverflow.
If A has any indexes or foreign keys, these won't be copied over - so you'll have to recreate them all manually. I'd recommend running SHOW CREATE TABLE A first and making note on its structure. Alternatively, you may consider creating the table new_A explicitly using the output of SHOW CREATE TABLE A as a template and then performing INSERT INTO new_A SELECT ... instead of CREATE TABLE new_A AS SELECT ... with the same query after this.

Replace one column of a table with another column of another table in SQL

I have a table with several columns Table1(Col A, Col B)
Now I have one more table with one column. Table2 (Col C)
What I want to do is:
Replace Col B of table1 with Col C of tabl 2.
Is it possible in SQL? I am using phpmyadmin to execute queries
Why I need to do this?
- I was playing around with the database structure and changed the type of text to integer which messed up the entries in the column
- Good thing: I have a backup excel file so now i am planning to replace the effected column to by the orginal values in the backedup excel file.
No can do.
You seem to be making an incorrect assumption, namely that the order of rows in a table is significant. Else what's confusing some of the commenters would be clear to you: there's no information in table2 to relate it to table1.
Since you still have the data in Excel, drop table2 and re-create it with rows having the key to table1. Then write a view to join them. Easiest is probably to insert that join result into a third table, and then drop the first two and rename the third.

SQL Entry's Duped

I messed up when trying to create a test Database and accidently duplicated everything inside of a certain table. Basically there is now 2 of every entry there was once before. Is there a simple way to fix this? (Using InnoDB tables)
Yet another good reason to use auto incrementing primary keys. That way, the rows wouldn't be total duplicates.
Probably the fastest way is to copy the data into another table, truncate the first table, and re-insert it:
create temporary table tmp as
select distinct *
from test;
truncate table test;
insert into test
select *
from tmp;
As a little note: in almost all cases, I recommend using the complete column list on an insert statement. This is the one case where it is optional. After all, you are putting all the columns in another table and just putting them back a statement later.

Deleting multiple rows from multiple tables MYSQL

I have this query that works fine. Its deletes records that are old based on current time.
$cleanacc_1 = "DELETE FROM $acc_1
WHERE `Scheduled` < DATE_SUB(UTC_TIMESTAMP(), INTERVAL 30 SECOND)";
$result = mysql_query($cleanacc_1);
However, there are over 100 tables (accounts) that need deleting and I was wondering if I can combine them into one query. If possible how?
This implies you create a new table for every account. Why are you not creating a record for each account within a single table?
For example...
create table account (id int unsigned primary key auto_increment, other fields...);
If you alter your table structure you will be able to delete individual account records with a single query...
delete from account where condition=true;
Individual transaction records for each account are then stored in another table and contain the account id they relate to...
create table transaction (id, account_id, other transaction fields);
If you don't change the database design you'll need to write PHP code that loops through each table and runs your delete query. This is very inefficient and I urge you to redesign the table as suggested.
If you don't understand why my table redsign suggestion is a better approach, post more information about your database and I'll explain in more detail with a working example.
No way to do that, AFAIK; anyways, I don't think it would be a big problem to run 100 queries, assuming you are not running that for each request or so..
Are you expecting performance issues? If that's the case, I'd probably use a cron job to run that query every X minutes..
You could setup a view of the tables and do then run the delete sql against the view. That should delete the underlying table data as well. Your table schema and permissions could have an affect whether this will work or not. Check out this answer, it might help as well.
Does deleting row from view delete row from base table - MYsql?
Please consider the following example.
I have three tables in following structure.
Table names : t1,t2,t3
Fields : Id, name
Im going to perform delete query with one condition which recode id must less than 10.
DELETE FROM t1, t2,t3 USING t1 INNER JOIN t2 INNER JOIN t3 WHERE t1.id<10 and t2.id<10 and t3.id<10.
The query has been successfully executed ( MySql ). I got the expected output.
So please try the same way with your condition.

Remove repeat rows from MySQL table

Is there a way to remove all repeat rows from a MySQL database?
A couple of years ago, someone requested a way to delete duplicates. Subselects make it possible with a query like this in MySQL 4.1:
DELETE FROM some_table WHERE primaryKey NOT IN
(SELECT MIN(primaryKey) FROM some_table GROUP BY some_column)
Of course, you can use MAX(primaryKey) as well if you want to keep the newest record with the duplicate value instead of the oldest record with the duplicate value.
To understand how this works, look at the output of this query:
SELECT some_column, MIN(primaryKey) FROM some_table GROUP BY some_column
As you can see, this query returns the primary key for the first record containing each value of some_column. Logically, then, any key value NOT found in this result set must be a duplicate, and therefore it should be deleted.
These questions / answers might interest you :
How to delete duplicate records in mysql database?
How to delete Duplicates in MySQL table.
And idea that's often used when you are working with a big table is to :
Create a new table
Insert into that table the unique records (i.e. only one version of the duplicates in the original table, generally using a select distinct)
and use that new table in your application ; or drop the old table and rename the new one to the old name.
Good thing with this principle is you have the possibility to verify what's in the new table before dropping the old one -- always nice to check that sort of thing ^^