Duplicate records of a MySQL table - mysql

Question

I have a table that has approximately 4 million records. I would like to make it have 240 million like so:
Add an additional column of type BIGINT,
Import 59 times the data I already have,
And for each 4 million group of records, have the additional column to have a different value
The value of the additional column would come from another table.
So I have these records (except that I have 4 millions of them and not just 3):
| id | value |
+----+-------+
| 1 | 123 |
| 2 | 456 |
| 3 | 789 |
And I want to achieve this (except that I want 60 copies and not just 3):
| id | value | data |
+----+-------+------+
| 1 | 123 | 1 |
| 2 | 456 | 1 |
| 3 | 789 | 1 |
| 4 | 123 | 2 |
| 5 | 456 | 2 |
| 6 | 789 | 2 |
| 7 | 123 | 3 |
| 8 | 456 | 3 |
| 9 | 789 | 3 |
I tried to export my data (using SELECT .. INTO OUTFILE ...), then re-import it (using LOAD DATA INFILE ...) but it is really painfully slow.
Is there a fast way to do this?
Thank you!

Answer 1

Sounds like you'd like to take the cartesian product of 2 tables and create a new table since you say The value of the additional column would come from another table? If so, something like this should work:
create table yourtable (id int, value int);
create table yournewtable (id int, value int, data int);
create table anothertable (data int);
insert into yourtable values (1, 123), (2, 456), (3, 789);
insert into anothertable values (1), (2), (3);
insert into yournewtable
select t.id, t.value, a.data
from yourtable t, anothertable a
SQL Fiddle Demo
Results:
ID VALUE DATA
1 123 1
2 456 1
3 789 1
1 123 2
2 456 2
3 789 2
1 123 3
2 456 3
3 789 3
Edit, Side Note -- it looks like your ID field in your new table is not suppose to keep repeating the same ids? If so, you can use an AUTO_INCREMENT field instead. However, this could mess up the original rows if they aren't sequential.

Answer 2

First, I would recommend that you create a new table. You can do this using a cross join:
create table WayBigTable as
select t.*, n
from table t cross join
(select 1 as n union all select 2 union all select 3 union all select 4 union all select 5 union all
. . .
select 60
) n;
I'm not sure why you would want a bigint for this column. If you really need that, you can cast to unsigned.

Answer 3

Hmm. You need a cross join of your table with a range. Something in a line of this:
INSERT INTO table (id,value,data) SELECT id, value from table
CROSS JOIN (SELECT 2 UNION SELECT 3 UNION ... SELECT 60) AS data;
Use this answer Generating a range of numbers in MySQL as reference on number range.

Answer 4

Here's one idea...
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
,value INT NOT NULL
);
INSERT INTO my_table VALUES
(1 ,123),
(2 ,456),
(3 ,789);
ALTER TABLE my_table ADD COLUMN data INT NOT NULL DEFAULT 1;
SELECT * FROM my_table;
+----+-------+------+
| id | value | data |
+----+-------+------+
| 1 | 123 | 1 |
| 2 | 456 | 1 |
| 3 | 789 | 1 |
+----+-------+------+
SELECT * FROM ints;
+---+
| i |
+---+
| 0 |
| 1 |
| 2 |
| 3 |
| 4 |
| 5 |
| 6 |
| 7 |
| 8 |
| 9 |
+---+
INSERT INTO my_table SELECT NULL,value,data+i2.i*10+i1.i+1 FROM my_table,ints i1,ints i2;
SELECT * FROM my_table;
+-----+-------+------+
| id | value | data |
+-----+-------+------+
| 1 | 123 | 1 |
| 2 | 456 | 1 |
| 3 | 789 | 1 |
| 4 | 123 | 2 |
| 5 | 456 | 2 |
| 6 | 789 | 2 |
| 7 | 123 | 3 |
| 8 | 456 | 3 |
...
...
| 296 | 456 | 97 |
| 297 | 789 | 97 |
| 298 | 123 | 98 |
| 299 | 456 | 98 |
| 300 | 789 | 98 |
| 301 | 123 | 99 |
| 302 | 456 | 99 |
| 303 | 789 | 99 |
+-----+-------+------+
303 rows in set (0.00 sec)
Note, for 240 million rows, this is still going to be a bit slow :-(