I have 2 tables (t1 with 581978 rows,t2 with 581196 rows) containing some common fields (f1,f2,f3). The tables normally have the same number of rows, but can get out of sync. When this happens I'd like to identify the rows in t1 that are not in t2. The following python code (just use the 'select stmt' for mysql without python) identified the additional 782 rows in t1 but on my ageing iMac took over 12hrs! Is there a more efficient method?
orphans = ("select t1.f1,t1.f2,t1.f3 from t1 where not exists " \
"(select 1 from t2 where t2.f1=t1.f1 and t2.f2=t1.f2 and " \
"t2.f3=t1.f3)")
cursor.execute(orphans)
for i,(f1,f2,f3) in enumerate(cursor):
print(i+1,"orphans {:<10} {:<10} {<:8}".format(f1,f2,f3))
db-fiddle code (Is there a free db-fiddle, seems I can't save without upgrading to pro?)
CREATE DATABASE if not exists SO_61403216;
use SO_61403216;
drop table if exists t1,t2;
create table t1 (f1 varchar(1),f2 int(2),f3 int(2),f4 int(2));
create table t2 (f1 varchar(1),f2 int(2),f3 int(2),f4 int(2),f5 int(2));
insert into t1 (f1,f2,f3,f4) values ("A",2,3,4);
insert into t1 (f1,f2,f3,f4) values ("B",2,3,4);
insert into t1 (f1,f2,f3,f4) values ("C",2,3,4);
insert into t1 (f1,f2,f3,f4) values ("D",2,3,4);
insert into t2 (f1,f2,f3,f4,f5) values ("A",2,3,44,55);
insert into t2 (f1,f2,f3,f4,f5) values ("B",2,3,99,99);
insert into t2 (f1,f2,f3,f4,f5) values ("C",2,3,99,99);
Query, which exposes the extra row in table t1 using both the "not exist" (preferred output) and "null" methods, which both take 0ms for these small tables, but same query takes hours with 500K+ rows - is there a better/quicker method?
use SO_61403216;
desc t1;
desc t2;
select * from t1;
select * from t2;
select t1.f1,t1.f2,t1.f3 from t1 where not exists (select 1 from t2 where t2.f1=t1.f1 and t2.f2=t1.f2 and t2.f3=t1.f3)
select * from t1 left join t2 on t1.f1=t2.f1 and t1.f2=t2.f2 and t1.f3=t2.f3 where t2.f1 is null;
I would like to know how I can insert new rows in my table 1 of table 2. The idea is that by comparing the two tables if in the second one you do not find the same ID in table 1 this inserts the new data in table 1.
This is the two table and the idea I want to do:
Tabla 1
ID-Name-Surname
1-Oriol-Molina
Tabla 2
ID-Name-Surname
1-Oriol-Molina
2-Ricard-Martin
And the result would be this:
Tabla 1
ID-Name-Surname
1-Oriol-Molina
2-Ricard-Martin
Tabla 2
ID-Name-Surname
1-Oriol-Molina
2-Ricard-Martin
Use the database to enforce data integrity. That is, if you don't want duplicate ids in the table, then declare a unique index/constraint:
create unique index unq_table1_id on table1(id);
Then, in MySQL, you can use on duplicate key update:
insert into table1 (id, name, surname)
select id, name, surname
from table2
on duplicate key update id = values(id);
The final statement is a no-op -- it does nothing except prevent an error.
The advantage of this approach is that the database will ensure that id is unique for any statement that inserts data into the table, not just this one.
You can use INSERT INTO .. SELECT with LEFT JOIN and IS NULL check, to fetch only those rows from Table2 which do no exist in Table1
INSERT INTO Table1 (ID, Name, Surname)
SELECT t2.ID, t2.Name, t2.Surname
FROM Table2 t2
LEFT JOIN Table1 t1 ON t1.ID = t2.ID
WHERE t1.ID IS NULL
You can try using a left join
insert into table1
select id, name, surname from table2 left join table1 on table2.id=table1.id
where table1.id is null
Consider the following:
QUERY
SELECT * FROM
`table1`,`table2`
WHERE `table1`.`RemoteID` = `table2`.`ID`
AND `table2`.`UserID`=1
How can I change it from a SELECT to DELETE from table1 where these records match? It must only delete from table1, not table2
In less specific terms, I want to delete all records from table1 where they match some criteria of both tables (discretely and relatively)
You can use IN with sub query
DELETE FROM table1
WHERE `table1`.`RemoteID` IN (
SELECT ID
FROM table2
WHERE `table2`.`UserID`=1)
Try this,
Delete
from table1
where Id in
(select table1.Id
from table1 t1, table2 t2
where t1.RemoteID = t2.ID
AND table2.UserID = 1)
I have two tables T1 and T2. I want to merge only those rows of T2 which is currently not there in T1 .
`Insert into T1 select * from (select * from T2 where id not in (select id from T1))`
Is there a better and faste way of achieving the same. ID column is unique across the table
Insert into T1
select * from T2
where id not in (select id from T1)
You could also join but then you'd need another subselect since MySQL does not want to select from a table it inserts at the same time without using a subselect.
I have two tables that have a one-to-many relationship table1 (one), table2 (many).
table1 has (t1_id) as KEY
table2 has (t2_id) as KEY and t2_t1_id as as refference to table1
Now in table1 I need a column (rand_t2_id) that holds an id from table2, I do not really care which one
This is the query I tried
INSERT INTO table1 (rand_t2_id)
SELECT t2_id
FROM table2
WHERE table1.t1_id = table2.t2_t1_id;
There also needs to be a limit build in somewhere if that is needed. I only need one id from table2
No luck here tho, anyone know a fix?
IMO you could try
UPDATE table1 t1 INNER JOIN table2 t2
ON t1.t1_id = t2.t2_t1_id;
SET t1.rand_t2_id = t2.t2_id
I assume you already have a column named rand_t2_id on your table1.
INSERT INTO table1 (t2_t1_id, rand_t2_id)
SELECT t2_id
FROM ( SELECT t2_id, t2_t1_id
FROM table2
ORDER BY RAND()) AS h
GROUP BY t2_t1_id
ON DUPLICATE KEY UPDATE rand_t2_id = VALUES(rand_t2_id)
I suppose that's what you're after?
This query would insert a random t2_id into relevant table1.