SQL: insert only new records - mysql

This must be very trivial but I can't seem to find the solution.
I work with two tables, both without any primary key.
I want to add all the records of the first table to the second table only if they don't exist.
Basically:
INSERT INTO Table2
SELECT Table1.*
FROM Table
WHERE "the record to be added doesn't already exists in Table2"

You could do something like this. You would need to check each relevant column - I have just put in 2 as an example. With a Not Exists clause you can check if a record already existed across multiple columns. With a NOT IN you would only be able to check if a record already existed against one column.
INSERT INTO Table2
SELECT t1.*
FROM Table1 t1
WHERE NOT EXISTS
(
SELECT 1
FROM table2 t2 WHERE
t2.col1 = t1.col1 AND
t2.col2 = t1.col2
)

you could make usage of the EXISTS function:
INSERT INTO Table2
SELECT Table1.*
FROM Table1
WHERE NOT EXISTS(SELECT * FROM table2 WHERE <your expression to compare the two tables goes here>)
But i would advise you to check the use of unique index for your tables

Just an idea - untested:
INSERT INTO Table2
SELECT *
FROM Table1
WHERE NOT EXISTS(SELECT * FROM Table2 WHERE Table2.Field1 = Table1.Field1 AND Table2.Field2 = Table1.Field2)
You must add every Field of both Tables in the WHERE clause of the NOT EXISTS Query

INSERT INTO X.TableX1
(ColumX1,ColumnX2)
SELECT DISTINCT c.[ColumnY1],c.[ColumnY2]
FROM Y.Table2 c INNER JOIN Database.z.Table3 i ON c.ColumnX1= i.ColumnY1
WHERE NOT EXISTS
(
SELECT *
FROM X.TableX1 WHERE
X.TableX1.ColumnX1=c.ColumnY1
)
This is Joining two tables and filtering the data required and updating only the new values to third table on every run

Related

How to compare 2 table with multiple column

I am trying to compare multiple column such as part, quantity, and type with another table. I tried with compare part id without problem but when I want to compare part id with quantity it will cause error such as return nothing.
Below coding is for compare part id only.
INSERT INTO `table3`
SELECT *
FROM table1
WHERE NOT EXISTS (SELECT *
FROM table2
WHERE table2.part = table1.part)
How can I compare part and quantity together to find unmatched record.
A handy method in MySQL is in with tuples:
INSERT INTO `table3` ( . . . )
SELECT . . .
FROM table1
WHERE (part, quantity) NOT IN (SELECT part, quantity FROM table2);
More importantly, you should explicitly list all the columns when using an insert. This helps prevent unexpected problems when inserting data.
There are 2 solutions -
1)
INSERT INTO `table3`
SELECT *
FROM table1
WHERE NOT EXISTS (SELECT *
FROM table2
WHERE table2.part = table1.part And table2.quantity=table1.quantity)
2)
INSERT INTO table2 (part, quantity, type)
SELECT part, quantity, type
FROM table1
ON DUPLICATE KEY UPDATE quantity= VALUES(quantity), type= VALUES(type)
Not sure if it's the best approach, but I'd be tempted to try a null-left-join on either a set of values or a concatenation - e.g.
select t1.*
from mytable1 as t1
left join mytable2 as t2 on t1.foo = t2.foo and t1.bar = t2.bar
where t1.id is null;
The only thing to note is that t1.id (and t2.id) are assumed to be the primary key - i.e. can never be null. The end result of the above should be all the rows in t1 that cannot be joined to a row in t2.

Value from one table on basis of another table

I have two table as table1 and table2 given below:
I want to have the value of only those table_name from table1 which has there id in print_table column in table 2.
I have implemented the following query but it returns only one value:
SELECT * FROM print_tabel_permission_admin WHERE id IN (select print_table from secondary)
Use FIND_IN_SET:
SELECT DISTINCT
t1.table_name
FROM table1 t1
INNER JOIN table2 t2
ON FIND_IN_SET(t1.id, t2.print_table) > 0;
Demo
You should probably move away from storing CSV data in your tables like this. Instead, break out the IDs in table2 onto separate rows. This will make your life easier.

I wanted to know the command to check if all the values in one field of a table is present in another table under a different field name

I have 2 tables. I want to find out whether the values present in the first table is there in another table with a different field name.
Here is how it looks,
Table1
BillNo
43529179
43256787
35425676
25467778
24354758
45754748
Table2
BNo
113104808
25426577
268579679
2542135464
252525232
235263663
I have 137 records in table1 that needs to be checked against table2.
Instead of doing it one by one using the following command,
Select * from Table2 where BNo = '43529179';
This gives the result for just the mentioned value. Is there a way to check for all the values in a single query?
Thanks!
You can use a sub-select to compare against:
Select * from Table2 where BNo IN (SELECT BillNo FROM Table1);
That will "evalaute" to something like Select * from Table2 where BNo IN (113104808, 25426577, 268579679, 2542135464, 252525232, ...);
Join the tables, and check how many matching records there are:
select
count(*) as Matches
from
Table1 as t1
inner join Table2 as t2 on t2.BNo = t1.BillNo
You can also use a left join to pick out the records in table 1 that has no matching record in table 2:
select
t1.BillNo
from
Table1 as t1
left join Table2 as t2 on t2.BNo = t1.BillNo
where
t2.BNo is null

SQL Query to find missing records between two tables and then update the second with the missing records from the first

Two tables. 8 fields in each. Both tables have the same data, one with
137,002 record (tablea) and one with 135,759 records (tableb). Both tables share a common primary field if three columns (qid, sid, aid).
Is there a single query that will.
1) compare tablea to tableb on the primary field
and
2) if the record is in tablea and not tableb copy the record from tablea to tableb
I would rather be able to update tableb with an sql query rather than writing a php loop to go through the 137,002 and do a compare on each one.
Thanks
That should be smth looking like:
insert into table2 (qid, sid ...)
select
t1.qid,
t1.sid,
...
from table1 t1
where
not exist (select t2.qid, t2.sid, ... from table2 t2 where t2.qid = t1.qid and t2.sid = t1.sid...)
INSERT INTO tableb AS b
(SELECT * FROM tablea AS a WHERE NOT EXISTS (SELECT * FROM tableb AS b2 WHERE b2.id = a.id))
Use merge...and use insert only....not update.
So, the following worked.
insert into f_step_ans (`qid`, `assid`, `sid`, `sas`, `cas`, `etim`, `stim`, `endtim`, `fil`)
select
t1.qid,
t1.assid,
t1.sid,
t1.sas,
t1.cas,
t1.etim,
t1.stim,
t1.endtim,
t1.fil
from f_step_ans_back t1
where
not exists (select t2.qid, t2.sid,t2.assid from f_step_ans as t2 where t2.qid = t1.qid and t2.assid = t1.assid and t2.sid = t1.sid)
1,588 records were moved from the f_step_ans_back table (old backup table) to the f_step_ans table (partially recovered backup + new data). Reporting shows that everything is working like it should be. Thank you all for the help.

Insert missing records from one table to another using mysql

I don't know why I am confused with this query.
I have two table: Table A with 900 records and Table B with 800 records. Both table need to contain the same data but there is some mismatch.
I need to write a mysql query to insert missing 100 records from Table A to Table B.
In the end, both Table A and Table B should be identical.
I do not want to truncate all the entries first and then do a insert from another table. So please any help is appreciated.
Thank you.
It is also possible to use LEFT OUTER JOIN for that. This will avoid subquery overhead (when system might execute subquery one time for each record of outer query) like in John Woo's answer, and will avoid doing unnecessary work overwriting already existing 800 records like in user2340435's one:
INSERT INTO b
SELECT a.* FROM a
LEFT OUTER JOIN b ON b.id = a.id
WHERE b.id IS NULL;
This will first select all rows from A and B tables including all columns from both tables, but for rows which exist in A and don't exist in B all columns for B table will be NULL.
Then it filter only such latter rows (WHERE b.id IS NULL),
and at last it inserts all these rows into B table.
I think you can use IN for this. (this is a simpliplification of your query)
INSERT INTO table2 (id, name)
SELECT id, name
FROM table1
WHERE (id,name) NOT IN
(SELECT id, name
FROM table2);
SQLFiddle Demo
AS you can see on the demonstration, table2 has only 1 records but after executing the query, 2 records were inserted on table2.
If it's mysql and the tables are identical, then this should work:
REPLACE INTO table1 SELECT * FROM table2;
This will insert the missing records into Table1
INSERT INTO Table2
(Col1, Col2....)
(
SELECT Col1, Col2,... FROM Table1
EXCEPT
SELECT Col1, Col2,... FROM Table2
)
You can then run an update query to match the records that differ.
UPDATE Table2
SET
Col1= T1.Col1,
Col2= T1.Col2,
FROM
Table T1
INNER JOIN
Table2 T2
ON
T1.Col1 = T2.Col1
Code also works when a group by and having clauses are used. Tested SQL 2012 (11.0.5058) Tab1 is source with new records, Tab 2 is the destination to be updated. Tab 2 also has an Identity column. (Yes folks, real world is not as neat and clean as the lab assignments)
INSERT INTO Tab2
SELECT a.T1,a.T2,a.T3,a.T4,a.Val1,a.Val2,a.Val3,a.Val4,-9,-9,-9,-9,MIN(hits) MinHit,MAX(hits) MaxHit,SUM(count) SumCnt, count(distinct(week)) WkCnt
FROM Tab1 a
LEFT OUTER JOIN Tab2 b ON b.t1 = a.t1 and b.t2 = a.t2 and b.t3 = a.t3 and b.t4 = a.t4 and b.val1 = a.val1 and b.val2 = a.val2 and b.val3 = a.val3 and b.val4 = a.val4
WHERE b.t1 IS NULL or b.Val1 is NULL
group by a.T1,a.T2,a.T3,a.T4,a.Val1,a.Val2,a.Val3,a.Val4 having MAX(returns)<4 and COUNT(distinct(week))>2 ;