insert ... select ... on duplicate key update + delete obsolete rows - mysql

Question

I need to update a table with pre-calculated values from tables where data can be added/updated/deleted.
I could use
insert into precalculated(...)
select ... from ...
on duplicate key update ...
to add/update the pre-calculated table but is there an optimized method to delete the obsolete rows ?

Answer 1

I think you should create a stored procedure that deletes the data of your related tables if and only if the records fulfill a condition.
There's not enough information in your question to design the procedure, but I can give you a little example:
delimiter $$
create procedure delete_orphans()
begin
declare id_orphan int;
declare done int default false;
declare cur_orphans cursor for
select distinct d.id
from data as d
left join precalculated as p on d.id = p.id
where p.id is null;
declare continue handler for not found set done = true;
open cur_orphans;
loop_delete_orphans: loop
fetch cur_orphans into id_orphan;
if done then
leave cur_orphans;
end if;
delete from data where id = id_orphan;
end loop;
close cur_orphans;
end$$
delimiter ;
This procedure will delete every row in the data table that does not have at least one related row in the precalculated table.
Of course, this approach might be inneficient, because it will delete the rows one by one, but as I said this is only an example. You can customize it to fit your needs.
You can call this procedure from a trigger if you want (with call delete_orphans()).
Hope this helps.

Answer 2

Since you are always adding or updating rows that exist in these other tables, and you want to remove any rows that don't exist, why don't you just :
DELETE FROM precalculated
insert into precalculated(...)
select ... from ...
on duplicate key update ...
Always starting clean means you don't have to worry about orphans later.

Answer 3

You could add triggers for insert, delete and update on the main tables that maintains precalculated.
When inserting or updating the same code can be used to calculate the values and issuing a replace into precalculated (...) values (...)
When deleting it's probably the same, with the addition that you'll also delete rows from precalculated that are orphans. Be smart here and use values from the original delete to query precalculated for orphans instead of doing a table scan.

Answer 4

I may have found my solution using rename.
so basically, I will do a simple insert select to the temporary table and then
rename precalculated to precalculated_temprename, precalculated_temp to precalculated, precalculated_temprename to precalculated_temp;
truncate precalculated_temp;
need some tests but it seems the rename operation is fast and atomic.