I need to create a change history of table rows when a certain field is changed. So what I wanted to do is create a trigger on table update. When the field txta changes, I want the whole row to get copied over to debugwhich is a cloned version of msser_210 with an added column for datetime at the end, without data. I would like to add NOW() on change so I would have a timestamp. This is what I have tried to far:
DELIMITER $$
CREATE TRIGGER history_trigger
BEFORE UPDATE ON msser_210
FOR EACH ROW
BEGIN
IF OLD.txta != NEW.txta
THEN
INSERT INTO `debug_history` (`idpm`,`posn`,`prnb`,`doid`,`ofcr`,`pidm`,`hitm`,`sitm`,`item`,`dsca`,`igid`,`kitm`,`leng`,`widt`,`hght`,`thik`,`radi`,`quas`,`wght`,`effc`,`colr`,`bdat`,`edat`,`back`,`cuid`,`intb`,`aggr`,`unqu`,`oqua`,`unsq`,`stoc`,`allo`,`hall`,`tqan`,`bqan`,`pkey`,`pric`,`cvqs`,`unsp`,`disc`,`dart`,`ksid`,`anhg`,`txta`,`txti`,`mndn`, `changedate`) VALUES (OLD.idpm,OLD.posn,OLD.prnb,OLD.doid,OLD.ofcr,OLD.pidm,OLD.hitm,OLD.sitm,OLD.item,OLD.dsca,OLD.igid,OLD.kitm,OLD.leng,OLD.widt,OLD.hght,OLD.thik,OLD.radi,OLD.quas,OLD.wght,OLD.effc,OLD.colr,OLD.bdat,OLD.edat,OLD.back,OLD.cuid,OLD.intb,OLD.aggr,OLD.unqu,OLD.oqua,OLD.unsq,OLD.stoc,OLD.allo,OLD.hall,OLD.tqan,OLD.bqan,OLD.pkey,OLD.pric,OLD.cvqs,OLD.unsp,OLD.disc,OLD.dart,OLD.ksid,OLD.anhg,OLD.txta,OLD.txti, OLD.mndn, NOW());
END IF;
END;
$$
Why I want to do this is because we are having (probably) a php script with a bug that writes the same text string into every field of the database but we don't know when or why it happens neither which script it does. Is there maybe a more elegant solution?
UPDATE: I found the option to "Track Changes" in phpMyAdmin, but apparently it does not track our programs php-issued UPDATE queries, the DROP and CREATE TABLE statements from PHP are tracked though. If I issue an UPDATE via phpMyAdmin, it is tracked though. Long story short I went back to my original plan with the trigger.
UPDATE2: found the answer out myself
Update: As per the OP's comment, clearly the context is very specific. An infrastructure team without access to (or the ability to feedback and direct the development team's) code needs a mechanism by which to log table changes on a production database.
Warnings about using triggers:
Triggers can be tricky to debug, not least because they're transparent and it is never obvious to someone new looking at your code that a trigger is performing some action behind the scenes. (I speak from experience.) They can also cause issues on replicated, multi-master and clustered installations. (Again, I speak from experience.) Also if they fail for some unrelated reason (e.g. the table they write to is broken), the entire transaction can/will fail (InnoDB) - which might not be what you want. (Especially with non-essential "debug" functions.)
Otherwise, triggers are a perfectly valid tool. And in your specific scenario, probably the best bet available to you.
There are several other options available to you, two of which I would highlight:
Stored procedures as an access layer to data
If you're very data centric and you already have business logic inside the database - (a hotly debated topic, I'm not here arguing that you should or should not have business logic in the database) then reading and writing to the database through stored procedures has a clear advantage.
Any transactionally tied logic can be inserted into these stored procedures such that the transactionally unsafe caller (PHP, being a common example) only needs to call 1 query (call sp_insert_tablename(123, 'abc')) and transactional safety can be enforced by the database.
Temporary debug logic can be added to these stored procedures and enabled/disabled by a flag in a settings table, session variable, final argument, whatever you please.
Data abstraction layer/library
Similar principle. Find a data abstraction layer for your client (assuming you have access to alter it's internals). For a PHP or .NET web app there are several popular choices, all of which allow you to override (extend through code inheritance) the save/delete operations to perform any additional actions you want - exactly as for stored procedures (but with the logic maintained inside models in the client).
If you want a specific example, you'll need to give us more information on what stack/language/framework(s) you're using
With both options, make sure you appropriately handle error scenarios.
The debug_history is a cloned via pypMyAdmin from the original table. It got an additional changedate column appended manually.
ALTER TABLE debug_history ADD COLUMN changedate DATETIME DEFAULT NULL;
I decided because there was no other way that I would have to type all the names myself. Because I am lazy I got a recent SQL dump, copied an INSERT INTO-Statement from the file that is used to rebuild msser_210 and altered the values.
I added an extra row with an autoincrement line, dropped the primary key and set the new primary key to the new row.
ALTER TABLE debug_history DROP PRIMARY KEY;
ALTER TABLE debug_history ADD COLUMN changenumber INT NOT NULL PRIMARY KEY AUTO_INCREMENT;
I now have a working changelog, triggered on change in txta field (Please see the question for the trigger with the original format). I renamed the txta column in the debug_history to txta_old and created a new column txta_new.
ALTER TABLE debug_history CHANGE txta txta_old TEXT NOT NULL $$
ALTER TABLE debug_history ADD COLUMN txta_new TEXT NOT NULL AFTER txta_old $$
Afterwards I had to modify the trigger because I manually had to copy all the names..
DROP TRIGGER history_trigger
DELIMITER $$
CREATE TRIGGER history_trigger
BEFORE UPDATE ON msser_210
FOR EACH ROW
BEGIN
IF OLD.txta != NEW.txta
THEN
INSERT INTO `debug_history` (`idpm`,`posn`,`prnb`,`doid`,`ofcr`,`pidm`,`hitm`,`sitm`,`item`,`dsca`,`igid`,`kitm`,`leng`,`widt`,`hght`,`thik`,`radi`,`quas`,`wght`,`effc`,`colr`,`bdat`,`edat`,`back`,`cuid`,`intb`,`aggr`,`unqu`,`oqua`,`unsq`,`stoc`,`allo`,`hall`,`tqan`,`bqan`,`pkey`,`pric`,`cvqs`,`unsp`,`disc`,`dart`,`ksid`,`anhg`,`txta_old`,`txta_new`,`txti`,`mndn`, `changedate`) VALUES (OLD.idpm,OLD.posn,OLD.prnb,OLD.doid,OLD.ofcr,OLD.pidm,OLD.hitm,OLD.sitm,OLD.item,OLD.dsca,OLD.igid,OLD.kitm,OLD.leng,OLD.widt,OLD.hght,OLD.thik,OLD.radi,OLD.quas,OLD.wght,OLD.effc,OLD.colr,OLD.bdat,OLD.edat,OLD.back,OLD.cuid,OLD.intb,OLD.aggr,OLD.unqu,OLD.oqua,OLD.unsq,OLD.stoc,OLD.allo,OLD.hall,OLD.tqan,OLD.bqan,OLD.pkey,OLD.pric,OLD.cvqs,OLD.unsp,OLD.disc,OLD.dart,OLD.ksid,OLD.anhg,OLD.txta,NEW.txta,OLD.txti, OLD.mndn, NOW());
END IF;
END;
$$
I'm trying to create a database with history in mind (experience shows you'll have to do this one day or another).
I've asked here database-design-how-to-handle-the-archive-problem but there's no better anser than the link here.
My problem is about where to do the code and technically, how (MySQL gives me headaches). First I've started doing this in Php: before doing any insert, duplicate the record mark it as "obsolete" then modify the record.
But there's a dependency problem (manytomany and manytoone associations must be updated as well) which implies coding (one way or another) all the dependancies and updates that come with the tables (which is not acceptable).
So I'm thinking about doing all the work on the database server side. This would greatly simplify my Php code.
The problem is that I have to "archive" the current record before modifying it. To do so, the code must be in a trigger "before update".
Here's my code:
DELIMITER ;;
DROP TRIGGER IF EXISTS produit_trigger_update_before;
CREATE TRIGGER produit_trigger_update_before
BEFORE UPDATE ON produit
FOR EACH ROW BEGIN
/* */
INSERT INTO produit SET
id_origine = OLD.id_origine,
date_v_creation = OLD.date_v_creation,
date_v_start = OLD.date_v_debut,
date_v_end = NOW(),
...
last_record = OLD.last_record;
/* Dependancies : */
SET #last=LAST_INSERT_ID();
UPDATE categorie_produit SET id_produit=#last
WHERE id_produit = OLD.id;
UPDATE produit_attribut SET id_produit=#last
WHERE id_produit = OLD.id;
END;;
DELIMITER ;;
If I get this code working, all my problems are gone. But damn it, it's not working:
mysql> update produit set importance=3;
ERROR 1442 (HY000): Can't update table 'produit' in stored function/trigger because it is already used by statement which invoked this stored function/trigger.
mysql> Bye
In this page there's a working sample, which uses INSTEAD OF UPDATE clause in the trigger. MySQL doesn't seem to support this.
So my question is both conceptual (= have you any other "principle" that could work) and/or technical (= can you make this trigger work).
If I get this code working, all my problems are gone. But damn it, it's not working:
As a rule you can't have a trigger on table A trigger inserts into table A - since that could cause an endless loop. (Trigger mutation in Oracle terms)
Personally I would not do this using triggers. Triggers can do "audit logging" - but this is not what you want here.
I suggest you solve it programatically - either with a PHP function or a MySQL stored procedure (whatever your preference) that you call something like "ModifyProduit".
The code would then do basically what you have the trigger above do. (It might be easier to just have the code set date_v_end on the current row, and then insert a completly new row. That way you don't have to mess around with updating your referenced tables)
you can do history of a table with an auxiliary table like this (i've done this for many tables on mysql and the speed is very good):
table produit_history has the same structure as produit + 2 additional columns: "history_start DATETIME NOT NULL" and "history_stop DATETIME DEFAULT NULL".
there are 3 triggers on produit table:
AFTER INSERT: in this trigger there is a simple insert into produit_history of the same data with history_start = NOW() and history_stop = NULL (NULL means the current row is valid)
AFTER UPDATE: this trigger performs two queries. The first is un update like this:
UPDATE produit_history set history_stop = NOW() WHERE id_origine = OLD.id_origine AND history_stop IS NULL;
The second query is an insert identical to the one in the AFTER INSERT trigger.
AFTER DELETE: this triggers there is a simple update which is identical to the one in the AFTER UPDATE.
You can then query this history table and obtain snapshots at whatever time you're interested in with the following where condition:
WHERE (history_start <= "interesting_time" AND (history_stop IS NULL OR history_stop > "interesting_time"))
Problem: I've got a table which holds certain records. After the insert has been done, I want to call an external program (php script) via MySQL's sys_* UDFs.
Now, the issue - the trigger I have passes the ID of the record to the script.
When I try to pull the data out via the script, I get 0 rows.
During my own testing, I came to a conclusion that the trigger invokes the php script and passes the parameters BEFORE the actual insert occured, thus I get no records for given ID.
I've tested this on MySQL 5.0.75 and 5.1.41 (Ubuntu OS).
I can confirm that parameters get passed to the script before actual insert happens because I've added sleep(2); to my php script and I've gotten the data correctly.
Without sleep(); statement, I'm receiving 0 records for given ID.
My question is - how to fix this problem without having to hardcode some sort of delay within the php script?
I don't have the liberty of assuming that 2 seconds (or 10 seconds) will be sufficient delay, so I want everything to flow "naturally", when one command finishes - the other gets executed.
I assumed that if the trigger is of type AFTER INSERT, everything within the body of the trigger will get executed after MySQL actually inserts the data.
Table layout:
CREATE TABLE test (
id int not null auto_increment PRIMARY KEY,
random_data varchar(255) not null
);
Trigger layout:
DELIMITER $$
CREATE TRIGGER `test_after_insert` AFTER INSERT ON `test`
FOR EACH ROW BEGIN
SET #exec_var = sys_exec(CONCAT('php /var/www/xyz/servers/dispatcher.php ', NEW.id));
END;
$$
DELIMITER ;
Disclaimer: I know the security issues when using sys_exec function, my problem is that the MySQL doesn't insert FIRST and THEN call the script with necessary parameters.
If anyone can shed some light on how to fix this or has a different approach that doesn't involve SELECT INTO OUTFILE and using FAM - I'd be very grateful. Thanks in advance.
Even if you use an AFTER trigger, the row isn't committed yet. But sys_exec() doesn't return until the php script exits, so the AFTER trigger can't complete, therefore you can't commit the INSERT either.
This is by design. After all, you may do more operations within the same transaction, or you may roll back the transaction. That's the problem with invoking external processes from a trigger: external processes can't see data within the scope of the transaction in the database.
You shouldn't do this task with a trigger. At best, you should use the trigger to set a "flag" column and then write an external process to look for rows with the flag set and then invoke that PHP script. That way only rows that have successfully been inserted AND committed will be processed.
If I understand it clearly, you insert a row in your DB. That invoke a trigger that launch an external command written in PHP. That command queries in its turn the same DB by using the id of the inserted row?
I don't think this is a problem of "delay".
The real "problem" is your initial insert and you external command connect to the same DB on two different sessions -- probably in two different transactions (depending your database engine and your transaction isolation level).
I assume, when the trigger in invoked the row insert is not yet committed to the DB. So the external command still see the DB as it was before.
BTW, if the above explanation is quite speculative -- what is more evident to me is that you should probably think about a different design than trying to made that work as it is.