I have a trigger that stores changes made in a separate table when a user edits the data. This data is written out on a web page beneath the current data in human readable format, i.e.
23/04/09 22:47 James Smith changed Tenant Name from "George Hill" to "George Hilling".
The trigger I have looks a bit like this - (this is the abridged version).
Two questions:
A) Is this quite costly performance-wise and if so is there a better approach to take?
B) Is there a tidier way to do this without all the IFs, using some sort of loop perhaps?
DELIMITER //
CREATE TRIGGER repair_history AFTER UPDATE ON repairs
FOR EACH ROW
BEGIN
INSERT INTO repair_edit SET repair_id=NEW.repair_id,
edit_date_time=NEW.edit_date_time, edited_by=NEW.edited_by;
IF OLD.tenant_name != NEW.tenant_name THEN
INSERT INTO repair_history SET edit_id=LAST_INSERT_ID(), field='tenant_name',
former_field_value=OLD.tenant_name, new_field_value=NEW.tenant_name;
END IF;
IF OLD.priority != NEW.priority THEN
INSERT INTO repair_history SET edit_id=LAST_INSERT_ID(), field='priority',
former_field_value=OLD.priority, new_field_value=NEW.priority;
END IF;
IF OLD.property_id != NEW.property_id THEN
INSERT INTO repair_history SET edit_id=LAST_INSERT_ID(), field='property_id',
former_field_value=OLD.property_id, new_field_value=NEW.property_id;
END IF;
IF OLD.type_id != NEW.type_id THEN
INSERT INTO repair_history SET edit_id=LAST_INSERT_ID(), field='type_id',
former_field_value=OLD.type_id, new_field_value=NEW.type_id;
END IF;
END; //
DELIMITER ;
A) Is this quite costly performance-wise
Profile it. Only you know what hardware you're using, and what volume of updates your database has to handle.
and if so is there a better approach to take?
Consider what's taking the mosst time in your trigger. Is it the comparisons, or the inserts?
B) Is there a tidier way to do this without all the IFs, using some sort of loop perhaps?
Unfortunately, I don't know of a way to index into the columns of pseudo tables NEW and OLD
Why not put a trigger on each column that can be updated and checked. This will have a performance hit though. What you may want to do is do a unit test where you have the trigger and don't have it, and see what the difference in time is when you are changing many rows quickly.
http://forums.mysql.com/read.php?99,174963,175381#msg-175381
Related
I successfully executed the sql code below from an msql editor (phpmyadmin), testing it with one customer (where Customer No=1). I need to now run the sql script for all the customers.
START TRANSACTION;
INSERT INTO `addresses` (`AddressLine1`,`CityID`,`ProvStateCode`,`AddressPostCode`,`CountryIso`) SELECT `Bill To Address`,`Bill To City`,`Bill To Province`,`Bill Code`,`Country` FROM `pdx_customers` where `Customer No`=1;
SELECT #last_id := LAST_INSERT_ID();
SELECT `Customer No` FROM `pdx_customers`
INSERT INTO `customer_addresses` (`CustID`,`AddressID`,`AddressTypeID`) Values(1,#last_id,1);
COMMIT;
It seems I would need to create a stored procedure ? In a loop, I need to get the Customer No dynamically for each row in the pdx_customers table, and enter into the Values clause in the insert command, i.e Values(#CustID,#last_id,1). Not sure how I would do this ?
Any help would be appreciated.
Thanks
This is a really common problem, and I would say that doing a loop in sql is almost never a good idea. Here is another option, which you may or may not consider good as it does introduce a new column. I've used it in some apps I've done, and its made things very simple. Does depend on your use case though so it wont be for everyone.
1) Firstly, add a new column to the address table, call it something that wont be confused by anyone looking at the table like TempInsertId.
2) When writing the new address, include the CustomerId in the TempInsertId column
3) Now you can easily read the AddressId and CustomerId back and write it into the CustomerAddress table
4) If you wish, do a final update to set the TempInsertId back to null.
As I said, not advocating in all cases, but it can be a very simple solution to the problem.
You can use the below statement to create a loop:
start transaction;
while counter < max do
insert into . . . ;
set counter=counter+1;
end while;
I have a source of data from where I extract some fields, among the fields there are some date fields and the source sends their values like this
#DD/MM/YYYY#
almost all the fields can be sent into the query with no modificaction, except this of course.
I have written a program the gets the data from an internet connection and sends it to the MySQL server and it's sending everything as it should, I am sure because I enabled general logging in the MySQL server and I can see all the queries are correct, except the ones with date fields.
I would like to avoid parsing the fields this way because it's a lot of work since it's all written in c, but if there is no way to do it, I understand and would accept that as an answer of course.
As an example suppose we had the following
INSERT INTO sometable VALUES ('#12/10/2015#', ... OTHER_VALUES ..., '#11/10/2015#');
in this case I send the whole thing as a query using mysql_query() from libmysqlclient.
In other cases I can split the parts of the message in something that is like an instruction and the parameters, something like this
iab A,B,C,#12/10/2015#,X,Y,#11/10/2015#
which could mean INSERT INTO table_a_something_b_whatever VALUES, and in this situation of course, I capture all the parameters and send a single query with a list of VALUES in it. Also in this situation, it's rather simple because I can handle the date like this
char date[] = "#11/10/2015#";
int day;
int month;
int year;
if (sscanf(date, "#%d/%d/%d#", &day, &month, &year) == 3)
{
/* it's fine, build a sane YYYY-MM-DD */
}
So the question is:
How can I tell the MySQL server in what format the date fields are?
Clarification to: Comment 1
Not necessarily INSERT, it's more complex than that. They are sometimes queries with all their parameters in it, sometimes they are just the parameters and I have to build the query. It's a huge mess but I can't do anything about it because it's a paid database and I must use it for the time being.
The real problem is when the query comes from the source and has to be sent as it is, because then there can be many occurrences. When I split the parameters one by one there is no real problem because parsing the above date format and generating the appropriate value of MySQL is quite simple.
You can use STR_TO_DATE() in MySQL:
SELECT STR_TO_DATE('#08/10/2015#','#%d/%m%Y#');
Use this as part of your INSERT process:
INSERT INTO yourtable (yourdatecolumn) VALUES (STR_TO_DATE('#08/10/2015#','#%d/%m%Y#'));
The only Thing I could imagine at the Moment would be to Change your Column-Type from DateTime to varchar and use a BEFORE INSERT Trigger to fix "wrong" Dates.
Something like this:
DELIMITER //
CREATE TRIGGER t1 BEFORE INSERT on myTable FOR EACH ROW
BEGIN
IF (NEW.myDate regexp '#[[:digit:]]+\/[[:digit:]]+\/[[:digit:]]+#') THEN
SET NEW.myDate = STR_TO_DATE(NEW.myDate,'#%d/%m/%Y#');
END IF;
END; //
DELIMITER ;
If you are just Need to run the Import in question once, use the Trigger to generate a "proper" dateTimeColumn out of the inserts - and drop the varchar-column afterwards:
('myDate' := varchar column to be dropped afterwards;`'myRealDate' := DateTime Column to Keep afterwards)
DELIMITER //
CREATE TRIGGER t1 BEFORE INSERT on myTable FOR EACH ROW
BEGIN
IF (NEW.myDate regexp '#[[:digit:]]+\/[[:digit:]]+\/[[:digit:]]+#') THEN
SET NEW.myRealDate = STR_TO_DATE(NEW.myDate,'#%d/%m/%Y#');
else
#assume a valid date representation
SET NEW.myRealDate = NEW.myDate;
END IF;
END; //
DELIMITER ;
Unfortunately you cannot use a Trigger to work on the datetime-column itself, because mysql will already mess up the NEW.myDate-Column.
I have recently switched over from OpenOffice Base to MySQL community. In ooBase you can customize integers upon entering them into the database. For example I could type 2013_00000 and then every number in this column would be formatted in this way (2013_00001, 2013_00002, ...). After playing around with MySQL community for a while I noticed that there is no obvious way to format a custom integer like this. I might be overlooking something basic, but if anyone knows how to do this please let me know. Thank you
In order to manipulate data before inserting them you need a trigger.
Something like that:
CREATE TRIGGER ins_sn BEFORE INSERT ON tbl
FOR EACH ROW
SET NEW.sn = CONCAT(LEFT(NEW.sn, 4), '_', RIGHT(NEW.sn,5));
If you need to update your rows, you need a second trigger:
CREATE TRIGGER upd_sn BEFORE UPDATE ON tbl
FOR EACH ROW
SET NEW.sn = CONCAT(LEFT(NEW.sn, 4), '_', RIGHT(NEW.sn,5));
See http://sqlfiddle.com/#!2/01428/1 for the complete example.
Here, I use the CONCAT(), LEFT() and RIGHT() functions for simplicity. Maybe your data require more complex manipulations (i.e.: padding). MySQL has a rich set of string functions. Anyway this is the principle.
Not the most flexible (here locked down to an underscore separator), but you can do it with a trigger, something like this;
CREATE TRIGGER trig_MyTable BEFORE INSERT ON MyTable
FOR EACH ROW
BEGIN
SELECT seq into #seq FROM MyTable_Seq FOR UPDATE;
SET NEW.id = #seq, #cutoff = LOCATE('_', #Seq);
UPDATE MyTable_Seq SET Seq =
CONCAT(LEFT(#seq, #cutoff),
LPAD(SUBSTR(#seq, #cutoff+1)+1, LENGTH(#seq)-#cutoff, '0'));
END;
//
An SQLfiddle to test with.
I notice that many people have said it's possible to create an injection attack, but my understanding is that is if someone is creating a query from a string, not parameters. In order to test the statement that Stored Procedures do not protect you against Injection Attacks, I am putting this example up in the hopes someone can show me a vulnerability if there is one.
Please note that I have built the code this way to easily insert a function that calls a procedure and embed it in a SELECT query. That means I cannot create a Prepared Statement. Ideally I'd like to keep my setup this way, as it is dynamic and quick, but if someone can create an injection attack that works, obviously that is not going to happen.
DELIMITER $$
#This procedure searches for an object by a unique name in the table.
#If it is not found, it inserts. Either way, the ID of the object
#is returned.
CREATE PROCEDURE `id_insert_or_find` (in _value char(200), out _id bigint(20))
BEGIN
SET #_value = _value;
SET #id = NULL;
SELECT id INTO _id FROM `table` WHERE name=_value;
IF _id IS NULL THEN
BEGIN
INSERT INTO `table` (`name`) VALUE (_value);
SELECT LAST_INSERT_ID() INTO _id;
END;
END IF;
END$$
CREATE FUNCTION `get_id` (_object_name char(200)) RETURNS INT DETERMINISTIC
BEGIN
SET #id = NULL;
call `id_insert_or_find`(_object_name,#id);
return #id;
END$$
The PHP Code
The PHP code I use here is:
(note, Boann has pointed out the folly of this code, below. I am not editing it for the sake of honoring the answer, but it will certainly not be a straight query in the code. It will be updated using ->prepare, etc. I still welcome any additional comments if new vulnerabilities are spotted.)
function add_relationship($table_name,$table_name_child) {
#This table updates a separate table which has
#parent/child relationships listed.
$db->query("INSERT INTO table_relationships (`table_id`,`tableChild_id`) VALUES (get_id('{$table_name}'),get_id('{$table_name_child}')");
}
The end result is
table `table`
id name
1 oak
2 mahogany
Now if I wanted to make oak the child of mahogany, I could use
add_relationship("mahogany","oak");
And if I wanted to make plastic the child of oak, I could use
add_relationship("oak","plastic");
Hopefully that helps give some framework and context.
It is not necessarily the stored procedure that is unsafe but the way you call it.
For example if you do the following:
mysqli_multi_query("CALL id_insert_or_find(" + $value + ", " + $id + ")");
then the attacker would set $value="'attack'" and id="1); DROP SCHEMA YOUR_DB; --"
then the result would be
mysqli_multi_query("CALL id_insert_or_find('attack', 1); DROP SCHEMA YOUR_DB; --)");
BOOM DEAD
Strictly speaking, that query should be written to escape the table names:
$db->query("INSERT INTO table_relationships (`table_id`,`tableChild_id`) " .
"VALUES (get_id(" . $db->quote($table_name) + ")," .
"get_id(" . $db->quote($table_name_child) . "))");
Otherwise, it would break out of the quotes if one of the parameters contained a single quote. If you only ever call that function using literal strings in code (e.g., add_relationship("mahogany", "oak");) then it is safe to not escape it. If you might ever call add_relationship using data from $_GET/$_POST/$_COOKIE or other database fields or files, etc, it's asking for trouble. I would certainly not let it pass a code review.
If a user could control the table name provided to that function then they could do, for example:
add_relationship("oak", "'+(SELECT CONCAT_WS(',', password_hash, password_salt) FROM users WHERE username='admin')+'");
Now you might say that there's no practical way to then extract that information if the resulting table name doesn't exist, but even then you could still extract information one binary bit at a time using a binary search and separate queries, just by breaking the query. Something like this (exact syntax not tested):
add_relationship("oak", "plastic'+(IF(ORD(SUBSTR(SELECT password_hash FROM users WHERE username='admin'),1,1)>=128, 'foo', ''))+'");
Really, it's easier to just escape the parameters and then you don't have to worry.
I am porting client DB to new one with different post titles and rows ID's , but he wants to keep the hits from old website,
he has over 500 articles in new DB , and updating one is not an issue with this query
UPDATE blog_posts
SET hits=8523 WHERE title LIKE '%slim charger%' AND category = 2
but how would I go by doing this for all 500 articles with 1 query ? I already have export query from old db with post title and hits so we could find the new ones easier
INSERT INTO `news_items` (`title`, `hits`) VALUES
('Slim charger- your new friend', 8523 )...
the only reference in both tables is product name word within the title everything else is different , id , full title ...
Make a tmp table for old data in old_posts
UPDATE new_posts LEFT JOIN old_posts ON new_posts.title = old_posts.title SET new_posts.hits = old_posts.hits;
Unfortunately that's not how it works, you will have to write a script/program that does a loop.
articles cursor;
selection articlesTable%rowtype;
WHILE(FETCH(cursor into selection)%hasNext)
Insert into newTable selection;
END WHILE
How you bridge it is up to you, but that's the basic pseudo code/PLSQL.
The APIs for selecting from one DB and putting into another vary by DBMS, so you will need a common intermediate format. Basically take the record from the first DB, stick it into a struct in the programming language of your choice, and prefrom an insert using those struct values using the APIs for the other DBMS.
I'm not 100% sure that you can update multiple records at once, but I think what you want to do is use a loop in combination with the update query.
However, if you have 2 tables with absolutely no relationship or common identifiers between them, you are kind of in a hard place. The hard place in this instance would mean you have to do them all manually :(
The last possible idea to save you is that the id's might be different, but they might still have the same order. If that is the case you can still loop through the old table and update the number table as I described above.
You can build a procedure that'll do it for you:
CREATE PROCEDURE insert_news_items()
BEGIN
DECLARE news_items_cur CURSOR FOR
SELECT title, hits
FROM blog_posts
WHERE title LIKE '%slim charger%' AND category = 2;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = TRUE;
OPEN news_items_cur;
LOOP
IF done THEN
LEAVE read_loop;
END IF;
FETCH news_items_cur
INTO title, hits;
INSERT INTO `news_items` (`title`, `hits`) VALUES (title, hits);
END LOOP;
CLOSE news_items_cur;
END;