I wonder why MySQL does not allow table locking inside a stored procedure.
I have the following SQL statements inside my stored procedure:
-- Total amount of money for comments
start transaction;
select #num_comments := count(*)
from `comment` c
where
c.user_id = user_id and
c.payment_rejection = 'NO' and
c.is_recorded = 0;
update `user` u set account_balance += u.comment_price * #num_comments where u.user_id = user_id;
update `comment` c set is_recorded = 1 where c.user_id = user_id and c.payment_rejection = 'NO' and c.is_recorded = 0;
commit;
So I have to lock table comment to prevent any writing to it for it may cause the number of rows selected in the first SQL statement be different from the number of actually updated.
Sir, in your code you can use ROW_COUNT() function instead of SELECT count(*)
According to documentation: http://dev.mysql.com/doc/refman/5.0/en/information-functions.html#function_row-count
ROW_COUNT() returns the number of rows changed, deleted, or inserted by the last statement if it was an UPDATE, DELETE, or INSERT. For other statements, the value may not be meaningful.
start transaction;
update `comment` c
set is_recorded = 1
where c.user_id = v_user_id
and c.payment_rejection = 'NO'
and c.is_recorded = 0;
SET #num_comments = row_count();
update `user` u
set account_balance += u.comment_price * #num_comments
where u.user_id = v_user_id;
commit;
In this way there is no need to lock tables, the number of rows cannot change between statements, and a bonus is a higher speed of the whole transaction.
Some comments:
The user_id column is ambiguous in the query:
where u.user_id = user_id;
and the update command updates the whole table instead of rows belong to one user only.
Don't use the same names for variables in the procedure as columns names in the table, the easiest way is to prepend some prefix to variable names to avoid ambiguity, for example:
where u.user_id = var_user_id;
Related
I'm having issues where an update statement should (to my knowledge) update 5 rows (I have selected 5 rows into a temp table and used an INNER JOIN in the update statement)
however when it comes to running the update statement it updates anything that could have been selected into the temp table not just the joined contents of the temp table itself.
I'm using the FOR UPDATE code in the selection statement to lock the rows, (as I'm expecting multiple queries to be aimed at this table at one time, (NOTE removing that does not change the error effect)
I've generalised the entire code base and it still has the same effect, I've been on this for the past few days and I'm sure that its just something silly I must be doing
Code description
TABLE `data`.`data_table Table to store data and to show it has been taken my the external program.
Stored Procedure `admin`.`func_fill_table debug code to populate the above table.
Stored Procedure `data`.`func_get_data Actual code designed to retrieve a batch size records, mark them as picked up and then return them to an external application.
Basic Setup Code
DROP TABLE IF EXISTS `data`.`data_table`;
DROP PROCEDURE IF EXISTS `admin`.`func_fill_table`;
DROP PROCEDURE IF EXISTS `data`.`func_get_data`;
DROP SCHEMA IF EXISTS `data`;
DROP SCHEMA IF EXISTS `admin`;
CREATE SCHEMA `admin`;
CREATE SCHEMA `data`;
CREATE TABLE `data`.`data_table` (
`identification_field_1` char(36) NOT NULL,
`identification_field_2` char(36) NOT NULL,
`identification_field_3` int(11) NOT NULL,
`information_field_1` int(11) NOT NULL,
`utc_action_time` datetime NOT NULL,
`utc_actioned_time` datetime DEFAULT NULL,
PRIMARY KEY (`identification_field_1`,`identification_field_2`,`identification_field_3`),
KEY `NC_IDX_data_table_action_time` (`utc_action_time`)
);
Procedure Creation
DELIMITER //
CREATE PROCEDURE `admin`.`func_fill_table`(
IN records int
)
BEGIN
IF records < 1
THEN SET records = 50;
END IF;
SET #processed = 0;
SET #action_time = NULL;
WHILE #processed < records
DO
SET #action_time = DATE_ADD(now(), INTERVAL FLOOR(RAND()*(45)-10) MINUTE);#time shorter for temp testing
SET #if_1 = UUID();
SET #if_2 = UUID();
INSERT INTO data.data_table(
identification_field_1
,identification_field_2
,identification_field_3
,information_field_1
,utc_action_time
,utc_actioned_time)
VALUES (
#if_1
,#if_2
,FLOOR(RAND()*5000+1)
,FLOOR(RAND()*5000+1)
,#action_time
,NULL);
SET #processed = #processed +1;
END WHILE;
END
//
CREATE PROCEDURE `data`.`func_get_data`(
IN batch int
)
BEGIN
IF batch < 1
THEN SET batch = 1; /*Minimum Batch Size of 1 */
END IF;
DROP TABLE IF EXISTS `data_set`;
CREATE TEMPORARY TABLE `data_set`
SELECT
`identification_field_1` as `identification_field_1_local`
,`identification_field_2` as `identification_field_2_local`
,`identification_field_3` as `identification_field_3_local`
FROM `data`.`data_table`
LIMIT 0; /* Create a temp table using the same data format as the table but insert no data*/
SET SESSION sql_select_limit = batch;
INSERT INTO `data_set` (
`identification_field_1_local`
,`identification_field_2_local`
,`identification_field_3_local`)
SELECT
`identification_field_1`
,`identification_field_2`
,`identification_field_3`
FROM `data`.`data_table`
WHERE
`utc_actioned_time` IS NULL
AND `utc_action_time` < NOW()
FOR UPDATE; #Select out the rows to process (up to batch size (eg 5)) and lock those rows
UPDATE
`data`.`data_table` `dt`
INNER JOIN
`data_set` `ds`
ON (`ds`.`identification_field_1_local` = `dt`.`identification_field_1`
AND `ds`.`identification_field_2_local` = `dt`.`identification_field_2`
AND `ds`.`identification_field_3_local` = `dt`. `identification_field_3`)
SET `dt`.`utc_actioned_time` = NOW();
# Update the table to say these rows are being processed
select ROW_COUNT(),batch;
#Debug output for rows altered (should be maxed by batch number)
SELECT * FROM
`data`.`data_table` `dt`
INNER JOIN
`data_set` `ds`
ON (`ds`.`identification_field_1_local` = `dt`.`identification_field_1`
AND `ds`.`identification_field_2_local` = `dt`.`identification_field_2`
AND `ds`.`identification_field_3_local` = `dt`. `identification_field_3`);
# Debug output of the rows that should have been modified
SELECT
`identification_field_1_local`
,`identification_field_2_local`
,`identification_field_3_local`
FROM
`data_set`; /* Output data to external system*/
/* Commit the in process field and allow other processes to access thoese rows again */
END;
//
Run Code
call `admin`.`func_fill_table`(5000);
call `data`.`func_get_data`(5);
You are misusing the sql_select_limit-setting:
The maximum number of rows to return from SELECT statements.
It only applies to select statements (to limit results sent to the client), not to insert ... select .... It is intended as a safeguard to prevent users to be accidentally flooded with millions of results, not as another limit function.
While in general, you cannot use a variable for limit, you can do it in a stored procedure (for MySQL 5.5+):
The LIMIT clause can be used to constrain the number of rows returned by the SELECT statement. LIMIT takes one or two numeric arguments, which must both be nonnegative integer constants, with these exceptions: [...]
Within stored programs, LIMIT parameters can be specified using integer-valued routine parameters or local variables.
So in your case, you can simply use
...
FROM `data`.`data_table`
WHERE `utc_actioned_time` IS NULL AND `utc_action_time` < NOW()
LIMIT batch
FOR UPDATE;
I'm a pretty new database student...
I have two tables. User, and Trip.
Whenever a new Trip is added to the database, I want it to trigger an update on the User table in the "totalMiles" column. I want it to take the Trip.userID and change the value in the "totalMiles" column of only the user with the same User.userID as the Trip.userID entry.
I want it to take the sum from "tripMiles" where the Trip.userID = User.userID. userID is a foreign key in the Trip table from the User table.
right now I just have:
CREATE DEFINER='root'#'localhost'TRIGGER 'DATABASE'.'Trip_AFTER_INSERT'
AFTER INSERT ON 'Trip' FOR EACH ROW
BEGIN
UPDATE User
SET totalMiles=sum(tripMiles)
WHERE Trip.userID = User.userID;
END
The error I get is "unknown column 'Trip.userID' in 'where clause'.
I googled and searched through a couple of posts here but I didn't really find anything.
Just change de "User.userID" on where clausule to "NEW.userID"
The "New" object means your new TRIP added...
Something like this:
CREATE TRIGGER 'Trip_AFTER_INSERT'
AFTER INSERT ON 'Trip' FOR EACH ROW
BEGIN
UPDATE User
SET totalMiles=sum(tripMiles)
WHERE User.userID = NEW.userID;
END
If you want more, see documentation here:
https://dev.mysql.com/doc/refman/5.7/en/trigger-syntax.html
and here
How to program a MySQL trigger to insert row into another table?
You have multiple problems with your statement, starting with the single quotes. One method is to recalculate the sum:
CREATE TRIGGER DATABASE.Trip_AFTER_INSERT
AFTER INSERT ON Trip FOR EACH ROW
BEGIN
UPDATE User u
SET totalMiles = (SELECT sum(t.tripMiles) FROM Trip t WHERE t.userId = u.userId)
WHERE u.userID = new.userID;
END;
The is overkill, though. If totalMiles is previously correct, then just adjust the value:
CREATE TRIGGER DATABASE.Trip_AFTER_INSERT
AFTER INSERT ON Trip FOR EACH ROW
BEGIN
UPDATE User u
SET totalMiles = u.totalMile + new.tripMiles - old.tripMailes;
WHERE u.userID = new.userID;
END;
I have a weird issue with my Trigger. There are 2 tables: Table A and Table B.
Whenever a row is inserted to Table A sum of a column in this table is inserted into Table B
It was working fine at first, but recently I noticed when >1 rows are inserted at the exact time for a user, the trigger returns sum in a weird way.
CREATE TRIGGER `update_something` AFTER INSERT ON `Table_A`
FOR EACH ROW BEGIN
DECLARE sum BIGINT(20);
SELECT IFNULL(SUM(number), 0) INTO sum FROM Table_A WHERE `user` = NEW.user;
UPDATE Table_B SET sum_number = sum WHERE id = NEW.id;
END
Example:
Table A
User X has a sum of 15 currently, then (with almost no delay in between):
Number 5 is inserted for him
Number 7 is inserted for him
Table B
On this table where we hold the sum, sum for this user was 15
Trigger updates this table in this way:
20
22 <--- Wrong, this should be 27
As you can see there isn't any number 2 inserted, it adds 7-5 = 2 for some reason.
How is that possible and why does it subtract 5 from 7 and add 2 to the sum instead of normally adding 7?
Edit 1:
Warning: This won't work, check the accepted answer instead
One of answers suggested select for update method.
Will this SELECT ... FOR UPDATE affect the performance negatively in a huge way?
CREATE TRIGGER `update_something` AFTER INSERT ON `Table_A`
FOR EACH ROW BEGIN
DECLARE sum BIGINT(20);
SELECT IFNULL(SUM(number), 0) INTO sum FROM Table_A WHERE `user` = NEW.user FOR UPDATE;
UPDATE Table_B SET sum_number = sum WHERE id = NEW.id;
END
Basically we only add FOR UPDATE to the end of SELECT line like this and it will perform Row Lock in InnoDB to fix the issue?
SELECT IFNULL(SUM(number), 0) INTO sum FROM Table_A WHERE user = NEW.user FOR UPDATE;
Edit 2 (Temporary Fix):
In case some one needs a very quick temporary fix for this before doing the actual & logical suggested fix: What I did was to put a random usleep(rand(1,500000)) before INSERT query in PHP to reduce the chance of simultaneous inserts.
The reason for this behaviour is that the inserted data is only committed to the database when the trigger finishes executing. So when both insert operations (5 and 7) execute the trigger in parallel, they read the data as it is in their transaction, i.e. the committed data with the changes made in their own transaction, but not the changes made in any other ongoing transaction.
The committed data in table A sums up to 20 for both transactions, and to that is added the record that is inserted in their own transaction. For the one this is 5, for the other it is 7, but as these records were not yet committed, the other transaction does not see this value.
That is why the sum is 20+5 for the one, and 20+7 for the other. The transactions then both update table B, one after the other (because table B will be locked during an update and until the end of the transaction), and the one that is latest "wins".
To solve this, don't read the sum from table A, but keep a running sum in Table B:
CREATE TRIGGER `update_something` AFTER INSERT ON `Table_A`
FOR EACH ROW BEGIN
UPDATE Table_B SET sum_number = sum_number + NEW.number WHERE id = NEW.id;
END;
/
I suppose you already have triggers for delete and update on Table_B, as otherwise you'd have another source of inconsistencies.
So these need to be (re)written too:
CREATE TRIGGER `delete_something` AFTER DELETE ON `Table_A`
FOR EACH ROW BEGIN
UPDATE Table_B SET sum_number = sum_number - OLD.number WHERE id = OLD.id;
END;
/
CREATE TRIGGER `update_something` AFTER UPDATE ON `Table_A`
FOR EACH ROW BEGIN
UPDATE Table_B SET sum_number = sum_number - OLD.number WHERE id = OLD.id;
UPDATE Table_B SET sum_number = sum_number + NEW.number WHERE id = NEW.id;
END;
/
This way you prevent to lock potentially many rows in your triggers.
Then, after you have done the above, you can fix the issues from the past, and do a one-shot update:
update Table_B
join (select id, user, ifnull(sum(number),0) sum_number
from Table_A
group by id, user) A
on Table_B.id = A.id
and Table_B.sum_number <> A.sum_number
set Table_B.sum_number = A.sum_number;
You get this because of the race condition in the trigger. Both triggeres are fired at the same time, thus SELECT returns the same value for both of them - 15. Then first trigger updates tha value adding 5 and resulting in 20, and then the second update is run with 15 + 7 = 22.
What you should do is use SELECT ... FOR UPDATE instead. This way if first trigger issues the select, then the second one will have to wait until first one finishes.
EDIT:
Your question made me think, and maybe using FOR UPDATE is not the best solution. According to documentation:
For index records the search encounters, SELECT ... FOR UPDATE locks the rows and any associated index entries, the same as if you issued an UPDATE statement for those rows.
And because you are selecting the sum of entries from Table A it will lock those entries, but will still allow inserting new ones, so the problem would not be solved.
It would be better to operate only on data from Table B inside the trigger, as suggested by trincot.
I have a table name tbl_tmp_trans
it contains every user transactions ever done ( and it's up to 6Mil right now !)
we have decided to keep only last 100 transaction per user in our database so we could keep the db clean
here is a query that i have came up with
delete from tbl_tmp_trans
where trans_id in
(
select trans_id
from
(
select trans_id
from tbl_faucets_transactions
order by date
group by user_id
limit 100
) foo
)
what am i doing wrong?
because after doing this my cpu reach 100% and mysql crashed.
Thanks in advance
P.S: our db is Mysql and table engine is Innodb
P.S2: We have about 120k and transction table have near 6 million record
I have a proposal... Hopefully, it might help you.
Alter your table:
alter table tbl_tmp_trans add column todel tinyint(1);
Implement a stored procedure to iterate through the table with a cursor and mark (set todel to 1) records that should be deleted. Example procedure to do that:
delimiter //
drop procedure if exists mark_old_transactions //
create procedure mark_old_transactions()
begin
declare done int default false;
declare tid int;
declare uid int;
declare last_uid int default 0;
declare count int default 0;
declare cur cursor for select trans_id, user_id from tbl_tmp_trans order by user_id, date desc;
declare continue handler for not found set done = true;
open cur;
repeat
fetch cur into tid, uid;
if (!done) then
if (uid!=last_uid) then
set count = 0;
end if;
set last_uid = uid;
set count = count + 1;
if (count > 100) then
update tbl_tmp_trans set todel=1 where trans_id=tid;
end if;
end if;
until done
end repeat;
close cur;
end //
Invoke the procedure, maybe do some simple checks (how many transactions you delete from the table, etc.), and delete the marked records.
call mark_old_transactions;
-- select count(*) from tbl_tmp_trans where todel=1;
-- select count(*) from tbl_tmp_trans;
delete from tbl_tmp_trans where todel=1;
Finally, remove the column that we just added.
alter table tbl_tmp_trans drop column todel;
Some notes:
Probably you have to iterate through all the records of the table
anyway, so you don't loose performance with the cursor.
If you have ~120K users and ~6M transactions, you have ~50 transactions per user on average. Which means, that probably you don't really
have too many users with transactions over 100, so the number of
updates (hopefully) won't be too many. => the procedure runs relatively fast.
Delete should be fast again with the new column.
In MySQL, is it possible to update the selected records on the same query?
For eg., If the query
SELECT *
FROM `table`
WHERE field = "value"
LIMIT 0,2
Return two rows, then on the same query, i need to increment the table's count field by 1. Is it possible?
Yes its possible you can write as UPDATE query as:
UPDATE my_table
SET count = count + 1
WHERE field = "value"
LIMIT 2;
or for LIMIT with offset try:
UPDATE my_table a
INNER JOIN (SELECT id FROM my_table WHERE field = "value" LIMIT 0, 2) b
ON a.id = b.id
SET count = count + 1;
It's not possible. The verb SELECT only retrieves data (without modifying it); and the verb UPDATE only modifies data (without retrieving it). There is no MySQL verb that will perform both actions. You will have to use two separate statements.
However, those two statements can be encapsulated within a transaction (if supported by your storage engine) to ensure that they are conducted atomically and/or could be invoked from within a stored procedure to simplify the command that must be issued by your client. Combining the two one would have:
DELIMITER ;;
CREATE PROCEDURE select_and_update(value TEXT)
BEGIN
START TRANSACTION;
SELECT * FROM `table` WHERE field = value LIMIT 0,2;
UPDATE `table` SET count = count + 1 WHERE ...;
COMMIT;
END;;
DELIMITER ;
Then your client need merely do:
CALL select_and_update('value');