I have a table which goes somewhat as follows:
Name1 (Key)|Name2(Key)|Total|LastUpdate
There will be hundreds of thousands of such records.
Now very frequently, my main program will query a source to get updated values. It will then update the total and last update. It may have to update hundreds of such rows.
Currently I have this:
Update mytable SET Total=[...] AND LastUpdate=[....] WHERE Name1='x' AND Name2='y';
Update mytable SET .....
I'm wondering if there is a faster way of updating the rows (similar to how you can Insert multiple rows at the same time).
The Totals will be completely different, but the LastUpdate time will be the same for each Update.
Update to Clarify:
The changes to total aren't just an increment, and don't depend on the current value - so its not deterministic in that regard. The source provides me a new value every second or so, and I have to put in a new one.
if its to slow to do the updates separatly, mayby you can do somthing like this
CREATE TABLE mytable_updates
(
Name1 VARCHAR(50) NOT NULL,
Name2 VARCHAR(50) NOT NULL,
total_increase INT NOT NULL
PRIMARY KEY(Name1, Name2)
)
INSERT INTO mytable_updates VALUES ('x', 'y', 5), ('x', 'z', 3);
...
UPDATE mytable_updates
LEFT JOIN mytable USING (Name1, Name2)
SET
Total = Total + total_increase,
LastUpdate = NOW();
Related
I have a scenerio. I have a field called card_no. Some entries are blank. We don't want to handle that. Some have 16 digit integers and some have the proper data I need. What I need is that I want to select all the records that are 16 digits integer and replace them with xxxxxxxxxxxxLAST4DIGITS (The credit card format). Can i do that via mysql?
You don't have to SELECT the rows.
You can apply an UPDATE to a subset of rows matching a condition.
UPDATE scenario
SET card_no = CONCAT('xxxxxxxxxxxx', RIGHT(card_no, 4))
WHERE LENGTH(card_no) = 16
Yes. But you have to have this field as char(4) as you might want to store a number 0002 as 0002 not just 2. So I would first update the datatype of the field and then update the field entries as desired.
-- update datatype
alter table `cards`
modify column `card_no` char(4);
-- update values in card_no
update `cards`
set `card_no` = LPAD(if(length(`card_no`) > 4, substr(`card_no`, -4, 4), `card_no`), 4, '0')
where `card_no` is not null;
The 'xxx' append operation you can do at application code side or else it will take extra memory in db.
There is a table with existing records with different entity ids, now need to insert records in the Table using Cursor and without hard coding the records.
Please find my script below.
DECLARE #txId varchar(50)
Declare #UserRole_id int
#txId = NEWID()
set #UserRole_id = (SELECT #UserRole_id = MAX(User_Role_Id) + 1 FROM USER_TB
INSERT INTO USER_ROLE_TB
(User_Role_id,
entity_Id,
User_role_code,
User_Role_description,
Print_Sequence,
Update_date,
Tx_id,
Add_date,
Add_Username,
Update_Username,
Active_Flag)
Values...
(#UserRole_id,entity_Id,'ACH ','AC-HEATING',7,GETUTCDATE(),#txID,GETDATE(),'30383212','30383212',1)
(#UserRole_id,entity_Id,'BRAKE TECH','BRAKE',8,GETUTCDATE(),#txID,GETDATE(),'30383212','30383212',1)
(#UserRole_id,Pos_entity_Id,'CASH OUT','CASH OUT TECHNICIAN',4,GETDATE(),#txID,GETDATE(),'30383212','30383212',1)
Here i have hardcoded the records what if i have to insert new records. Im not sure how do i do it.
i got to insert in these column without harding coding them and using cursor.
Not so familiar with Cursor. Please help.
How can I can write the query this way, that the result of the CASE WHEN - statement is adapted to every row. So that in every row the result will be 5. Thank you very much!
CREATE TABLE DATA
(`Person` CHAR(4),
`Apples` INT(1),
`Tomatoes` INT(1),
`Result` INT(1)
);
INSERT INTO DATA
(Person, Apples, Tomatoes)
VALUES ('Mark' , 1, 2),
('Sepp', 2, 3),
('Carl', 3, 1);
UPDATE DATA
SET `Result` = CASE WHEN (`Person` = 'Sepp') THEN (`Apples` + `Tomatoes`) END;
Table of result as it should be
SQL fiddle demonstration
If you want all rows to get the value from Sepps row you can do it using a subquery.
The "normal" way would be to do this:
UPDATE DATA
SET Result = (SELECT Apples + Tomatoes FROM DATA WHERE Person = 'Sepp')
But this will most likely give you an error with MySQL (can't specify target table for update) and a workaround is to introduce another level in the query which forces a temporary table to be used, like this:
UPDATE DATA
SET Result = (
SELECT Value FROM (
SELECT Apples + Tomatoes AS Value
FROM DATA WHERE Person = 'Sepp'
) t
);
Problem to insert running total in MySQL transactional database. need your help for solutions and opinion. Table structure of my table is,
create table `wtacct` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`ACCOUNT_NO` varchar(16),
`AMOUNT` float(16,2),
`BALANCE` float(16,2)
);
[Please note other fields have been removed to make it simple example]
I am doing Transaction as,
Dr 10 USD from account 1001 and
Cr 10 USD to account 2002
Insert query
INSERT INTO wtacct (ID, ACCOUNT_NO, AMOUNT, BALANCE)
VALUES ('', 1001, -10, 100), ('', 2002, 10, 5000);
I want the Balance as,
BALANCE of Account no 1001 = Last transaction Balance of account 1001 - 10.
My solutions and limitations
Solution 1
In insert statement put sub query in balance field:
select balance from wtacct where account_no=1001 and id in(select max(id) from wtacct where account_no=1001)
Limitation: Mysql does not support same table select query (wtacct) where inserting the data (wtacct).
Solution 2
Using insert into select statement
insert into wtacct select '' ID, 1001 ACCOUNT_NO, -10 AMOUNT, (BALANCE-10) BALANCE where account_no=1001 and id in(select max(id) from wtacct where account_no=1001)
Limitation: For first transaction there is no record in wtacct for the account 1001 so select query will not return any record for first transaction.
Solution 3
Taking balance in variable and use it in insert statement.
select #balance1001 :=balance from wtacct
where account_no=1001 and id in(select max(id) from wtacct where account_no=1001)
select #balance2002 :=balance from wtacct
where account_no=2002 and id in(select max(id) from wtacct where account_no=2002)
INSERT INTO wtacct (ID, ACCOUNT_NO, AMOUNT, BALANCE)
VALUES ('', 1001, -10, #balance1001-10), ('', 2002, 10, #balance2002+10);
Limitation: there is a chance to be change the balance in time between select and insert query execution. also its costly, 3 query execution required.
Solution 4
Insert and then update Balance
INSERT INTO wtacct (ID, ACCOUNT_NO, AMOUNT, BALANCE)
VALUES ('', 1001, -10, 0);
UPDATE wtacct set balance = (ifnull(Select balance from wtacct where account_no=1001 and id in(select max(id) from wtacct where id <last_insert_id() and account_no=1001),0) -10)
where id =last_insert_id() and account_no=1001
........
Limitation: query is costly. its required 4 (two insert and 2 update) query execution. note last_insert_id() is php function
Solution 5
Using a trigger on insert statement. In the trigger, the balance will be updated calculating last transaction value and insert amount.
Limitation: Trigger not support transaction behavior and may fail.
Please give your solution and opinion on the above solutions. Please note in the above example their may be some syntax error/error. Please ignore them.
A big limitation I didn't see listed is a potential race condition, where two rows are being inserted into the table at the same time. There's a chance that the two inserts will both get the current "balance" from the same previous row.
One question: do you also have a separate "current balance" table that keeps a single value of the current "balance" for each account? Or are you only relying on the "balance" from the previous transaction.
Personally, I would track the current balance on a separate "account balance" table. And I would use BEFORE INSERT/UPDATE triggers to maintain the value in that row, and use that to return the current balance for the account.
For example, I would define a trigger like this which gets fired when a row is inserted into `wtacct` table:
CREATE TRIGGER wtacct_bi
BEFORE INSERT ON wtacct
FOR EACH ROW
BEGIN
IF NEW.amount IS NULL THEN
SET NEW.amount = 0;
END IF
;
UPDATE acct a
SET a.balance = (#new_balance := a.balance + NEW.amount)
WHERE a.account_no = NEW.account_no
;
SET NEW.balance = #new_balance
;
END$$
The setup for that trigger...
CREATE TABLE acct
( account_no VARCHAR(16) NOT NULL PRIMARY KEY
, balance DECIMAL(20,2) NOT NULL DEFAULT 0
) ENGINE=InnoDB
;
CREATE TABLE wtacct
( id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT
, account_no VARCHAR(16) NOT NULL COMMENT 'FK ref acct.account_no'
, amount DECIMAL(20,2) NOT NULL
, balance DECIMAL(20,2) NOT NULL
, FOREIGN KEY FK_wtacct_acct (account_no) REFERENCES acct (account_no)
ON UPDATE CASCADE ON DELETE RESTRICT
) ENGINE=InnoDB
;
My reason for using a separate "current balance" table is that there is only one row for the given account_no, and that row retains the current balance of the account.
The UPDATE statement in the trigger should obtain an exclusive lock on the row being updated. And that exclusive lock prevents any other UPDATE statement from simultaneously updating the same row. The execution of the UPDATE statement will add the `amount` from the current transaction row being inserted to the current balance.
If we were using Oracle or PostgreSQL, we could use a RETURNING clause to get the value that was assigned to the \'balance\' column.
In MySQL we can do a wonky workaround, using a user-defined variable. The new value we are going to assign to the column is first assigned to the user_defined variable, and then that is assigned to the column.
And we can assign the value of the user-defined variable to the `balance` column of the row being inserted into `wtacct`.
The purpose of this approach is to make the retrieval and update of the current balance in a single statement, to avoid any race conditions.
The UPDATE statement locates the row, obtains an exclusive (X) lock on the row, retrieves the current balance (value from the \'balance\' column), calculates the new current balance, and assigns it back to the \'balance\' column. Then continues to hold the lock until the transaction completes.
Once the trigger completes, the INSERT statement (which initially fired the trigger) proceeds, attempting to insert the new row into `wtacct`. If that fails, then all of the changes made by the INSERT statement and execution of the trigger are rolled back, keeping everything consistent.
Once a COMMIT or ROLLBACK is issued by the session, the exclusive (X) lock held on the row(s) in `acct` are released, and other sessions can obtain locks on that row in `acct`.
I have done it using Store Procedure for MySql
CREATE DEFINER=`root`#`%` PROCEDURE `example_add`(IN dr Int, IN cr Int)
BEGIN
DECLARE LID int;
Declare Balance decimal(16,2);
INSERT INTO example (Debit,Credit)
VALUES (dr, cr);
SET LID = LAST_INSERT_ID();
SET Balance = (select SUM(Debit) - SUM(Credit) as Balance from example);
UPDATE Example SET Balance = Balance WHERE ID = LID;
END
Use it example_add(10,0) or example_add(0,15) then select and see the result.
I'm currently working on a project with a MySQL Db of more than 8 million rows. I have been provided with a part of it to test some queries on it. It has around 20 columns out of which 5 are of use to me. Namely: First_Name, Last_Name, Address_Line1, Address_Line2, Address_Line3, RefundID
I have to create a unique but random RefundID for each row, that is not the problem. The problem is to create same RefundID for those rows whose First_Name, Last_Name, Address_Line1, Address_Line2, Address_Line3 as same.
This is my first real work related to MySQL with such large row count. So far I have created these queries:
-- Creating Teporary Table --
CREATE temporary table tempT (SELECT tt.First_Name, count(tt.Address_Line1) as
a1, count(tt.Address_Line2) as a2, count(tt.Address_Line3) as a3, tt.RefundID
FROM `tempTable` tt GROUP BY First_Name HAVING a1 >= 2 AND a2 >= 2 AND a3 >= 2);
-- Updating Rows with First_Name from tempT --
UPDATE `tempTable` SET RefundID = FLOOR(RAND()*POW(10,11))
WHERE First_Name IN (SELECT First_Name FROM tempT WHERE First_Name is not NULL);
This update query keeps on running but never ends, tempT has more than 30K rows. This query will then be run on the main DB with more than 800K rows.
Can someone help me out with this?
Regards
The solutions that seem obvious to me....
Don't use a random value - use a hash:
UPDATE yourtable
SET refundid = MD5('some static salt', First_Name
, Last_Name, Address_Line1, Address_Line2, Address_Line3)
The problem is that if you are using an integer value for the refundId then there's a good chance of getting a collision (hint CONV(SUBSTR(MD5(...),1,16),16,10) to get a SIGNED BIGINT). But you didn't say what the type of the field was, nor how strict the 'unique' requirement was. It does carry out the update in a single pass though.
An alternate approach which creates a densely packed seguence of numbers is to create a temporary table with the unique values from the original table and a random value. Order by the random value and set a monotonically increasing refundId - then use this as a look up table or update the original table:
SELECT DISTINCT First_Name
, Last_Name, Address_Line1, Address_Line2, Address_Line3
INTO temptable
FROM yourtable;
set #counter=-1;
UPDATE temptable t SET t,refundId=(#counter:=#counter + 1)
ORDER BY r.randomvalue;
There are other solutions too - but the more efficient ones rely on having multiple copies of the data and/or using a procedural language.
Try using the following:
UPDATE `tempTable` x SET RefundID = FLOOR(RAND()*POW(10,11))
WHERE exists (SELECT 1 FROM tempT y WHERE First_Name is not NULL and x.First_Name=y.First_Name);
In MySQL, it is often more efficient to use join with update than to filter through the where clause using a subquery. The following might perform better:
UPDATE `tempTable` join
(SELECT distinct First_Name
FROM tempT
WHERE First_Name is not NULL
) fn
on temptable.First_Name = fn.First_Name
SET RefundID = FLOOR(RAND()*POW(10,11));