Update MySQL table any time another table changes - mysql

I am trying to reduce the number of queries my application uses to build the dashboard and so am trying to gather all the info I will need in advance into one table. Most of the dashboard can be built in javascript using the JSON which will reduce server load doing tons of PHP foreach, which was resulting in excess queries.
With that in mind, I have a query that pulls together user information from 3 other tables, concatenates the results in JSON group by family. I need to update the JSON object any time anything changes in any of the 3 tables, but not sure what the "right " way to do this is.
I could set up a regular job to do an UPDATE statement where date is newer than the last update, but that would miss new records, and if I do inserts it misses updates. I could drop and rebuild the table, but it takes about 16 seconds to run the query as a whole, so that doesn't seem like the right answer.
Here is my initial query:
SET group_concat_max_len = 100000;
SELECT family_id, REPLACE(REPLACE(REPLACE(CONCAT("[", GROUP_CONCAT(family), "]"), "\\", ""), '"[', '['), ']"', ']') as family_members
FROM (
SELECT family_id,
JSON_OBJECT(
"customer_id", c.id,
"family_id", c.family_id,
"first_name", first_name,
"last_name", last_name,
"balance_0_30", pa.balance_0_30,
"balance_31_60", pa.balance_31_60,
"balance_61_90", pa.balance_61_90,
"balance_over_90", pa.balance_over_90,
"account_balance", pa.account_balance,
"lifetime_value", pa.lifetime_value,
"orders", CONCAT("[", past_orders, "]")
) AS family
FROM
customers AS c
LEFT JOIN accounting AS pa ON c.id = pa.customer_id
LEFT JOIN (
SELECT patient_id,
GROUP_CONCAT(
JSON_OBJECT(
"id", id,
"item", item,
"price", price,
"date_ordered", date_ordered
)
) as past_orders
FROM orders
WHERE date_ordered < NOW()
GROUP BY customer_id
) AS r ON r.customer_id = c.id
where c.user_id = 1
) AS results
GROUP BY family_id
I briefly looked into triggers, but what I was hoping for was something like:
create TRIGGER UPDATE_FROM_ORDERS
AFTER INSERT OR UPDATE
ON orders
(EXECUTE QUERY FROM ABOVE WHERE family_id = orders.family_id)
I was hoping to create something like that for each table, but at first glance it doesn't look like you can run complex queries such as that where we are creating nested JSON.
Am I wrong? Are triggers the right way to do this, or is there a better way?

As a demonstration:
DELIMITER $$
CREATE TRIGGER orders_au
ON orders
AFTER UPDATE
FOR EACH ROW
BEGIN
SET group_concat_max_len = 100000
;
UPDATE target_table t
SET t.somecol = ( SELECT expr
FROM ...
WHERE somecol = NEW.family_id
ORDER BY ...
LIMIT 1
)
WHERE t.family_id = NEW.family_id
;
END$$
DELIMITER ;
Notes:
MySQL triggers are row level triggers; a trigger is fired for "for each row" that is affected by the triggering statement. MySQL does not support statement level triggers.
The reference to NEW.family_id is a reference to the value of the family_id column of the row that was just updated, the row that the trigger was fired for.
MySQL trigger prohibits the SQL statements in the trigger from modifying any rows in the orders table. But it can modify other tables.
SQL statements in a trigger body can be arbitrarily complex, as long as its not a bare SELECT returning a resultset, or DML INSERT/UPDATE/DELETE statements. DDL statements (most if not all) are disallowed in a MySQL trigger.

Related

MySQL - make "Update" a permanent change to a table?

So I have two tables - "Horses" and "Results". "Horses" lists a bunch of information about each horse, including a spot called "LTE", which totals from an "Earnings" field from the "Results" table. "Results" listed all the results of recent horse shows. I use the following code to calculate LTE -
UPDATE horses
SET horses.LTE = ( SELECT SUM(results.earnings)
FROM results WHERE horses.hname=results.hname )
Which works wonderfully - it updates the LTE column. However...I have to run this code EVERY time I add new data to the "Results" table. I will be adding data month for...well, pretty much ever.
I don't want to have to run this code every time. Is there a way to make the code "permanent," in a sense that the LTE field KNOWS it just calculates whenever new information is added? Or does MySQL not work this way?
Here's a peek at my tables with some data in them.
MySQL doesn't work that way. But you can achieve such functionality by using triggers. For example, you can update your sum each time there's insert in respective table (and/or update), this way you will always have your sum 'cached' and you can recalculate if needed.
If you want a bit more analytics on sum changes, you can follow this pattern, I think it will be helpful
You can use a TRIGGER on INSERT, UPDATE and DELETE to update your table:
-- trigger for INSERT (new rows on table result).
DELIMITER |
CREATE TRIGGER ins_result AFTER INSERT ON results
FOR EACH ROW
BEGIN
UPDATE horses SET horses.LTE = (
SELECT SUM(results.earnings) FROM results WHERE horses.hname = results.hname
);
END;
|
-- trigger for UPDATE (changed rows on table result).
CREATE TRIGGER upd_result AFTER UPDATE ON results
FOR EACH ROW
BEGIN
UPDATE horses SET horses.LTE = (
SELECT SUM(results.earnings) FROM results WHERE horses.hname = results.hname
);
END;
|
-- trigger for DELETE (removed rows on table result).
CREATE TRIGGER del_result AFTER DELETE ON results
FOR EACH ROW
BEGIN
UPDATE horses SET horses.LTE = (
SELECT SUM(results.earnings) FROM results WHERE horses.hname = results.hname
);
END;
|
Another solution could be a VIEW:
CREATE VIEW v_horses AS
SELECT h.*, SUM(r.earnings) AS 'LTE'
FROM horses h INNER JOIN results r ON h.hname = r.hname
GROUP BY h.hname;
With the created VIEW you can get the information about the horses with the following query:
SELECT * FROM v_horses;
In your case you doesn't use a DECIMAL column. So you have to convert the VARCHAR column to SUM the earnings. So in your case you have to use the following VIEW:
CREATE VIEW v_horses AS
SELECT h.*, SUM(CAST(REPLACE(REPLACE(r.earnings, ',', ''), '$', '') AS DECIMAL)) AS 'LTE'
FROM horses h INNER JOIN results r ON h.hname = r.hname
GROUP BY h.hname;
-- for a specific year (like 2017)
CREATE VIEW v_horses AS
SELECT h.*, SUM(CAST(REPLACE(REPLACE(r.earnings, ',', ''), '$', '') AS DECIMAL)) AS 'LTE'
FROM horses h INNER JOIN results r ON h.hname = r.hname
WHERE DATE_FORMAT(STR_TO_DATE(r.`Date`, '%c/%e/%y'), '%Y') = 2017
GROUP BY h.hname;
-- grouped by year (so you can use WHERE on the VIEW):
CREATE VIEW v_horses AS
SELECT h.*, SUM(CAST(REPLACE(REPLACE(r.earnings, ',', ''), '$', '') AS DECIMAL)) AS 'LTE',
DATE_FORMAT(STR_TO_DATE(r.`Date`, '%c/%e/%y'), '%Y') AS 'year'
FROM horses h INNER JOIN results r ON h.hname = r.hname
GROUP BY h.hname, DATE_FORMAT(STR_TO_DATE(r.`Date`, '%c/%e/%y'), '%Y');
Note: In case of using this VIEW you have to remove the column LTE from table horses.

MySQL: bulk updating in table

I'm using MySQL 5.6 and I have this issue.
I'm trying to improve my bulk update strategy for this case.
I have a table, called reserved_ids, provided by an external company, to assign unique IDs to its invoices. There is no other way to make this; I can't use auto_increment fields or simulated sequences.
I have this PL pseudocode to make this assignment:
START TRANSACTION;
OPEN invoice_cursor;
read_loop: LOOP
FETCH invoice_cursor INTO internalID;
IF done THEN
LEAVE read_loop;
END IF;
SELECT MIN(SECUENCIAL)
INTO v_secuencial
FROM RESERVED_IDS
WHERE COUNTRY_CODE = p_country_id AND INVOICE_TYPE = p_invoice_type;
DELETE FROM RESERVED_IDS WHERE SECUENCIAL = v_secuencial;
UPDATE MY_INVOICE SET RESERVED_ID = v_secuencial WHERE INVOICE_ID = internalID;
END LOOP read_loop;
CLOSE invoice_cursor;
COMMIT;
So, it's take one - remove - assign, then take next - remove - assign... and so on.
This works, but it's very very slow.
I don't know if there is any approach to make this assignment in a faster way.
I'm looking for something like INSERT INTO SELECT..., but with UPDATE statement, to assign 1000 or 2000 IDs directly, and no one by one.
Please, any suggestion is very helpful for me.
Thanks a lot.
EDIT 1: I have added WHERE clause details, because it was requested by user #vmachan . In the UPDATE...INVOICE clause, I don't filter by other criteria, because I have the direct and indexed invoice ID, which I want to update. Thanks
Finally, I have this solution. It's much faster than my initial approach.
The UPDATE query is
set #a=0;
set #b=0;
UPDATE MY_INVOICE
INNER JOIN
(
select
F.invoice_id,
I.secuencial as RESERVED_ID,
CONCAT_WS(/* format your final invoice ID */) AS FINAL_MY_INVOICE_NUMBER
FROM
(
select if(#a, #a:=#a+1, #a:=1) as current_row, internal_id
from MY_INVOICE
where reserved_id is null
order by internal_id asc
limit 2000
) F
INNER JOIN
(
SELECT if(#b, #b:=#b+1, #b:=1) as current_row, secuencial
from reserved_ids
order by secuencial asc
limit 2000
) I USING (CURRENT_ROW)
) TEMP MY_INVOICE.internal_id=TEMP.INTERNAL_ID
SET MY_INVOICE.RESERVED_ID = TEMP.RESERVED_ID, MY_INVOICE.FINAL_MY_INVOICE_NUMBER=TEMP.FINAL_MY_INVOICE_NUMBER
So, with autogenerated and correlated secuencial numbers #a and #b, we can join two different and no related tables like MY_INVOICE and RESERVED_IDs.
If you want to check this solution, please execute this tricky update following these steps:
Execute #a and then the first inner select in an isolated way: select if(#a, #a:=#a+1, ...
Execute #b and then the second inner select in an isolated way: select if(#b, #b:=#b+1, ...
Execute #a, #b and the big select that builds the TEMP auxiliar table: select F.invoice_id, ...
Execute the UPDATE
Finally, remove the assigned IDs from RESERVED_ID table.
Assignation time reduced drastically. My initial solution was one by one; with this, you assign 2000 (or more) in one single (ok, and a little tricky) update.
Hope this helps.

Loop through column and update it with MySQL?

I want to loop through some records and update them with an ad hoc query in MySql. I have a name field, so I just want to loop though all of them and append a counter to each name, so it will be name1, name2, name3. Most examples I see use stored procs, but I don't need a stored proc.
As a stepping stone on your way to developing an UPDATE statement, first generate a SELECT statement that generates the new name values to your liking. For example:
SELECT t.id
, t.name
, CONCAT(t.name,s.seq) AS new_name
FROM ( SELECT #i := #i + 1 AS seq
, m.id
FROM mytable m
JOIN (SELECT #i := 0) i
ORDER BY m.id
) s
JOIN mytable t
ON t.id = s.id
ORDER BY t.id
To unpack that a bit... the #i is a MySQL user variable. We use an inline view (aliased as i) to initialize #i to a value of 0. This inline view is joined to the table to be updated, and each row gets assigned an ascending integer value (aliased as seq) 1,2,3...
We also retrieve a primary (or unique) key value, so that we can match each of the rows from the inline view (one-to-one) to the table to be updated.
It's important that you understand how that statement is working, before you attempt writing an UPDATE statement following the same pattern.
We can now use that SELECT statement as an inline view in an UPDATE statement, for example:
UPDATE ( SELECT t.id
, t.name
, CONCAT(t.name,s.seq) AS new_name
FROM ( SELECT #i := #i + 1 AS seq
, m.id
FROM mytable m
JOIN (SELECT #i := 0) i
ORDER BY m.id
) s
JOIN mytable t
ON t.id = s.id
ORDER BY t.id
) r
JOIN mytable u
ON u.id = r.id
SET u.name = r.new_name
SQL Fiddle demonstration here:
http://sqlfiddle.com/#!2/a8796/1
I had to extrapolate, and provide a table name (mytable) and a column name for a primary key column (id).
In the SQL Fiddle, there's a second table, named prodtable which is identical to mytable. SQL Fiddle only allows SELECT in the query pane, so in order to demonstrate BOTH the SELECT and the UPDATE, I needed two identical tables.
CAVEAT: be VERY careful in using MySQL user variables. I typically use them only in SELECT statements, where the behavior is very consistent, with careful coding. With DML statements, it gets more dicey. The behavior may not be as consistent in DML, the "trick" is to use a SELECT statement as an inline view. MySQL (v5.1 and v5.5) will process the query for the inline view and materialize the resultset as a temporary MyISAM table.
I have successfully used this technique to assign values in an UPDATE statement. But (IMPORTANT NOTE) the MySQL documentation does NOT specify that this usage or MySQL user variables is supported, or guaranteed, or that this behavior will not change in a future release.
Have the names stored in a table. Do a join against the names and update in the second table you want to.
Thanks

pure MySQL loop to update multiple rows

I want to update multiple rows based on a SELECT sql query.
I want to do it ALL IN AN SQL SHELL!
Here is my select:
SELECT #myid := id, #mytitle := title
FROM event
WHERE pid>0 GROUP BY title
ORDER BY start;
Then, I want to do an update with this pseudocode:
foreach($mytitle as $t)
BEGIN
UPDATE event
SET pid=$myid
WHERE title=$t;
END
But I don't know how to ake a loop in SQL.
Maybe there's a way to make it in a single sql query?
I DON'T WANT ANY PHP!!! ONLY SQL SHELL CODE!!!
I want to update every rows with a pid with the id of the first occurence of an event. Start is a timestamp
I think this should do what you want, but if it doesn't (I'm not sure about joining a subquery in an UPDATE query) then you can use a temporary table instead.
UPDATE
event
JOIN (
SELECT
MIN(pid) AS minPID,
title
FROM
event
WHERE
pid > 0
GROUP BY
title
) AS findPIDsQuery ON event.title = findPIDsQuery.title
SET
event.pid = findPIDsQuery.minPID
Pure SQL doesn't really have "loops", per se: it's a set-based descriptive language. I believe the following update will do what you want (though your problem statements leaves much to be desired—we know nothing about the underlying schema).
update event t
set pid = ( select min(id)
from event x
where x.title = t.title
and x.pid > 0
group by x.title
having count(*) > 1
)
Cheers!

MySQL Stored Procedure | How to write it?

This time I have a MySQL question, I'm trying to create a stored procedure which will execute a prepared statement, the goal is to get a ranged list from a table("order_info"), the list is divided by "pages", each page is determined by a record count and should be ordered using a particular field sorted either 'ASC' or 'DESC', each record represents an "order" the catch here is that the procedure returns the orders of a particular group, the the order is associated to a user which belongs to a group. Here's what I've done so far:
CREATE DEFINER=`root`#`%` PROCEDURE `getGroupOrders`(IN grp INT,
IN page INT,
IN count INT,
IN ord TINYINT,
IN srt VARCHAR(4)
)
BEGIN
PREPARE prepGroupOrders FROM
"SELECT oi.* FROM `dbre`.`order_info` oi
INNER JOIN `dbre`.`users` usr
ON oi.`username` = usr.`username` AND usr.`id_group` = ?
ORDER BY ? ? LIMIT ?, ?";
SET #g := grp;
SET #cnt := count;
SET #start := #page*count ;
SET #orderBy := ord;
SET #sortBy := srt;
EXECUTE prepGroupOrders USING #g,#orderBy,#sortBy,#start,#cnt;
END
I get a syntax error when executing this, even though the editor does not higlight any errors and lets me save the procedure,I think that one of the follwing may be happening:
I am incorrectly usng the `ASC` or `DESC` since it is a SQL reserved word.
I read somewhere that Prepared statement are for only ONE SQL query, and since I have nested queries it can't be done.
I've tested this standard query:
SELECT oi.* FROM `dbre`.`order_info` oi
INNER JOIN `dbre`.`users` usr
ON oi.`username` = usr.`username` AND usr.`id_group` = 1
ORDER BY `status` DESC LIMIT 5, 10;
And it gives me the results I want. SO how would I design the procedure?
Any help is truly appreciated.
This may not necessarily solve your issue but, you can probably clean that query up a bit, eliminate the subquery and get something that should perform a little better.
SELECT oi.*
FROM `dbre`.`order_info` oi
INNER JOIN `dbre`.`users` u
ON oi.username = u.username
AND u.id_group = 1
ORDER BY `status` DESC
LIMIT 5, 10;