MySQL batch queries with limit - mysql

I need to change the ownership of several possessions in my MySQL table. The thing that's tripping me up is that there are several identical goods and I don't want to change all of them.
Right now, I'm doing it with queries like this:
UPDATE possessions SET citizen_id=1 WHERE citizen_id=2 AND good_id=8 LIMIT 1
UPDATE possessions SET citizen_id=2 WHERE citizen_id=4 AND good_id=2 LIMIT 1
UPDATE possessions SET citizen_id=4 WHERE citizen_id=3 AND good_id=5 LIMIT 2
There are, some times, a lot of these and they're all so similar that I feel like there should be a way to submit them in a batch. The examples I can find that show batch updates don't allow setting individual limits on each update like I need to.
Do you know of any way I can make this process faster?

The following method relies on the fact that the possessions table has a primary key and citizen_id is not part of it. Here's the idea:
Put all the parameters of the update (citizen_id and good_id to filter on, the new values of citizen_id and the numbers of rows to update) into some storage, a dedicated table, perhaps, or a temporary table.
Assign row numbers to possessions rows partitioning on (citizen_id, good_id), then join the ranked row set to the parameter table to filter the original full set on citizen_id and good_id, as well as the number of rows.
Join possessions and the result of the previous join on the primary key values and update citizen_id with the new values.
In MySQL's SQL, the above might look like this:
UPDATE possessions AS p
INNER JOIN
(
SELECT
#r := #r * (#c = p.citizen_id AND #g = p.good_id) + 1 AS r,
p.possession_id,
#c := p.citizen_id AS citizen_id,
#g := p.good_id AS good_id
FROM
possessions AS p
CROSS JOIN
(SELECT #r := 0, #c := 0, #g := 0) AS x
ORDER BY
p.citizen_id,
p.good_id
) AS f ON p.possession_id = f.possession_id
INNER JOIN
possession_updates AS u ON u.citizen_id = f.citizen_id AND u.good_id = f.good_id
SET
p.citizen_id = u.new_citizen_id
WHERE
f.r <= u.row_count
;
The possessions_update is the table containing the parameter values.
The query uses a known method of row numbering that employs variables, which is implemented in the f subquery.
I don't have MySQL so I can't test this properly from the performance point of view, but at least you can see from this SQL Fiddle demo that the method works. (The UPDATE statement is in the schema script, because SQL Fiddle doesn't allow data modification statements in the right-side script for MySQL. The right side just returns the post-UPDATE contents of possessions.)

Related

MySQL 5.5.56 Temporary table always empty when used in a trigger but works when manually running the query

Can someone help or clear things for me. I've got this SQL code that I need to run on a trigger that doesn't work. But works when manually running the code on an SQL client.
SET #sr_id = NEW.purchase_id; /* SET #sr_id = 123456 when run manually */
SET #ndi = (SELECT COUNT(a.id) FROM purchase_rewards a LEFT JOIN item b ON b.id = a.item_id WHERE a.unit_id IS NOT NULL AND COALESCE(b.is_privileged,0) = 0 AND a.purchase_id = #sr_id);
SET #res = #ndi - CEIL(#ndi/2);
DROP TEMPORARY TABLE IF EXISTS for_removal;
CREATE TEMPORARY TABLE for_removal
SELECT ID FROM (
SELECT a.id, #rownum := #rownum + 1 AS `rank` FROM purchase_rewards a LEFT JOIN item b ON b.id = a.item_id
WHERE a.purchase_id = #sr_id AND COALESCE(b.is_privileged,0) = 0
) ft CROSS JOIN (SELECT #rownum := 0) r WHERE `rank` <= #res;
DELETE ta FROM purchase_rewards ta INNER JOIN for_removal tb ON ta.id = tb.id WHERE ta.purchase_id = #sr_id;
The code queries the purchased items that are not "privileged", putting a rank column on each and removing half of them. You only get rewarded for half of it, that's the point. The software was created by someone else with no source code so this is a piggy back system behind it.
I placed debug codes in between each to see if the connection changes or the results where empty but all is good except for the last part. Before the delete part I added a debug code:
SET #icount = (SELECT COUNT(ID) FROM for_removal);
INSERT INTO debug_log SET `log` = #icount;
and the result is that the table is always empty. I also tried converting the code into a stored procedure but I'm getting the same problem. Only running the code manually where it works.
I'm currently settling on CURSOR and loop-deletes which works, but it is slower when there are hundreds of items.
Sample Data: dbfiddle
Thanks!
Based on the comments above, the answer is to set the #rownum variable before the query.
SET #rownum = 0;
CREATE TEMPORARY TABLE ...
The reason is that you can't depend on the order of table evaluation in the CROSS JOIN. If the subquery is evaluated before the initialization of #rownum, then #rownum will be NULL, and any attempt to increment it with #rownum := #rownum + 1 will also yield NULL. So rank will have NULL on every row, and no rows will satisfy the WHERE clause.
As for why this works in the MySQL client but not in the trigger, I have a theory:
The session variable #rownum will keep its value if you test your query multiple times. So if you set it to some non-NULL value once in a session, then test the ranking query in the same session subsequently, it will increment.
But if you run it as part of a trigger, it will likely be a brand new session each time, and the value of #rownum will be initially NULL.

MySQL: Incorrect usage of UPDATE and ORDER BY occuring at a position decoration update query

My Ruby plugin acts_as_list for sorting generates gaps in the position column when I insert Thing 2.0 at position 2 of a list from 1-3 and I want to get this list.
formatted_position;position;name
1;1;Thing 1
2;2;Thing 2.0;
3;4;Thing 2
4;5;Thing 3
So I tried ...
UPDATE user_ranking_items JOIN (SELECT #rownum := 0) r SET formatted_position = #rownum := #rownum + 1 WHERE ranking_id = 1 ORDER BY position ASC
But I got the MySQL exception Incorrect usage of UPDATE and ORDER BY.
How to handle this?
P.S.: Selecting works and returns the list from above ...
SELECT position, #rownum := #rownum + 1 AS formatted_position FROM user_ranking_items JOIN (SELECT #rownum := 0) r WHERE ranking_id = 1 ORDER BY position ASC;
The "trick" is to wrap your query that's working (with the user variable and the order by) as an inline view (MySQL calls it a derived table). Reference that whole view as a row source in the UPDATE statement.
Since you've already got the query that returns the result you want, you just need to have it return a unique id (e.g. the primary key) so you can do the JOIN between the inline view back to the original row in the target of the UPDATE.
Something like this:
UPDATE user_ranking_items t
JOIN (
-- your query here
) s
ON s.id = t.id
SET t.col = s.new_col_val
(This works because MySQL first materializes that inline view, as a temporary MyISAM table, and then the UPDATE references the temporary MyISAM table.)

how do I find a random number within a range while excluding numbers

I'm currently assigning records to servers using a modulus of the primary key which is resulting to an even distribution. Sometimes I need to take one of the servers down and I don't want the records to build up while it's offline.
Current SQL
UPDATE table
SET server_id = ((primary_key % 10000) + 1)
WHERE server_id IS NULL;
I am trying to avoid a second SQL statement so if you have another idea, I'd love to hear it.
I think you want something like the following:
update table t join
(select s.*, #rn := #rn + 1 as FakeServerNumber, NumServers
from servers s cross join
(select count(*) as NumServers, #rn := -1
from servers
where isup = 'isup'
) const
where s.isup = 'isup'
) s
on t.primary_key % s.NumServer = s.rn
set t.server_id = s.server_id
where t.server_id is null;
This adds a sequential number to all the "up" servers. It uses this for the join back the modulus expression on the primary key. You can then insert the actual server information into your table.

Reorder a MYSQL table

I have a MySql table with a 'Order' field but when a record gets deleted a gap appears
how can i update my 'Order' field sequentially ?
If possible in one query 1 1
id.........order
1...........1
5...........2
4...........4
3...........6
5...........8
to
id.........order
1...........1
5...........2
4...........3
3...........4
5...........5
I could do this record by record
Getting a SELECT orderd by Order and row by row changing the Order field
but to be honest i don't like it.
thanks
Extra info :
I also would like to change it this way :
id.........order
1...........1
5...........2
4...........3
3...........3.5
5...........4
to
id.........order
1...........1
5...........2
4...........3
3...........4
5...........5
In MySQL you can do this:
update t join
(select t.*, (#rn := #rn + 1) as rn
from t cross join
(select #rn := 0) const
order by t.`order`
) torder
on t.id = torder.id
set `order` = torder.rn;
In most databases, you can also do this with a correlated subquery. But this might be a problem in MySQL because it doesn't allow the table being updated as a subquery:
update t
set `order` = (select count(*)
from t t2
where t2.`order` < t.`order` or
(t2.`order` = t.`order` and t2.id <= t.id)
);
There is no need to re-number or re-order. The table just gives you all your data. If you need it presented a certain way, that is the job of a query.
You don't even need to change the order value in the query either, just do:
SELECT * FROM MyTable WHERE mycolumn = 'MyCondition' ORDER BY order;
The above answer is excellent but it took me a while to grok it so I offer a slight rewrite which I hope brings clarity to others faster:
update
originalTable
join (select originalTable.ID,
(#newValue := #newValue + 10) as newValue
from originalTable
cross join (select #newValue := 0) newTable
order by originalTable.Sequence)
originalTable_reordered
on originalTable.ID = originalTable_reordered.ID
set originalTable.Sequence = originalTable_reordered.newValue;
Note that originalTable.* is NOT required - only the field used for the final join.
My example assumes the field to be updated is called Sequence (perhaps clearer in intent than order but mainly sidesteps the reserved keyword issue)
What took me a while to get was that "const" in the original answer was not a MySQL keyword. (I'm never a fan of abbreviations for that reason -- the can be interpreted many ways at times especially at these very when it is best they not be misinterpreted. Makes for verbose code I know but clarity always trumps convenience in my books.)
Not quite sure what the select #newValue := 0 is for but I think this is a side effect of having to express a variable before it can be used later on.
The value of this update is of course an atomic update to all the rows in question rather than doing a data pull and updating single rows one by one pragmatically.
My next question, which should not be difficult to ascertain, but I've learned that SQL can be a trick beast at the best of times, is to see if this can be safely done on a subset of data. (Where some originalTable.parentID is a set value).

Loop through column and update it with MySQL?

I want to loop through some records and update them with an ad hoc query in MySql. I have a name field, so I just want to loop though all of them and append a counter to each name, so it will be name1, name2, name3. Most examples I see use stored procs, but I don't need a stored proc.
As a stepping stone on your way to developing an UPDATE statement, first generate a SELECT statement that generates the new name values to your liking. For example:
SELECT t.id
, t.name
, CONCAT(t.name,s.seq) AS new_name
FROM ( SELECT #i := #i + 1 AS seq
, m.id
FROM mytable m
JOIN (SELECT #i := 0) i
ORDER BY m.id
) s
JOIN mytable t
ON t.id = s.id
ORDER BY t.id
To unpack that a bit... the #i is a MySQL user variable. We use an inline view (aliased as i) to initialize #i to a value of 0. This inline view is joined to the table to be updated, and each row gets assigned an ascending integer value (aliased as seq) 1,2,3...
We also retrieve a primary (or unique) key value, so that we can match each of the rows from the inline view (one-to-one) to the table to be updated.
It's important that you understand how that statement is working, before you attempt writing an UPDATE statement following the same pattern.
We can now use that SELECT statement as an inline view in an UPDATE statement, for example:
UPDATE ( SELECT t.id
, t.name
, CONCAT(t.name,s.seq) AS new_name
FROM ( SELECT #i := #i + 1 AS seq
, m.id
FROM mytable m
JOIN (SELECT #i := 0) i
ORDER BY m.id
) s
JOIN mytable t
ON t.id = s.id
ORDER BY t.id
) r
JOIN mytable u
ON u.id = r.id
SET u.name = r.new_name
SQL Fiddle demonstration here:
http://sqlfiddle.com/#!2/a8796/1
I had to extrapolate, and provide a table name (mytable) and a column name for a primary key column (id).
In the SQL Fiddle, there's a second table, named prodtable which is identical to mytable. SQL Fiddle only allows SELECT in the query pane, so in order to demonstrate BOTH the SELECT and the UPDATE, I needed two identical tables.
CAVEAT: be VERY careful in using MySQL user variables. I typically use them only in SELECT statements, where the behavior is very consistent, with careful coding. With DML statements, it gets more dicey. The behavior may not be as consistent in DML, the "trick" is to use a SELECT statement as an inline view. MySQL (v5.1 and v5.5) will process the query for the inline view and materialize the resultset as a temporary MyISAM table.
I have successfully used this technique to assign values in an UPDATE statement. But (IMPORTANT NOTE) the MySQL documentation does NOT specify that this usage or MySQL user variables is supported, or guaranteed, or that this behavior will not change in a future release.
Have the names stored in a table. Do a join against the names and update in the second table you want to.
Thanks