I'm trying to write a function to SELECT the least-recently fetched value from a table in my database. I do this by SELECTing a row and then immediately changing the last_used field.
Because this involves a SELECT and UPDATE, I'm trying to do this with locks. The locks are to ensure that concurrent executions of this query won't operate on the same row.
The query runs perfectly fine in phpMyAdmin, but fails in Magento. I get the following error:
SQLSTATE[HY000]: General error
Error occurs here:
#0 /var/www/virtual/magentodev.com/htdocs/lib/Varien/Db/Adapter/Pdo/Mysql.php(249): PDOStatement->fetch(2)
Here is my model's code, including the SQL query:
$write = Mage::getSingleton('core/resource')->getConnection('core_write');
$sql = "LOCK TABLES mytable AS mytable_write WRITE, mytable AS mytable_read READ;
SELECT #val := unique_field_to_grab FROM mytable AS mytable_read ORDER BY last_used ASC LIMIT 1;
UPDATE mytable AS mytable_write SET last_used = unix_timestamp() WHERE unique_field_to_grab = #val LIMIT 1;
UNLOCK TABLES;
SELECT #val AS val;";
$result = $write->raw_fetchrow($sql, 'val');
I've also tried using raw_query and query instead of raw_fetchrow with no luck.
Any thoughts on why this doesn't work? Or is there a better way to accomplish this?
EDIT: I'm starting to think this may be related to the PDO driver, which Magento is definitely using. I think phpMyAdmin is using mysqli, but I can't confirm that.
Probably a function that Magento uses doesn't support multiple sql statements.
Call each statement separately.
exec("LOCK TABLES mytable AS mytable_write WRITE, mytable AS mytable_read READ");
exec("SELECT #val := unique_field_to_grab FROM mytable AS mytable_read ORDER BY last_used ASC LIMIT 1");
exec("UPDATE mytable AS mytable_write SET last_used = unix_timestamp() WHERE unique_field_to_grab = #val LIMIT 1");
exec("UNLOCK TABLES");
exec("SELECT #val AS val");
Use appropriate functions instead of exec().
Related
I'm writing an application to:
select a small recordset from a table of subscribers (150k records);
update those rows to indicate that an email is in the process of being sent;
send email to the subscribers in the recordset;
update the rows again to indicate that the email has been sent.
The wrinkle is that the table is simultaneously being accessed by multiple clients to distribute the email workload, which is why there is the intermediate update (to indicate in-process) is used -- to keep the different clients from selecting the same rows, which results in multiple emails being sent to the same subscriber. I've applied some randomizing logic to reduce the likelihood of two clients working with the same data, but it still happens occasionally.
So now I am looking at using SELECT ... FOR UPDATE in order to lock the relevant rows (so another client won't select them). My question: is it better to write the UPDATE statement based on the IDs of the SELECT...FOR UPDATE statement, or to create a loop to update each row individually?
Here's what I've got so far:
DELIMITER $$
CREATE DEFINER=`mydef`#`%` PROCEDURE `sp_SubscribersToSend`(v_limit INTEGER)
BEGIN
START TRANSACTION;
SELECT _ID, email, date_entered, DATE_FORMAT(date_entered, '%b %e, %Y') AS 'date_entered_formatted'
FROM _subscribers
WHERE send_state = 'Send'
AND status = 'Confirmed'
LIMIT v_limit
FOR UPDATE;
[[UPDATE _subscribers SET send_state = 'Sending' WHERE _ID IN (...?)]]
[[OR]]
[[Loop through the resultset and update each row?]]
COMMIT;
END
Seems like a single UPDATE is going to be more efficient; what is the best way to turn the _ID column of the resultset into a comma-delimited list for the IN() clause? (I've been doing this client-side before this) -- or is there a better way altogether?
Instead of trying to create a comma-delimited list, just do an UPDATE with the same criteria as the SELECT
START TRANSACTION;
UPDATE _subscribers
SET send_state = 'Sending'
WHERE send_state = 'Send'
AND status = 'Confirmed'
ORDER BY <something>
LIMIT v_limit;
SELECT _ID, email, date_entered, DATE_FORMAT(date_entered, '%b %e, %Y') AS 'date_entered_formatted'
FROM _subscribers
WHERE send_state = 'Send'
AND status = 'Confirmed'
ORDER BY <something>
LIMIT v_limit;
COMMIT;
The ORDER BY clause is necessary to ensure that both queries process the same rows; if you use LIMIT without ORDER BY, they could select a different subset of rows.
Thanks to Barmar, I took a different tack in the stored procedure:
SET #IDs := null;
UPDATE _subscribers
SET send_state = 'Sending'
WHERE send_state = 'Send'
AND status = 'Confirmed'
AND (SELECT #IDs := CONCAT_WS(',', _ID, #IDs) )
LIMIT v_limit;
SELECT CONVERT(#IDs USING utf8);
As Barmar suggested, it does an UPDATE but also concatenates the IDs of the rows being updated into a variable. Just SELECT that variable, and it gives you a comma-delimited list that can be passed into a PREPARE statement. (I had to use CONVERT because SELECTing the variable was returning a binary/blob value). So...this does not use SELECT...FOR UPDATE as I was originally intending, but it does ensure that the different clients won't be working with the same rows.
I have a query running on a large table that worked when I did limit 100. When I remove the limit I get:
[Err] 126 - Incorrect key file for table '/tmp/#sql_5e2d_6.MYI'; try
to repair it
I checked with server admin basically the /tmp file fills up quickly.
Is there a way to set up the query to flush the table as it goes along? Or run say 100 records, stop, re-run? The query is pretty simple:
select distinct a,
min(b) N_b
from K
group by a;
At the end of the day what I am trying to do is delete from a large table duplicate records, keeping the record with the lowest value in b. This was the initial select statement.
You can use something like,
*** Edit according to the language you use.
$sql = true;
int i = 0;
While($sql){
$sql = "select distinct a,
min(b) N_b
from K
group by a LIMIT i, i+99";
//Do whatever you want.
i=i+100;
}
I'm having a trouble with something that looks like simple thing. I'm trying to find first row that satisfies WHERE part of query and UPDATE it.
UPDATE Donation SET Available=0 WHERE Available != 0 and BloodGroup='" + bloodGroup + "' LIMIT 1"
bloodGroup is variable that gets filled automatically using C# and it keeps string value of selected blood group.
When I try to run this I get incorrect syntax near 'limit'.
What I'm doing wrong? Is it possible using LIMIT like during UPDATE query?
During debugging I got query like this:
UPDATE Donation SET Available=0 WHERE Available != 0 AND BloodGroup='AB-' LIMIT 1
Because C# is often used with SQL Server, perhaps the question is mistagged. The syntax looks fine for MySQL.
In SQL Server, you can do this as:
UPDATE TOP (1) Donation
SET Available = 0
WHERE Available <> 0 AND BloodGroup = 'AB-';
Note that this chooses an arbitrary matching row, as does your original query (there is no order by).
It is not safe to use limit in update queries.
Please refer
http://bugs.mysql.com/bug.php?id=42415
The documentation states that any UPDATE statement with LIMIT clause is considered unsafe since the order of the rows affected is not defined: http://dev.mysql.com/doc/refman/5.1/en/replication-features-limit.html
However, if "ORDER BY PK" is used, the order of rows is defined and such a statement could be logged in statement format without any warning.
You can use like this way limit in Update Queries like these
UPDATE messages SET test_read=1
WHERE id IN (
SELECT id FROM (
SELECT id FROM messages
ORDER BY date_added DESC
LIMIT 5, 5
) tmp
);
Also please
Can you try it? way of getting row_number
UPDATE Donation d1 join (SELECT id,(SELECT #Row:=0) as row,(#Row := #Row + 1) AS row_number FROM Donation where Available <> 0 AND BloodGroup='AB-') d2
ON d1.id=d2.id
SET d1.Available='three'
WHERE d1.Available <> 0 AND d1.BloodGroup='AB-' AND d2.row_number='1'
In trying to avoid deadlocks and synchronize requests from multiple services, I'm using ROWLOCK, READPAST. My question is where should I put it in a query that includes a CTE, a subquery and an update statement on the CTE? Is there one key spot or should all three places have it (below)? Or maybe there's a better way to write such a query so that I can select ONLY the rows that will be updated.
alter proc dbo.Notification_DequeueJob
#jobs int = null
as
set nocount on;
set xact_abort on;
declare #now datetime
set #now = getdate();
if(#jobs is null or #jobs <= 0) set #jobs = 1
;with q as (
select
*,
dense_rank() over (order by MinDate, Destination) as dr
from
(
select *,
min(CreatedDt) over (partition by Destination) as MinDate
from dbo.NotificationJob with (rowlock, readpast)
) nj
where (nj.QueuedDt is null or (DATEDIFF(MINUTE, nj.QueuedDt, #now) > 5 and nj.CompletedDt is null))
and (nj.RetryDt is null or nj.RetryDt < #now)
and not exists(
select * from dbo.NotificationJob
where Destination = nj.Destination
and nj.QueuedDt is not null and DATEDIFF(MINUTE, nj.QueuedDt, #now) < 6 and nj.CompletedDt is null)
)
update t
set t.QueuedDt = #now,
t.RetryDt = null
output
inserted.NotificationJobId,
inserted.Categories,
inserted.Source,
inserted.Destination,
inserted.Subject,
inserted.Message
from q as t
where t.dr <= #jobs
go
I don't have an answer off-hand, but there are ways you can learn more.
The code you wrote seems reasonable. Examining the actual query plan for the proc might help verify that SQL Server can generate a reasonable query plan, too.
If you don't have an index on NotificationJob.Destination that includes QueuedDt and CompletedDt, the not exists sub-query might acquire shared locks on the entire table. That would be scary for concurrency.
You can observe how the proc behaves when it acquires locks. One way is to turn on trace flag 1200 temporarily, call your proc, and then turn off the flag. This will generate a lot of information about what locks the proc is acquiring. The amount of info will severely affect performance, so don't use this flag in a production system.
dbcc traceon (1200, -1) -- print detailed information for every lock request. DO NOT DO THIS ON A PRODUCTION SYSTEM!
exec dbo.Notification_DequeueJob
dbcc traceoff (1200, -1) -- turn off the trace flag ASAP
I want to update multiple rows based on a SELECT sql query.
I want to do it ALL IN AN SQL SHELL!
Here is my select:
SELECT #myid := id, #mytitle := title
FROM event
WHERE pid>0 GROUP BY title
ORDER BY start;
Then, I want to do an update with this pseudocode:
foreach($mytitle as $t)
BEGIN
UPDATE event
SET pid=$myid
WHERE title=$t;
END
But I don't know how to ake a loop in SQL.
Maybe there's a way to make it in a single sql query?
I DON'T WANT ANY PHP!!! ONLY SQL SHELL CODE!!!
I want to update every rows with a pid with the id of the first occurence of an event. Start is a timestamp
I think this should do what you want, but if it doesn't (I'm not sure about joining a subquery in an UPDATE query) then you can use a temporary table instead.
UPDATE
event
JOIN (
SELECT
MIN(pid) AS minPID,
title
FROM
event
WHERE
pid > 0
GROUP BY
title
) AS findPIDsQuery ON event.title = findPIDsQuery.title
SET
event.pid = findPIDsQuery.minPID
Pure SQL doesn't really have "loops", per se: it's a set-based descriptive language. I believe the following update will do what you want (though your problem statements leaves much to be desired—we know nothing about the underlying schema).
update event t
set pid = ( select min(id)
from event x
where x.title = t.title
and x.pid > 0
group by x.title
having count(*) > 1
)
Cheers!