MS SQL Server lock all records where column X = y - sql-server-2008

I'm working with a legacy table which I cannot change. It looks similar to:
CREATE TABLE foo
(
Id int IDENTITY(1,1) not null,
OwnerId int not null,
OwnerRecordId int not null,
SomeColumn varchra(50) not null,
CONSTRAINT ix_foo_OwnerId (OwnerId)
-- Ideally the following constraint would exist, but it doesn't. It is enforced
-- with code alone. There are currently duplicates, which should not
-- not exist, but they prevent creation of the unique index.
--CONSTRAINT ux_foo UNIQUE (OwnerId, OwnerRecordId)
)
OwnerRecordId is similar to an identity column within each OwnerId:
Id
OwnerId
OwnerRecordId
1
16
1
2
16
2
3
16
3
4
57
1
Now I would like to copy all records from ownerId 16 to ownerId 57. So OwnerId 57 would end up with 3 new records, and their OwnerRecordId would range from 2 - 4.
While this copying is taking places, other processes might be creating new records.
I thought about doing the following, but the sub-query seems slow:
insert into foo (OwnerId, SomeColumn, OwnerRecordId)
select
(57, SomeColumn, (select isnull(max(OwnerRecordId), 0) + 1
from foo where ownerId = 57)
from foo
where OwnerId = 16
Then I thought I could lock the table where OwnerId = 57. If I could do this I could lock those records, get the current maximum, and then use ROW_NUMBER in my select and add that to the MAX value I grabbed once.
Only, I can't seem to prevent other users from selecting from the table short of a table lock. Is there a way to lock records where colun OwnerId = 57? Doing so would prevent others from geting the current max(OwnerRecordId) + 1 value.
Perhas there is a better approach?
Certainly the unique index should be added, I can't do that at this point though.

The following code should hopefully do the correct amount of locking
insert into foo WITH (SERIALIZABLE)
(OwnerId, SomeColumn, OwnerRecordId)
select
57,
SomeColumn,
(select isnull(max(OwnerRecordId), 0) + 1
from foo with (SERIALIZABLE, UPDLOCK)
where ownerId = 57)
from foo
where OwnerId = 16;
SERIALIZABLE (which is a synonym for HOLDLOCK) will cause a range lock over all the rows where ownerId = 57, and UPDLOCK will cause that lock to be held until the end of the transaction.
You need an index on (OwnerId) with OwnerRecordId as another key column or as an INCLUDE, otherwise the whole table will get locked.
Do not fall into the trap of using XLOCK, it doesn't work unless you are actually modifying that table reference.
You say you have many different IDs to copy, in which case it would be more performant to do it in bulk.
Dump the list into a temp table (or a Table Valued Parameter), then do a joined update. Something like this
CREATE TABLE #tmp (SourceId int, TargetId int, primary key (SourceId, TargetId))
-- insert using statements or BULK INSERT or SqlBulkCopy etc
insert into foo WITH (SERIALIZABLE)
(OwnerId, SomeColumn, OwnerRecordId)
select
t.TargetId,
f.SomeColumn,
ISNULL(f2.mx, 0) + ROW_NUMBER() OVER (PARTITION BY f.OwnerId ORDER BY f.OwnerRecordId)
FROM foo f
JOIN #tmp t ON t.SourceId = f.OwnerId
LEFT JOIN (
SELECT f2.OwnerId, mx = MAX(f2.OwnerRecordId)
FROM foo f2 WITH (SERIALIZABLE, UPDLOCK)
GROUP BY f2.OwnerId
) f2 ON f2.OwnerId = t.TargetId;

Related

How to insert a record in mysql even the key is missing

I have a table like this:
id ctr
A 1
I'd like to insert a row of id A and update ctr to 2
The new value of ctr which is 2 is based on the previous value increased by 1.
The table would look like this.
id ctr
A 1
A 2
Now I'd like to insert another row of id B. But since B does not exist in the table it will have a counter initialized to 1. So the table will look like below.
id ctr
A 1
A 2
B 1
Is there a way to do this in one sql statement? (I prefer mysql)
Thanks a lot.
edited:
Actually the table is very simple. the actual table would look like this:
id amt balance ctr
A 100 100 1
A 150 250 2
B 50 50 1
As with the ctr the balance is amt+the previous balance.
It is all inserts and no updates.
Thanks.
You can create the primary key on id with ctr, and set the ctr auto-increment.
So every time you insert a record with id = A, the ctr will automatically increase.
`ctr` int(11) NOT NULL AUTO_INCREMENT,
...
PRIMARY KEY (`id`,`ctr`)
You can use COALESCE( MAX(ID)+1, 1 ) in order to initialize the INSERT Statement for a non-existing letter's ID as
SET #ctr = 'B';
INSERT INTO tab
SELECT COALESCE(MAX(ID)+1,1),#ctr FROM tab WHERE ctr = #ctr
Demo
even for the cases the unique key does not exist, otherwise using
INSERT INTO ...
SELECT ... FROM ...
ON DUPLICATE KEY UPDATE ...
would fit better to use.
You can express in one query as as:
insert into t (id, ctr)
select id, coalesce(max(ctr) + 1, 0)
from (select 'a' as id union all
select 'b' as id
) x left join
t
using (id)
group by id;

Update with Subquery never completes

I'm currently working on a project with a MySQL Db of more than 8 million rows. I have been provided with a part of it to test some queries on it. It has around 20 columns out of which 5 are of use to me. Namely: First_Name, Last_Name, Address_Line1, Address_Line2, Address_Line3, RefundID
I have to create a unique but random RefundID for each row, that is not the problem. The problem is to create same RefundID for those rows whose First_Name, Last_Name, Address_Line1, Address_Line2, Address_Line3 as same.
This is my first real work related to MySQL with such large row count. So far I have created these queries:
-- Creating Teporary Table --
CREATE temporary table tempT (SELECT tt.First_Name, count(tt.Address_Line1) as
a1, count(tt.Address_Line2) as a2, count(tt.Address_Line3) as a3, tt.RefundID
FROM `tempTable` tt GROUP BY First_Name HAVING a1 >= 2 AND a2 >= 2 AND a3 >= 2);
-- Updating Rows with First_Name from tempT --
UPDATE `tempTable` SET RefundID = FLOOR(RAND()*POW(10,11))
WHERE First_Name IN (SELECT First_Name FROM tempT WHERE First_Name is not NULL);
This update query keeps on running but never ends, tempT has more than 30K rows. This query will then be run on the main DB with more than 800K rows.
Can someone help me out with this?
Regards
The solutions that seem obvious to me....
Don't use a random value - use a hash:
UPDATE yourtable
SET refundid = MD5('some static salt', First_Name
, Last_Name, Address_Line1, Address_Line2, Address_Line3)
The problem is that if you are using an integer value for the refundId then there's a good chance of getting a collision (hint CONV(SUBSTR(MD5(...),1,16),16,10) to get a SIGNED BIGINT). But you didn't say what the type of the field was, nor how strict the 'unique' requirement was. It does carry out the update in a single pass though.
An alternate approach which creates a densely packed seguence of numbers is to create a temporary table with the unique values from the original table and a random value. Order by the random value and set a monotonically increasing refundId - then use this as a look up table or update the original table:
SELECT DISTINCT First_Name
, Last_Name, Address_Line1, Address_Line2, Address_Line3
INTO temptable
FROM yourtable;
set #counter=-1;
UPDATE temptable t SET t,refundId=(#counter:=#counter + 1)
ORDER BY r.randomvalue;
There are other solutions too - but the more efficient ones rely on having multiple copies of the data and/or using a procedural language.
Try using the following:
UPDATE `tempTable` x SET RefundID = FLOOR(RAND()*POW(10,11))
WHERE exists (SELECT 1 FROM tempT y WHERE First_Name is not NULL and x.First_Name=y.First_Name);
In MySQL, it is often more efficient to use join with update than to filter through the where clause using a subquery. The following might perform better:
UPDATE `tempTable` join
(SELECT distinct First_Name
FROM tempT
WHERE First_Name is not NULL
) fn
on temptable.First_Name = fn.First_Name
SET RefundID = FLOOR(RAND()*POW(10,11));

mysql insert with value equal to primary key + 1

I have an auto increment column ID, and for some situation I wanted the other column to be equal to the primary key + 1 value
ID | other
1 | 2
2 | 3
3 | 4
4 | 123 (some situation, it is not always plus 1)
How can I achieve this?
Here's what I have tried
INSERT INTO table (`ID`,`other`) VALUES ('',(SELECT MAX(ID)+1 FROM table))
But that returns an error
You can't specify target table 'table' for update in FROM clause
Try Below query:
ALTER TABLE dbo.table ADD
Column AS ([ID]+1)
GO
It will definitely work
Using a normal AUTO_INCREMENT column as id, I cannot think of a way to do this in MySQL. Triggers, which otherwise would have been an option, don't work well with AUTO_INCREMENT columns.
The only way I see is to do two commands for an INSERT;
INSERT INTO bop (value) VALUES ('These values should be 1 and 2');
UPDATE bop SET other = id+1 WHERE id = LAST_INSERT_ID();
An SQLfiddle to test with.
The closest I'm getting to what you're looking for is to generate sequences separately from AUTO_INCREMENT using a function, and use that instead to generate the table id;
DELIMITER //
CREATE TABLE bop (
id INT UNIQUE,
other INT,
value VARCHAR(64)
)//
CREATE TABLE bop_seq ( seq INT ) // -- Sequence table
INSERT INTO bop_seq VALUES (1) // -- Start value
CREATE FUNCTION bop_nextval() RETURNS int
BEGIN
SET #tmp = (SELECT seq FROM bop_seq FOR UPDATE);
UPDATE bop_seq SET seq = seq + 1;
RETURN #tmp;
END//
CREATE TRIGGER bop_auto BEFORE INSERT ON bop
FOR EACH ROW
SET NEW.id = bop_nextval(), NEW.other=NEW.id + 1;
//
That'd let you do inserts and have it autonumber like you want. The FOR UPDATE should keep the sequence transaction safe, but I've not load tested so you may want to do that.
Another SQLfiddle.
I solved this by updating 2 times the DB..
I wanted to do +1 from 19 till ..
UPDATE `table` SET `id`=`id`+101 WHERE id <= 19
UPDATE `table` SET `id`=`id`-100 WHERE id <= 119 AND id >= 101

Unique constraint on 2 columns, but in normal and reverse order

Couldn't find any answer, so I'm writing this open question. I'm curious if is there any possibility to create such UNIQUE constraint on 2 columns in SQL Server 2008 table, that "normal" and "reverse" duplicates wouldn't be allowed.
Example:
ID1 = 10, ID2 = 20 -- existing row
Trying to add a pair of values:
ID1 = 10, ID2 = 20 -- not allowed because of the UNIQUE key
ID1 = 20, ID2 = 10 -- allowed
The second row will be inserted (of course it's not a duplicate). And that's the issue. Can any key/constraint/whatever be set on a table to disallow above insertion? I.e. something using an expression instead of list of columns? For now I use a trigger which checks for such "duplicates", but I just wonder if is there any simpler solution.
Thanks,
Peter P.
CREATE TABLE dbo.test
(ID1 int , ID2 int ,
CONSTRAINT ID_UK UNIQUE(ID1,ID2),
)
GO
IF EXISTS (SELECT name FROM sysobjects
WHERE name = 'check_val' AND type = 'TR')
DROP TRIGGER check_val
GO
CREATE TRIGGER check_val
ON dbo.test
FOR INSERT, UPDATE
AS
if exists ( select i.ID1 ,i.ID2 from inserted i inner join dbo.test t
on t.ID2=i.ID1 and t.ID1=i.ID2 )
RAISERROR ('duplicate values',
16, 1)
ROLLBACK TRANSACTION
GO
insert dbo.test
select 10,20
union
select 20,10
I just needed the exact same thing (just a couple years later)
I decided to go with a check constraint that requires ID1 to be less than ID2
i know that's kinda of hacky, which is why i'm not convinced it's better than the trigger.
now when i insert data, ID1 has to be the smaller else the check constraint fails, that coupled with the unique constraint ensures only one instance of the combination exists.
I'm not anti-trigger, just prefer to not use them unless i really need to
--create your table
create table dbo.test
(
ID1 int not null,
ID2 int not null
)
go
--create a unique constraint the prevents duplicate of id1 and id2
create unique index test_ID1_ID2_uindex
on test (ID1, ID2)
go
--create a check that ensures id1 is less than id2, disallowing something like Id1=20, id2=10
ALTER TABLE dbo.Test
ADD CONSTRAINT CHK_Test_ID1_Less_ID2 CHECK (ID1<ID2);
GO
--this will fail
insert dbo.test
select 10,20
union
select 20,10

A multitude of the same id in an WHERE id IN () statement

I have a simple query that increases the value of a field by 1.
Now I used to loop over all id's and fire a query for each of them, but now that things are getting a bit resource heavy I wanted to optimize this. Normally I would just do
UPDATE table SET field = field + 1 WHERE id IN (all the ids here)
but now I have the problem that there are id's that occur twice (or more, I can't know that on forehand).
Is there a way to have the query run twice for id 4 if the query looks like this:
UPDATE table SET field = field + 1 WHERE id IN (1, 2, 3, 4, 4, 5)
Thanks,
lordstyx
Edit: sorry for not being clear enough.
The id here is an auto inc field, so it are all unique ID's. the id's that have to be updated are indirectly comming from users, so I can't predict which id is going to occur how often.
If there are the ID's (1, 2, 3, 4, 4, 5) I need the field of row with id 4 to be incremented with 2, and all the rest with 1.
If (1, 2, 3, 4, 4, 5) comes from a SELECT id ... query, then you can do something like this:
UPDATE yourTable
JOIN
( SELECT id
, COUNT(id) AS counter
....
GROUP BY id
) AS data
ON yourTable.id = data.id
SET yourTable.field = yourTable.field + data.counter
;
Since the input comes from users, perhaps you can manipulate it a bit. Change (1, 2, 3, 4, 4, 5) to (1), (2), (3), (4), (4), (5).
Then (having created a temporary table):
CREATE TABLE tempUpdate
( id INT )
;
Do the following procedure:
add the values in the temporary table,
run the update and
delete the values.
Code:
INSERT INTO TempUpdate
VALUES (1), (2), (3), (4), (4), (5)
;
UPDATE yourTable
JOIN
( SELECT id
, COUNT(id) AS counter
FROM TempUpdate
GROUP BY id
) AS data
ON yourTable.id = data.id
SET yourTable.field = yourTable.field + data.counter
;
DELETE FROM TempUpdate
;
No. But you could perform something like
UPDATE table
SET field = field + (LENGTH(',1,2,3,4,4,5,') - LENGTH(REPLACE(',1,2,3,4,4,5,', CONCAT(',', id, ','), ''))) / LENGTH(CONCAT(',', id, ','))
WHERE id IN (1, 2, 3, 4, 4, 5)
if you need row with id = 4 specifically to be incremented twice
Here is solution you wanted, but I'm not sure this is what you need.
Let's say that your talbe is called test. You want to increase id. I've added a field idwas to easily show what was the id before the query:
CREATE TABLE `test` (
`id` int(10) unsigned NOT NULL auto_increment,
`idwas` int(8) unsigned default NULL,
PRIMARY KEY (`id`)
) ;
Let's fill it with data:
truncate table test;
insert into test(id) VALUES(1),(3),(15);
update test set idwas = id;
Now let's say that you have user input 1,3,5,3, so:
id 1 should be increased by 1
id 3 should be increased by 2
id 5 is missing, nothing to increase.
row with id 15 should not be changed because not in user input
We'll put the user input in a variable to be easier to use it:
SET #userInput = '1,3,5,3';
then do the magic:
SET #helperTable = CONCAT(
'SELECT us.id, count(us.id) as i FROM ',
'(SELECT ',REPLACE(#userInput, ',',' AS `id` UNION ALL SELECT '),
') AS us GROUP BY us.id');
SET #stmtText = CONCAT(
' UPDATE ',
'(',#helperTable,') AS h INNER JOIN test as t ON t.id = h.id',
' SET t.id = t.id + h.i');
PREPARE stmt FROM #stmtText;
EXECUTE stmt;
And this is the result:
mysql> SELECT * FROM test;
+----+-------+
| id | idwas |
+----+-------+
| 2 | 1 |
| 5 | 3 |
| 15 | 15 |
+----+-------+
3 rows in set (0.00 sec)
If it's reasonable, you could try doing a combination of what you had before and what you have now.
In whatever is creating this list, separate it into (depending on the language's constructs) some type of array. Follow this by sorting it,finding how many multiples of each there are, and doing whatever else you need to to get the following: an array with (increment-number => list of ids), so you do one query for each increment amount. Thus, your example becomes
UPDATE table SET field = field + 1 WHERE id IN (1, 2, 3, 5)
UPDATE table SET field = field + 2 WHERE id IN (4)
In php, for example, I would take the array, sort the array, use the content of the array as the keys for another array of the form (id => count), and then fold that over into the (count => list of ids) array.
It's not that efficient, but is definitely better than one query per id. It's also probably better than using iteration and string manipulation in SQL. Unless you're forced to use SQL to do everything (which it sounds like you're not), I wouldn't use it to do everything, when it's overly awkward to do so.
You could use the following:
create temporary table temp1 (id integer);
insert into temp1 (id) values (1),(2),(3),(4),(4),(5);
update your_table set your_field = your_field + (select count(*) from temp1 where id = your_table.id)
This solution requires you to format the id list like (1),(2),(3),(4),(4),(5) but I don't think that is a problem, right?
This worked on my test database, hope it works for you too!
Regards,
Arthur