Related
I'm reading the MySQL docs for hours but I still cannot answer to myself a couple of pretty simple questions... :(
Here is my (simplified) scenario: I have two tables in a database: tablea and tableb, both tables use the InnoDB storage engine. tablea (which is my main table) has a PRIMARY index (id) with autoincrement. Now here is what I want to achieve and please keep in mind that the following business logic can be and will be run concurrently:
I start a transaction:
START TRANSACTION
BEGIN
then I check if an id exists in tablea if yes, I SELECT the row FOR UPDATE, let's call the id I am looking for myid :
SELECT `id` FROM `tablea` WHERE `id`='myid' FOR UPDATE;
if the above SELECT returns no rows, I simply ROLLBACK the transaction and exit from my function. In other words I'm done when myid is not present in tablea.
On the other hand when myid exists then first I need to update some values in tablea:
UPDATE `tablea` SET `somefield`='somevalue' WHERE `id`='myid';
then I need to check if myid also exists in tableb:
SELECT * FROM `tableb` WHERE `id`='myid' FOR UPDATE;
my first question is about the above SELECT statement: Is it okay to do another SELECT FOR UPDATE here (on tableb) ??? Or "FOR UPDATE" is not needed here when dealing with tableb, because I already started a transaction and also acquired a lock based on a row in tablea ??? Can someone please answer this?
The last SELECT statement above either returns a row from tableb (and locks that row for update) or it turns out that myid does not exist in tableb.
When myid is present in tableb then I just need to update some values in that row, it's simple:
UPDATE `tableb` SET `somefieldintableb`='somevaluefortableb' WHERE `id`='myid';
On the other hand when myid is not in tableb I need to insert it, and here comes my 2nd question: Should I lock tableb before I issue my INSERT INTO statement, like this:
LOCK TABLES `tableb` WRITE;
INSERT INTO `tableb` (`id`,`somefieldintableb`) VALUES ('myid','somevaluefortableb');
UNLOCK TABLES `tableb`;
and then finally, I do:
COMMIT
My goal is this: Since the above described function (with the MySQL transaction) will run in many instances in parallel, I want to prevent any of those instances updating the same row in either tablea or tableb at the same time. I also want to prevent double-insertion of myid into tableb, hence I thought about using LOCK TABLES when myid was not found in tableb.
So I have two questions: Should I do a SELECT ... FOR UPDATE within my already started transaction when I want to update tableb or locking tableb with SELECT ... FOR UPDATE is unnecessary, because holding the lock on tablea already "protects" tableb too from simultaneous UPDATEs in this case ??? Thanks to the way I started my transaction, I mean.
2nd question: When I need to INSERT a new row into tableb should I lock the whole table for that insertion? Or is that something that is totally unnecessary in this case? (Do I need LOCK TABLES tableb or not?)
I would appreciate if an expert can answer these two questions for me, because reading the various docs and examples online simply won't help me answering these questions. :(
I would do it this way:
BEGIN;
SELECT a.`id` AS a_id, b.`id` AS b_id
FROM `tablea` AS a LEFT OUTER JOIN `tableb` AS b ON a.id=b.id
WHERE a`id`='myid'
FOR UPDATE;
Now you have row locks on both tablea and tableb if rows exist. If the SELECT returns nothing, you know the id is not present in tablea. If the SELECT returns a row with a value for a_id, but a NULL for b_id, then you know it's present in tablea and not in tableb.
If the row is present in both tables, this locks rows in both tables simultaneously. If you do it in two steps, you might risk a race condition and deadlock.
Try the INSERT and use ON DUPLICATE KEY UPDATE:
INSERT INTO `tableb` (id, somefieldintableb) VALUES ('myid', 'somevaluefortableb')
ON DUPLICATE KEY UPDATE `somefieldintableb`='somevaluefortableb';
If the row with your desired id value is not present, this will insert it. If the row is present, this will update the row. And you're sure to have access to an existing row, because your SELECT FOR UPDATE locked it earlier.
Don't use table locks if you can avoid it. That's a sure way to create a bottleneck in your application.
Re your comments:
Yes, you can use extra join conditions for the date column.
You don't have to update all the columns when you use ON DUPLICATE KEY UPDATE. You can leave most of them alone if the row exists, and just update one, or a few, or whatever.
Also you can reference the value you tried to insert.
INSERT INTO `tableb` (id, date, col1, col2, col3, col4, col5, col6)
VALUES ('myid', $a_date, ?, ?, ?, ?, ?, ?)
ON DUPLICATE KEY UPDATE col4=VALUES(col4);
For more details, I recommend reading http://dev.mysql.com/doc/refman/5.7/en/insert-on-duplicate.html
I have two tables, in which I want the following logic to occur:
if (any row with a specific id exist in table1)
{
1. Delete the row from table1
2. insert some data into the table2 with the id as one of the values
3. return success somehow (for me to verify in java)
}
else
{
return fail
}
I'm sure this can be expressed in a clever manner, but I can not figure out how!
Can somebody help me translate this from my procedural way of thinking?
Greetings
Depending on the language (java?) you're using:
PreparedStatement stmt = conn.prepareStatement("select * from table1 where id = ?");
stmt.setInt(1, id);
ResultSet rs = stmt.executeQuery();
if (rs.first()) {
// same for insert and delete
insert into table2 (id, col2, col3) values(?, ?, ?);
delete from table1 where id = ?;
return true;
} else {
return false;
}
After some research I found this post. I slightly modified Mike's answer and ended up with this query:
START TRANSACTION;
INSERT INTO table1(col1, col2, col3, id)
SELECT * FROM (SELECT 'value1', 'value2,'valu3', 'id') AS tmp
WHERE EXISTS (
SELECT id FROM table2 WHERE id='123'
) LIMIT 1;
DELETE FROM table2 WHERE id='123';
COMMIT;
If id exists in table2, then the insert will be performed in table1 and deleted from table 2. Else, the insert will not be performed and the delete will not find any rows with the id 123 - so it will not be deleted. I also use START TRANSACTION and COMMIT to temporary disable AUTO COMMIT mode and thereby ensure that either all transactions occur, or none (in case of failure). I can then, in Java, check how many rows that where affected and see if the update where executed or not.
In MySQL I am trying to copy a row with an autoincrement column ID=1 and insert the data into same table as a new row with column ID=2.
How can I do this in a single query?
Use INSERT ... SELECT:
insert into your_table (c1, c2, ...)
select c1, c2, ...
from your_table
where id = 1
where c1, c2, ... are all the columns except id. If you want to explicitly insert with an id of 2 then include that in your INSERT column list and your SELECT:
insert into your_table (id, c1, c2, ...)
select 2, c1, c2, ...
from your_table
where id = 1
You'll have to take care of a possible duplicate id of 2 in the second case of course.
IMO, the best seems to use sql statements only to copy that row, while at the same time only referencing the columns you must and want to change.
CREATE TEMPORARY TABLE temp_table ENGINE=MEMORY
SELECT * FROM your_table WHERE id=1;
UPDATE temp_table SET id=0; /* Update other values at will. */
INSERT INTO your_table SELECT * FROM temp_table;
DROP TABLE temp_table;
See also av8n.com - How to Clone an SQL Record
Benefits:
The SQL statements 2 mention only the fields that need to be changed during the cloning process. They do not know about – or care about – other fields. The other fields just go along for the ride, unchanged. This makes the SQL statements easier to write, easier to read, easier to maintain, and more extensible.
Only ordinary MySQL statements are used. No other tools or programming languages are required.
A fully-correct record is inserted in your_table in one atomic operation.
Say the table is user(id, user_name, user_email).
You can use this query:
INSERT INTO user (SELECT NULL,user_name, user_email FROM user WHERE id = 1)
This helped and it supports a BLOB/TEXT columns.
CREATE TEMPORARY TABLE temp_table
AS
SELECT * FROM source_table WHERE id=2;
UPDATE temp_table SET id=NULL WHERE id=2;
INSERT INTO source_table SELECT * FROM temp_table;
DROP TEMPORARY TABLE temp_table;
USE source_table;
For a quick, clean solution that doesn't require you to name columns, you can use a prepared statement as described here:
https://stackoverflow.com/a/23964285/292677
If you need a complex solution so you can do this often, you can use this procedure:
DELIMITER $$
CREATE PROCEDURE `duplicateRows`(_schemaName text, _tableName text, _whereClause text, _omitColumns text)
SQL SECURITY INVOKER
BEGIN
SELECT IF(TRIM(_omitColumns) <> '', CONCAT('id', ',', TRIM(_omitColumns)), 'id') INTO #omitColumns;
SELECT GROUP_CONCAT(COLUMN_NAME) FROM information_schema.columns
WHERE table_schema = _schemaName AND table_name = _tableName AND FIND_IN_SET(COLUMN_NAME,#omitColumns) = 0 ORDER BY ORDINAL_POSITION INTO #columns;
SET #sql = CONCAT('INSERT INTO ', _tableName, '(', #columns, ')',
'SELECT ', #columns,
' FROM ', _schemaName, '.', _tableName, ' ', _whereClause);
PREPARE stmt1 FROM #sql;
EXECUTE stmt1;
END
You can run it with:
CALL duplicateRows('database', 'table', 'WHERE condition = optional', 'omit_columns_optional');
Examples
duplicateRows('acl', 'users', 'WHERE id = 200'); -- will duplicate the row for the user with id 200
duplicateRows('acl', 'users', 'WHERE id = 200', 'created_ts'); -- same as above but will not copy the created_ts column value
duplicateRows('acl', 'users', 'WHERE id = 200', 'created_ts,updated_ts'); -- same as above but also omits the updated_ts column
duplicateRows('acl', 'users'); -- will duplicate all records in the table
DISCLAIMER: This solution is only for someone who will be repeatedly duplicating rows in many tables, often. It could be dangerous in the hands of a rogue user.
If you're able to use MySQL Workbench, you can do this by right-clicking the row and selecting 'Copy row', and then right-clicking the empty row and selecting 'Paste row', and then changing the ID, and then clicking 'Apply'.
Copy the row:
Paste the copied row into the blank row:
Change the ID:
Apply:
insert into MyTable(field1, field2, id_backup)
select field1, field2, uniqueId from MyTable where uniqueId = #Id;
A lot of great answers here. Below is a sample of the stored procedure that I wrote to accomplish this task for a Web App that I am developing:
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON
-- Create Temporary Table
SELECT * INTO #tempTable FROM <YourTable> WHERE Id = Id
--To trigger the auto increment
UPDATE #tempTable SET Id = NULL
--Update new data row in #tempTable here!
--Insert duplicate row with modified data back into your table
INSERT INTO <YourTable> SELECT * FROM #tempTable
-- Drop Temporary Table
DROP TABLE #tempTable
You can also pass in '0' as the value for the column to auto-increment, the correct value will be used when the record is created. This is so much easier than temporary tables.
Source:
Copying rows in MySQL
(see the second comment, by TRiG, to the first solution, by Lore)
I tend to use a variation of what mu is too short posted:
INSERT INTO something_log
SELECT NULL, s.*
FROM something AS s
WHERE s.id = 1;
As long as the tables have identical fields (excepting the auto increment on the log table), then this works nicely.
Since I use stored procedures whenever possible (to make life easier on other programmers who aren't too familiar with databases), this solves the problem of having to go back and update procedures every time you add a new field to a table.
It also ensures that if you add new fields to a table they will start appearing in the log table immediately without having to update your database queries (unless of course you have some that set a field explicitly)
Warning: You will want to make sure to add any new fields to both tables at the same time so that the field order stays the same... otherwise you will start getting odd bugs. If you are the only one that writes database interfaces AND you are very careful then this works nicely. Otherwise, stick to naming all of your fields.
Note: On second thought, unless you are working on a solo project that you are sure won't have others working on it stick to listing all field names explicitly and update your log statements as your schema changes. This shortcut probably is not worth the long term headache it can cause... especially on a production system.
INSERT INTO `dbMyDataBase`.`tblMyTable`
(
`IdAutoincrement`,
`Column2`,
`Column3`,
`Column4`
)
SELECT
NULL,
`Column2`,
`Column3`,
'CustomValue' AS Column4
FROM `dbMyDataBase`.`tblMyTable`
WHERE `tblMyTable`.`Column2` = 'UniqueValueOfTheKey'
;
/* mySQL 5.6 */
Try this:
INSERT INTO test_table (SELECT null,txt FROM test_table)
Every time you run this query, This will insert all the rows again with new ids. values in your table and will increase exponentially.
I used a table with two columns i.e id and txt and id is auto increment.
I was looking for the same feature but I don't use MySQL. I wanted to copy ALL the fields except of course the primary key (id). This was a one shot query, not to be used in any script or code.
I found my way around with PL/SQL but I'm sure any other SQL IDE would do. I did a basic
SELECT *
FROM mytable
WHERE id=42;
Then export it to a SQL file where I could find the
INSERT INTO table (col1, col2, col3, ... , col42)
VALUES (1, 2, 3, ..., 42);
I just edited it and used it :
INSERT INTO table (col1, col2, col3, ... , col42)
VALUES (mysequence.nextval, 2, 3, ..., 42);
insert into your_table(col1,col2,col3) select col1+1,col2,col3 from your_table where col1=1;
Note:make sure that after increment the new value of col1 is not duplicate entry if col1 is primary key.
CREATE TEMPORARY TABLE IF NOT EXISTS `temp_table` LIKE source_table;
DELETE FROM `purchasing2` ;
INSERT INTO temp_table SELECT * FROM source_table where columnid = 2;
ALTER TABLE temp_table MODIFY id INT NOT NULL;
ALTER TABLE temp_table DROP PRIMARY KEY;
UPDATE temp_table SET id=NULL ;
INSERT INTO source_table SELECT * FROM temp_table;
DROP TEMPORARY TABLE IF EXISTS temp_table ;
Dump the row you want to sql and then use the generated SQL, less the ID column to import it back in.
Is there a way to do an insert under a count condition, something like:
INSERT INTO my_table (colname) VALUES('foo') IF COUNT(my_table) < 1
Basically I want to insert a single default record if the table is currently empty. I'm using mysql.
Use SELECT instead of VALUES to be able to expand the query with a WHERE clause.
EXISTS is a better & faster test than COUNT
INSERT INTO my_table (colname)
SELECT 'foo'
WHERE NOT EXISTS (SELECT * FROM my_table)
One way would be to place a unique key on a column. Then execute a REPLACE:
REPLACE [LOW_PRIORITY | DELAYED]
[INTO] tbl_name [(col_name,...)]
{VALUES | VALUE} ({expr | DEFAULT},...),(...),...
REPLACE works exactly like INSERT,
except that if an old row in the table
has the same value as a new row for a
PRIMARY KEY or a UNIQUE index, the old
row is deleted before the new row is
inserted
This is easier to read:
INSERT INTO my_table (colname)
SELECT 'foo' FROM DUAL
WHERE NOT EXISTS (SELECT * FROM my_table);
The lack of a VALUES is mitigated by the SELECT FROM DUAL which will provide the values. the FROM DUAL is not always required, but it doesn't hurt to include it for that weird configurations where it is required (like the installation of Percona I am using).
The NOT EXISTS is faster than doing a count which can be slow on a table with a large number of rows.
I have a table with columns record_id (auto inc), sender, sent_time and status.
In case there isn't any record of a particular sender, for example "sender1", I have to INSERT a new record otherwise I have to UPDATE the existing record which belongs to "user1".
So if there isn't any record already stored, I would execute
# record_id is AUTO_INCREMENT field
INSERT INTO messages (sender, sent_time, status)
VALUES (#sender, time, #status)
Otherwise I would execute UPDATE statement.
Anyway.. does anyone know how to combine these two statements in order to insert a new record if there isn't any record where the field sender value is "user1" otherwise update the existing record?
MySQL supports the insert-on-duplicate syntax, f.e.:
INSERT INTO table (key,col1) VALUES (1,2)
ON DUPLICATE KEY UPDATE col1 = 2;
If you have solid constraints on the table, then you can also use the REPLACE INTO for that. Here's a cite from MySQL:
REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted.
The syntax is basically the same as INSERT INTO, just replace INSERT by REPLACE.
INSERT INTO messages (sender, sent_time, status) VALUES (#sender, time, #status)
would then be
REPLACE INTO messages (sender, sent_time, status) VALUES (#sender, time, #status)
Note that this is a MySQL-specific command which doesn't occur in other DB's, so keep portability in mind.
As others have mentioned, you should use "insert...on duplicate key update", sometimes referred to as an "upsert". However, in your specific case you don't want to use a static value in the update, but rather the values you pass in to the values clause of the insert statement.
Specifically, I think you want to update two columns if the row already exists:
1) sent_time
2) status
In order to do this, you would use an "upsert" statement like this (using your example):
INSERT INTO messages (sender, sent_time, status)
VALUES (#sender, time, #status)
ON DUPLICATE KEY UPDATE
sent_time = values(sent_time),
status = values(status);
Check out "Insert on Duplicate Key Update".
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
UPDATE table SET c=c+1 WHERE a=1;
One options is using on duplicate update syntax
http://dev.mysql.com/doc/refman/5.1/en/insert-on-duplicate.html
Other options is doing select to figure out if record exists and then doind inser/update accordingly. Mind that if you're withing transaction select will not explicitly terminate the transaction so it's safe using it.
use merge statement :
merge into T1
using T2
on (T1.ID = T2.ID)
when matched
then update set
T1.Name = T2.Name
when not matched
then insert values (T2.ID,T2.Name);