I have a table in a MySQL database from which I want to select some data and insert into another table. The query is pretty straightforward
QUERY-1
Insert into ArchiveTable
select * from OriginalTable where ColumnName<utc_timestamp();
Now I want to delete the records from OriginalTable which were inserted in the ArchiveTable
QUERY-2
delete from OriginalTable
where ID in(select ID from OriginalTable where ColumnName<utc_timestamp();
The problem here is that, while QUERY-1 executes, there would be records which would fullfill the ColumnName
condition but wont be picked up by the query since it has already started executing.
Now on executing the delete statement(QUERY-2), those extra rows would be deleted from OriginalTable but I would have no record of them in the ArchiveTable since it was not picked up by the previous query.
In SQLServer this can be resolved by using the OUTPUT clause, but I am not sure what would be the best approach to handle this scenario.
Any help in this regard would be highly appreciated
Related
I would like to query a database table for some of it's oldest entries and update them with a second query afterwards.
But how can I prevent that another process (that does the same) will return the same rows by the SELECT query and the UPDATE part will modify the entries twice?
As far as I see a simple transaction cannot prevent this from happening.
Use the SELECT ... FOR UPDATE mechanism to do this (see http://dev.mysql.com/doc/refman/5.0/en/innodb-locking-reads.html)
I have a left join query that shows all the fields from a primary table (tblMarkers) and the values from a second table (tblLocations) where there is matching record.
tblLocations does not have a record for every id in tblMarkers
$query ="SELECT `tblMarkers`.*,`tblLocation`.*,`tblLocation`.`ID` AS `markerID`
FROM
`tblMarkers`
LEFT JOIN `tblLocation` ON `tblMarkers`.`ID` = `tblLocation`.`ID`
WHERE
`tblMarkers`.`ID` = $id";
I am comfortable with using UPDATE to update the tblMarkers fields but how do I update or INSERT a record into tblLocations if the record does not exist yet in tblLocations.
Also, how do I lock the record I ma working on to prevent someone else from doing an update at the same time?
Can I also use UPDATE tblMarkers * or do I have to list every field in the UPDATE statement?
Unfortunately you might have to implement some validation in your outside script. There is an IF statement in SQL, but I'm not sure if you can trigger different commands based on it's outcome.
Locking
In terms of locking, you have 2 options. for MyISAM tables, you can only lock the entire table using http://dev.mysql.com/doc/refman/5.0/en/lock-tables.html
LOCK TABLE users;
For InnoDB tables, there is no explicit 'lock' for single rows, however you can use transactions, to get exclusive rights during the operation. http://dev.mysql.com/doc/refman/5.0/en/innodb-locks-set.html
Update
There might be some shorthand notation, but I think you have to list every field in your query. Alternatively, you can always read the entire row, delete it and insert again using shorthand INSERT query. It all depends on how many fields you've got.
I have N number of records in a table ,i wanna move all records from one table other say old table as table1 and new as table2 .I have a query with sub query to select the records from the table for insertion .
Assuming as 10000 records While inserting on 6000 record it gets some exception in it , it got to an end,but still the table2 is empty , Here i wanna know that is the 5999 records where it would have been inserted in a databse ?
Thanks in advance ,,
if its unworthy to answer or any cause let me know the reason to down vote i can improve it
I have a query with sub query to select the records from the table for insertion
I assume you have some INESRT INTO table2(<COLUMN LIST>) SELECT <COLUMN LIST> FROM table1 WHERE ... that you are running to move the records.
If so, the INSERT statement is run as part of a transaction and will be committed only if the statement is executed successfully, i.e. if it is able to INSERT all the records returned by that SELECT query. Otherwise, the transaction gets rolled back and no records will be inserted.
Here i wanna know that is the 5999 records where it would have been inserted in a database?
These records would have been inserted into the worktable in tmp location while executing the INSERT statement. It would have been committed to the main table if everything had gone well.
Does anyone know what would be more efficient and use less resources:
Method 1-- Using a single SELECT statement to get data from one table and then iterating through it to execute multiple UPDATEs on another table. E.G. (pseudo-code, execute() runs query):
Query1_resultset = execute("SELECT item_id, sum(views) as view_count FROM tableA WHERE condition=1");
while(Query1_resultset as row) {
execute("UPDATE tableB SET view_count=row.view_count WHERE id=row.item_id");
}
Method 2-- Use a single INSERT.. ON DUPLICATE KEY UPDATE statement with a nested SELECT statement. E.G.:
INSERT INTO tableB (id, view_count) SELECT item_id, SUM(views) as view_count FROM tableA WHERE condition=1 ON DUPLICATE KEY UPDATE view_count=VALUES(view_count);
Note: ID on tableB is a primary key. There actually won't be any INSERTS because I know the key will exist. So it's all UPDATEs. Just using this statement to pass in a single query rather than multiple.
I'm really curious as to why either would be more efficient. Is it the number of queries that determines how quickly it will run? Where is the bottleneck?
I'm looking for something that will scale (the number of rows being updated grows daily).
Any ideas?
Thanks
It depens on your update/insert ratio. If you have lots of inserts and only a couple of updates than the INSERT ... ON DUPLICATE KEY UPDATE statement will be faster.
If you mainly have updates, than you would be better off with an UPDATE statement and an insert as fallback (if there was no update). You could use the multi table update clause to do it with a single update instead of a select followed by an update by the way. If you're doing both a SELECT and an UPDATE than the INSERT will definately be faster.
I think INSERT.. ON DUPLICATE KEY UPDATE is more efficient (otherwise, it wouldn't make much sense to add such an extension). By the way, your first example is not exactly the same as the second one - you neither use transactions nor you lock the table, so it's possible that the record returned by SELECT will not exist by the time you execute UPDATE.
Is there a way to remove all repeat rows from a MySQL database?
A couple of years ago, someone requested a way to delete duplicates. Subselects make it possible with a query like this in MySQL 4.1:
DELETE FROM some_table WHERE primaryKey NOT IN
(SELECT MIN(primaryKey) FROM some_table GROUP BY some_column)
Of course, you can use MAX(primaryKey) as well if you want to keep the newest record with the duplicate value instead of the oldest record with the duplicate value.
To understand how this works, look at the output of this query:
SELECT some_column, MIN(primaryKey) FROM some_table GROUP BY some_column
As you can see, this query returns the primary key for the first record containing each value of some_column. Logically, then, any key value NOT found in this result set must be a duplicate, and therefore it should be deleted.
These questions / answers might interest you :
How to delete duplicate records in mysql database?
How to delete Duplicates in MySQL table.
And idea that's often used when you are working with a big table is to :
Create a new table
Insert into that table the unique records (i.e. only one version of the duplicates in the original table, generally using a select distinct)
and use that new table in your application ; or drop the old table and rename the new one to the old name.
Good thing with this principle is you have the possibility to verify what's in the new table before dropping the old one -- always nice to check that sort of thing ^^