Efficiently Update MySQL Records - mysql

Example Data:
PostID DateTime DataTimeProper UserID UserName IPAddress
1234567.page#00008912 07/25/2013 14:50:21 NULL 00000001 TestUser 127.0.0.1
2468012.page#04208002 07/28/2013 18:42:13 NULL 03209827 BobTest 127.0.0.2
I'm looking for the most efficient way to update every record in a table (millions) where the DateTimeProper column IS NULL with the value being inserted coming from a str_to_date of the DateTime column.
SELECT STR_TO_DATE(`DateTime`,'%m/%e/%Y %H:%i:%s');

It's simply
UPDATE yourTable SET DateTimeProper = STR_TO_DATE(`DateTime`,'%m/%e/%Y %H:%i:%s')
WHERE DateTimeProper IS NULL;
It takes as long as it takes and it's a one time operation anyway, right?
If you really insist, you can make yourself the trouble to update in chunks, like (assuming you have an auto_increment column named id or something)
UPDATE yourTable SET DateTimeProper = STR_TO_DATE(`DateTime`,'%m/%e/%Y %H:%i:%s')
WHERE DateTimeProper IS NULL
AND id BETWEEN 0 AND 10000;
and then
UPDATE yourTable SET DateTimeProper = STR_TO_DATE(`DateTime`,'%m/%e/%Y %H:%i:%s')
WHERE DateTimeProper IS NULL
AND id BETWEEN 10000 AND 20000;
and so on. Some people do that in the hope, that the chunks fit into memory, but in my opinion it's not worth the trouble. MySQL already does a good job at that.

Try this
UPDATE table SET DataTimeProper =
(SELECT STR_TO_DATE(DateTime,'%m/%e/%Y %H:%i:%s'))
WHERE DataTimeProper IS NULL
Although you should consider changing your DateTime column to be of type DATETIME instead of string, as this will make it more efficient. Especially as you're dealing with tables of multi-million rows.

Related

How to update timestamp values in column by adding a specific time (seconds) to the existing timestamp using mysql?

I am using mysql and pma. I have a table mytable and a column time, storing ~17K individual values, i.e. timestamps (integers).
I need to update each by adding 962758 to each timestamp. What does the SQL command for that look like?
SELECT (*) FROM `mytable` t1
UPDATE `mytable` SET time = + 962758
PROFIT? :)
Would you need a SELECT statement for that or does it work with UPDATE only?
I cant use php for that in this case.
Considering that it's TIMESTAMP datatype, you can say
UPDATE `mytable` SET time = time + INTERVAL 962758 seconds;
Per your comment, since it's of INT type; you can just do the addition likewise you are already doing.
UPDATE `mytable` SET `time` = `time` + 962758;
If the data is stored as a datetime value, then you simply can use:
select timestampadd(second, 962758, time)
If the value is a unix timestamp, then it is already in seconds, and you can just add 962758.
According to your comment the field is int
UPDATE `mytable`
SET time = time + 962758;
And no need select

Setting TIME field to NULL with Zend_Db

I came across what looks like an odd issue with either Zend_Db or PHP's PDO MySQL driver, that perhaps stems from my lack of knowledge of those two.
Let's assume I have a MySQL Table with a NULLable TIME field. If I run a query like this in MySQL client:
UPDATE mytable SET mytime = NULL WHERE id = 1;
Everything works as expected and mytime field will hold NULL as value after this query.
However, if I run the exact same query in PHP through the Zend_Db_Adapter, the mytime field is set to '0:0:0' after such query:
$db->getConnection()->exec('UPDATE mytable SET mytime = NULL WHERE id = 1');
How do I set that TIME field to NULL?
I'm using PHP5.3 with PDO MySQL driver, Zend Framework 1.11 and MySQL 5.1.
What you have should work, i.e.:
$db->getConnection()->exec('UPDATE mytable SET mytime = NULL WHERE id = 1');
That should work. I tested it.
Caveat
If the data type time is NOT NULL, then trying to set it to NULL will cause the value NULL to be defaulted to 00:00:00, which may be unexpected behaviour e.g.:
CREATE TABLE `test` (
`time` datetime NOT NULL
);
Trying to insert or update the time field above with NULL will cause the value 00:00:00 to be inserted.
This is similar for date, datetime, and a few other data types. e.g. Trying to set a data type datetime which is NOT NULL to NULL will default its value to 0000-00-00 00:00:00.
NOTE: Mysql will not throw an error when you try to set a NULL value to a NUT NULL data type, you can change this behaviour by setting MySQL's SQL_MODE to STRICT_ALL_TABLES: see this stackoverflow question.
The Fix
Change the field to allow NULL and it should be fine:
CREATE TABLE `test` (
`time` datetime DEFAULT NULL
);
Now the time field can be set to NULL.
use new Zend_Db_Expr('NULL') instead of NULL :
$zend_null = new Zend_Db_Expr('NULL');
$db->getConnection()->exec("UPDATE mytable SET mytime = $zend_null WHERE id = 1");

Generate GUID in MySQL for existing Data?

I've just imported a bunch of data to a MySQL table and I have a column "GUID" that I want to basically fill down all existing rows with new and unique random GUID's.
How do I do this in MySQL ?
I tried
UPDATE db.tablename
SET columnID = UUID()
where columnID is not null
And just get every field the same
I had a need to add a guid primary key column in an existing table and populate it with unique GUID's and this update query with inner select worked for me:
UPDATE sri_issued_quiz SET quiz_id=(SELECT uuid());
So simple :-)
I'm not sure if it's the easiest way, but it works. The idea is to create a trigger that does all work for you, then, to execute a query that updates your table, and finally to drop this trigger:
delimiter //
create trigger beforeYourTableUpdate BEFORE UPDATE on YourTable
FOR EACH ROW
BEGIN
SET new.guid_column := (SELECT UUID());
END
//
Then execute
UPDATE YourTable set guid_column = (SELECT UUID());
And DROP TRIGGER beforeYourTableUpdate;
UPDATE
Another solution that doesn't use triggers, but requires primary key or unique index :
UPDATE YourTable,
INNER JOIN (SELECT unique_col, UUID() as new_id FROM YourTable) new_data
ON (new_data.unique_col = YourTable.unique_col)
SET guid_column = new_data.new_id
UPDATE once again:
It seems that your original query should also work (maybe you don't need WHERE columnID is not null, so all my fancy code is not needed.
The approved solution does create unique IDs but on first glance they look identical, only the first few characters differ.
If you want visibly different keys, try this:
update CityPopCountry set id = (select md5(UUID()));
MySQL [imran#lenovo] {world}> select city, id from CityPopCountry limit 10;
+------------------------+----------------------------------+
| city | id |
+------------------------+----------------------------------+
| A Coruña (La Coruña) | c9f294a986a1a14f0fe68467769feec7 |
| Aachen | d6172223a472bdc5f25871427ba64e46 |
| Aalborg | 8d11bc300f203eb9cb7da7cb9204aa8f |
| Aba | 98aeeec8aa81a4064113764864114a99 |
| Abadan | 7aafe6bfe44b338f99021cbd24096302 |
| Abaetetuba | 9dd331c21b983c3a68d00ef6e5852bb5 |
| Abakan | e2206290ce91574bc26d0443ef50fc05 |
| Abbotsford | 50ca17be25d1d5c2ac6760e179b7fd15 |
| Abeokuta | ab026fa6238e2ab7ee0d76a1351f116f |
| Aberdeen | d85eef763393862e5fe318ca652eb16d |
+------------------------+----------------------------------+
I'm using MySQL Server version: 5.5.40-0+wheezy1 (Debian)
select #i:=uuid();
update some_table set guid = (#i:=uuid());
Just a minor addition to make as I ended up with a weird result when trying to modify the UUIDs as they were generated. I found the answer by Rakesh to be the simplest that worked well, except in cases where you want to strip the dashes.
For reference:
UPDATE some_table SET some_field=(SELECT uuid());
This worked perfectly on its own. But when I tried this:
UPDATE some_table SET some_field=(REPLACE((SELECT uuid()), '-', ''));
Then all the resulting values were the same (not subtly different - I quadruple checked with a GROUP BY some_field query). Doesn't matter how I situated the parentheses, the same thing happens.
UPDATE some_table SET some_field=(REPLACE(SELECT uuid(), '-', ''));
It seems when surrounding the subquery to generate a UUID with REPLACE, it only runs the UUID query once, which probably makes perfect sense as an optimization to much smarter developers than I, but it didn't to me.
To resolve this, I just split it into two queries:
UPDATE some_table SET some_field=(SELECT uuid());
UPDATE some_table SET some_field=REPLACE(some_field, '-', '');
Simple solution, obviously, but hopefully this will save someone the time that I just lost.
Looks like a simple typo. Didn't you mean "...where columnId is null"?
UPDATE db.tablename
SET columnID = UUID()
where columnID is null
I faced mostly the same issue.
Im my case uuid is stored as BINARY(16) and has NOT NULL UNIQUE constraints.
And i faced with the issue when the same UUID was generated for every row, and UNIQUE constraint does not allow this. So this query does not work:
UNHEX(REPLACE(uuid(), '-', ''))
But for me it worked, when i used such a query with nested inner select:
UNHEX(REPLACE((SELECT uuid()), '-', ''))
Then is produced unique result for every entry.
MYsql
UPDATE tablename SET columnName = UUID()
oracle
UPDATE tablename SET columnName = SYS_GUID();
SQLSERVER
UPDATE tablename SET columnName = NEWID();;
UPDATE db.tablename SET columnID = (SELECT UUID()) where columnID is not null
// UID Format: 30B9BE365FF011EA8F4C125FC56F0F50
UPDATE `events` SET `evt_uid` = (SELECT UPPER(REPLACE(#i:=UUID(),'-','')));
// UID Format: c915ec5a-5ff0-11ea-8f4c-125fc56f0f50
UPDATE `events` SET `evt_uid` = (SELECT UUID());
// UID Format: C915EC5A-5FF0-11EA-8F4C-125FC56F0F50
UPDATE `events` SET `evt_uid` = (SELECT UPPER(#i:=UUID()));
I got this error when using mysql as sql_mode = "". After some testing, I decided that the problem was caused by this usage. When I tested on the default settings, I found that this problem was not there.
Note: Don't forget to refresh your connection after changing the mode.
SELECT CONCAT(SUBSTRING(REPLACE(UUID(),'-',''), 1, 5), SUBSTRING(UPPER(REPLACE(UUID(),'-','')), 4, 5), SUBSTRING('##$%(*&', FLOOR(RAND()*(1-8))+8, 1)) pass
I did this SELECT, five character lower case, five character upper case and one special character.

Update a field based off of the current value (in mysql)

I have a table similar to
CREATE TABLE `mytable` (
`ID` int(11) unsigned NOT NULL AUTO_INCREMENT,
`Time` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00',
KEY `time` (`Time`),
) ENGINE=MyISAM AUTO_INCREMENT=2373485 DEFAULT CHARSET=latin1
I had a weird issue with daylight savings time, and now I need to update rows with IDs 2370144 through 2373391 so that the Time values are six hours less than their current values.
I can select the affected rows with
SELECT * FROM mytable WHERE ID >= 2370144 AND ID <= 2373391
How do I update these entries so that the new timestamp is six hours less than the old value?
I think this will work
UPDATE mytable SET Time = date_sub(Time, INTERVAL 6 HOUR) WHERE id BETWEEN 2370144 AND 2373391;
UPDATE mytable
SET `Time` = (`Time` - INTERVAL 6 HOUR)
WHERE ID >= 2370144
AND ID <= 2373391
Expanding on this a little bit, when feasible I would typically run a SQL query to generate a .sql file that contains one update statement per row, then execute that sql file to update the rows. Since you are only updating about 3,000 rows this should be feasible for you.
This dump and load approach has a couple of benefits:
You can save the SQL script as an
audit record of what you changed.
You can include both the ID and the Time
value in the SQL script. That way if
you accidentally run the script more
than once you don't end up changing
the value to something incorrect. For
example, if you ran my original
update twice the values would end up
6 hours too low, but if you use the
dump-and-load approach and run the
script twice, the second time it
won't change the records because the
where clause will no longer match.
Here's an example of the dump-and-load approach:
select concat('update mytable set `Time` = ''',
`Time` - interval 6 hour,
''' where id = ',
id,
' and `Time` = ''',
`Time`,
''';') as sql_stmt
into outfile '/tmp/mytable.update.dstfix.20110315.sql'
from mytable
WHERE ID >= 2370144
AND ID <= 2373391;
\. /tmp/mytable.update.dstfix.20110315.sql

MySQL (DEFAULT + ON UPDATE) TIMESTAMPs

I've a table where I've two fields:
dateCreated
dateUpdated
I want both fields to hold timestamps, dateCreated to have a DEFAULT CURRENT TIMESTAMP flag and dateUpdated to have a ON UPDATE CURRENT TIMESTAMP behavior, however seems that I can't have more than one timestamps field types on the same table.
It would be really useful if the database could take care of this for me, can I circumvent this issue somehow?
YES WE CAN.
You can use a trigger to update the field, like so:
create trigger CreationTimestamp after insert on MyTable for each row
begin
update MyTable set CreateField = UpdateField where id = new.id
end
This way, you can set your auto-update field using the native MySQL method, and use this to capture the creation date.