Can MySQL convert a stored UTC time to local time-zoned time directly in a normal select statement?
Let's say you have some data with a timestamp (UTC).
CREATE TABLE `SomeDateTable` (
`id` int(11) NOT NULL auto_increment,
`value` float NOT NULL default '0',
`date` datetime NOT NULL default '0000-00-00 00:00:00',
PRIMARY KEY (`id`)
)
Then when I
"select value, date from SomeDateTable";
I of course get all the dates as in their stored UTC form.
But let's say that I would like to have them in another timezone (with DST),
can I then add some magic to the select query so that I get all the dates back in the selected timezone?
"select value, TIMEZONE(date, "Europe/Berlin") from SomeDateTable";
Or must I do this in some other layer on top, like in some php code?
(it seems to be how most people have solved this problem).
If your MySQL installation allows you to use CONVERT_TZ it is a very clean solution,
this example shows how to use it.
SELECT CONVERT_TZ( '2010-01-01 12:00', 'UTC', 'Europe/Stockholm' )
However I don't know if this is a good way since some MySQL installation is missing this function, use with care.
Yup, there's the convert_tz function.
For those unable to configure the mysql environment (e.g. due to lack of SUPER access) to use human-friendly timezone names like "America/Denver" or "GMT" you can also use the function with numeric offsets like this:
CONVERT_TZ(date,'+00:00','-07:00')
One can easily use
CONVERT_TZ(your_timestamp_column_name, 'UTC', 'your_desired_timezone_name')
For example:
CONVERT_TZ(timeperiod, 'UTC', 'Asia/Karachi')
Plus this can also be used in WHERE statement and to compare timestamp i would use the following in Where clause:
WHERE CONVERT_TZ(timeperiod, 'UTC', '{$this->timezone}') NOT BETWEEN {$timeperiods['today_start']} AND {$timeperiods['today_end']}
select convert_tz(now(),##session.time_zone,'+03:00')
For get the time only use:
time(convert_tz(now(),##session.time_zone,'+03:00'))
1. Correctly setup your server:
On server, su to root and do this:
# mysql_tzinfo_to_sql /usr/share/zoneinfo | mysql mysql
(Note that the command at the end is of course mysql , and, you're sending it to a table which happens to have the same name: mysql.)
Next, you can now # ls /usr/share/zoneinfo .
Use that command to see all the time zone info on ubuntu or almost any unixish server.
(BTW that's the convenient way to find the exact official name of some time zone.)
2. It's then trivial in mysql:
For example
mysql> select ts, CONVERT_TZ(ts, 'UTC', 'Pacific/Tahiti') from example_table ;
+---------------------+-----------------------------------------+
| ts | CONVERT_TZ(ts, 'UTC', 'Pacific/Tahiti') |
+---------------------+-----------------------------------------+
| 2020-10-20 16:59:57 | 2020-10-20 06:59:57 |
| 2020-10-20 17:02:59 | 2020-10-20 07:02:59 |
| 2020-10-20 17:30:08 | 2020-10-20 07:30:08 |
| 2020-10-20 18:36:29 | 2020-10-20 08:36:29 |
| 2020-10-20 18:37:20 | 2020-10-20 08:37:20 |
| 2020-10-20 18:37:20 | 2020-10-20 08:37:20 |
| 2020-10-20 19:00:18 | 2020-10-20 09:00:18 |
+---------------------+-----------------------------------------+
I propose to use
SET time_zone = 'proper timezone';
being done once right after connect to database. and after this all timestamps will be converted automatically when selecting them.
Related
So, in Laravel's app.php I have the following timezone set:
'timezone' => 'America/Denver',
In MySQL settings I've got the same timezone. When I run select now() I get the current Denver time.
However, when I create a record in any table in the database, the created_at field (with default value set to CURRENT_TIMESTAMP) somehow ends up 5 hours ahead of Denver.
I believe it's somehow defaulting to UTC time, but I am not sure. All online resources I've found related to this issue claim that setting the timezone in Laravel should do the trick.
What else can I do to make sure I get the correct timezone saved in CURRENT_TIMESTAMP?
I don't think server-wide PHP settings should have precedent over what's set in MySQL or in Laravel in this matter, but I have still gone ahead and tried editing the timezone in php.ini to America/Denver and no luck. It was previously commented out (not set to UTC).
Use
SET SESSION time_zone = 'America/Denver';
In a raw query (DB::select(DB::raw("SET SESSION time_zone = 'America/Denver'")) before inserting and updating.
Test case
CREATE TABLE test (
id INT
, created_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO test (id) VALUES(1);
SET SESSION time_zone = 'America/Denver';
INSERT INTO test (id) VALUES(2);
Possible results
| id | created_at |
| --- | ------------------- |
| 1 | 2019-03-04 13:57:31 |
| 2 | 2019-03-04 06:57:31 |
see demo
Eloquent creates a new Carbon object when it sets the timestamp for created_at, it doesn’t use MySQL’s default. This should use date_default_timezone_set, which Laravel is setting.
Rather obvious answer is: have you tried clearing your config cache?
php artisan config:clear
As an aside it is generally advisable to always use UTC across everything, and only convert it to a local timezone at the last possible moment.
From Carbon:
// PS: we recommend you to work with UTC as default timezone and only use
// other timezones (such as the user timezone) on display
I have an old PHP system, using MySQL 5.5.47 as DB.
The guys who have created the system, have taken a strange decision.
In some cases, they saved a date value without day - for example '2018-01-00'. The field type is DATE.
A lot of queries use where clause like this: UNIX_TIMESTAMP(DATE(<DATE>)) BETWEEN 1514757600 AND 1546207210, where <DATE> is a column which contains records like '2018-01-00', '2018-02-00', etc.
The two timestamps represent dates 2018-01-01 and 2018-12-31.
On production, this type of queries run without issue.
On my local machine, they do not return any results.
What I found is if I run the command: SELECT UNIX_TIMESTAMP( DATE( '2018-01-00' ) ) on production the result is 1514757600, but on my local machine it returns 0.
I'm using a Docker compose to reproduce the production as close as possible. Initially, I have used MySQL 5.6 for local development when I hit this issue, I tried with MySQL 5.5.62, but the result is same.
Does anyone know how I can set up my local MySQL to work as the production one?
Query on production:
mysql> SELECT DATE('2018-01-00'), UNIX_TIMESTAMP(DATE('2018-01-00')), UNIX_TIMESTAMP('2018-01-00');
+--------------------+------------------------------------+------------------------------+
| DATE('2018-01-00') | UNIX_TIMESTAMP(DATE('2018-01-00')) | UNIX_TIMESTAMP('2018-01-00') |
+--------------------+------------------------------------+------------------------------+
| 2018-01-00 | 1514757600 | 0 |
+--------------------+------------------------------------+------------------------------+
Query on local:
mysql> SELECT DATE('2018-01-00'), UNIX_TIMESTAMP(DATE('2018-01-00')), UNIX_TIMESTAMP('2018-01-00');
+--------------------+------------------------------------+------------------------------+
| DATE('2018-01-00') | UNIX_TIMESTAMP(DATE('2018-01-00')) | UNIX_TIMESTAMP('2018-01-00') |
+--------------------+------------------------------------+------------------------------+
| 2018-01-00 | 0 | 0 |
+--------------------+------------------------------------+------------------------------+
It turns out to be a bug in Mysql prior 5.5.48. In the release notes of 5.5.48, there is a statement about fixing a bug related to the unix_timestamp function.
https://dev.mysql.com/doc/relnotes/mysql/5.5/en/news-5-5-48.html
When an invalid date was supplied to the UNIX_TIMESTAMP() function using the STR_TO_DATE() function, no check was performed before converting it to a timestamp value. (Bug #21564557)
Our database uses '0000-00-00 00:00:00' as the default value for many datetime and timestamp fields. MySQL has apparently decided that they only want us to use a valid date or null for these types of fields.
However, the '0000-00-00 00:00:00' values used to be acceptable and our code checks for this value. When I setup a new server, I edit the /etc/mysql/mysql.conf.d/mysqld.cnf file and add one line to the [mysqld] section.
sql_mode=NO_ENGINE_SUBSTITUTION
Today I have attempted to setup a new server. I added the sql_mode=NO_ENGINE_SUBSTITUTION to the MySQL configuration and restarted the mysql service. However, this new server only gets errors.
UPDATE example_table SET active = 1 WHERE example_table_id = 1;
ERROR 1292 (22007): Incorrect datetime value: '0000-00-00 00:00:00' for column 'my_date_field' at row 1
I could of course update the database to have NULL values or a "valid" default date such as '1970-01-01 00:00:01', but this would break existing code that checks the data for '0000-00-00 00:00:00'.
Additional Example Information:
Server type: MySQL
Server version: 5.7.22-0ubuntu0.16.04.1 - (Ubuntu)
Protocol version: 10
innodb_version: 5.7.22
SELECT my_date_field FROM example_table WHERE active = 1;
+---------------------+
| my_date_field |
+---------------------+
| 0000-00-00 00:00:00 |
+---------------------+
DESC example_table;
+------------------------------------+--------------+------+-----+---------------------+
| Field | Type | Null | Key | Default |
+------------------------------------+--------------+------+-----+---------------------+
| my_date_field | datetime | NO | | 0000-00-00 00:00:00 |
| active | tinyint(2) | NO | | 1 |
+------------------------------------+--------------+------+-----+---------------------+
I've got other machines on the same version of MySQL working with just sql_mode=NO_ENGINE_SUBSTITUTION and I'm almost to the point of just imaging one of those for this new machine, until such time as the code is updated to look for a valid date or null instead of '0000-00-00 00:00:00'.
The NO_ENGINE_SUBSTITUTION SQL mode doesn't have anything to do with dates. It only did the trick for you because by setting it you also removed the server's default mode, which in modern versions is something on the line of TRADITIONAL, which is a combination mode that, among others, includes NO_ZERO_IN_DATE and NO_ZERO_DATE.
You possibly just want to set a legacy mode for this application at session level, e.g.:
SET ##SESSION.sql_mode = '';
date = Date.today
query = "insert into custom_reports(name,description,created_at) values(#{report_name}, #{report_content}, #{date})"
ActiveRecord::Base.connection.execute(query);
it inserts
0000-00-00 00:00:00
So thought of going the mysql timestamp way. Is it possible to do?
This should work
query = "insert into custom_reports(name,description,created_at) values('#{report_name}', '#{report_content}', '#{Time.now}')"
Try, changing
date = Date.today
to
date = Time.now
Apart from the above I tend to not use variables that may or may not accidentally be a keyword (especially when I am not sure), so instead of naming variable as date, I would use d
Why do you write that SQL manually? Thats is more complicated and insecure, because you do not escape the strings in the query. On the other side ActiveRecord will take care of the created_at column.
I would prefer to have a basic model for that:
class CustomReport < ActiveRecord::Base
end
CustomRepor.create!(name: report_name, description: report_content)
Mysql "Date" type columns should use
Date.now
Mysql "Time" type columns should use
Time.now
Mysql "DateTime" type columns have to use
DateTime.now
I had to find out what is the data format on my created_at field
rails db
select created_at from users;
+---------------------+
| created_at |
+---------------------+
| 2017-12-12 00:51:19 |
| 2017-12-12 00:51:20 |
| 2017-12-12 00:51:22 |
| 2017-12-12 00:51:23 |
+---------------------+
Then I have to run the raw query according to the format I'm using.
date = Time.now.strftime("%Y-%m-%d %H:%M:%S")
query = "insert into custom_reports(name,description,created_at) values(#{report_name}, #{report_content}, #{date})"
ActiveRecord::Base.connection.execute(query);
Try Changing the timestamp to date time format:
Time.at(1335437221)
convert
2012-04-26 12:47:01 +0200
Delphi XE2 and MySql.
My previous question led to the recommendation that I should be using MySql's native TIMESTAMP datatype to store date/time.
Unfornately, I can't seem to find any coding examples, and I am getting weird results.
Given this table:
mysql> describe test_runs;
+------------------+-------------+------+-----+---------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------+-------------+------+-----+---------------------+-------+
| start_time_stamp | timestamp | NO | PRI | 0000-00-00 00:00:00 | |
| end_time_stamp | timestamp | NO | | 0000-00-00 00:00:00 | |
| description | varchar(64) | NO | | NULL | |
+------------------+-------------+------+-----+---------------------+-------+
3 rows in set (0.02 sec)
I woudl like to :
declare a variable into which I can store the result of SELECT CURRENT_TIMESTAMP - what type should it be? TSQLTimeStamp?
insert a row at test start which has start_time_stamp = the variable above
and end_time_stamp = some "NULL" value ... "0000-00-00 00:00:00"? Can I use that directly, or do I need to declare a TSQLTimeStamp and set each field to zero? (there doesn't seem to be a TSQLTimeStamp.Clear; - it's a structure, not a class
upadte the end_time_stamp when the test completes
calcuate the test duration
Can somene please point me at a URL with some Delphi code whcich I can study to see how to do this sort of thing? GINMF.
I don't know why you want to hassle around with that TIMESTAMP and why you want to retrieve the CURRENT_TIMESTAMP just to put it back.
And as already stated, it is not a good advice to use a TIMESTAMP field as PRIMARY KEY.
So my suggestion is to use this TABLE SCHEMA
CREATE TABLE `test_runs` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`start_time_stamp` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
`end_time_stamp` timestamp NULL DEFAULT NULL,
`description` varchar(64) NOT NULL,
PRIMARY KEY (`id`)
);
Starting a test run is handled by
INSERT INTO test_runs ( description ) VALUES ( :description );
SELECT LAST_INSERT_ID() AS id;
and to finalize the record you simply call
UPDATE test_runs SET end_time_stamp = CURRENT_TIMESTAMP WHERE id = :id
just declare a TSQLQuery (or the correct component for the data access layer of your choice), attach it to a valid connection and populate it's SQL property with:
select * from test_runs;
double click on the query to launch it's fields editor and select add all fields from the contextual menu of that editor.
It will create the correct field type, according to the data access layer and driver you're using to access your data.
Once that's done, if you need to use the value in code, usually you do it by using the AsDateTime property of the field, so you just use a plain TDateTime Delphi type and let the database access layer deal with the specific database details to store that field.
For example, if your query object is named qTest and the table field is named start_time_stamp, your Delhi variable associated with that persistent field will be named qTeststart_time_stamp, so you can do something like this:
var
StartTS: TDateTime;
begin
qTest.Open;
StartTS := qTeststart_time_stamp.AsDateTime;
ShowMessage('start date is ' + DateTimeToStr(StartTS));
end;
If you use dbExpress and are new to it, read A Guide to Using dbExpress in Delphi database applications
I don't know about MySQL, but if the TField subclass generated is a TSQLTimeStampField, you will need to use the type and functions in the SqlTimSt unit (Data.SqlTimSt for XE2+).
You want to declare the local variables as TSQLTimeStamp
uses Data.SQLTmSt....;
....
var
StartTS: TSQLTimeStamp;
EndTS: TSQLTimeStamp;
begin
StartTS := qTeststart_time_stamp.AsSQLTimeStamp;
SQLTmSt also includes functions to convert to and from TSQLTimeStamp, e.g. SQLTimeStampToDateTime and DateTimeToSQLTimeStamp.
P.S. I tend to agree that using a timestamp as a primary key is likely to cause problems. I would tend to use a auto incrementing surrogate key as Sir Rufo suggests.