I have the following query from Group OHLC-Stockmarket Data into multiple timeframes - Mysql.
SELECT
FLOOR(MIN(`timestamp`)/"+period+")*"+period+" AS timestamp,
SUM(amount) AS volume,
SUM(price*amount)/sum(amount) AS wavg_price,
SUBSTRING_INDEX(MIN(CONCAT(`timestamp`, '_', price)), '_', -1) AS `open`,
MAX(price) AS high,
MIN(price) AS low,
SUBSTRING_INDEX(MAX(CONCAT(`timestamp`, '_', price)), '_', -1) AS `close`
FROM transactions_history -- this table has 3 columns (timestamp, amount, price)
GROUP BY FLOOR(`timestamp`/"+period+")
ORDER BY timestamp
In my select statement, FLOOR(MIN(timestamp)/"+period+")*"+period+" AS timestamp,
I am trying to understand what it is doing. and
I need to convert this back to a mysql date/time Y-M-D H:i:s string or a UTC timestamp for parsing via javascript.
Let's assume that +period+ is 86400 (The number of seconds in a day)
Let's assume that the timestamp is '2015-12-08 20:58:58'
From what I can see, it takes the timestamp, which internally is stored as an integer and divides by 86400.
'2015-12-08 20:58:58' / 86400 = 233231576.4566898000
It then uses the FLOOR operation which would make it 233231576 then multiplies by 86400 again (I assume that this is to ensure rounding to the day)
I end up with 20151208166400.
So that's the 8th December 2015 but I also have 166400 which I have no idea what it is?
So now the second part of the question is, how to convert this integer to 2015-12-08 %H:%i:%s or even a UTC timestamp for parsing via Javascript.
I mentioned the problem in the comment, but not a fix. The problem is that the proposed code is for a unix timestamp, not a datetime value.
This can be fixed by doing appropriate conversions
SELECT FROM_UNIXTIME(FLOOR(UNIX_TIMESTAMP(MIN(timestamp)) / $period) * $period)
This gives you the flexibility of have arbitrary numbers of seconds for the groupings.
You're right that FLOOR(timestamp / 86400) * 86400 is a crude way of rounding a UNIX-style timestamp (seconds since 1970-01-01 00:00UTC) to midnight on the present day UTC.
If that's what you're trying to do, I suggest you try this kind of MySQL code:
SELECT DATE_FORMAT(DATE(`timestamp`), '%Y-%m-%d'),
...
GROUP BY DATE(`timestamp`)
This uses MySQL's built in date arithmetic to turn a timestamp into midnight.
But you should be careful of one thing. Those timestamps are all stored in UTC (f/k/a Greenwich Mean Time). When you do date arithmetic with them, or pull them out of the database to use them, they're automatically converted to local time according to your MySQL time zone settings.
It is rounding timestampt to period (e.g day).
DATE_FORMAT( DATE( FLOOR(MIN(timestamp)/"+period+")*"+period+" ) , '%Y-%m-%d %H:%i:%s' )
If period==day consider using only MySQL period rounding by DAY().
Convert a Date object to a string, according to universal time:
var d = new Date();
var n = d.toUTCString();
The result of n will be:
Mon, 28 Dec 2015 12:57:32 GMT
Related
I need to modify the output of a datetime field so that it returns the first day of the week in which that date falls. For example, if the date is 9/2/2016, it should return 8/29/2016 because that's the Monday of that week (I want Monday, not Sunday). However, I also need to convert the timestamp to a different timezone. The result is that I end up having to convert the timezone twice:
CONVERT_TZ(timestamp, '+00:00', '+05:00') - INTERVAL WEEKDAY(CONVERT_TZ(timestamp, '+00:00', '+05:00')) day
I can't simply perform the INTERVAL calculation on the UTC datetime and then convert the timezone on the result because the timezone conversion may affect the result of the WEEKDAY function, e.g. a datetime of 2016-9-5 00:00 UTC will actually fall on 9/4 for EST, thus causing it to be part of a different week.
Is there a way to avoid having to make two calls to CONVERT_TZ in the SELECT statement?
You can put it into a subquery.
SELECT converted - INTERVAL WEEKDAY(converted) AS day
FROM (SELECT CONVERT_TZ(timestamp, '+00:00', '+05:00') AS converted
FROM yourTable) AS x
Another way is to assign a user variable. To be able to use it twice, put the assignment into the condition part of an IF() expression -- that ensures that the assignment will be done before the uses.
IF(#converted := CONVERT_TZ(timestamp, '+00:00', '+05:00'),
#converted - INTERVAL(WEEKDAY(#converted) DAY,
NULL) AS day
I have read various questions here on Stackoverflow about the use of FROM_UNIXTIME but none directly deal with what I am trying to do. I have one timestamp in a variable coming from php (that has been reformatted - e.g. 25 March 2014) to a function which uses a database query to determine if there are other entries in the database that have the same date (not time). I've run across various methods for formatting and comparing timestamp entries using MySql and ended up with the following but I understand that it isn't very efficient. Does anyone know of a better way to accomplish this?
FROM_UNIXTIME(sd.timestart, "%e %M %Y") = ?'
where the variable in my array for comparison is the date format listed above. This accomplishes what I want but, again, I don't think it is the most efficient way to get this done. Any advice and/or ideas will be much appreciated.
*EDIT*
My timestamp is stored as an integer so I'm trying to use:
$thissessiondate = strtotime($date->timestart, strtotime('today'));
and
$tomorrowdate = strtotime($date->timestart, strtotime('tomorrow'));
to do trim to midnight but get an error (strtotime() expects parameter 2 to be long) and when I move 'today' to the first argument position, I get a conversion to 11 pm instead of 0:00...? I'm making some progress but my very incomplete knowledge of both PHP and MySQL are holding me back.
If you can avoid it, don't wrap columns used in predicates in expressions.
Have your predicates on bare columns to make index range scans possible. You want the datatype conversion to happen over on the literal side of the predicate, wherever possible.
The STR_TO_DATE function is the most convenient for this.
Assuming the timestart column is DATE, DATETIME or TIMESTAMP (which it really should be, if it represents a point in time.)
WHERE sd.timestart >= STR_TO_DATE( ? , "%e %M %Y")
AND sd.timestart < STR_TO_DATE( ? , "%e %M %Y") + INTERVAL 1 DAY
Effectively, what that's doing is taking the string passed in as the first argument to the STR_TO_DATE function, MySQL is going to convert that string to a DATETIME, based on the format specified as the second argument. And that effectively becomes a literal that MySQL can use to compare to the stored values in the column.
If there's an appropriate index available, MySQL will consider an index range scan operation to satisfy that predicate.
You'd need to pass in the same value twice, but that's not really a problem.
On the second line, we're just adding a day to the same value. So what MySQL is seeing is this:
WHERE sd.timestart >= STR_TO_DATE( '25 March 2014' , "%e %M %Y")
AND sd.timestart < STR_TO_DATE( '25 March 2014' , "%e %M %Y") + INTERVAL 1 DAY
In terms of performance, that's equivalent to:
WHERE sd.timestart >= '2014-03-15 00:00:00'
AND sd.timestart < '2014-03-16 00:00:00'
If you do it the other way around, and wrap timestart in a function, that's going to require MySQL to evaluate the function on every single row (or at least, on every row that isn't filtered out by another predicate first.)
IMPORANT NOTE
Be aware that MySQL interprets datetime values as being in the timezone of the MySQL connection, which defaults to the timezone setting of the MySQL server. MySQL is going to interpret datetime literals in the current setting of the timezone. For example, if MySQL timezone is set to +00:00, then datetime literals will be interpreted as UTC.
I assumed the format string matches the data being passed in, I don't use %e or %m. The %Y is a four digit year. (The list of format elements is in the MySQL documentation, under the DATE_FORMAT function.
Reference: http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_str-to-date
If your timestart column is INTEGER or other numeric datatype, representing a number of seconds or some other unit of time since the beginning of an era, you can use the same approach for performance benefits.
In the predicate, reference bare columns from the table, and do any conversions required on the literal side.
If you aren't using MySQL functions to do the conversion to "seconds since Jan 1, 1970 UTC" when rows are inserted (which is really what the TIMESTAMP datatype is doing internally), then I wouldn't recommend using MySQL functions to do the conversion in the query either.
If you're doing the conversion from date and time to an integer type "timestamp" in PHP, then I'd do the inverse conversion in PHP as well, and do the trimming to midnight and the adding of a day in PHP.
In that case, your MySQL query would be of the simple form:
WHERE sd.timestart >= ?
AND sd.timestart < ?
Where you would pass in the appropriate integer values, to compare to the INTEGER timestamp column.
Note that MySQL does provide a function for converting to "seconds since Jan 1 1970 UTC", so if timestart is seconds since Jan 1 1970 UTC, then something like this is valid:
WHERE sd.timestart >= UNIX_TIMESTAMP(STR_TO_DATE( '25 March 2014' , "%e %M %Y"))
AND sd.timestart < UNIX_TIMESTAMP(STR_TO_DATE( '25 March 2014' , "%e %M %Y") + INTERVAL 1 DAY)
BUT... again, be aware of timezone conversion issues; if the MySQL database has a different timezone setting than the web server. If you are going to store "integer", then I wouldn't muck that up with the conversion that MySQL does, which may not be exactly the same as the conversion functions the web server does.
If you store your date as an int timestamp, you can do this
round(sd.timestart/86400)=round(UNIX_TIMESTAMP(NOW())/86400)
This will get everything in your database that is from the same day.
For example:
SELECT id FROM uploads WHERE (approved=0 OR approved is NULL) AND round(uploads.date/86400)<=round(UNIX_TIMESTAMP(NOW())/86400) order by uploads.date DESC LIMIT 20
Will display all the uploads for today and the days before, without showing the future uploads. 86400 is the number of seconds in one day.
I have a table with the following columns:
|start_date |TZ |
|Dec 2, 2012 |Eastern |
|Dec 2, 2012 |GMT |
Note 1: our server is in UTC time.
Note 2:The column start_date is a date field, not a timestamp field. Dec 2nd 2012 implicitly means "2012-12-02 00:00:00"
Note 3: The above table is actually multiple normalized tables, but for simplicity, I de-normalized it.
Note 4: I can put anything into the TZ table to make this easy.
I would like to select from my_table where start_date <= now()
However, this doesn't work because of timezone. If the current date/time is
Dec 1st Eastern at 9PM (which is Dec 2nd 1AM UTC), the above query will return both results,
but I really only want the 2nd one. This is further complicated by daylight savings.
Ideally, I would like a query that does the following:
select * from my_table where convert_to_utc_timestamp(start_date,tz) <= now()
The above method would convert start_date to a timestamp and then convert it to the right timezone.
How would I do this in SQL?
There are two functions you'll probably find useful.
The first is:
STR_TO_DATE(start_date,'%M %d,%Y')
That will get your string, in the specified format, converted to a MySQL DATE datatype.
If you have the mysql.time_zone_name et al. tables populated, you can use the function:
CONVERT_TZ()
(need to check that CONVERT_TZ takes a DATE and will return a DATETIME or TIMESTAMP, or include a time component in the string being converted to get a DATETIME, e.g.
STR_TO_DATE( CONCAT(start_date,' 00:00:00'),'%M %d,%Y %T')
Wrap that expression in the CONVERT_TZ() function, e.g.
CONVERT_TZ( datetime_expr ,'US/Eastern','GMT')
To make use of the values stored in your TZ column, those are going to need to match, or you need to come up with a way to match to, the values stored in the mysql.time_zone_name table.
I have a table 't' with date(yyyy-mm-dd), hour(1-12), minute(00-59), ampm(a/p), and timezone(pst/est) fields.
How can I select the rows that are <= now()? (ie. already happened)
Thank you for your suggestions!
edit: this does it without attention to the hour/minute/ap/tz fields:
SELECT * FROM t.date WHERE date <= now()
Here's one way to do it - combine all your seconds, minutes, etc into a date and compare to NOW(), making sure you do the comparison in the same time-zone. (Untested):
SELECT *
FROM t
LEFT JOIN y ON t.constant=y.constant
WHERE CONVERT_TZ(STR_TO_DATE(CONCAT(date,' ',hour,':',minute,' 'ampm),
'%Y-%m-%d %l:%i %p' ),
timezone,"SYSTEM") < NOW();
If your hour is 01 - 12 not 1-12 then use %h instead of %l in the STR_TO_DATE.
The STR_TO_DATE tries to stick your date and time columns together and convert them into a date.
The CONVERT_TZ(...,timezone,"SYSTEM") converts this date from whatever timezone is specified in the timezone column to system time.
This is then compared to NOW(), which is always in system time.
As an aside, perhaps you should make a single column date using MySQL's date datatype, as it's a lot easier to do arithmetic on that!
For reference, here is a summary of very useful mysql date functions where you can read up on those featuring in this answer.
Good luck!
SELECT * FROM t
WHERE `date`<=DATE_SUB(curdate(), INTERVAL 1 DAY)
OR (
`date`<=DATE_ADD(curdate(), INTERVAL 1 DAY)
AND
CONVERT_TZ(CAST(CONCAT(`date`,' ',IF(`hour`=12 AND ampm='a',0,if(ampm='a',`hour`,`hour`+12)),':',`minute`,':00') AS DATETIME),'GMT',`timezone`)<=NOW()
)
Rationale for date<=DATE_[ADD|SUB](curdate(), INTERVAL 1 DAY):
The fancy conversion is quite an expensive operation, so we don't want it to run on the complete table. This is why we pre-select against an UNCHANGED date field (possibly using an index). In no timezone can an event being more than a day in current timezone's past be in the future, and in no timezone can an event more than a day in the curent timezone's future be in the past.
I wasn't able to find out (googling, reading mysql reference manual) how to get value of DATETIME in seconds in MySQL.
I dont mean to extract seconds from datetime, but to convert it into seconds.
If by "convert to seconds", you mean "convert to an UNIX Timestamp" (i.e. number of seconds since 1970-01-01), then you can use the UNIX_TIMESTAMP function :
select UNIX_TIMESTAMP(your_datetime_field)
from your_table
where ...
And, for the sake of completness, to convert from an Unix Timestamp to a datetime, you can use the FROM_UNIXTIME function.
If you want to have the difference between two DATETIME values, use TIMESTAMPDIFF:
TIMESTAMPDIFF(unit,datetime_expr1,datetime_expr2)
Returns datetime_expr2 – datetime_expr1, where datetime_expr1 and datetime_expr2 are date or datetime expressions. One expression may be a date and the other a datetime; a date value is treated as a datetime having the time part '00:00:00' where necessary. The unit for the result (an integer) is given by the unit argument. The legal values for unit are the same as those listed in the description of the TIMESTAMPADD() function.
mysql> SELECT TIMESTAMPDIFF(MONTH,'2003-02-01','2003-05-01');
-> 3
mysql> SELECT TIMESTAMPDIFF(YEAR,'2002-05-01','2001-01-01');
-> -1
mysql> SELECT TIMESTAMPDIFF(MINUTE,'2003-02-01','2003-05-01 12:05:55');
-> 128885
unit can also be HOUR which is what you asked for in one of the comments.
The unit argument can be any of the following:
MICROSECOND
SECOND
MINUTE
HOUR
DAY
WEEK
MONTH
QUARTER
YEAR
The level of usefulness of some of the other options will of course be determined by the granularity of the data. For instance, "MICROSECOND" will only have limited use if you are not storing microseconds in your DATETIME values.
Use TIME_TO_SEC in previous versions for mysql
SELECT TIME_TO_SEC(time column) FROM table
i used in mysql
TO_SECONDS(your date goes here) method to convert date to seconds from year 0
http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html
The function UNIX_TIMESTAMP(datetime) returns the unix time, which happens to be the number of seconds since 1-Jan-1970 0000 UTC. That may be what you need, but not if you're dealing with dates of birth, historical dates, or dates after 2037.
Starting in mysql 5.5.0 you can use to_seconds()
TO_SECONDS(FIELD_NAME)
FIELD_NAME must be DATETIME type
I have created my own query for your problem:
SELECT HOUR(`colname`) * 3600 + MINUTE(`colname`) * 60 + SECOND(`colname`)
FROM widgets
WHERE id = 1;
Use id = 1 if you have to take a specific row.
The output will be in seconds.