Firstly sorry for my english, it's my first post here, and my english isn't as well as i wish but i hope it'll be enough to get a answer.
So how some of you maybe now reddit put their own source code on github and i want to use (a little modified by me) version of sql schema with a hotness algorithm. The problem is that schema is written in psgsql and my database use mysql engine.
I tried to convert schema manually but i give up with no effects, so i try again with misc tools and apps, but not even one of them support converting of procedures & functions, and the problem is that i need exactly just that one option.
So, is anyone of you can help me convert the hotness function from there:
create or replace function hot(ups integer, downs integer, date timestamp with time zone) returns numeric as $$
select round(cast(log(greatest(abs($1 - $2), 1)) + sign($1 - $2) * (date_part('epoch', $3) - 1134028003) / 45000.0 as numeric), 7)
$$ language sql immutable;
to mysql schema, i would be very grateful :)
Once again sorry for my language, i now that i underestimates the standard :)
I just wrote this function below, and it seem to work.
CREATE FUNCTION hot (ups INT(10),downs INT(10),d TIMESTAMP)
RETURNS DOUBLE DETERMINISTIC
RETURN ROUND(LOG(GREATEST(ABS(ups-downs), 1)) + SIGN(ups-downs)*(UNIX_TIMESTAMP(d) - 1134028003) / 45000.0,7);
I compared its outputs to the outputs of this Python code, everything seem OK.
Sample run:
SELECT hot(20,10,'2013-01-01 00:00:00');
Query OK, 0 rows affected (0.00 sec)
+----------------------------------+
| hot(20,10,'2013-01-01 00:00:00') |
+----------------------------------+
| 4957.1202962 |
+----------------------------------+
1 row in set (0.00 sec)
I don't know the MySQL syntax for user defined functions, but some PostgreSQL specific parts are:
date_part('epoch', $3)
Number of seconds of $3 since the epoch i.e. since 1970-01-01 00:00:00.
1134028003
Number of seconds from epoch to 2005-12-08 07:46:43.
Perhaps this is useful for finding MySQL equivalents.
Related
Is there any sql command which I can insert into the stated query so I can convert the timestamp. Although it could be done separately which I have seen so far but I am trying to find something which I can add to the stated query as that would be helpful because I am using other queries to retrieve the data as well. If you any other questions please do mention. Addition: rating_timestamp contains both time and date.
SELECT rating_id,
rating_postid,
rating_posttitle,
rating_rating,
rating_timestamp,
rating_username,
rating_userid
FROM wp_ratings;
In cases of date arithmetic, it is especially important to specify the DBMS you are using - Oracle's math is different from Postgres' math is different from SQL Server's math is different from MySQL's math is...
This assumes that you are using SQL Server. Since there is no built in command to do this conversion, you need to create your own function to do that. The function below takes a UNIX / Linux timestamp and converts it to an SQL Server datetime.
CREATE FUNCTION dbo.fn_ConvertToLocalDateTime (#unixdate BIGINT)
RETURNS DATETIME
AS
BEGIN
DECLARE #UTCTimeOffset BIGINT
,#LocalDatetime DATETIME;
SET #UTCTimeOffset = DATEDIFF(second, GETUTCDATE(), GETDATE())
SET #LocalDatetime = DATEADD(second, #unixdate + #UTCTimeOffset, CAST('1970-01-01 00:00:00' AS datetime))
RETURN #LocalDatetime
END;
GO
I wast sure about about Sql version before. This worked perfectly for me.
FROM_UNIXTIME(rating_timestamp,'%h:%i:%s %D %M %Y')
I am a super noob in postgresql. But I need to define a func that would map: int --> datetime
After reading documentation I've come up with this:
CREATE FUNCTION fut(num integer) RETURNS datetime
-- convert a UNIX time integer into the datetime timestamp
AS $$ select timestamp 'epoch'+interval '1 second'*num; $$
LANGUAGE plpgsql;
select fut(500);
But it returns
The cursor is not located inside a statement!
Could you please point me to what I am doing wrong here?
As far as I knew Redshift doesn't even permit user-defined functions. Yeah: http://docs.aws.amazon.com/redshift/latest/dg/c_unsupported-postgresql-features.html : User-defined functions and stored procedures. So I think you're plain out of luck.
A search for Redshift epoch to timestamp or redshift to_timestamp finds that lots of other people have looked into this already:
http://yiyujia.blogspot.com.au/2014/04/redshift-convert-integer-to-timestamp.html
http://www.valkrysa.com/tech/2015/1/30/amazon-redshift-converting-unix-epoch-time-into-timestamps
etc.
The most sensible answers are those that rely on:
TIMESTAMP 'epoch' + myunixtime * INTERVAL '1 Second '
which is what you appear to be already doing. This the best you are going to get, because Redshift does not support user-defined functions.
As far as I understand from your function, you want to convert unix time (i.e., the number of seconds since 00:00:00 UTC, 1 January 1970) saved as a value of integer type, to a value of valid postgresql date/time type.
If this is so, you do not have to create a new function for such a conversion, just use the predefined postgresql function to_timestamp, for example:
# select to_timestamp(500);
to_timestamp
------------------------
1970-01-01 03:08:20+03
(1 row)
I googled a lot for this, and don't see anyone talking about it, so it must be a simple issue, but still it has me stumped.
This performance_schema table - http://dev.mysql.com/doc/refman/5.6/en/events-statements-current-table.htm has timer_start and TIMER_END columns. Accordintg to the documentation " The TIMER_START and TIMER_END values indicate when event timing started and ended" .
One small problem. It's a bigint and not a date. How do I convert it to a date?
I saw one blogger suggest that it's the number of time units since the server was started. In my case statements are supposed to be measured to a nanosecond (10^9). So if I have a timer_start value of 3723676792253626000 that would mean 3723676792 s which would be unlikely since the server uptime is 3723716 s. a simple comparison of the number of digits in these two numbers would lead me to think that the unit of time is really picoseconds (10^12).
so the question is :
1. is timer_start really the number of units from the last restart?
2. if so, why is it in picoseconds when setup_timers indicates nanoseconds?
TIA
Here is a corrected version for MySQL 5.7:
SELECT
DATE_SUB(NOW(), INTERVAL (SELECT VARIABLE_VALUE FROM performance_schema.global_status WHERE VARIABLE_NAME='UPTIME') - TIMER_START*10e-13 second) AS `start_time`,
ROUND(timer_wait*10E-10, 3) AS `wait in (ms)`,
sql_text,
current_schema
FROM performance_schema.events_statements_history_long;
The fixes are:
get global_status from performance_schema (introduced in MySQL 5.7)
fix wait in (ms) (10E-8 vs. 10E-10) + formatting; example test is the following SQL query: SELECT SLEEP(0.5)
the returned columns
uses the "long" history table (events_statements_history_long)
ok, I was able to partially figure it out. Answer to the first question is yes. Here is a query that converts the timer_start value to a time stamp that a human can recognize:
select
date_sub(now(),INTERVAL (select VARIABLE_VALUE from information_schema.global_status where variable_name='UPTIME')-TIMER_START*10e-13 second) `start_time`
,timer_wait/10E-8 `wait in (ms)`
,timer_wait
,sql_text
,digest_text
from performance_schema.events_statements_history
guys I need help.
I am using Mysql / phpmyadim.
I have db with table which stores name and code id of people.
+--------+---------+
| Name | code_id |
+--------+---------+
| Nazeer | MD-1 |
+--------+---------+
I have 10 contacts and ids. I am using php program which used to generate automatic code.
recently i imported more records in to db from excel file and record increase to 5000+.
My php automatic code stopped generating codes giving me syntax error on code id.
I figured out that my excel import was having code id like MD-1, MD-2, etc. and my program used automatic code for number in 3 digits since my record is over thousands which 4 digit it give syntax error.
I did some research on solving that and the answer was to change all 2 digit numbers eg. "MD-1" ~ "MD-99" TO "MD-001" ~ "MD-099" and my program will work again.
so the question is how do i do that in phpmyadmin sql to change it. I need to keep 'MD-' and add '0' then add back the corresponding number.
thanks and appreciate your help in advance.
Regrds.
this sql will update all your data, but like I said in comments, you better off fixing your php code instead.
WARNING : this sql only works assuming all your data are in the format of [MD-xxx] with 3 or less numbers in it
UPDATE your_table SET
code_id=case length(substr(code_id,4))
WHEN 1 THEN concat("MD-00",substr(code_id,4))
WHEN 2 THEN concat("MD-0",substr(code_id,4))
ELSE code_id END;
I assume that you want to update the content MD-1 to MD-001 and MD-99 to MD-099. To do that you can write a PHP code to retrieve the rows one by one and have to match patterns and then update. Here are some useful links. link 1
HINT : you can check 5 digit string and then add another 0 in the position of 3.(use [exploid] to split by "-" and then concat with "-0" 2) There are no way to do the same only by using MYSQl since it's not a programming language. And other thing is PHP is not a program. It's a programming language.
run UPDATE query and use CONCAT function :
for ($x=0; $x=<upto>; $x++){
UPDATE <table_name>
SET <columnname>= CONCAT('MD-',0,$x)
WHERE <columnname>= CONCAT('MD-',$x)
}
Below simple update command can help you.
UPDATE mytable
SET code_id=IF(LENGTH(code_id)=4,CONCAT(SUBSTRING_INDEX(code_id,'-',1),'-00',SUBSTRING_INDEX(code_id,'-',-1)),IF(LENGTH(code_id)=5,CONCAT(SUBSTRING_INDEX(code_id,'-',1),'-0',SUBSTRING_INDEX(code_id,'-',-1)),code_id));
I have a small Django project that imports data dumps from MongoDB into MySQL. Inside these Mongo dumps are dates stored in epoch time. I would expect epoch time to be the same regardless of time zone but what I am seeing is that the Django TIME_ZONE setting has an effect on the data created in MySQL.
I have been testing my database output with the MySQL UNIX_TIMESTAMP function. If I insert a date with the epoch of 1371131402880 (this includes milliseconds) I have my timezone set to 'America/New_York', UNIX_TIMESTAMP gives me 1371131402, which is the same epoch time excluding milliseconds. However if I set my timezone to 'America/Chicago' I get 1371127802.
This is my code to convert the epoch times into Python datetime objects,
from datetime import datetime
from django.utils.timezone import utc
secs = float(epochtime) / 1000.0
dt = datetime.fromtimestamp(secs)
I tried to fix the issue by putting an explict timezone on the datetime object,
# epoch time is in UTC by default
dt = dt.replace(tzinfo=utc)
PythonFiddle for the code
I've tested this Python code in isolation and it gives me the expected results. However it does not give the correct results after inserting these object into MySQL through a Django model DateTimeField field.
Here is my MySQL query,
SELECT id, `date`, UNIX_TIMESTAMP(`date`) FROM table
I test this by comparing the unix timestamp column in the result of this query against the MongoDB JSON dumps to see if the epoch matches.
What exactly is going on here? Why should timezone have any effect on epoch times?
Just for reference, I am using Django 1.5.1 and MySQL-python 1.2.4. I also have the Django USE_TZ flag set to true.
I am no python or Django guru, so perhaps someone can answer better than me. But I will take a guess at it anyway.
You said that you were storing it in a Django DateTimeField, which according to the documents you referenced, stores it as a Python datetime.
Looking at the docs for datetime, I think the key is understanding the difference between "naive" and "aware" values.
And then researching further, I came across this excellent reference. Be sure the read the second section, "Naive and aware datetime objects". That gives a bit of context to how much of this is being controlled by Django. Basically, by setting USE_TZ = true, you are asking Django to use aware datetimes instead of naive ones.
So then I looked back at you question. You said you were doing the following:
dt = datetime.fromtimestamp(secs)
dt = dt.replace(tzinfo=utc)
Looking at the fromtimestamp function documentation, I found this bit of text:
If optional argument tz is None or not specified, the timestamp is converted to the platform’s local date and time, and the returned datetime object is naive.
So I think you could do this:
dt = datetime.fromtimestamp(secs, tz=utc)
Then again, right below that function, the docs show utcfromtimestamp function, so maybe it should be:
dt = datetime.utcfromtimestamp(secs)
I don't know enough about python to know if these are equivalent or not, but you could try and see if either makes a difference.
Hopefully one of these will make a difference. If not, please let me know. I'm intimately familiar with date/time in JavaScript and in .Net, but I'm always interested in how these nuances play out differently in other platforms, such as Python.
Update
Regarding the MySQL portion of the question, take a look at this fiddle.
CREATE TABLE foo (`date` DATETIME);
INSERT INTO foo (`date`) VALUES (FROM_UNIXTIME(1371131402));
SET TIME_ZONE="+00:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
SET TIME_ZONE="+01:00";
select `date`, UNIX_TIMESTAMP(`date`) from foo;
Results:
DATE UNIX_TIMESTAMP(`DATE`)
June, 13 2013 13:50:02+0000 1371131402
June, 13 2013 13:50:02+0000 1371127802
It would seem that the behavior of UNIX_TIMESTAMP function is indeed affected by the MySQL TIME_ZONE setting. That's not so surprising, since it's in the documentation. What's surprising is that the string output of the datetime has the same UTC value regardless of the setting.
Here's what I think is happening. In the docs for the UNIX_TIMESTAMP function, it says:
date may be a DATE string, a DATETIME string, a TIMESTAMP, or a number in the format YYMMDD or YYYYMMDD.
Note that it doesn't say that it can be a DATETIME - it says it can be a DATETIME string. So I think the actual value being implicitly converted to a string before being passed into the function.
So now look at this updated fiddle that converts explicitly.
SET TIME_ZONE="+00:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
SET TIME_ZONE="+01:00";
select `date`, convert(`date`, char), UNIX_TIMESTAMP(convert(`date`, char)) from foo;
Results:
DATE CONVERT(`DATE`, CHAR) UNIX_TIMESTAMP(CONVERT(`DATE`, CHAR))
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371131402
June, 13 2013 13:50:02+0000 2013-06-13 13:50:02 1371127802
You can see that when it converts to character data, it strips away the offset. So of course, it makes sense now that when UNIX_TIMESTAMP takes this value as input, it is assuming the local time zone setting and thus getting a different UTC timestamp.
Not sure if this will help you or not. You need to dig more into exactly how Django is calling MySQL for both the read and the write. Does it actually use the UNIX_TIMESTAMP function? Or was that just what you did in testing?