Issue inserting timestamp value in MySQL - mysql

There are some other questions where people have problems with timestamp being all zeros. I have checked them and this is not a duplicate.
I declare a table like this:
CREATE TABLE `my_db`.`my_table` (
`time_stamp` timestamp NOT NULL,
`author` varchar() NOT NULL,
`text` text NOT NULL,
`md5` int(11) NOT NULL,
PRIMARY KEY (`time_stamp`)
) ;
I also have a second table which will have a timestamp as primary key and they should have the same value.
Coding in Delphi, I use SELECT CURRENT_TIMESTAMP which returns something like '19/6/2010 4:56:17 AM' which I then use in an INSERT statement. The INSERT succeeds, but the timestamp is all zeros.
What am I doing wrong?
Here's the INSERT code:
sqlCommand := 'INSERT INTO my_db.my_table(time_stamp, author, text, md5) VALUES ("'
+ timestamp +
'", "mawg", ' +
'"Hello, world"' +
0 +
'");';
Result := DoSQlCommandWithNoResultSet(sqlCommand, AdoConnection);
Insertion will be extremely low rate, one entry every few weeks or maybe months, so I am happy with timestamp as primary key. I am keeping "versions" of things, so timestamp makes sense to me.
I am begging to think that this is an ADO problem, although I would expect ADO to just "pass through". I don't see any other solution. In a console, the output is "correct", but when run through ADO in Delphi then it is wrong
Can I specify to MySQL how it ought to format its dates?

After reviewing the MySQL documentation, it appears that if your timestamp value is incorrectly formatted, it would normally cause the timestamp to be '0000-00-00 00:00:00'.
In any case, you don't need to specify a timestamp value—that's the benefit of TIMESTAMP over DATETIME. And even if you did, you can simply set it to NOW() instead of running an unnecessary SELECT statement.
Edit:
Also, I know you said you thought this through, but have you considered daylight savings time? This could cause two records to have the same timestamp when the clock is set back during autumn.
Edit 2:
K, I don't know why I didn't catch this earlier, but that timestamp format you gave is incorrect. Try inserting a valid timestamp like '2010/06/19 4:56:17'. MySQL has pretty relaxed parsing of date & time values, but it always has to be year-month-date and hour-minute-seconds.
Edit 3:
Alright, there seems to be a little confusion over this, so I'm gonna post this quote from the MySQL 5.0 doc page on the DATETIME format:
For values specified as strings that include date part delimiters, it is not necessary to specify two digits for month or day values that are less than 10. '1979-6-9' is the same as '1979-06-09'. Similarly, for values specified as strings that include time part delimiters, it is not necessary to specify two digits for hour, minute, or second values that are less than 10. '1979-10-30 1:2:3' is the same as '1979-10-30 01:02:03'.

Have a look at MySQL date functions. They are very extensive and allow a high flexibility.
Besides all that, I would recommend re-thinking your table structure. A timestap as a primary key is not exactly what you want. When you have high traffic, it CAN happen, that the timestamp is the same. Also if you are saving 2 or more records in a row, the timestamp will be the same. Furthermore, your MD5 column is set to int(11). MD5 hashes use mixed characters, so i would rather go with varchar(32).
CREATE TABLE `my_db`.`my_table` (
`id` int(11) NOT NULL,
`timestamp` timestamp NOT NULL,
`author` varchar(100) NOT NULL,
`text` text NOT NULL,
`md5` varchar(32) NOT NULL,
PRIMARY KEY (`id`)
) ;

AFAIR 19/6/2010 4:56:17 AM is not a valid date format for MySQL date types. You should convert it to 2010-06-19 04:56:14 (see doc).

Related

Why mysql query is slow without quotation mark?

The table DDL as flows:
CREATE TABLE `video` (
`short_id` varchar(50) NOT NULL,
`prob` float DEFAULT NULL,
`star_id` varchar(50) NOT NULL,
`qipu_id` int(11) NOT NULL,
`cloud_url` varchar(100) DEFAULT NULL,
`is_identical` tinyint(1) DEFAULT NULL,
`quality` varchar(1) DEFAULT NULL,
PRIMARY KEY (`short_id`),
KEY `ix_video_short_id` (`short_id`),
KEY `sid` (`star_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
The video table has 4.5 million lines.
I execute the same query in mysql shell client as flows. except in where clause the star_id equal to a value with quatation mark, another not as flows.
select * from video where star_id="215343405";
12914 rows in set (0.22 sec)
select * from video where star_id=215343405;
12914 rows in set (3.17 sec)
the one with quatation mark is 10x faster then another(I have create index on star_id).i watch out the slow one does not use the index. I just wonder how mysql process the query?
mysql> explain select * from video where star_id=215343405;
Thanks advance!
This is answered in the manual:
For comparisons of a string column with a number, MySQL cannot use an
index on the column to look up the value quickly. If str_col is an
indexed string column, the index cannot be used when performing the
lookup in the following statement:
SELECT * FROM tbl_name WHERE str_col=1;
The reason for this is that there are many different strings that may convert to the value 1, such as '1', ' 1', or '1a'.
If you do not use Quotation marks mysql uses the value as an int and must convert the value for every record. Therefor the db needs a lot of time.
The quotes define the expression as a string, whereas without the single quote it is evaluated as a number. This means that MySQL is forced to perform a Type Conversion to convert the number to a CHAR to do a proper comparison.
As the doc above says,
For comparisons of a string column with a number, MySQL cannot use an
index on the column to look up the value quickly. If str_col is an
indexed string column, the index cannot be used when performing the
lookup...
However, the inverse of that is not true and while the index can be used, using a string as a value causes a poor execution plan (as illustrated by jkavalik's sqlfiddle) where using where is used instead of the faster using index condition. The main difference between the two is that the former requires a row lookup and the latter can get the data directly from the index.
You should definitely modify the column data type (assuming it truly is only meant to contain numbers) to the appropriate data type ASAP, but make sure that no queries are actually using single quotes, otherwise you'll be back where you started.

Efficient MySQL query for huge set of data

Say i have a table like below:
CREATE TABLE `hadoop_apps` (
`clusterId` smallint(5) unsigned NOT NULL,
`appId` varchar(35) COLLATE utf8_unicode_ci NOT NULL,
`user` varchar(64) COLLATE utf8_unicode_ci NOT NULL,
`queue` varchar(35) COLLATE utf8_unicode_ci NOT NULL,
`appName` varchar(255) COLLATE utf8_unicode_ci DEFAULT NULL,
`submitTime` datetime NOT NULL COMMENT 'App submission time',
`finishTime` datetime DEFAULT NULL COMMENT 'App completion time',
`elapsedTime` int(11) DEFAULT NULL COMMENT 'App duration in milliseconds',
PRIMARY KEY (`clusterId`,`appId`,`submitTime`),
KEY `hadoop_apps_ibk_finish` (`finishTime`),
KEY `hadoop_apps_ibk_queueCluster` (`queue`,`clusterId`),
KEY `hadoop_apps_ibk_userCluster` (`user`(8),`clusterId`),
) ENGINE=InnoDB DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
mysql> SELECT COUNT(*) FROM hadoop_apps;
This would return me a count 158593816
So I am trying to understand what is inefficient about the below query and how I can improve it.
mysql> SELECT * FROM hadoop_apps WHERE DATE(finishTime)='10-11-2013';
Also, what's the difference between these two queries?
mysql> SELECT * FROM hadoop_apps WHERE user='foobar';
mysql> SELECT * FROM hadoop_apps HAVING user='foobar';
WHERE DATE(finishTime)='10-11-2013';
This is a problem for the optimizer because anytime you put a column into a function like this, the optimizer doesn't know if the order of values returned by the function will be the same as the order of values input to the function. So it can't use an index to speed up lookups.
To solve this, refrain from putting the column inside a function call like that, if you want the lookup against that column to use an index.
Also, you should use MySQL standard date format: YYYY-MM-DD.
WHERE finishTime BETWEEN '2013-10-11 00:00:00' AND '2013-10-11 23:59:59'
What is the difference between [conditions in WHERE and HAVING clauses]?
The WHERE clause is for filtering rows.
The HAVING clause is for filtering results after applying GROUP BY.
See SQL - having VS where
If WHERE works, it is preferred over HAVING. The former is done earlier in the processing, thereby cutting down on the amount of data to shovel through. OK, in your one example, there may be no difference between them.
I cringe whenever I see a DATETIME in a UNIQUE key (your PK). Can't the app have two rows in the same second? Is that a risk you want to take.
Even changing to DATETIME(6) (microseconds) could be risky.
Regardless of what you do in that area, I recommend this pattern for testing:
WHERE finishTime >= '2013-10-11'
AND finishTime < '2013-10-11' + INTERVAL 1 DAY
It works "correctly" for DATE, DATETIME, and DATETIME(6), etc. Other flavors add an extra midnight or miss parts of a second. And it avoids hassles with leapdays, etc, if the interval is more than a single day.
KEY `hadoop_apps_ibk_userCluster` (`user`(8),`clusterId`)
is bad. It won't get past user(8). And prefixing like that is often useless. Let's see the query that tempted you to build that key; we'll come up with a better one.
158M rows with 4 varchars. And they sound like values that don't have many distinct values? Build lookup tables and replace them with SMALLINT UNSIGNED (2 bytes, 0..64K range) or other small id. This will significantly shrink the table, thereby making it faster.

Setting a column as timestamp in MySql workbench?

This might be a really elementary question, but I've never created a table with TIMESTAMP() before, and I'm confused on what to put as the parameters. For example, here:
I just randomly put TIMESTAMP(20), but what does the 20 as a parameter signify here? What should be put in here?
I googled the question, but didn't really come up with anything so... Anyway I'm new to sql, so any help would be greatly appreciated, thank you!!
EDIT
As of MySQL 5.6.4, datatype TIMESTAMP(n) specifies n (0 up to 6) decimal digits of precision for fractional seconds.
Before MySQL 5.6, MySQL did not support fractional seconds stored as part of a TIMESTAMP datatype.
Reference: https://dev.mysql.com/doc/refman/5.6/en/fractional-seconds.html
We don't need to specify a length modifier on a TIMESTAMP. We can just specify TIMESTAMP by itself.
But be aware that the first TIMESTAMP column defined in the table is subject to automatic initialization and update. For example:
create table foo (id int, ts timestamp, val varchar(2));
show create table foo;
CREATE TABLE `foo` (
`id` INT(11) DEFAULT NULL,
`ts` TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
`val` VARCHAR(2) DEFAULT NULL
)
What goes in parens following a datatype depends on what the datatype is, but for some datatypes, it's a length modifier.
For some datatypes, the length modifier affects the maximum length of values that can be stored. For example, VARCHAR(20) allows up to 20 characters to be stored. And DECIMAL(10,6) allows for numeric values with four digits before the decimal point and six after, and effective range of -9999.999999 to 9999.999999.
For other types, the length modifier it doesn't affect the range of values that can be stored. For example, INT(4) and INT(10) are both integer, and both can store the full range of values for allowed for the integer datatype.
What that length modifier does in that case is just informational. It essentially specifies a recommended display width. A client can make use of that to determine how much space to reserve on a row for displaying values from the column. A client doesn't have to do that, but that information is available.
EDIT
A length modifier is no longer accepted for the TIMESTAMP datatype. (If you are running a really old version of MySQL and it's accepted, it will be ignored.)
Thats the precision my friend, if you put for example (2) as a parameter, you will get a date with a precision like: 2015-12-29 00:00:00.00, by the way the maximum value is 6.
This syntax seems to be from old version of MySQL, prior to 4.1. It has been removed completely from 5.5 https://dev.mysql.com/doc/refman/5.0/en/upgrading-from-previous-series.html
So no point in specifying a width here, as it may be ignored. What version are you running?
MySQL 5.7 appears to support this syntax. The argument passed is the precision. TIMESTAMP(3) will allow millisecond precision. 6 is the highest amount of allowed precision.
reference: http://dev.mysql.com/doc/refman/5.7/en/datetime.html
In MySQL workbench 8.0
TIMESTAMP
doesn't work, you need to add wole statement (if u don't want to update timestamp in future)
TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP
than u have e.g :
2020-01-08 19:10:05
but if you want that TIMESTAMP could be modify with the record update than you use :
TIMESTAMP NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP

Storing time in MySQL

I have data in the format of both "2013-01-17 18:46:47 -0800" and "1358477089" ...I'm wondering what is the best way to store this in a mysql db, that allows me to select results within a certain month, week, day etc.. using mysql's own functions.
Currently my create table code is like this.. the "timestamp" needs changing.
visible
CREATE TABLE IF NOT EXISTS `votes` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`from` varchar(32) NOT NULL,
`username` varchar(32) NOT NULL,
`address` varchar(16) NOT NULL,
`timestamp` varchar(32) NOT NULL,
PRIMARY KEY (`id`)) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Best way is to use MySQL built-in DATETIME type.
MySQL offers lots of function which will allow you to select results within a certain month, week, day, whatever you need.
See great list of functions here:
http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html
As hek2mgl and other guys mentioned, there is also TIMESTAMP.
MySQL converts TIMESTAMP values from the current time zone to UTC for storage, and back from UTC to the current time zone for retrieval. (This does not occur for other types such as DATETIME.)
If you store a TIMESTAMP value, and then change the time zone and retrieve the value, the retrieved value is different from the value you stored.
I preffer and advice you to use DATETIME.
If you use a timestamp your field should be an "integer" not a varchar. This provides better perfomance (for example if you use an index for this column).
If you do not need to have dates before 1970 I would suggest to use a timestamp, not a datetime. It is easier to use.
PHP
$timestamp = date('U');
MySQL
INSERT INTO table SET timestamp = UNIX_TIMESTAMP()

MYSQL: How to convert VARCHAR column containing dates to a DATE column?

I have a large table of birthdays that I want to convert from a varchar column to a date column.
The table SQL is:
CREATE TABLE `birthdays` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`uid` bigint(20) DEFAULT NULL,
`birthday_date` varchar(255) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM
The birthday date is stored in one of two ways: strings like "05/26/1994" or sometimes "03/14" (for people who don't want to reveal their birth year).
What is the best way to create another table and store the birthdays in a date column? Is it possible to do this just using MySQL (and avoid using PHP or some other intermediary)?
I have found a STR_TO_DATE function in MySQL. Should I use this?
Thanks!
SELECT IF(birthday_date RLIKE '[0-9]{2}/[0-9]{2}/[0-9]{4}',STR_TO_DATE(birthday_date,'%m/%d/%Y'),STR_TO_DATE(birthday_date,'%m/%d'));
This will result in dates like 0000-03-14 for rows that have no year entered. Your server needs to be configured to allow invalid dates though (see http://dev.mysql.com/doc/refman/5.0/en/server-sql-mode.html )
If you convert your column to DateTime then you will not be able to store dates like "03/14" in which year is missing. So instead I suggest to keep this as it is and probably have another column for storing the dateTime if you really need that.
Also have internal trigger to convert the datestrings from varchar column to dateTime column.
yes, you have to use the method STR_TO_DATE for your purpose.