i am trying to load csv data file into mysql , which later on i need to pass to HDFS but while updating the file
mysql > load data infile '/var/lib/mysql-files/online_retail.csv' into table retail fields terminated by ',' ignore 1 lines;
ERROR : "Incorrect date value: '12/1/2010 8:26' for column
'invoicedate' at row 1"
mysql> DESCRIBE retail;
+-------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+-------+
| InvoiceNo | int(11) | YES | | NULL | |
| stockcode | varchar(20) | YES | | NULL | |
| Description | varchar(500) | YES | | NULL | |
| Quantity | int(11) | YES | | NULL | |
| invoicedate | date | YES | | NULL | |
| UnitPrice | double | YES | | NULL | |
| CustomerId | int(11) | YES | | NULL | |
| Country | varchar(30) | YES | | NULL | |
+-------------+--------------+------+-----+---------+-------+
If you give the data type as "DATE", Date format should be in the form of "YYYY-MM-DD" or "YY-MM-DD".
Either make the data type as STRING, or transform the data to the YYYY-MM-DD format...
I.e from 12/1/2010 8:26 to 2010/12/01
Hope this helps
As said eggyal and as documented under Date and Time Literals:
MySQL recognizes DATE values in these formats:
As a string in either 'YYYY-MM-DD' or 'YY-MM-DD' format. A “relaxed” syntax is permitted: Any punctuation character may be used as the delimiter between date parts. For example, '2012-12-31', '2012/12/31', '2012^12^31', and '2012#12#31' are equivalent.
As a string with no delimiters in either 'YYYYMMDD' or 'YYMMDD' format, provided that the string makes sense as a date. For example, '20070523' and '070523' are interpreted as '2007-05-23', but '071332' is illegal (it has nonsensical month and day parts) and becomes '0000-00-00'.
As a number in either YYYYMMDD or YYMMDD format, provided that the number makes sense as a date. For example, 19830905 and 830905 are interpreted as '1983-09-05'.
And for your specific case, a few lines below :
MySQL recognizes DATETIME and TIMESTAMP values in these formats:
As a string in either 'YYYY-MM-DD HH:MM:SS' or 'YY-MM-DD HH:MM:SS' format. A “relaxed” syntax is permitted here, too: Any punctuation
character may be used as the delimiter between date parts or time
parts. For example, '2012-12-31 11:30:45', '2012^12^31 11+30+45',
'2012/12/31 11*30*45', and '2012#12#31 11^30^45' are equivalent.
The only delimiter recognized between a date and time part and a fractional seconds part is the decimal point.
The date and time parts can be separated by T rather than a space. For example, '2012-12-31 11:30:45' '2012-12-31T11:30:45' are
equivalent.
As a string with no delimiters in either 'YYYYMMDDHHMMSS' or 'YYMMDDHHMMSS' format, provided that the string makes sense as a date.
For example, '20070523091528' and '070523091528' are interpreted as
'2007-05-23 09:15:28', but '071122129015' is illegal (it has a
nonsensical minute part) and becomes '0000-00-00 00:00:00'.
As a number in either YYYYMMDDHHMMSS or YYMMDDHHMMSS format, provided that the number makes sense as a date. For example,
19830905132800 and 830905132800 are interpreted as '1983-09-05 13:28:00'
.
Therefore your expression is not a valid MySQL neither a date nor a timestamp literal. In order to comply with one of the literal formats detailed above, you must either create a script that makes your time data comply with the DATETIME or TIMESTAMP format and probably update your database to change the invoicedate type.
Related
My SQL knowledge is rather weak and I come from procedural programming, so bear with me. I have a database that contains data from a weather station - these are collected each minute and the (important part of the) table is
MariaDB [weather]> describe readings;
+------------------+------------+------+-----+-------------------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------------+------------+------+-----+-------------------+-------+
| time | timestamp | NO | PRI | CURRENT_TIMESTAMP | |
| inside_temp | float | YES | | NULL | |
| outside_temp | float | YES | | NULL | |
+------------------+------------+------+-----+-------------------+-------+
I want to find all days where the outside_temp was not lower and not larger than some values.
I can code it externally using MySQL for queries like
select min(outside_temp), max(outside_temp) from readings where date(time)='2022-01-27';
and iterating over all days in the database to check temperature values for each day separately, but I wonder if it is possible to do the selection just using MySQL command (I suppose it is, just beyond my imagination).
Something like select date(time), min(outside_temp), max(outside_temp) from readings group by date(time); would give you all timestamps that meet the requirements
How to update DATETIME column in mysql
Tried using: (and many others)
I do not really care how the date is formatted in the database however need to be able to update the current row with the 01/01/2001 01:01 format
update contacts set replydate=STR_TO_DATE('1/9/2020 13:32', '%m/%d/%Y hh:mm') where id='3';
The date is not the current date, these are all different dates from a spread sheet, that all have the same formatting.
MariaDB [ddcontactsdb]> describe contacts;
+----------------+-----------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+----------------+-----------------+------+-----+-------------------+-----------------------------+
| replydate | datetime | YES | | NULL | |
+----------------+-----------------+------+-----+-------------------+-----------------------------+
You must use this format:
update contacts set replydate=STR_TO_DATE('1/9/2020 13:32', '%m/%d/%Y %H:%i') where id='3';
because m stands for month and i for minutes, also H for hour 00-23.
I have been given the task of reading in a .txt file into a mySQL table.
Here is what I did.
CREATE TABLE PERSON(PIDM INT, FNAME VARCHAR(20), LNAME VARCHAR(20), SSN INT(8), DOB DATE, GENDER CHAR(2));
LOAD DATA LOCAL INFILE '/pathtoTextFile'
INTO TABLE PERSON
FIELDS TERMINATED BY ', '
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(PIDM, FNAME, LNAME, SSN, #DATE, GENDER)
SET DOB = STR_TO_DATE(#DATE, '%m-%d-%Y');
But I get several warnings from doing this.
including:
+---------+------+-----------------------------------------------------------------+
| Level | Code | Message |
+---------+------+-----------------------------------------------------------------+
| Warning | 1411 | Incorrect datetime value: '12/14/1957' for function str_to_date |
| Warning | 1411 | Incorrect datetime value: '01/13/1969' for function str_to_date |
| Warning | 1264 | Out of range value for column 'SSN' at row 3 |
| Warning | 1411 | Incorrect datetime value: '02/14/1976' for function str_to_date |
| Warning | 1366 | Incorrect integer value: '' for column 'SSN' at row 4 |
| Warning | 1411 | Incorrect datetime value: '03/12/1989' for function str_to_date
I haven't been able to figure out why I get an invalid date from the str_to_date function/method.
This is supposed to be a learning excercise but I have been at it for like 2 hours and haven't gotten anywhere. Does anyone have any tips or any idea of what I'm doing wrong?
Thank you
P.S. The purpose is not to change the .txt file for it to work, but to be able to handle incorrect inputs.
It depends on the source data but if there is a mixture of MM/DD/YYYY and MM-DD-YYYY you could replace the unwanted delimiter with the wanted delimiter, then convert to date.
Note: because Month First and Day First formats are "ambiguous" (e.g. 7/8/2014 could be a date in July or August) they should be avoided whenever possible.
I am parsing an xml file that has following data structure:
<row Id="253858" UserId="40883" Name="Scholar" Date="2009-03-08T01:52:32.570" />
<row Id="253860" UserId="19483" Name="Supporter" Date="2009-03-08T01:57:31.733" />
<row Id="253861" UserId="74951" Name="Autobiographer" Date="2009-03-08T02:02:32.390" />
I used a ruby script to parse this data and insert them into a mysql database. Here is how my data table looks like:
+---------+-------------+------+-----+-------------------+-----------------------------+
| Field | Type | Null | Key | Default | Extra |
+---------+-------------+------+-----+-------------------+-----------------------------+
| id | int(11) | NO | PRI | NULL | |
| user_id | int(11) | NO | | NULL | |
| name | varchar(40) | YES | | NULL | |
| created | timestamp | NO | | CURRENT_TIMESTAMP | on update CURRENT_TIMESTAMP |
+---------+-------------+------+-----+-------------------+-----------------------------+
This xml file has large number of records and until the parser gets to 3rd record I don't get any error and all the data get parse correctly and insert into the table:
| 253857 | 23658 | Critic | 2009-03-08 01:52:32 |
| 253858 | 40883 | Scholar | 2009-03-08 01:52:33 |
| 253860 | 19483 | Supporter | 2009-03-08 01:57:32 |
+--------+---------+--------------------+---------------------+
But when we get to the record with row Id="253861" I get the following mysql error:
load.rb:21:in `execute': Incorrect datetime value: '2009-03-08T02:02:32.390' for column 'created' at row 1 (Mysql::Error)
from load.rb:21:in `on_start_element'
from load.rb:133:in `parse'
from load.rb:133:in `<main>'
incase if you need the ruby method that insert records:
def on_start_element(element, attributes)
if element == 'row'
#st.execute(attributes['Id'], attributes['UserId'], attributes['Name'], attributes['Date'])
end
end
end
I don't think this is related to script, because I extracted the record from xml file and tried to insert directly into my mysql table and I got the following error:
mysql> insert into badge values(253861,74951,'Autobiographer','2009-03-08T02:02:32.390');
ERROR 1292 (22007): Incorrect datetime value: '2009-03-08T02:02:32.390' for column 'created' at row 1
I also tried to insert the record that is after the above record in the xml file and I got the same results:
mysql> insert into badge values(253862,49628,'Teacher','2009-03-08T02:12:30.807');
ERROR 1292 (22007): Incorrect datetime value: '2009-03-08T02:12:30.807' for column 'created' at row 1
So something in that date string makes mysql unhappy. What I couldn't figure out is that date and previous records that has the same date structure didn't have any problem. Hope I have explain the problem clear with information.
Since you said you insert dates in a couple of places I would suggest you write and helper method to do date conversions that you can reuse everywhere you need to insert dates
require 'date'
def iso8601_to_mysql_datetime(date)
DateTime.parse(date).to_time.strftime("%F %T")
end
iso8601_to_mysql_datetime('2009-03-08T02:02:32.390')
=> "2009-03-08 02:02:32"
NOTE: The above converts a ISO8601 into a string that MySQL understands
MySQL date and time literal documentation can be found here:
http://dev.mysql.com/doc/refman/5.6/en/date-and-time-literals.html
I'm having an interesting issue with Mysql DATE format.
I have this table :
| id | int(11) | NO | PRI | NULL | auto_increment |
| file_path | varchar(255) | YES | | NULL | |
| date_export | date | YES | | NULL | |
When i'm updating a row using the date function : NOW(), the date is updated with this format :
'2014-01-23'
But when i'm using another date format, like hand-written one like :
update backup_conf_allied set date_export='2014-23-01' where file_path='IDF-952584-SW1' ;
The date_export column transforms into :
'0000-00-00'
Warning table tells me that :
| Warning | 1265 | Data truncated for column 'date_export' at row 3628 |
Why? The date format is the same as NOW() function.
Thanks.
Posted query
update backup_conf_allied set `date_export='2014-23-01'` where file_path='IDF-952584-SW1' ;
What it should be
update backup_conf_allied set `date_export='2014-01-23'` where file_path='IDF-952584-SW1' ;
MySQL Support DATE format as 'YYYY-MM-DD' , Year then Month then Date, So you are updating a Date column with wrong value "2014-23-01" , There are only 12 months in year, you are using month 23 which is invalid that's MySQL is converting it to ZERO DATE (0000-00-00)
I had a similar problem recently, my issue was that I was trying to upload a datetime object as a date object. Although that's not the same issue that Gui O was having, if you run into this, make sure that you're actually trying to upload a date object.
I'm having this same problem but for a different reason. My MySQL column has a datetime type, but my datetime values are from Python, and they look like this: `'2021-04-01 03:58:50.088087'.
So the result is to truncate before the period:
t = t[0:19]
e.g.:
>>> datetime.datetime.now().isoformat()
'2021-04-03T08:28:41.602373'
>>> datetime.datetime.now().isoformat()[0:19]
'2021-04-03T08:28:44'
>>>