Date imported from csv into mysql as 0000-00-00 - mysql

I have some data saved as txt file. I am saving the txt file as csv in order to import it into a database using my sql workbench.
what I am doing is the following:
LOAD DATA LOCAL INFILE '/path/to/csv/file.csv' INTO TABLE mytable FIELDS TERMINATED BY ',' ENCLOSED BY '"' lines terminated by '\n';
But one of my column is a date, and it is imported as 0000-00-00
How to import it in a good way ?
Edit
Here is what my csv contains:
id task hoursWorked begindate enddate
0 task1 15 11/17/2012 11/18/2012
1 task2 20 11/18/2012 11/20/2012
2 task3 20 12/4/2012 12/5/2013
3 task4 22 1/5/2013 1/7/2013

Please have a try with this one:
LOAD DATA LOCAL INFILE '/path/to/csv/file.csv'
INTO TABLE mytable
LINES TERMINATED BY '\n'
(id, task, hoursWorked, #var1, #var2)
SET begindate = STR_TO_DATE(#var1, '%m/%d/%Y'),
enddate = STR_TO_DATE(#var2, '%m/%d/%Y');
For more info see LOAD DATA and STR_TO_DATE
Note: I deleted the FIELDS TERMINATED BY ',' ENCLOSED BY '"' part, cause I neither see , nor " in your CSV. But if it works fine for you the way it is, feel free to revert :)

The default date format is YYYY-MM-DD:
mysql> SELECT ##date_format;
+---------------+
| ##date_format |
+---------------+
| %Y-%m-%d |
+---------------+
... thus MySQL won't recognise stuff like 11/17/2012 as a proper date. In theory, you should be able to change the default format, but I'm not sure it can be done in session scope and I wouldn't recommend to change it for the whole server. It's better to make the transformation yourself. The trick is to insert the value into a variable rather than a column.
Additionally, there're two other issues:
Your CSV file contains a header line.
Your fields are not separated by ,.
Assuming your file uses tabs as separators, the complete command would be:
LOAD DATA LOCAL INFILE '/path/to/csv/file.csv'
INTO TABLE mytable
FIELDS TERMINATED BY '\t' ENCLOSED BY '"' LINES TERMINATED BY '\n'
IGNORE 1 LINES
(id, task, hoursWorked, #tmp_begindate, #tmp_enddate)
SET begindate = STR_TO_DATE(#tmp_begindate, '%m/%d/%Y'),
enddate = STR_TO_DATE(#tmp_enddate, '%m/%d/%Y');
MySQL doesn't actually allow to change ##date_format anyway:
mysql> SET ##date_format='%d/%m/%Y';
ERROR 1238 (HY000): Variable 'date_format' is a read only variable
As the MySQL 5.6 manual explains:
This variable is unused. It is deprecated as of MySQL 5.6.7 and will
be removed in a future MySQL release.
Also, at Date and Time Types we can read:
MySQL retrieves values for a given date or time type in a standard
output format, but it attempts to interpret a variety of formats for
input values that you supply (for example, when you specify a value to
be assigned to or compared to a date or time type). For a description
of the permitted formats for date and time types, see Section 10.1.3,
“Date and Time Literals”. It is expected that you supply valid values.
Unpredictable results may occur if you use values in other formats.
Although MySQL tries to interpret values in several formats, date
parts must always be given in year-month-day order (for example,
'98-09-04'), rather than in the month-day-year or day-month-year
orders commonly used elsewhere (for example, '09-04-98', '04-09-98').

If it didn't work, just add columns to your CVS for year, month and day and separate day, month and year of your date, and use the following:
set date_column = concat(#year , '-' , #month , '-' , #day)

Related

MySql Error Code: 1292. Incorrect date value: '2000 January' for column 'Dates' at row 1

My data looks like this:
Year Month Country Sales Dates
2018 March Malta 5000 2018 March
Trying to concatenate Year and Month in one field.
CREATE TABLE data (
Year INT,
Month VARCHAR (15),
Country VARCHAR (40),
Sales FLOAT,
Dates DATE
);
SET GLOBAL sql_mode = '';
UPDATE data SET Dates = STR_TO_DATE(Dates, '%Y %M');
LOAD DATA INFILE 'data.csv' INTO TABLE data
FIELDS TERMINATED BY ';'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
I also tried to use
UPDATE data SET Month = STR_TO_DATE(Month, '%M');
But either way I am getting :
Error Code: 1292. Incorrect date value: '2000 January' for column 'Dates' at row 1
(same to Month AND while in Command Line client and Workbench)
So I created additional column manually, but it looks like I cannot format it the way it's necessary.
If I used
UPDATE data SET Dates = str_to_date(concat(Year,' ',Month), '%Y %M');
without the manually created column, MySql returned:
Unknown column 'Dates' in 'field list'"
Using MySql Workbench 8.0.30

Insert a 4-digit as Time in MySql

My data contains hour & minute of time and I want to load this data into MySql database.
Sample Data: 305 -- This means Hours=03 & Minutes=05
But when I upload the data into MySql column of type TIME, it is stored as 00:03:05 (HH:MM:SS).
Whereas, I want it to be stored as 03:05:00.
I can make changes to my data using Python & then load into MySql. However, I was wondering if there is a way to do it using MySql itself.
It is not up to MySQL to guess what you mean by "305". It's up to you to convert/format yor data to "03:05:00", then MySQL will undertand it properly.
insert into try (elasp) values ("03:05:00")
See MySQL documentation: https://dev.mysql.com/doc/refman/8.0/en/time.html
While converting to TIME datatype the numeric value is assumed to have HHMMSS format.
MySQL 8.0 Reference Manual / ... / The TIME Type:
Be careful about assigning abbreviated values to a TIME column. MySQL interprets abbreviated TIME values with colons as time of the day. That is, '11:12' means '11:12:00', not '00:11:12'. MySQL interprets abbreviated values without colons using the assumption that the two rightmost digits represent seconds (that is, as elapsed time rather than as time of day). For example, you might think of '1112' and 1112 as meaning '11:12:00' (12 minutes after 11 o'clock), but MySQL interprets them as '00:11:12' (11 minutes, 12 seconds). Similarly, '12' and 12 are interpreted as '00:00:12'.
If you want to use HHMM format during convertion then simply multiply the value by 100.
CREATE TABLE test (int_source INT, time_destination TIME);
INSERT INTO test (int_source) VALUES ('305');
SELECT * FROM test;
UPDATE test SET time_destination = int_source * 100;
SELECT * FROM test;
✓
✓
int_source | time_destination
---------: | :---------------
305 | null
✓
int_source | time_destination
---------: | :---------------
305 | 03:05:00
db<>fiddle here
Or, if the value to be converted to TIME datatype has some string type you may concatenate '00' to it:
SET time_destination = CONCAT(int_source, '00')

Date Time Conversion in MySQL

I imported a CSV file into MySQL. The CSV file contains Date/Time in the format 'mm/dd/yyyy hh:mm'.
It gets imported into MySQL only as text. So I did import it as text. I then created another field hoping to convert this text value to date time in this format 'mm-dd-yyyy hh:mm' OR in this format 'YYYY-mm-dd hh:mm'. But it does not work. Below is my code.
ALTER TABLE table1 ADD COLUMN StartDateNEW DATETIME;
SET SQL_SAFE_UPDATES = 0;
UPDATE table1 SET StartDateNEW = STR_TO_DATE(StartDate, '%m/%e/Y %H:%i');
SET SQL_SAFE_UPDATES = 1;
Sample Data:
Some more sample data:
I have been trying this for over an hour now. Can someone please help?
If you are loading data using the LOAD DATA INFILE syntax, it should be possible to handle conversion on the fly.
Assuming that your source csv file looks like:
StartDate, EndDate, Value
"1/10/2012 10:05", "1/11/2012 11:51", abc
"1/13/2012 08:00", "1/15/2012 09:01", abc
You can defined columns StartDate and EndDate as datetime datatype, and simply do:
LOAD DATA INFILE '/path/to/my/file.csv' INTO TABLE mytable
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n' -- or '\n'
IGNORE 1 LINES
(#StartDate, #EndDate, Value)
SET
StartDate = STR_TO_DATE(#StartDate, '%c/%e/%Y %k:%i'),
EndDate = STR_TO_DATE(#EndDate, '%c/%e/%Y %k:%i')
;
NB: and if you are curently not using LOAD DATA INFILE... I would still recommend migrating your code to use it. This is the proper way to do this in MySQL, it works on all clients.... and it is very fast (see this link).
Based on your sample data, the correct format string for STR_TO_DATE is
%c/%e/%Y %k:%i
%c allows for single digit month numbers
%e allows for single digit day numbers
%Y four digit year
%k allows for single digit hours
%i two digit minutes
All the format strings are described in the manual for DATE_FORMAT.
Demo on dbfiddle
A quick hack is as follows:
Open CSV File in Excel
Select the date column
Change date format to yyyy-mm-dd using cell format popup
Save file
Import to MySQL
This was you don't need to import in text form and then try to convert the data into DateTime.

Laravel RAW query with IGNORE doesn't work as expected

I have a project for travel agent where I need to insert date that support incomplete date. So there are 3 possible format:
YYYY-MM-DD
YYYY-MM-00
YYYY-00-00
00 means month or day is not set.
In my database (MariaDB v10.1.28) I can run this query and it sets the date as I wish:
UPDATE IGNORE departure SET flight = "2019-00-00" WHERE id = "10"
I use IGNORE to bypass strict mode in config/database.php. In Laravel, when I tried to run this raw SQL query:
DB::update(UPDATE IGNORE departure SET flight = ? WHERE id = ?', ['2019-00-00', '10']);
said record will have its departure changed to 0000-00-00. Is this a bug? There is a workaround to use 3 tinyInt but changing structure might mess everything else up.

mysql data loading error (date order)

I have a large .txt file including 20 millions of lines of strings like:
"CS1221|123.10|17.02.2012 09:10:23,5676"
The first is customer id, then separated by "|" we have $ amount of transactions and finally date and time (dd.mm.yyyy hh:mm:ss,ssss).
I am trying to load it to Mysql table but it isn't accepting this ordering as TIMESTAMP (it accepts YYYY-MM-DD hh:mm:ss,ssss)
Is there any piece of code written in mysql that helps me?
You can use the STR_TO_DATE method to convert that date format. Try something like this:
SELECT STR_TO_DATE('17.02.2012 09:10:23,5676', '%d.%m.%Y %H:%i:%s,%f');
Should yield:
2012-02-17 09:10:23.567600
So your INSERT query would look something like:
INSERT INTO your_table (all, relevant, field_names) VALUES ("CS1221", "123.10", STR_TO_DATE('17.02.2012 09:10:23,5676', '%d.%m.%Y %H:%i:%s,%f'));