Importing .csv and converting to dates in mysql - mysql

I am trying to import a CSV file with date information in the format MMM-YY (e.g., Jan-92). My ultimate goal is to have the information in a date format in SQL. The format doesn't matter that much to me, but I was thinking something like dd-mm-yy (e.g., 16-01-92). I've tried a lot and looked around the forum, but can't figure it out.
Right now, I am loading the data from the CSV into a column "period_month" as a VARCHAR field.
Then, my conversion code looks like
UPDATE PERIODS
SET period_month = DATE(str_to_date(period_month,'%M-%Y'));
I end up with a field of VARCHAR type in the format yyyy-mm-dd, with the dd field set to '00'. (e.g., 1992-01-00)
I really just want this to be in a date format that I can export to another program like R for analysis.
Any help appreciated.

try
UPDATE PERIODS
SET period_month =DATE(str_to_date('Jan-92','%M-%Y')+1)
or
UPDATE PERIODS
SET period_month =DATE(str_to_date('period_month','%M-%Y')+1)

use date_format function of mysql query

Related

How to convert date from MM/dd/yyyy to yyyy-MM-dd in MySQL? Error Code: 1411. Incorrect datetime value: '' for function str_to_date

I have a large dataset with employees' time entries. The current date format is MM/dd/yyyy. However, I need to convert all the dates into yyyy-MM-dd format.
I have tried the following:
Update human_resources.timekeeping
Set Actual_Date = str_to_date(Actual_Date,'%d-%m-%Y');
Got the errror messsage Error Code: 1411. Incorrect datetime value: '' for function str_to_date.
My SQL version is 5.7.18-log.
I tried to view SQL mode using SELECT ##sql_mode; and I got NO ENGINE SUBSTITUTION.
I have tried to retrieve the value like shown below and it was working fine.
Converting varchar mm/dd/yy to date format yyyy-mm-dd
However, updating the data would not work. I need to update the actual records, not insert new records.
Hope someone can help me regarding this. Thank you in advance!
EDIT: The data type for Actual_Date is VARCHAR.
Apologies if my explanation may be a bit confusing. But I am using this data set to display and filter time entries in a gridview. When I am filtering dates, say for example (01/15/2022-01/25/2022), data from 2021 is also being displayed. When I tried to manually change the format of some of my data in sql to yyyy-MM-dd, my code seemed to be working fine. The problem is there are a lot of data in this table, which is why manually updating the format is impossible. What is the first thing that I need to do? I'm sorry this is all still a bit confusing for me.
My apologies if you have already taken the following things into consideration but I thought them worth mentioning.
Given that you say this is a "large dataset" I assume this is a table that is currently in use. Does the existing application rely on the Actual_Date being in that string format? Does it rely on a fixed number of columns in the table? Some poorly written applications can be very brittle when it comes to changing underlying data structure.
You may want to consider creating a copy of the current table, modifying the structure of the copy, and replacing the original with a view with the same columns and formats as the original. This way you get improved data but reduce risk to existing application.
In the title and first line of your question you state that the current format is MM/dd/yyyy
Update human_resources.timekeeping Set Actual_Date = str_to_date(Actual_Date,'%m/%d/%Y');
Your separator is / not -
%d-%m-%Y >> %d/%m/%Y

Simple way to unify multiple date formats?

I need to import a large file of csv data into MySQL, and when I attempted to use MySQL's unix_timestamp function to import the dates, about half of the records didn't make it.
As far as I can tell, the datetime values are formatted with either a single first "month" digit or two of them, and the same goes with the day of the month (e.g. 6/6/2014 3:48PM vs. 12/16/2014 3:48PM) This throws off the import completely (well about half of the records won't import).
I'm trying to convert this into a unix_timestamp.
Now I know I could write a script with a regex to fix something like this, but I am wondering is there a simpler way to do a mass import like this? For the record, I am using my text editor to write the sql statements from the csv as "insert into" statements. This is where I tried to use date formatting but it seems to only accept one format.
Any way to do this with such a minor difference in input?
Actually, despite my comment, something like this might work:
COALESCE(STR_TO_DATE(val, "formatcandidate1")
, STR_TO_DATE(val, "formatcandidate2")
, STR_TO_DATE(val, "formatcandidate3")
, STR_TO_DATE(val, "formatcandidate4")
, [etc...]
) AS dateVal
There are online tools to do this kind of stuff
reports.zoho.com is one of them
In this tool you can import data applying a specific date format and skip the other rows.
and you can do the same for all the type of formats that are present in your file
and finally you can export the data with same date format for all the data
ask me any doubts if you have any regarding this :)

How to find Date format in MySQL

I am looking to simply find out what is the current Date format that I have in m MYSql Database. This should be easy but I am not finding the command or function to do this. I simply want to query for date format and have it return what it is currently set at, basically; DD:MM:YY or YYYY:MM:DD or whatever the format currently is set at for a particular DB.
Thank You
By default, the date format is YYYY-MM-DD. However you can convert any date with the CONVERT()-function. Make sure to look this function up.
You'll have lots of possibilities.

How to make the default date format dd/mm/yyyy in SQL Server 2008?

I have an Excel file that contains a column full of dates in the dd/mm/yyyy format. When I try to import it using openrowset, it said that there was a datatype mismatch. I have a table where the date is defined as type date. Now, I know that the default date format in SQL Server is yyyy-mm-dd. How can I avoid this conflict? Is there a way I can make the default date type be dd/mm/yyyy? I need to do this import operation everyday and it has to be automated and so I cannot afford it to fail in between. I tried using sp_addlanguage to make it British as the default date type is dd/mm/yyyy there, but it didn't work :(. I'm using SQL Server 2008 and Windows 7, if that is of any help. Please help me out! Thanks!
You could CONVERT the incoming data before you insert it. So, in the openrowset statement, where you select the field, you could surround it with a CONVERT statement. Here's an example:
print convert(date,'19/07/2010',103)
This is a UK style date, but if you run it you can see that it's converted it to SQL-friendly format.

MySqlImport - Import a date field not in the proper format

I have a csv file that has a date field in a format like (among other fields):
17DEC2009
When I do a mysqlimport, the other fields are imported properly, but this field remains 0000-00-00 00:00:00
How can I import this date properly? Do I have to run a sed/awk command on the file first to put it into a proper format? If so, what would that be like? Does the fact that the month is spelled out instead of a number matter?
STR_TO_DATE() enables you to convert a string to a proper DATE within the query. It expects the date string, and a format string.
Check the examples in the manual entry to figure out the correct format.
I think it should be along the lines of %d%b%Y (However the %b is supposed to produce Strings like Dec instead of DEC so you will have to try out whether it works).
I had this issue in the past. What I had to do was to utilize LOAD DATA and set the appropriate expression here -
[SET col_name = expr,...]
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Here is the approach I took to solve similar problem. My use case was bit complex with so many columns, but making here simple to present the solution.
I have Persons table with (Id int autogen, name varchar(100),DOB date), and few million of data(name,DOB) needs to be populated from CSV file.
Created additional column in persons table with name like (varchar_DOB varchar(25)).
Imported data using mysqlimport utility into columns(name,varchar_DOB).
Executed update query that updated DOB column using str_to_date(varchar_DOB,'format') function.
Now, I have expected data populated DOB column.
The same logic could be applied in doing even other kind of data formatting like double,time_stamp etc.