I am trying to load an xml file containing timestamp entries in the "20120924 22:12" format into MySql.
I am using the LOAD XML feature. Of course it's not working because MySQL is expecting "2012-09-24 22:12". If I was using LOAD FILE I would add
SET tmstmp = str_to_date(#var3, '%Y%m%d %h:i%')}
to my command, where the tmstmp data is in column 3 of the tile. So for XML I'd like to use
SET tmstmp = str_to_date(#tmstmp, '%Y%m%d %h:i%')}
where tmstmp is the tag containing my timestamp data. But this doesn't work. #tmstmp is empty. How do I access tags in SET statements under LOAD XML? The MySQL documentation just defers to the LOAD FILE documentation but it's not there of course. Thanks for any help.
The format is wrong (i% must be %i). So, change '%Y%m%d %h:i%' with '%Y%m%d %h:%i' and try to load data again.
Also %h should be %H because you use 24-hour format.
Related
I have a column of data, let's call it bank_date, that I receive from an external vendor as a csv file every day. As such the dates in that column show as '1/1/2020'.
I am trying to upload that raw csv file directly to SQL daily. We used to store the SQL bank_date format as text, but we have converted it to a Data data type, and now it keeps zero'ing out every time, with some sort of truncate / "datetime value incorrect" error.
I have now tested 17 different versions of utilizing STR_TO_date (mostly), CAST, and CONVERT, and feel like I'm close, but I'm not quite getting the syntax right.
Also for reference, I did find 2 other workarounds that are successful, but my boss specifically wants it uploaded and converted directly through the import process (not manipulating the raw csv data) for safety reasons. For reference:
Workaround 1: Convert csv date column to the YYYY-MM-DD format and save file. The issue with this is that if you try to open that CSV file again, it auto-changes the date format back to the standard mm/dd/yyyy. If someone doesn't know to watch out for this and is re-opening the csv file to double check something, they're gonna find an error when they upload, and the problem is not easy to identify.
Workaround 2:Create an extra dummy_date column in the table that is formatted as a text data type and upload as normal. Then copy and paste the data into the correct bank_date column using a str_to_date function as follows: UPDATE dummy_date SET bank_date = STR_TO_DATE(dummy_date, ‘%c/%e/%Y’); The issue with this is that it just creates extra unnecessary data that can be confused when other people may not know that 1 of the columns is not intended for querying.
Here is my current code:
USE database_name;
LOAD DATA LOCAL INFILE 'C:/Users/Shelly/Desktop/Date Import.csv'
INTO TABLE bank_table
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS
(bank_date, bank_amount)
SET bank_date = str_to_date(bank_date,'%Y-%m-%d');
The "SET" line is what I cannot work out on syntax to convert a csv's 1/5/2020' to SQL's 2020-1-5 format. Every test I've made either produces 0000-00-00 or nulls the column cells. I'm thinking maybe I need to tell SQL how to understand the csv's format in order for it to know how to convert it. Newbie here and stuck.
You need to specify a format for a date that is in the file, not a "required" one:
SET bank_date = str_to_date(bank_date,'%c/%e/%Y');
I'm trying to change the format of the general log file generated by MySQL 5.1.73.
the actual timestamp format for "event_time is:
"210909 10:32:12 12 Connect user#localhost on database"
Is there a possibility for changing its timestamp format ?
I would like the format to follow the format: 2015-04-14 22:52:11 or even containing the timezone following the RFC 3339. "1937-01-01T12:00:27.87+00:2" (UTC)
I found a solution by adding the mysql-general.log file as inputFile in rsyslog. Applying a template allow me to rewrite the log with a correct timestamp format.
Thanks for your replies / comments
I am trying to load data into a MySQL table and all of my timestamps (datetime field type) are being loaded as null. An example value I am trying to load is "2002-04-26 19:31:15.200000000" and I have specified the format as '%Y-%c-%d %H:%i:%s.%f' (statement below).
Since null is loading I am making an assumption that I have the format incorrect and that is causing the null to be loaded. What is the correct format that I should be specifying?
set DateAdded = STR_TO_DATE(#DateAdded,'%Y-%c-%d %H:%i:%s.%f')
The complete LOAD INFILE statement:
LOAD DATA INFILE 'CANDIDATES.txt' INTO TABLE dbo.CANDIDATE
FIELDS TERMINATED BY '|'
ignore 1 lines
(ResumeKey,Address1,Address2,City,Candidate_Type,Candidate_Type_Changed_UserID,State,Country,#DateAdded,#DateModified,Degree,Email,FirstName,GPA,GradYear,HomePhone,JobTitle,LastName,Locale,Major,MiddleName,OrgName,OtherPhone,SchoolName,Zip,Active_Flag,SecureCandidate_Flag,CandidateStackingField,CellPhone,Homepage,FaxNumber,BRUID)
set
DateAdded = STR_TO_DATE(#DateAdded,'%Y-%c-%d %H:%i:%s.%f'),
DateModified = STR_TO_DATE(#DateModified,'%Y-%c-%e %H:%i:%s.%f');
The issue was that the file that was being loaded was not in utf-8 format. MySQL can't load uff-16 files.
To convert the file on linux you would do this:iconv -f utf-16le -t utf-8 (file in uff-15le format) > (name of new uff-8 file)
if you need to find the current encoding of a file you can use file -ib filename.
After converting the file it loaded without issue.
i'm importing 1m+ records into my table from a csv file.
Works great using the load data local infile method.
However, the dates are all different formats.
A quick google lead me to this function:
STR_TO_DATE
However, when I implement that, I get nothing, an empty insert. here's my SQ cut down to include one date (I've 4 with the same issue) and generic column names:
load data local infile 'myfile.csv' into table `mytable`
fields terminated by '\t'
lines terminated by '\n'
IGNORE 1 LINES
( `column name 1`
, `my second column`
, #temp_date
, `final column`)
SET `Get Date` = STR_TO_DATE(#temp_date, '%c/%e/%Y')
If I do:
SET `Get Date` = #temp_date
The date from the csv is captured in the the format it was in the file.
However when I try the first method, my table column is empty. I've changed the column type to varchar (255) from timestamp to captre whatever is going in, but ultimatly, I want to capture y-m-d H:i:s (Not sure if STR_TO_DATE can do that?)
I'm also unsure as to why I need the # symbol.. google failed me there.
So, my questions are:
Why do I need the # symbol to use this function?
Should the data format ('%c/%e/%Y') be the format of the inputted data or my desired output?
Can I capture time in this way too?
sorry for the large post!
Back to Google for now...
Why do I need the # symbol to use this function?
The # symbol means that you are using a variable, so the read string isnt put right away into the table but into a memory pice that lets you operate with it before inserting it. More info in http://dev.mysql.com/doc/refman/5.0/en/user-variables.html
Should the data format ('%c/%e/%Y') be the format of the inputted data or my desired output?
Its the format of the inputted data, more info in http://dev.mysql.com/doc/refman/5.5/en/date-and-time-functions.html#function_str-to-date
Can I capture time in this way too?
You should be able to as long as you chose the correct format, something like
STR_TO_DATE(#temp_date,'%c/%e/%Y %h:%i:%s');
I had this problem. What solved it for me was making sure I accounted for whitespace that weren't delimiters in my load file. So if ',' is the delimiter:
..., 4/29/2012, ...
might be interpreted as " 4/29/2012"
So should be
...,4/29/2012,...
I am exporting a csv file in to mysql db using load data infile syntax.
the date in csv is in 2009/10/31 7:8:57.0 format. Is there any way to convert this while loading to something like 2009-10-31 07:08:57 ?
Execute TO_CHAR(TO_DATE(datefromcsv, 'YYYY/MM/DD HH:MI:SS.FF'), 'YYYY-MM-DD HH:MI:SS') when you are doing the INSERT into the db.
(usual caveats apply here) A regular expression might be what you need. Substitute / with - and remove the trailing .0.
I am surprised, though, that mysql can't handle dates like the one you provided. See for example the mySql manual. Have you tried feeding it to mysql and seeing what happens?