I am trying to load data from a CSV into a database in MySQL workbench. The table I am loading into has an auto increment ID column. I am trying to get the query to recognize that I want it to keep the first column as Null, and I put NULL as the value in the CSV, but I cannot get the SET ... NULL command to recognize the name of the ID column. Here is the SQL I am using:
load data infile 'filenam.csv'
INTO TABLE table_name
fields Terminated By ','
LINES TERMINATED BY ',,'
SET column_name = null
I suspect I am making a simple syntax error that is causing the problem. But I cannot figure out what it is.
If you put NULL as the value in the CSV file then you shouldn't need the "SET column_name = null" in the statement. AFAIK, the SET value should be used to supply values not derived from the input file or to perform calculations to the value before insertion. This statement should work fine since you said you specified NULL in the CSV. However, make sure you specified NULL "properly" according to the documentation. I always use \N in my import files.
LOAD DATA INFILE 'filename.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY ',,'
Secondly, you can discard the NULL specified in the CSV file by assigning it to a user variable and then specifying the column value with SET. However, you need to specify a column list
LOAD DATA INFILE 'filename.csv'
INTO TABLE table_name (#dummy, column_2, column_3)
FIELDS TERMINATED BY ','
LINES TERMINATED BY ',,'
SET column_name = NULL
I have one other thought based on the MySQL docs and dependent upon how your server is configured. Comment if this does not work and I will provide more options.
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
Related
I have a dataset as follows,
Date,Time,ID,Name,Count
01-MAY-2009,00:00,4,Town Hall (West),209
01-MAY-2009,02:00,17,Collins Place (South),28
01-MAY-2009,23:00,18,Collins Place (North),36
For this I have created table with following schema,
CREATE TABLE table_name(date DATE, time TIME, id int, name VARCHAR(50), count int);
And for loading the table from ".csv" file as,
LOAD DATA INFILE '/home/cloudera/dataset.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(#var1, #var2, id, name, count)
SET date = STR_TO_DATE(#var1, '%d-%b-%Y')
SET time = TIME(#var2, "%H:%i");
But I get an error as,
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SET time = TIME(#var2, "%H:%i")' at line 1
I can't understand what the error is. I have tried going through MySQL website and documentation, but can't make out what the correct format is. Can anyone please me. Thanks in advance.
I don't think that you even need to be using the TIME function here. Your current hour:minute string time literals should be good enough, q.v. the documentation:
Be careful about assigning abbreviated values to a TIME column. MySQL interprets abbreviated TIME values with colons as time of the day. That is, '11:12' means '11:12:00', not '00:11:12'
As valid literals, your times would be interpreted as having a zero second component. So, if you just insert those time strings as is, I think it should work. Try the following code:
LOAD DATA INFILE '/home/cloudera/dataset.csv'
INTO TABLE table_name
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
(#var1, Time, id, name, count)
SET date = STR_TO_DATE(#var1, '%d-%b-%Y');
Tim is right in pointing out that you do not need to convert your time data in the load data infile statement.
Just to answer why you get a syntax error: load data infile can only have a single set clause, in which assignments to multiple columns are separated by comma. Your code has multiple set clauses, hence it fails.
Also, the time() function does not have a 2nd parameter for a pattern. The function you need to use is called str_to_date().
So, it should look like as follows:
...
SET date = STR_TO_DATE(#var1, '%d-%b-%Y'), time = TIME(str_to_date(#var2, "%H:%i"));
I have many files, for example: 20170319, 20170320 ...
For each file I have 2 columns, one for username and the other was data.
I've created a table
create table A(user varchar(35), date date, data varchar(35), primary key(user, date));
Then, I want to load those files into database, and use filename as specific date in date field.
Can I still use sth like:
Load data infile '20170320' into table A
The answer is that you cannot do this in MySQL alone, you need to use an external program or script that builds the load data infile statements with the appropriate SET clause derived from the name of the file:
The SET clause can be used to supply values not derived from the input
file. The following statement sets column3 to the current date and
time:
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, column2)
SET column3 = CURRENT_TIMESTAMP;
The reason for this is that:
load data infile cannot use the file name as an input variable
neither MySQL prepared statements using the prepare statement, nor stored procedures are allowed to use the load data infile statement.
I am trying to insert some data using Load data infile into a mysql table that already has some data. The table contains id and name. My csv file contains three fields: id, name and code. The table schema also has these three fields, but currently, the table has NULL for the code field. I am trying to insert the code from csv to an existing row in the table if it matches the name, else I am trying to insert a complete new row.
The code I have tried is as follows:
LOAD DATA INFILE 'table1.csv'
INTO TABLE table1
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(#code, #name, #other_columns)
IF EXISTS (SELECT * FROM table1 where name=#name);
BEGIN
set code=#Code;
END
ELSE
BEGIN
set code=#Code, name=#name;
END
By doing so, I am getting a mysql syntax error, but am unable to figure it out. Can anyone point me in the right direction, or suggest to me another approach? I have thousands of new rows to insert and thousands of existing rows to modify based on the certain field, (name in this case).
MySQL does not allow the LOAD DATA INFILE statement inside a stored program, which is where the IF statement appears. Break up your task into two parts. First, LOAD DATA INFILE into a temporary table. Then create a stored program that replaces the loaded data into your table1 from the temporary table.
I have tried the suggestion in the question below but I still have syntax errors.
How to LOAD DATA INFILE in mysql with first col being Auto Increment?
create table db.test
(ai_id int(11) auto_increment primary key,
field varchar(5))
LOAD DATA LOCAL INFILE 'C:\\Users\\nick\\Desktop\\test\\book1.csv'
INTO TABLE db.test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
(field)
SET ai_id = NULL
IGNORE 1 lines;
I am having trouble reconciling this seemingly very simple syntax error, any assistance greatly appreciated!
EDIT:
error code: 1064: You have error in SQL syntax; check syntax around 'ignore 1 lines' line 8.
datasource is a csv with one column "field" with five rows "one"-"five"(all five rows are characters not int)
This syntax is correct, I tested its working(in MySQL 5.6). please verify your input file.
The following works. It appears the order of commands is what threw it off.
LOAD DATA LOCAL INFILE 'C:\\Users\\nshatz\\Desktop\\test\\book1.csv'
INTO TABLE db.test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 lines
(field);
I'm importing a CSV file with dotted german dates into a MySQL database. I want the dates in the CSV to automatically be formatted correctly to the correct data type fields used by MySQL.
I'm using Sequel Pro for the import. I gather I'm supposed to use the STR_TO_DATE function, but I just can't wrap my head around how to use Add Value or Expression in the program.
German date
Here are the dates in the CSV file:
DD.MM.YYYY e.g.: 28.01.1978
MySQL date
Here is what I want to end up with in the database:
YYYY-MM-DD
e.g.: 1978-01-28
Here's what I've tried
I put in STR_TO_DATE('$5', '%d.%m.%Y'); into Add Value or Expression, but this only gives the following error message:
[ERROR in row 1] You have an error in your SQL syntax; check the manual that corresponds to your
MySQL server version for the right syntax to use near '06.04.1997'', '%d.%m.
%Y');,'2KMX','43354997')' at line 2
Any ideas?
You need import the date field in a varchar fields (temp_varchar_field) first, after that, you can use something like:
update table_name set final_date=STR_TO_DATE(temp_varchar_field,'%d.%m.%Y');
You should do something like:
Create a temporary field: alter table table_name add column temp_varchar_field varchar(10);
Import, using Sequel Pro, the CVS file but using the temp_varchar_field for the date.
update table_name set final_date=STR_TO_DATE(temp_varchar_field,'%d.%m.%Y');
Delete the temp field if everything was imported properly. Using: alter table_name drop column temp_varchar_field;
I just got it to work with this piece of SQL-code:
load data local infile 'myfile.csv' into table `mytable`
fields terminated by ','
enclosed by '"'
lines terminated by '\n'
(surname, name, #germandate, telephone, etc)
set birthyear = STR_TO_DATE(#germandate , "%d.%m.%Y")
;
The clue here being the #germandate variable which is turned into the default MySQL date by setting the respective column with STR_TO_DATE(). No hacks needed! :)
It is easier if your CSV import would contain a date as a MySQL string, but it can be done otherwise too:
Step 1:
Define a varchar(10) for your german dates and import the data.
Step 2:
Add another field to your table:
ALTER TABLE `yourtable`
ADD COLUMN `your_mysql_date` DATE NULL;
Step 3:
Move the data:
UPDATE yourtable
SET your_mysql_date = CONCAT(
RIGHT(your_german_date,4),
'-',
MID(your_german_date,4,2),
'-',
LEFT(your_german_date,2)
);
...done!
There might be an easier way to solve this, but this way you have alot of control over the data and the formatting.