I am trying to import data from a CSV file to MySQL. I have a column called attendance whereby there is a possibility that it is a null value in the excel file. Therefore, when importing the data to MySQL, I want to convert these null value in the excel file to a 0 value in MySQL. However, I keep getting an error called "Incorrect Integer value" for the attendance column. I was wondering is there issue with my definition in this line:
SET
attendance = NULLIF(#one, "0");
SQL:
CREATE TABLE IF NOT EXISTS students(
id INT AUTO_INCREMENT,
name DATE,
course INT,
attendance INT,
PRIMARY KEY (id)
);
LOAD DATA INFILE
'C:/Users/ben/OneDrive/Desktop/studentslist.csv'
INTO TABLE students
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY "\n"
IGNORE 1 ROWS
(id, name, course, #dummy, #one)
SET
attendance = NULLIF(#one, "0");
Related
I have the data saved in my csv excel. The data have some null value in row and column. I want to save this data into my database in MySQL. But null value is causing problem in saving the data to MySQL. This is the query for creating the table -
create table student (
Std_ID int,
Roll_NO int,
First_Name varchar(10) NOT NULL,
Last_Name varchar(10),
Class int,
constraint test_student primary key (Std_ID)
);
...and it ran successfully. Now I want to save my data from csv to this table using the query -
load data infile 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\new.csv' into table student fields terminated by ',' lines terminated by '\n' ignore 1 lines;
...and is giving me the error msg -
ERROR 1366 (HY000): Incorrect integer value: '' for column 'XXX' at row X.
For the reference you can use this data.
The same can be find below.
LOAD DATA INFILE 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\new.csv'
INTO TABLE student
FIELDS TERMINATED BY ','
LINES TERMINATED by '\n'
IGNORE 1 LINES
-- specify columns, use variables for the columns where incorrect value may occur
(Std_ID, #Roll_NO, First_Name, Last_Name, #Class)
-- use preprocessing, replace empty string with NULL but save any other value
SET Roll_NO = NULLIF(#Roll_NO, ''),
Class = NULLIF(#Class, '');
If some column in CSV is empty string then NULL value will be inserted into according column of the table.
Std_ID is not preprocessed because it is defined as PRIMARY KEY, and it cannot be NULL.
UPDATE
OP provides source file sample. Viewing it in HEX mode shows that the file is Windows-style text file, and hence the lines terminator is '\r\n'. After according edition the file is imported successfuly.
I want to load data from a CSV file into a table in my database. However, the first two columns are an ID (AUTO INCREMENT, PRIMARY KEY) and CURRENT DATE (CURRENT DATE ON UPDATE) field. The source file will therefore not have these fields.
How would I get the file to load? The below code doesn't work. I have also tried leaving out id and action_date from the fields in the brackets.
LOAD DATA INFILE '/upliftments//UPLIFTMENTS_20160901.csv'
INTO TABLE database.return_movements
CHARACTER SET latin1
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
IGNORE 1 LINES
( id
, action_date
, current_code
, serial_number
, new_code
);
I am trying to import data from a csv file into MySQL using LOAD DATA LOCAL INFILE. All columns except the date columns which have timestamp as their datatype are imported correctly. I am getting the error 1265: data truncated for date column and it inserts 0000-00-00 00:00:00 for all values.This has been asked before but I did not find a perfect solution for this. I have also tried various solutions posted for this type of question but none have worked for me.
table create statement :
CREATE TABLE MySchema.response
(
`id` int,
`version` int,
`name` varchar(500),
`date_created` timestamp,
`last_updated` timestamp,
`count` int,
);
loading data into table:
LOAD DATA LOCAL INFILE 'C:/response.csv'
INTO TABLE MySchema.response
FIELDS TERMINATED BY ',' optionally ENCLOSED by '"'
ignore 1 lines
Sample Data in CSV file
id version name date_created last_updated count
1, 0, xyz, 5/3/2013 1:18, 5/3/2013 1:18, 2
2, 0, abc, 5/3/2013 1:18, 5/3/2013 1:18, 1
Date columns in your sample are not in MySQL's default format and thus are not identified as dates. You need to try something like the following, in order to state how the dates should be interpreted:
LOAD DATA LOCAL INFILE 'C:/response.csv'
INTO TABLE MySchema.response
FIELDS TERMINATED BY ',' optionally ENCLOSED by '"'
IGNORE 1 lines
(id, version, name, #date_created, #last_updated, count)
SET date_created = STR_TO_DATE(#date_created, '%d/%c/%Y %k:%i'),
last_updated = STR_TO_DATE(#last_updated, '%d/%c/%Y %k:%i');
Check MySQL date format documentation for specifiers that suit your case (probably the ones I added in the sample above).
i use below script for insert data to sql from textpad.
#!/bin/bash
mysql --utest -ptest test << EOF
LOAD DATA INFILE 'test.txt'
INTO TABLE content_delivery_process
FIELDS TERMINATED BY ',';
EOF
in my test file i have a format like,
cast , date , name , buy
i can insert but i need format like below,
S.NO | date | name | buy | cast
You can specify the columns you want to import:
From the MySQL Manual:
MySQL LOAD DATA INFILE
The following example loads all columns of the persondata table:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata;
By default, when no column list is provided at the end of the LOAD
DATA INFILE statement, input lines are expected to contain a field for
each table column.
If you want to load only some of a table's columns, specify a column
list:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata (col1,col2,...);
You must also specify a column list if the order of the fields in the
input file differs from the order of the columns in the table.
Otherwise, MySQL cannot tell how to match input fields with table
columns.
You would include "FIELDS TERMINATED BY '|';" at the end to import data delimited with a '|' symbol.
Hope this helps.
create table [YOUR TABLE] ( `S.NO` INT AUTO_INCREMENT, date DATETIME, name VARCHAR(50), buy VARCHAR(50), cast VARCHAR(50));
Load data local infile 'test.txt' ignore into table [YOUR TABLE] fields terminated by ',' lines terminated by '\n'(cast , date , name , buy);
I'm using LOAD DATA INFILE to upload a .csv into a table.
This is the table I have created in my db:
CREATE TABLE expenses (entry_id INT NOT NULL AUTO_INCREMENT, PRIMARY KEY(entry_id),
ss_id INT, user_id INT, cost FLOAT, context VARCHAR(100), date_created DATE);
This is some of the sample data I'm trying to upload (some of the rows have data for every column, some are missing the date column):
1,1,20,Sandwiches after hike,
1,1,45,Dinner at Yama,
1,2,40,Dinner at Murphys,
1,1,40.81,Dinner at Yama,
1,2,1294.76,Flight to Taiwan,1/17/2011
1,2,118.78,Grand Hyatt # Seoul,1/22/2011
1,1,268.12,Seoul cash withdrawal,1/8/2011
Here is the LOAD DATA command which I can't get to work:
LOAD DATA INFILE '/tmp/expense_upload.csv'
INTO TABLE expenses (ss_id, user_id, cost, context, date)
;
This command completes, uploads the correct number of rows into the table but every field is NULL. Anytime I try to add FIELDS ENCLOSED BY ',' or LINES TERMINATED BY '\r\n' I get a syntax error.
Other things to note: the csv was created in MS Excel.
If anyone has tips or can point me in the right direction it would be much appreciated!
First of all I'd change FLOAT to DECIMAL for cost
CREATE TABLE expenses
(
entry_id INT NOT NULL AUTO_INCREMENT PRIMARY KEY,
ss_id INT,
user_id INT,
cost DECIMAL(19,2), -- use DECIMAL instead of FLOAT
context VARCHAR(100),
date_created DATE
);
Now try this
LOAD DATA INFILE '/tmp/sampledata.csv'
INTO TABLE expenses
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n' -- or \r\n
(ss_id, user_id, cost, context, #date_created)
SET date_created = IF(CHAR_LENGTH(TRIM(#date_created)) > 0,
STR_TO_DATE(TRIM(#date_created), '%m/%d/%Y'),
NULL);
What id does:
it uses correct syntax for specifying fields and columns terminators
since your date values in the file are not in a proper format, it first reads a value to a user/session variable then if it's not empty it converts it to a date, otherwise assigns NULL. The latter prevents you from getting zero dates 0000-00-00.
Here is my advice. Load the data into a staging table where all the columns are strings and then insert into the final table. This allows you to better check the results along the way:
CREATE TABLE expenses_staging (entry_id INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY(entry_id),
ss_id varchar(255),
user_id varchar(255),
cost varchar(255),
context VARCHAR(100),
date_created varchar(255)
);
LOAD DATA INFILE '/tmp/expense_upload.csv'
INTO TABLE expenses_staging (ss_id, user_id, cost, context, date);
This will let you see what is really being loaded. You can then load this data into the final table, doing whatever data transformations are necessary.