Error in saving csv to MySQL table due to NULL values - mysql

I have the data saved in my csv excel. The data have some null value in row and column. I want to save this data into my database in MySQL. But null value is causing problem in saving the data to MySQL. This is the query for creating the table -
create table student (
Std_ID int,
Roll_NO int,
First_Name varchar(10) NOT NULL,
Last_Name varchar(10),
Class int,
constraint test_student primary key (Std_ID)
);
...and it ran successfully. Now I want to save my data from csv to this table using the query -
load data infile 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\new.csv' into table student fields terminated by ',' lines terminated by '\n' ignore 1 lines;
...and is giving me the error msg -
ERROR 1366 (HY000): Incorrect integer value: '' for column 'XXX' at row X.
For the reference you can use this data.
The same can be find below.

LOAD DATA INFILE 'C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\new.csv'
INTO TABLE student
FIELDS TERMINATED BY ','
LINES TERMINATED by '\n'
IGNORE 1 LINES
-- specify columns, use variables for the columns where incorrect value may occur
(Std_ID, #Roll_NO, First_Name, Last_Name, #Class)
-- use preprocessing, replace empty string with NULL but save any other value
SET Roll_NO = NULLIF(#Roll_NO, ''),
Class = NULLIF(#Class, '');
If some column in CSV is empty string then NULL value will be inserted into according column of the table.
Std_ID is not preprocessed because it is defined as PRIMARY KEY, and it cannot be NULL.
UPDATE
OP provides source file sample. Viewing it in HEX mode shows that the file is Windows-style text file, and hence the lines terminator is '\r\n'. After according edition the file is imported successfuly.

Related

Set null value in CSV file to 0 in mysql

I am trying to import data from a CSV file to MySQL. I have a column called attendance whereby there is a possibility that it is a null value in the excel file. Therefore, when importing the data to MySQL, I want to convert these null value in the excel file to a 0 value in MySQL. However, I keep getting an error called "Incorrect Integer value" for the attendance column. I was wondering is there issue with my definition in this line:
SET
attendance = NULLIF(#one, "0");
SQL:
CREATE TABLE IF NOT EXISTS students(
id INT AUTO_INCREMENT,
name DATE,
course INT,
attendance INT,
PRIMARY KEY (id)
);
LOAD DATA INFILE
'C:/Users/ben/OneDrive/Desktop/studentslist.csv'
INTO TABLE students
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY "\n"
IGNORE 1 ROWS
(id, name, course, #dummy, #one)
SET
attendance = NULLIF(#one, "0");

Loading a table from a csv file: Error code 1292 Incorrect DATA value, but it's correct

I have a csv file with dates written as YYYY-MM-DD but I'm getting an incorrect date value while importing the file. I created the table and tried to import a csv file as it as follows:
CREATE TABLE afiliados (
afiliado_id INT PRIMARY KEY,
fecha DATE,
nombre VARCHAR(30),
municipio VARCHAR(20),
dni_impreso VARCHAR(2),
ficha_bien_llenada VARCHAR(2),
comentario VARCHAR(100)
);
DESCRIBE afiliados;
LOAD DATA INFILE '/var/lib/mysql-files/fichis.csv'
INTO TABLE afiliados
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
And here is what the csv file looks like
When I execute the second part, i get this:
Error Code: 1292. Incorrect date value: '' for column 'fecha' at row 1
I'm following this tutorial
http://www.mysqltutorial.org/import-csv-file-mysql-table/
I'm not sure if it's the correct to use ENCLOSED BY "" or LINES TERMINATED BY '\n' but the guy in the tutorial had a pretty similar file to mine. Either way, erasing these terms yields the same result.
EDIT: Managed to solve it by importing the file using a wizard that MySQL Workbench has, but I'm still wondering how to do it manually.

using linux shell script insert dump data to Mysql

i use below script for insert data to sql from textpad.
#!/bin/bash
mysql --utest -ptest test << EOF
LOAD DATA INFILE 'test.txt'
INTO TABLE content_delivery_process
FIELDS TERMINATED BY ',';
EOF
in my test file i have a format like,
cast , date , name , buy
i can insert but i need format like below,
S.NO | date | name | buy | cast
You can specify the columns you want to import:
From the MySQL Manual:
MySQL LOAD DATA INFILE
The following example loads all columns of the persondata table:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata;
By default, when no column list is provided at the end of the LOAD
DATA INFILE statement, input lines are expected to contain a field for
each table column.
If you want to load only some of a table's columns, specify a column
list:
LOAD DATA INFILE 'persondata.txt' INTO TABLE persondata (col1,col2,...);
You must also specify a column list if the order of the fields in the
input file differs from the order of the columns in the table.
Otherwise, MySQL cannot tell how to match input fields with table
columns.
You would include "FIELDS TERMINATED BY '|';" at the end to import data delimited with a '|' symbol.
Hope this helps.
create table [YOUR TABLE] ( `S.NO` INT AUTO_INCREMENT, date DATETIME, name VARCHAR(50), buy VARCHAR(50), cast VARCHAR(50));
Load data local infile 'test.txt' ignore into table [YOUR TABLE] fields terminated by ',' lines terminated by '\n'(cast , date , name , buy);

Load Data Infile Error with date

I am trying to load a .csv file that has 5 columns into a table that has the same corresponding columns plus a PK. Dates are data type DATE and all others are Varchar(), except PK.
Here is my load data Import:
LOAD DATA INFILE 'C:\Events_Upload.csv'
INTO TABLE db.events
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
Ignore 1 lines
(
Event_Name,
Event_Handle,
Create_Date,
Retire_Date,
Event_Desc
)
The error is :
Error Code: 1292. Incorrect date value: '' for column 'Retire_Date' at row 1
Row one in CSV looks like:
Time to Send Survey,SurvESend,2013-04-10,,Time for the system to send out the Esurveys
How can I make this field NULL and not blank for uploading from CSV?
There is nothing as null in CSV file. What you can do is may be provide some default value for the missing column and replace it with null by the query after insertion into the table.
Or you can have a look at skipping-empty-csv-objects
check this Export null to .csv and https://superuser.com/questions/390031/how-to-write-null-into-csv-from-excel-for-blank-fields

error 1265. Data truncated for column when trying to load data from txt file

I have table in mysql table
table looks like
create table Pickup
(
PickupID int not null,
ClientID int not null,
PickupDate date not null,
PickupProxy varchar (40) ,
PickupHispanic bit default 0,
EthnCode varchar(2),
CategCode varchar (2) not null,
AgencyID int(3) not null,
Primary Key (PickupID),
FOREIGN KEY (CategCode) REFERENCES Category(CategCode),
FOREIGN KEY (AgencyID) REFERENCES Agency(AgencyID),
FOREIGN KEY (ClientID) REFERENCES Clients (ClientID),
FOREIGN KEY (EthnCode) REFERENCES Ethnicity (EthnCode)
);
sample data from my txt file
1065535,7709,1/1/2006,,0,,SR,6
1065536,7198,1/1/2006,,0,,SR,7
1065537,11641,1/1/2006,,0,W,SR,24
1065538,9805,1/1/2006,,0,N,SR,17
1065539,7709,2/1/2006,,0,,SR,6
1065540,7198,2/1/2006,,0,,SR,7
1065541,11641,2/1/2006,,0,W,SR,24
when I am trying to submit it by using
LOAD DATA INFILE 'Pickup_withoutproxy2.txt' INTO TABLE pickup;
it throws error
Error Code: 1265. Data truncated for column 'PickupID' at row 1
I am using MySQL 5.2
This error means that at least one row in your Pickup_withoutproxy2.txt file has a value in its first column that is larger than an int (your PickupId field).
An Int can only accept values between -2147483648 to 2147483647.
Review your data to see what's going on. You could try to load it into a temp table with a varchar data type if your txt file is extremely large and difficult to see. Easy enough to check for an int once loaded in the database.
Good luck.
You're missing FIELDS TERMINATED BY ',' and it's assuming you're delimiting by tabs by default.
I had same problem. I wanted to edit ENUM values in table structure. Problem was because of rows that was saved before and new ENUM values doesn't contain saved values.
Solution was updating old saved rows in MySql table.
I had this issue when trying to convert an existing varchar column to enum. For me the issue was that there were existing values for that column that were not part of the enum's list of accepted values. So if your enum will only allow values, say ('dog', 'cat') but there is a row with bird in your table, the MODIFY COLUMN will fail with this error.
I have met this problem with a column that has ENUM values('0','1').
When I was trying to save a new record, I was assigning value 0 for the ENUM variable.
For the solution: I have changed ENUM variable value from 0 to 1, and 1 to 2.
The reason is that mysql expecting end of the row symbol in the text file after last specified column, and this symbol is char(10) or '\n'. Depends on operation system where text file created or if you created your text file yourself, it can be other combination (Windows uses '\r\n' (chr(13)+chr(10)) as rows separator). Thus, if you use Windows generated text file, add following suffix to your LOAD command: “ LINES TERMINATED BY '\r\n' ”. Otherwise, check how rows are separated in your text file. On default mysql expecting char(10) as rows separator.
I had the same problem, my mistake was that I was trying to load a value that I hadn't defined previously to an ENUM().
For example, ENUM('sell','lend') and I was trying to load the value return to that column. I needed to load it, so I added it to the ENUM and load it again.
I have seen the same warning when my data has extra space, tabs, newlines or other characters in my column which is decimal(10,2) to solve that, I had to remove those characters from value.
here is how I handled it.
LOAD DATA LOCAL INFILE 'c:/Users/Hitesh/Downloads/InventoryMasterReportHitesh.csv'
INTO TABLE stores_inventory_tmp
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(#col1, #col2, #col3, #col4, #col5)
SET sku = TRIM(REPLACE(REPLACE(REPLACE(REPLACE(#col1,'\t',''), '$',''), '\r', ''), '\n', ''))
, product_name = TRIM(REPLACE(REPLACE(REPLACE(REPLACE(#col2,'\t',''), '$',''), '\r', ''), '\n', ''))
, department_number = TRIM(REPLACE(REPLACE(REPLACE(REPLACE(#col3,'\t',''), '$',''), '\r', ''), '\n', ''))
, department_name = TRIM(REPLACE(REPLACE(REPLACE(REPLACE(#col4,'\t',''), '$',''), '\r', ''), '\n', ''))
, price = TRIM(REPLACE(REPLACE(REPLACE(REPLACE(#col5,'\t',''), '$',''), '\r', ''), '\n', ''))
;
I've got that hint from this answer
I was facing the same issue, while importing csv files into mysql xampp. I have used LINES TERMINATED BY '\r' instead of LINES TERMINATED BY '\n' and LINES TERMINATED BY '\r\n'.
This error can also be the result of not having the line,
FIELDS SPECIFIED BY ','
(if you're using commas to separate the fields) in your MySQL syntax, as described in this page of the MySQL docs.