Create sample database and table to use with.
create database `test`;
use `test`;
create table `test` (`value` float(10,2) null);
insert into test (value) values (null);
select value from test;
+-------+
| value |
+-------+
| NULL |
+-------+
1 row in set (0.00 sec)
Now i want to create a txt file ,when you load it ,null data can be loaded into the test table.
delete from test;
Here is my data file named test.txt showed in vim(:set list).
Load the file into table.
LOAD DATA LOCAL INFILE "test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n' \w;
select value from test;
+-------+
| value |
+-------+
| 0.00 |
| 0.00 |
| 0.00 |
| 0.00 |
| 0.00 |
+-------+
How to load null data for value field with file in my case?
How to create this kind of data file,the value field is null after you load it?
The runtime environment:win10+mysql-5.7.
select version();
+-----------+
| version() |
+-----------+
| 5.7.22 |
+-----------+
1 row in set (0.03 sec)
Show test.txt file in vim with set list.
\N$
\N^I$
case:without set
Load it without (value) SET value = (CASE WHEN #value IS NULL THEN NULL ELSE #value END).
LOAD DATA LOCAL INFILE "f:/test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n' \W;
OUTPUT:
Warning (Code 1265): Data truncated for column 'value' at row 1
Warning (Code 1262): Row 2 was truncated; it contained more data than there were input columns
select * from test;
+-------+
| value |
+-------+
| 0.00 |
| NULL |
+-------+
case:with set
Load it with (value) SET value = (CASE WHEN #value IS NULL THEN NULL ELSE #value END).
delete from test;
LOAD DATA LOCAL INFILE "f:/test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
(value)SET value = (CASE WHEN #value IS NULL THEN NULL ELSE #value END) \W;
OUTPUT:
Warning (Code 1265): Data truncated for column 'value' at row 1
Warning (Code 1262): Row 2 was truncated; it contained more data than there were input columns
select * from test;
+-------+
| value |
+-------+
| NULL |
| NULL |
+-------+
Thank #Schwern.
LOAD DATA LOCAL INFILE "f:/test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\r\n' \W;
Either works fine with no warnings.
You can use \N.
The problem is all your input lines have an extra empty field. Tab terminates the field, like a comma. You've used it as if it encloses it like quotes. For example, your first line ^Inull^I is three fields. The first is empty. The second contains null. And the third is empty.
MySQL should have said something like
Query OK, 5 rows affected, 8 warnings (0.01 sec)
Records: 5 Deleted: 0 Skipped: 0 Warnings: 8
You can read those warnings with show warnings.
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1265 | Data truncated for column 'value' at row 1 |
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
| Warning | 1265 | Data truncated for column 'value' at row 2 |
| Warning | 1262 | Row 2 was truncated; it contained more data than there were input columns |
| Warning | 1265 | Data truncated for column 'value' at row 3 |
| Warning | 1265 | Data truncated for column 'value' at row 4 |
| Warning | 1262 | Row 4 was truncated; it contained more data than there were input columns |
| Warning | 1265 | Data truncated for column 'value' at row 5 |
+---------+------+---------------------------------------------------------------------------+
All you need is this:
\N$
Note that lines terminated by is sensitive to Unix vs Windows newlines. From the docs...
If you have generated the text file on a Windows system, you might have to use LINES TERMINATED BY '\r\n' to read the file properly, because Windows programs typically use two characters as a line terminator.
This explains the warnings you are getting.
Related
I'm doing mysql in my terminal as well as in my cmd.
I have a table :
mysql> create table stu (sno int, name char(10));
It's CSV file (b.csv):
sno, name
1,a
2,b
3,"c,d"
I imported that in mysql :
mysql> load data local infile 'd://b.csv' into table stu fields terminated by ',' enclosed by '"' lines terminated by '\n' ignore 1 rows;
Query OK, 3 rows affected (0.05 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from stu;
+------+------+
| sno | name |
+------+------+
| 1 | a |
| 2 | b |
| 3 | c,d |
+------+------+
3 rows in set (0.00 sec)
When I change the default delimiter alone :
I have another file with a custom delimiter and a custom quotechar (a.csv):
*rno*-*name*
*1*-*ron*
*3*-*vince*
*5*-*Abi-nav*
when I import this into mysql,
mysql> load data local infile 'd://a.csv' into table stu1 fields terminated by "-"
enclosed by "*" lines terminated by '\n' ignore 1 rows;
Query OK, 2 rows affected, 1 warning (0.05 sec)
Records: 2 Deleted: 0 Skipped: 0 Warnings: 1
mysql> show warnings;
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
+---------+------+---------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select * from stu1;
+------+------------+
| sno | name |
+------+------------+
| 1 | ron*
*3 |
| 5 | *Abi-nav*
+------+------------+
2 rows in set (0.00 sec)
I get this problem only when I change the default quotechar from " into other characters in both Ubuntu and in Windows.
Why I get this warning and this output??? How to solve this?
Thanks in advance
EDIT :
I tried as said by in the answer:
mysql> select version();
+-------------------------+
| version() |
+-------------------------+
| 5.7.31-0ubuntu0.16.04.1 |
+-------------------------+
1 row in set (0.04 sec)
mysql> load data local infile '/home/data.csv' into table stu fields terminated by "-" enclosed by '*' lines terminated by '\n' ignore 1 rows;
Query OK, 3 rows affected (0.19 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from stu;
+------+---------+
| sno | name |
+------+---------+
| 1 | ron |
| 3 | vince |
| 5 | Abi-nav |
+------+---------+
3 rows in set (0.00 sec)
Then here you go :
mysql> load data local infile '/home/a.csv' into table stu fields terminated by "-" enclosed by '|' lines terminated by '\n' ignore 1 rows;
Query OK, 2 rows affected, 1 warning (0.09 sec)
Records: 2 Deleted: 0 Skipped: 0 Warnings: 1
mysql> show warnings;
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
+---------+------+---------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select * from stu;
+------+------------+
| sno | name |
+------+------------+
| 1 | ron|
|3 |
| 5 | |Abinav|
|
+------+------------+
2 rows in set (0.00 sec)
Again I got the warning. Why that happens???
*-*name*
*1*-*ron*
*3*-*vince*
*5*-*Abi-nav*
mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.20 |
+-----------+
1 row in set (0.00 sec)
mysql> create table stu1 (sno int, name char(10));
Query OK, 0 rows affected (1.23 sec)
mysql> load data local infile '/data/data.csv' into table stu1 fields terminated by "-" enclosed by "*" lines terminated by '\n' ignore 1 rows;
Query OK, 3 rows affected (0.07 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from stu1;
+------+---------+
| sno | name |
+------+---------+
| 1 | ron |
| 3 | vince |
| 5 | Abi-nav |
+------+---------+
3 rows in set (0.00 sec)
it is working in mysql version - 8.0.20
Under windows i needed to change termination. so that it runs
MySQL 8.0.22
Load data infile 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/a.csv'
into table stu
fields terminated by "-"
enclosed by "*"
lines terminated by '\r\n'
ignore 1 LINES;
The main problem that i see, is that we can't see your csv file further you seem to switch between operating system Windows and linux.
Like i showed in my example the csv was produced in windows and had so \r\n as line termination, and this is what the MySQL interpreter had problems with, when i used \n\allone.
It had nothing to do with field termination or enclosure.
Your problem can be due to line termination (on Windows \r\n, on Unix/Linux \n)
You can try with a program like Notepad++, set the line termination to \n.
After delete all line break and recreate the line break.
If the load statement works, that was the issue.
Recently updated to mysql Ver 8.0.19 for osx10.15 on x86_64 (Homebrew)
I am trying to load data from a tsv file to add entries to an existing table. Everything seems to be working fine, except the values which are "NULL" (as in it literally says that, it's not empty) are being converted to 0000-00-00 00:00:00. There are no errors but I do get the warning"Data truncated for column 'beginexp' at row 2".
Here's what I've tried, to no avail:
ALTER TABLE infomegatask
ALTER beginexp SET DEFAULT NULL; # same for field 'endhit'
ALTER TABLE infomegatask MODIFY COLUMN beginexp DATETIME NULL; #even though it was already nullable to begin with when I checked the schema
#also set sql_mode="NO_ZERO_DATE"
Here is a redacted snippet of what the tsv looks like (sensitive data has been obscured):
uniqueid assignmentid workerid hitid ipaddress browser platform language cond counterbalance codeversion beginhit beginexp endhit bonus status mode datastring
XXXXX:YYYYY ZZZZZ AAAAA BBBBB CCCCC chrome windows UNKNOWN 0 0 4.2 2020-04-22 16:32:24 NULL NULL 0 4 live
mysql> LOAD DATA LOCAL INFILE 'test/DB/tsv/infomegatask_round_9g.tsv' INTO TABLE infomegatask FIELDS TERMINATED BY '\t' ENCLOSED BY '' LINES TERMINATED BY '\n' IGNORE 1 LINES;
Query OK, 24 rows affected, 13 warnings (0.21 sec)
Records: 24 Deleted: 0 Skipped: 0 Warnings: 13
mysql> warnings
Show warnings enabled.
mysql> show warnings
-> ;
+---------+------+------------------------------------------------+
| Level | Code | Message |
+---------+------+------------------------------------------------+
| Warning | 1265 | Data truncated for column 'beginexp' at row 2 |
| Warning | 1265 | Data truncated for column 'endhit' at row 2 |
| Warning | 1265 | Data truncated for column 'beginexp' at row 4 |
| Warning | 1265 | Data truncated for column 'endhit' at row 4 |
| Warning | 1265 | Data truncated for column 'beginexp' at row 7 |
| Warning | 1265 | Data truncated for column 'endhit' at row 7 |
| Warning | 1265 | Data truncated for column 'beginexp' at row 9 |
| Warning | 1265 | Data truncated for column 'endhit' at row 9 |
| Warning | 1265 | Data truncated for column 'beginexp' at row 10 |
| Warning | 1265 | Data truncated for column 'endhit' at row 10 |
| Warning | 1265 | Data truncated for column 'endhit' at row 14 |
| Warning | 1265 | Data truncated for column 'beginexp' at row 20 |
| Warning | 1265 | Data truncated for column 'endhit' at row 20 |
+---------+------+------------------------------------------------+
13 rows in set (0.00 sec)
I mean the workaround I'm going with for now is to just update these zero datetimes to NULL, but it's a bit annoying and I would just like to figure out what is going wrong.
In the LOAD DATA syntax you can convert columns.
So in your case:
LOAD DATA
INTO TABLE xx ( .., #beginexp , ... )
...
SET beginexp = IF(#beginexp='NULL', NULL, #beginexp),
...
I have the following table :
desc tr_tmp;
+-------+------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+------+------+-----+---------+-------+
| aya | text | NO | | NULL | |
+-------+------+------+-----+---------+-------+
1 row in set (0.002 sec)
But When I load data from file via :
LOAD DATA INFILE '/home/mohsen/codes/haq/translation-tmp/ber.mensur.txt'
INTO TABLE tr_tmp
FIELDS TERMINATED BY ''
ENCLOSED BY '"'
LINES TERMINATED BY '\n' (aya);
I get the following warnings:
/usr/local/lib/python3.7/dist-packages/pymysql/cursors.py:329: Warning: (1265, "Data truncated for column 'aya' at row 1218")
self._do_get_result()
/usr/local/lib/python3.7/dist-packages/pymysql/cursors.py:329: Warning: (1265, "Data truncated for column 'aya' at row 1266")
self._do_get_result()
/usr/local/lib/python3.7/dist-packages/pymysql/cursors.py:329: Warning: (1265, "Data truncated for column 'aya' at row 1611")
After above warnings, It can't load rest of file.
Here's the content of a very simple text file to show my problem. Somehow, I end up using "|" as the separator. But that's not the problem...
10|Antonio
11|Carolina
12|Diana
13|Alejandro
Here is the code I use to create that very simple table and to load the file into it.
CREATE TABLE IF NOT EXISTS names
(
id INT NOT NULL PRIMARY KEY AUTO_INCREMENT,
name VARCHAR(100)
);
LOAD DATA LOCAL INFILE
'C:\\Users\\Carolina\\Desktop\\Tmp\\TablasCSV\\names.csv'
INTO TABLE names
CHARACTER SET 'utf8'
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\r\n';
Here is the result of a simple select:
mysql> SELECT * FROM names;
+----+-----------+
| id | name |
+----+-----------+
| 1 | Antonio |
| 11 | Carolina |
| 12 | Diana |
| 13 | Alejandro |
+----+-----------+
4 rows in set (0.00 sec)
It has worked fine for me until I encounter the case, in which the first record value for the "id" was not "1".
I always get "1" after the load.
Any one has noticed this problem?
Is this a bug?
So far, I am fixing the record using an UPDATE command after the load, but I don't like it!!!!
I made a table which name is 'test' in mysql in my PC like belows.
create table test(
telnum varchar(20) not null,
reg_date datetime not null default '0000-00-00 00:00:00',
remarks text,
primary key(telnum)
);
And I uploaded a file named 1.txt into table 'test'.
1.txt's contents are like belows :
01011112222
01022223333
01033334444
And 'load data infile' syntax are like belows :
load data infile "c:/temp/1.txt"
ignore into table test;
But there is a problem.
Every phone numbers were cut like below.
Query OK, 3 rows affected, 3 warnings (0.00 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 3
mysql> select * from test;
+-------------+---------------------+---------+
| telnum | reg_date | remarks |
+-------------+---------------------+---------+
|12222 | 0000-00-00 00:00:00 |
|23333 | 0000-00-00 00:00:00 |
|34444 | 0000-00-00 00:00:00 |
+-------------+---------------------+---------+
3 rows in set (0.00 sec)
Only 5 characters are remained from 11 characters.
6 characters disappeared.
And second problem is 'warnings'. Reason of warnings is like below.
mysql> show warnings;
+---------+------+---------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------+
| Warning | 1264 | Out of range value for column 'reg_date' at row 1 |
| Warning | 1264 | Out of range value for column 'reg_date' at row 2 |
| Warning | 1264 | Out of range value for column 'reg_date' at row 3 |
+---------+------+---------------------------------------------------+
3 rows in set (0.00 sec)
I did'nt put anything into the reg_date field.
But the message says out of range value for column 'reg_date'.
What are the reasons and how can I solve these problems?
Try to change the line delimiter. On Windows it's usually \r\n rather then \n which is default if you omit LINES TERMINATED BY clause
LOAD DATA INFILE "/tmp/1.txt"
IGNORE INTO TABLE test
LINES TERMINATED BY '\n'