Is it possible, and if so how, can I SELECT ... INTO OUTFILE and have it not enclose with any character.
So far this doesn't work:
SELECT hour_stamp,
day_stamp,
month_stamp,
hour,
day,
month,
year,
quarter,
day_of_week,
week_of_year,
SUBSTR(hour_text,1,24),
SUBSTR(day_text,1,24)
FROM date_dim
INTO OUTFILE '/media/ssd0/temp/dates.tsv'
FIELDS TERMINATED BY '\t'
ENCLOSED BY '';
I'm not sure if the engine matters in this case, but it may be important to note that I am using InfoBright on a Linux machine.
The output is as follows:
1293840000000 1293840000000 1293840000000 0 1 1 2011 1 5 52 "2011-01-01T00:00:00" "2011-01-01T00:00:00"
1293843600000 1293840000000 1293840000000 1 1 1 2011 1 5 52 "2011-01-01T01:00:00" "2011-01-01T00:00:00"
1293847200000 1293840000000 1293840000000 2 1 1 2011 1 5 52 "2011-01-01T02:00:00" "2011-01-01T00:00:00"
1293850800000 1293840000000 1293840000000 3 1 1 2011 1 5 52 "2011-01-01T03:00:00" "2011-01-01T00:00:00"
Adding OPTIONALLY ENCLOSED BY '' might have the desired effect.
Try it without anything:
SELECT hour_stamp,
day_stamp,
month_stamp,
hour,
day,
month,
year,
quarter,
day_of_week,
week_of_year,
SUBSTR(hour_text,1,24),
SUBSTR(day_text,1,24)
FROM date_dim
INTO OUTFILE '/media/ssd0/temp/dates.tsv';
Here is a sample
mysql> desc veto.prova;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| provaemail | varchar(255) | NO | | NULL | |
+------------+--------------+------+-----+---------+----------------+
2 rows in set (0.05 sec)
mysql> select * from veto.prova;
+----+--------------------------+
| id | provaemail |
+----+--------------------------+
| 1 | redwards#logicworks.net |
| 2 | rolandoedwards#yahoo.com |
+----+--------------------------+
2 rows in set (0.00 sec)
mysql> select id,provaemail from prova into outfile 'C:/lwdba/prova.txt';
Query OK, 2 rows affected (0.01 sec)
mysql>
What does it look like on disk ???
C:\>cd lwdba
C:\LWDBA>type prova.txt
1 redwards#logicworks.net
2 rolandoedwards#yahoo.com
C:\LWDBA>
I tried something weird. I terminated with \0
mysql> select id,provaemail,substr(provaemail,1,5) from prova into outfile 'C:/lwdba/prova9.txt' fields terminated by '\0';
Query OK, 2 rows affected, 1 warning (0.00 sec)
mysql> show warnings;
+---------+------+------------------------------------------------------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+------------------------------------------------------------------------------------------------------------------------+
| Warning | 1475 | First character of the FIELDS TERMINATED string is ambiguous; please use non-optional and non-empty FIELDS ENCLOSED BY |
+---------+------+------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select id,provaemail,substr(provaemail,1,5) from prova into outfile 'C:/lwdba/prova8.txt' fields enclosed by '\0';
Query OK, 2 rows affected (0.00 sec)
mysql>
The files look like the this:
C:\LWDBA>type prova9.txt
1 redwards#logicworks.net redwa
2 rolandoedwards#yahoo.com rolan
C:\LWDBA>type prova8.txt
1 redwards#logicworks.net redwa
2 rolandoedwards#yahoo.com rolan
C:\LWDBA>
I used \0 because it is a null character.
The double quote anomaly you are seeing is probably due to the InfoBright Storage Engine and how it renders character output of function calls.
Here is weird suggestion, but I don't know if it will work...
If you make a subquery, data are always stored in a MySQL temp table. Alter the query:
mysql> select * from (select id,provaemail,substr(provaemail,1,5) as stuff from prova) A
-> A into outfile 'C:/lwdba/prova444.txt' fields terminated by '\0' enclosed by '\0';
Query OK, 2 rows affected (0.00 sec)
mysql>
In you case, that would be
SELECT * FROM (
SELECT hour_stamp,
day_stamp,
month_stamp,
hour,
day,
month,
year,
quarter,
day_of_week,
week_of_year,
SUBSTR(hour_text,1,24) ht,
SUBSTR(day_text,1,24) dt
FROM date_dim) A
INTO OUTFILE '/media/ssd0/temp/dates.tsv';
See if that does something different
Did you try with "Null"?
SELECT hour_stamp,
day_stamp,
month_stamp,
hour,
day,
month,
year,
quarter,
day_of_week,
week_of_year,
SUBSTR(hour_text,1,24),
SUBSTR(day_text,1,24)
FROM date_dim
INTO OUTFILE '/media/ssd0/temp/dates.tsv'
FIELDS TERMINATED BY '\t'
ENCLOSED BY 'NULL';
Related
I'm doing mysql in my terminal as well as in my cmd.
I have a table :
mysql> create table stu (sno int, name char(10));
It's CSV file (b.csv):
sno, name
1,a
2,b
3,"c,d"
I imported that in mysql :
mysql> load data local infile 'd://b.csv' into table stu fields terminated by ',' enclosed by '"' lines terminated by '\n' ignore 1 rows;
Query OK, 3 rows affected (0.05 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from stu;
+------+------+
| sno | name |
+------+------+
| 1 | a |
| 2 | b |
| 3 | c,d |
+------+------+
3 rows in set (0.00 sec)
When I change the default delimiter alone :
I have another file with a custom delimiter and a custom quotechar (a.csv):
*rno*-*name*
*1*-*ron*
*3*-*vince*
*5*-*Abi-nav*
when I import this into mysql,
mysql> load data local infile 'd://a.csv' into table stu1 fields terminated by "-"
enclosed by "*" lines terminated by '\n' ignore 1 rows;
Query OK, 2 rows affected, 1 warning (0.05 sec)
Records: 2 Deleted: 0 Skipped: 0 Warnings: 1
mysql> show warnings;
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
+---------+------+---------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select * from stu1;
+------+------------+
| sno | name |
+------+------------+
| 1 | ron*
*3 |
| 5 | *Abi-nav*
+------+------------+
2 rows in set (0.00 sec)
I get this problem only when I change the default quotechar from " into other characters in both Ubuntu and in Windows.
Why I get this warning and this output??? How to solve this?
Thanks in advance
EDIT :
I tried as said by in the answer:
mysql> select version();
+-------------------------+
| version() |
+-------------------------+
| 5.7.31-0ubuntu0.16.04.1 |
+-------------------------+
1 row in set (0.04 sec)
mysql> load data local infile '/home/data.csv' into table stu fields terminated by "-" enclosed by '*' lines terminated by '\n' ignore 1 rows;
Query OK, 3 rows affected (0.19 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from stu;
+------+---------+
| sno | name |
+------+---------+
| 1 | ron |
| 3 | vince |
| 5 | Abi-nav |
+------+---------+
3 rows in set (0.00 sec)
Then here you go :
mysql> load data local infile '/home/a.csv' into table stu fields terminated by "-" enclosed by '|' lines terminated by '\n' ignore 1 rows;
Query OK, 2 rows affected, 1 warning (0.09 sec)
Records: 2 Deleted: 0 Skipped: 0 Warnings: 1
mysql> show warnings;
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
+---------+------+---------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> select * from stu;
+------+------------+
| sno | name |
+------+------------+
| 1 | ron|
|3 |
| 5 | |Abinav|
|
+------+------------+
2 rows in set (0.00 sec)
Again I got the warning. Why that happens???
*-*name*
*1*-*ron*
*3*-*vince*
*5*-*Abi-nav*
mysql> select version();
+-----------+
| version() |
+-----------+
| 8.0.20 |
+-----------+
1 row in set (0.00 sec)
mysql> create table stu1 (sno int, name char(10));
Query OK, 0 rows affected (1.23 sec)
mysql> load data local infile '/data/data.csv' into table stu1 fields terminated by "-" enclosed by "*" lines terminated by '\n' ignore 1 rows;
Query OK, 3 rows affected (0.07 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> select * from stu1;
+------+---------+
| sno | name |
+------+---------+
| 1 | ron |
| 3 | vince |
| 5 | Abi-nav |
+------+---------+
3 rows in set (0.00 sec)
it is working in mysql version - 8.0.20
Under windows i needed to change termination. so that it runs
MySQL 8.0.22
Load data infile 'C:/ProgramData/MySQL/MySQL Server 8.0/Uploads/a.csv'
into table stu
fields terminated by "-"
enclosed by "*"
lines terminated by '\r\n'
ignore 1 LINES;
The main problem that i see, is that we can't see your csv file further you seem to switch between operating system Windows and linux.
Like i showed in my example the csv was produced in windows and had so \r\n as line termination, and this is what the MySQL interpreter had problems with, when i used \n\allone.
It had nothing to do with field termination or enclosure.
Your problem can be due to line termination (on Windows \r\n, on Unix/Linux \n)
You can try with a program like Notepad++, set the line termination to \n.
After delete all line break and recreate the line break.
If the load statement works, that was the issue.
Create sample database and table to use with.
create database `test`;
use `test`;
create table `test` (`value` float(10,2) null);
insert into test (value) values (null);
select value from test;
+-------+
| value |
+-------+
| NULL |
+-------+
1 row in set (0.00 sec)
Now i want to create a txt file ,when you load it ,null data can be loaded into the test table.
delete from test;
Here is my data file named test.txt showed in vim(:set list).
Load the file into table.
LOAD DATA LOCAL INFILE "test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n' \w;
select value from test;
+-------+
| value |
+-------+
| 0.00 |
| 0.00 |
| 0.00 |
| 0.00 |
| 0.00 |
+-------+
How to load null data for value field with file in my case?
How to create this kind of data file,the value field is null after you load it?
The runtime environment:win10+mysql-5.7.
select version();
+-----------+
| version() |
+-----------+
| 5.7.22 |
+-----------+
1 row in set (0.03 sec)
Show test.txt file in vim with set list.
\N$
\N^I$
case:without set
Load it without (value) SET value = (CASE WHEN #value IS NULL THEN NULL ELSE #value END).
LOAD DATA LOCAL INFILE "f:/test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n' \W;
OUTPUT:
Warning (Code 1265): Data truncated for column 'value' at row 1
Warning (Code 1262): Row 2 was truncated; it contained more data than there were input columns
select * from test;
+-------+
| value |
+-------+
| 0.00 |
| NULL |
+-------+
case:with set
Load it with (value) SET value = (CASE WHEN #value IS NULL THEN NULL ELSE #value END).
delete from test;
LOAD DATA LOCAL INFILE "f:/test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\n'
(value)SET value = (CASE WHEN #value IS NULL THEN NULL ELSE #value END) \W;
OUTPUT:
Warning (Code 1265): Data truncated for column 'value' at row 1
Warning (Code 1262): Row 2 was truncated; it contained more data than there were input columns
select * from test;
+-------+
| value |
+-------+
| NULL |
| NULL |
+-------+
Thank #Schwern.
LOAD DATA LOCAL INFILE "f:/test.txt"
INTO TABLE test
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\r\n' \W;
Either works fine with no warnings.
You can use \N.
The problem is all your input lines have an extra empty field. Tab terminates the field, like a comma. You've used it as if it encloses it like quotes. For example, your first line ^Inull^I is three fields. The first is empty. The second contains null. And the third is empty.
MySQL should have said something like
Query OK, 5 rows affected, 8 warnings (0.01 sec)
Records: 5 Deleted: 0 Skipped: 0 Warnings: 8
You can read those warnings with show warnings.
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1265 | Data truncated for column 'value' at row 1 |
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
| Warning | 1265 | Data truncated for column 'value' at row 2 |
| Warning | 1262 | Row 2 was truncated; it contained more data than there were input columns |
| Warning | 1265 | Data truncated for column 'value' at row 3 |
| Warning | 1265 | Data truncated for column 'value' at row 4 |
| Warning | 1262 | Row 4 was truncated; it contained more data than there were input columns |
| Warning | 1265 | Data truncated for column 'value' at row 5 |
+---------+------+---------------------------------------------------------------------------+
All you need is this:
\N$
Note that lines terminated by is sensitive to Unix vs Windows newlines. From the docs...
If you have generated the text file on a Windows system, you might have to use LINES TERMINATED BY '\r\n' to read the file properly, because Windows programs typically use two characters as a line terminator.
This explains the warnings you are getting.
I want to import the second column of a CSV file into MySQL. Here's the CSV file:
Name,Source,Follows
John,Youtube,Y
Kat,FB,N
Jacob,Twitter,N
Here's the code I have so far:
DROP TABLE temp;
CREATE TABLE temp
(ID INT AUTO_INCREMENT primary key,
sn VARCHAR(50)
);
DESCRIBE temp;
LOAD DATA LOCAL INFILE '/Users/...temp.csv' INTO TABLE Person
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
IGNORE 1 LINES
(#col2) set sn=#col2;
SELECT * FROM temp;
However, I get that there is an empty set.
Try:
File: /path/to/temp.csv:
Name,Source,Follows
John,Youtube,Y
Kat,FB,N
Jacob,Twitter,N
MySQL Command-Line:
mysql> DROP TABLE IF EXISTS `temp`;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE TABLE IF NOT EXISTS `temp` (
-> `ID` INT AUTO_INCREMENT PRIMARY KEY,
-> `sn` VARCHAR(50)
-> );
Query OK, 0 rows affected (0.00 sec)
mysql> LOAD DATA LOCAL INFILE '/path/to/temp.csv'
-> INTO TABLE `temp`
-> FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
-> IGNORE 1 LINES
-> (#`null`, `sn`, #`null`);
Query OK, 3 rows affected (0.00 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 0
mysql> SELECT `ID`, `sn`
-> FROM `temp`;
+----+---------+
| ID | sn |
+----+---------+
| 1 | Youtube |
| 2 | FB |
| 3 | Twitter |
+----+---------+
3 rows in set (0.00 sec)
UPDATE
mysql> LOAD DATA LOCAL INFILE '/tmp/temp.csv'
-> INTO TABLE `temp`
-> FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
-> IGNORE 1 LINES
-> (#`null`, `sn`);
Query OK, 3 rows affected, 3 warnings (0.00 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 3
mysql> SHOW WARNINGS;
+---------+------+---------------------------------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------------------------------+
| Warning | 1262 | Row 1 was truncated; it contained more data than there were input columns |
| Warning | 1262 | Row 2 was truncated; it contained more data than there were input columns |
| Warning | 1262 | Row 3 was truncated; it contained more data than there were input columns |
+---------+------+---------------------------------------------------------------------------+
3 rows in set (0.00 sec)
mysql> SELECT `ID`, `sn`
-> FROM `temp`;
+----+---------+
| ID | sn |
+----+---------+
| 1 | Youtube |
| 2 | FB |
| 3 | Twitter |
+----+---------+
3 rows in set (0.00 sec)
I have one table
test
ID text sum
-----------------------
1 1_2_3 0
2 2_3_4_5 0
i want to update this table as
ID text sum
------------------------
1 1_2_3 6
2 2_3_4_5 14
how to write the query or function/procedure.
You should really NORMALIZE your data,but assuming you are forced to work with it:
UPDATE tableName SET sum=SUBSTRING_INDEX(text,'_',1) +
SUBSTRING_INDEX(SUBSTRING_INDEX(CONCAT(text,'_0'),'_',2),'_',-1) +
SUBSTRING_INDEX(SUBSTRING_INDEX(CONCAT(text,'_0'),'_',3),'_',-1) +
SUBSTRING_INDEX(SUBSTRING_INDEX(CONCAT(text,'_0'),'_',4),'_',-1) +
SUBSTRING_INDEX(SUBSTRING_INDEX(CONCAT(text,'_0'),'_',5),'_',-1);
Use SUBSTRING _INDEX to isolate each number,CONCAT is used to give a 0 if the number of expressions is larger than the number of values.
The fiddle
Try with this solution...
Hope this will help you....
SELECT SUM(Trim( Left(Name, InStr(Name, "_") - 1)) +
Trim( Mid(Name, InStr(Name, "_") + 1)) +
Trim(Right(Name, InStr(Name, ",") + 1))) as SUM FROM TEST;
Where the table structure is like this:
Id | Name |
1 | 1_2_3 |
2 | 5_8_10 |
It's resulted as
SUM
6
23
The best way to go about this is to make each text field an SQL statement.
First, here is sample data
mysql> drop table if exists prabhu;
Query OK, 0 rows affected (0.27 sec)
mysql> create table prabhu
-> (
-> id int not null auto_increment primary key,
-> text varchar(128),
-> sum int default 0
-> );
Query OK, 0 rows affected (0.56 sec)
mysql> insert into prabhu (text) values ('1_2_3'),('2_3_4_5');
Query OK, 2 rows affected (0.09 sec)
Records: 2 Duplicates: 0 Warnings: 0
mysql> select * from prabhu;
+----+---------+------+
| id | text | sum |
+----+---------+------+
| 1 | 1_2_3 | 0 |
| 2 | 2_3_4_5 | 0 |
+----+---------+------+
2 rows in set (0.00 sec)
mysql>
Here is a query to make each row produce an SQL statement to update the sum column
mysql> SELECT CONCAT('UPDATE prabhu SET sum=',
-> REPLACE(text,'_','+'),' WHERE id=',id,';') sqlstmt FROM prabhu;
+-------------------------------------------+
| sqlstmt |
+-------------------------------------------+
| UPDATE prabhu SET sum=1+2+3 WHERE id=1; |
| UPDATE prabhu SET sum=2+3+4+5 WHERE id=2; |
+-------------------------------------------+
2 rows in set (0.00 sec)
mysql>
Now, pipe the output of the query back into mysql and execute each line
C:\>mysql -Dtest -ANe"SELECT CONCAT('UPDATE prabhu SET sum=',REPLACE(text,'_','+'),' WHERE id=',id,';') sqlstmt FROM pra
bhu" | mysql -Dtest
C:\>mysql -Dtest -Ae"SELECT * FROM prabhu"
+----+---------+------+
| id | text | sum |
+----+---------+------+
| 1 | 1_2_3 | 6 |
| 2 | 2_3_4_5 | 14 |
+----+---------+------+
C:\>
Give it a Try !!!
I made a table which name is 'test' in mysql in my PC like belows.
create table test(
telnum varchar(20) not null,
reg_date datetime not null default '0000-00-00 00:00:00',
remarks text,
primary key(telnum)
);
And I uploaded a file named 1.txt into table 'test'.
1.txt's contents are like belows :
01011112222
01022223333
01033334444
And 'load data infile' syntax are like belows :
load data infile "c:/temp/1.txt"
ignore into table test;
But there is a problem.
Every phone numbers were cut like below.
Query OK, 3 rows affected, 3 warnings (0.00 sec)
Records: 3 Deleted: 0 Skipped: 0 Warnings: 3
mysql> select * from test;
+-------------+---------------------+---------+
| telnum | reg_date | remarks |
+-------------+---------------------+---------+
|12222 | 0000-00-00 00:00:00 |
|23333 | 0000-00-00 00:00:00 |
|34444 | 0000-00-00 00:00:00 |
+-------------+---------------------+---------+
3 rows in set (0.00 sec)
Only 5 characters are remained from 11 characters.
6 characters disappeared.
And second problem is 'warnings'. Reason of warnings is like below.
mysql> show warnings;
+---------+------+---------------------------------------------------+
| Level | Code | Message |
+---------+------+---------------------------------------------------+
| Warning | 1264 | Out of range value for column 'reg_date' at row 1 |
| Warning | 1264 | Out of range value for column 'reg_date' at row 2 |
| Warning | 1264 | Out of range value for column 'reg_date' at row 3 |
+---------+------+---------------------------------------------------+
3 rows in set (0.00 sec)
I did'nt put anything into the reg_date field.
But the message says out of range value for column 'reg_date'.
What are the reasons and how can I solve these problems?
Try to change the line delimiter. On Windows it's usually \r\n rather then \n which is default if you omit LINES TERMINATED BY clause
LOAD DATA INFILE "/tmp/1.txt"
IGNORE INTO TABLE test
LINES TERMINATED BY '\n'