I am trying to import a csv file that is delimited by tabs.
Here is my query
LOAD DATA LOCAL INFILE 'c:/news.csv'
INTO TABLE news
FIELDS TERMINATED BY '\t'
LINES TERMINATED BY '\r'
(url, storyid, title, date, details, category, author);
What happens is only the first column is loaded, (url).
The rest shows NULL. I have tried lines terminated by \n as well. Same result.
Any advice?
Table structure for table `news`
--
CREATE TABLE IF NOT EXISTS `news` (
`url` varchar(62) DEFAULT NULL,
`storyid` int(15) DEFAULT NULL,
`title` varchar(255) DEFAULT NULL,
`date` date DEFAULT NULL,
`details` longtext,
`category` varchar(255) DEFAULT NULL,
`author` varchar(110) DEFAULT NULL
)
It depends on the exact format of your .csv file but for Windows .csv format I always use
LINES TERMINATED BY '\r\n'
also (again depending on the data) try
FIELDS ESCAPED BY '\\' TERMINATED BY '\t' OPTIONALLY ENCLOSED BY '\"'
If you're unsure of the exact nature of the data sometimes it is better to view it in hexadecimal to see how the lines are really terminated. I use Hexedit - http://www.hexedit.com/
Hope this helps.
Dermot
Like I said in the comments you can use '/r/n' for a new line.
However your csv file contains only 1 column, namely a full line of text.
That is probably also why only the first table column is filled and the rest is null.
LOAD DATA LOCAL INFILE 'c:/news.csv'
INTO TABLE news
COLUMNS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
(url, storyid, title, date, details, category, author)
This worked.
Turned out that even though it looks tab separated, it is comma separated. Dermot was right that you need to view it in hexadecimal view to see how it is really deliminated.
Related
I have this table:
CREATE TABLE `country` (
`name` VARCHAR(60) NOT NULL,
`code` VARCHAR(3) UNIQUE NOT NULL,
PRIMARY KEY (`code`)
);
As you can see the primary key of this table is the word code
When I try to select a specific code in this table, that is 2 characters long, it cannot find anything.
On the other hand, when I select a 3 characters long code like this:
select * from `country` where `code` = "TZA";
I get the result I want
I searched for my variable in the table (for example the code "AL") and it appears to be registered.
Why is this happening and how could I make it work?
Thank you in advance!
I am importing my data from a csv file that looks like this:
LOAD DATA LOCAL INFILE 'path_to_file\\countries.csv'
INTO TABLE `country`
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(`name`, `code`);
I have tried selecting with a space in the end of the code and on the front of it:
select * from `country` where `code` = 'AL ';
select * from `country` where `code` = ' AL';
But they output nothing
The real solution is:
When importing this CSV file you should use:
LOAD DATA LOCAL INFILE 'path_to_file\\countries.csv'
INTO TABLE `country`
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS
(`name`, `code`);
Because your lines seems to be terminated the way Windows terminates lines.
my text file is like this.
https://drive.google.com/open?id=1faW_OkO7_VoEQL_gndFIOrJv2e4Ycuzo
and my table is here.
CREATE TABLE news(
num INT auto_increment primary key,
link VARCHAR(150),
date INT,
title VARCHAR(150) unique,
description TEXT
);
i try
LOAD DATA INFILE 'test.txt'
INTO TABLE news
CHARACTER SET utf8mb4
FIELDS
TERMINATED BY ', ' OPTIONALLY ENCLOSED BY '"'
LINES
TERMINATED BY '\n' (link, date, title, description);
but it not working what is my mistake?
Your input file has four columns looking like this
url,month,title,"description": data
A CSV import will include the word "description": at the beginning of each description column.
So remove OPTIONALLY ENCLOSED BY '"' from your command, and you should be all set.
By the way, make sure your table definition mentions the utf8mb4 character set (or some characters from your input may not get represented correctly). For best results, don't rely on the server defaults to choose character set.
CREATE TABLE news(
num INT auto_increment primary key,
link VARCHAR(150),
date INT,
title VARCHAR(150) unique,
description TEXT
)
COLLATE 'utfmb4_general_ci';
This error is because of Optionally Enclosed by '. I have faced the same issue before and got resolved it by making a simple adjustment.
Just use the Enclosed by clause immediately after 'Fields' keyword.
Hence your code would be like something:
LOAD DATA INFILE 'test.txt'
INTO TABLE news
CHARACTER SET utf8mb4
FIELDS OPTIONALLY ENCLOSED BY '"'
TERMINATED BY ', '
LINES
TERMINATED BY '\n' (link, date, title, description);
I am trying to import a small data set of Berlin street addresses using MySQL's LOAD DATA statement. The problem is that after the import runs, all of the beautiful ß characters in the German street names have become ß sets.
Here's the create-table statement I used for this table:
CREATE TABLE `subway_distances` (
`STN` varchar(255) DEFAULT NULL,
`HNR` int(9) DEFAULT NULL,
`Lat` decimal(36,15) DEFAULT NULL,
`Lon` decimal(36,15) DEFAULT NULL,
`Distance` decimal(45,20) DEFAULT NULL
) ENGINE=MyISAM DEFAULT CHARSET=utf8
... and here is my MySQL shell code:
charset utf8;
TRUNCATE TABLE subway_distances;
LOAD DATA LOCAL INFILE '/path/to/output.csv'
INTO TABLE berlin.subway_distances
FIELDS TERMINATED BY ',' ENCLOSED BY '"' ESCAPED BY '\\';
SELECT * FROM subway_distances LIMIT 0,10;
I have looked at output.csv in vim, and the eszett character appears to be fine there.
I am assuming that I simply need a different encoding declaration in MySQL, but I'm not sure where to start.
I am also assuming that collation doesn't matter yet, since I'm not comparing values -- just purely trying to get a valid import.
I found an answer to this relatively quickly. It looks like I just need to specify the CHARACTER SET value in my LOAD DATA statement. So the new statement looks like this:
LOAD DATA LOCAL INFILE '/path/to/output.csv'
INTO TABLE berlin.subway_distances
CHARACTER SET 'utf8'
FIELDS TERMINATED BY ',' ENCLOSED BY '"' ESCAPED BY '\\';
I have a CSV file with one column with the next data:
"2015-01-01",
...
...
"2015-03-27"
I created mysql table that way:
CREATE TABLE `my_tbl` ( `id` int(11) NOT NULL AUTO_INCREMENT, `date` DATE NOT NULL, PRIMARY KEY (`id`) );
I am trying to insert data using the next command:
LOAD DATA INFILE '/tmp/myFile.csv' INTO TABLE my_tbl FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (#col1) set date=#col1;
The problem: When checking my_tbl I see that all records are '0000-00-00'
What am I doing wrong and how can I fix it?
Did I define the table as it should be (e.g. Maybe it was better to define timestamp etc.)
Spot the difference:
[..snip..] BY '\n' (#col1) set date=#co1;
^---------------^
It appears I should have added FIELDS ENCLOSED BY '\"'. So the LOAD query should be this way:
LOAD DATA INFILE '/tmp/myFile.csv' INTO TABLE my_tbl FIELDS ENCLOSED BY '\"' LINES TERMINATED BY '\n' (#col1) set date=#col1;
Tnx for user #Marc B for the hint
I have ASCII files with a static number of characters for each line with no delimiters. I'd like to use LOAD DATA INFILE to import into my table.
Example of file:
USALALABAMA
USARARKANSAS
USFLFLORIDA
The structure for this table:
country Char(2)
state Char(2)
name Varchar(70)
CREATE TABLE `states` (
`country` char(2) COLLATE latin1_general_ci NOT NULL,
`state` char(2) COLLATE latin1_general_ci NOT NULL,
`name` varchar(70) COLLATE latin1_general_ci NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1_general_ci COLLATE=latin1_general_ci;
Is it possible to specify a start and end position for each column?
According to the documentation, you can load a fixed format file without using a temporary table.
If the FIELDS TERMINATED BY and FIELDS ENCLOSED BY values are both empty (''), a fixed-row (nondelimited) format is used. With fixed-row format, no delimiters are used between fields (but you can still have a line terminator). Instead, column values are read and written using a field width wide enough to hold all values in the field. For TINYINT, SMALLINT, MEDIUMINT, INT, and BIGINT, the field widths are 4, 6, 8, 11, and 20, respectively, no matter what the declared display width is.
The positions are derived from the columns definitions, which in your case match the structure of the file. So you just need to do:
LOAD DATA INFILE 'your_file' INTO TABLE your_table
FIELDS TERMINATED BY ''
LINES TERMINATED BY '\r\n'
SET name = trim(name);
First create a temporary table which you will load all lines into it, then you can load the data from the temporary table into the main table and split to fields using substring
Something like this:
CREATE TEMPORARY TABLE tmp_lines
(countrystring TEXT);
LOAD DATA INFILE 'yourfilegoeshere' INTO TABLE tmp_lines
FIELDS TERMINATED BY ''
LINES TERMINATED BY '\r\n';
INSERT INTO main_table SELECT SUBSTRING(countrystring,1,2), SUBSTRING(countrystring,3, 2), SUBSTRING(countrystring,5) from tmp_lines;
Another way to do this is just assigning a variable and splitting it direct in your load.
LOAD DATA INFILE 'yourfilegoeshere' INTO TABLE main_table
LINES TERMINATED BY '\r\n' (#_var)
set
field1=TRIM(SUBSTR(#_var from 1 for 2)),
field2=TRIM(SUBSTR(#_var from 3 for 2)),
field3=TRIM(SUBSTR(#_var from 5 for 70));
Just be sure not to specify any field separator, otherwise you will have to use more variables, note that I'm using TRIM to clean data in the same statement.