I try to import a CSV file in my database, I don't have any error but no rows are insert
I checked my field name
My table structure is:
CREATE TABLE IF NOT EXISTS `tmp` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`test` int(11) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
and my CSV looks like:
id;test
1;11
2;22
The command I use to import the file is
LOAD DATA LOCAL INFILE '.../test.csv' INTO TABLE tmp FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n' IGNORE 1 ROWS
It should import data into the tmp table, but just return No error, 0 row inserted
It could be because you did not specify the column to add the data in or the command "enclosed by". You could try adding those 2 lines you your code as shown below:
LOAD DATA INFILE '/path/to/test.csv'
INTO TABLE tmp
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS
(id,test);
I used this article's step as reference for importing csv, you could refer to it for additional detail and explanation:
https://blog.terresquall.com/2021/11/importing-a-csv-file-into-an-sql-table/
If your CSV is in a notepad like mine during testing, a "data truncated" warning will appear because there is a newline character at the end of each line. It can be ignored as it does not affect the results.
Related
I am storing the contents of text files in a table
CREATE TABLE Pages
(
ID int(11) unsigned NOT NULL,
Text mediumtext COMPRESSED,
PRIMARY KEY(ID)
) ENGINE=ARIA DEFAULT CHARSET=utf8 COLLATE utf8_general_ci ROW_FORMAT=DYNAMIC
I try to INSERT each file's contents directly via LOAD DATA INFILE
LOAD DATA INFILE 'file.txt' INTO TABLE table
FIELDS TERMINATED BY '\0' LINES TERMINATED BY '' (Text)
SET ID=$id
The problem is that if I ideally use TERMINATED BY '', it gives the error
You can't use fixed rowlength with BLOBs; please use 'fields
terminated by'
I used '\0' assuming the null character does not exist in the text file. Although it works, is there a more standard way to do so?
I have created a table that has the following definition:
CREATE TABLE vacayhome.photo (
id INT NOT NULL AUTO_INCREMENT,
url_path CHAR NOT NULL,
caption CHAR NOT NULL,
space_type CHAR NOT NULL,
is_main BOOLEAN NOT NULL,
listing_id INT NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY (listing_id) REFERENCES listing (id)
ON UPDATE RESTRICT
);
The data I'm trying to insert to this table has the following form:
url_path,caption,space_type,is_main,listing_id
JFK/015ec48e93480.jpg,An interior designer dream,,true,10000000
JFK/9184bd57e9f80.jpg,"Ready for you, YASS",,false,10000000
BCN/5ccd9b138c76.jpg,"Stay -- you're welcome",,false,10000001
BCN/5fbb3a5ac2b30.jpg,Warm sunlight throughout,,false,10000001
And this is my LOAD DATA statement into the empty table from the MariaDB and MySQL documentation:
ALTER TABLE photo DISABLE KEYS;
BEGIN;
LOAD DATA INFILE '/path/to/photos.csv' INTO TABLE photo
FIELDS
OPTIONALLY ENCLOSED BY '"'
TERMINATED BY ','
LINES
TERMINATED BY '\n'
IGNORE 1 ROWS (url_path,caption,space_type,is_main,listing_id);
SET id=NULL;
COMMIT;
ALTER TABLE photo ENABLE KEYS;
The OPTIONALLY ENCLOSED BY is being used for the CHAR columns that have a comma in the text, and the IGNORE 1 ROWS is used to ignore the header row.
When I try to load the data, I get the following error:
ERROR 1193 (HY000) at line 33: Unknown system variable 'id'
I've also tried adding SET id = NULL from answers in other StackOverflow posts like this one. What am I doing wrong?
You have an extra semicolon before the SET ID=NULL; directive.
Below code ran successfully in my local MySQL environment:
ALTER TABLE photo DISABLE KEYS;
BEGIN;
LOAD DATA LOCAL INFILE '/path/to/photos.csv' INTO TABLE photo
FIELDS
OPTIONALLY ENCLOSED BY '"'
TERMINATED BY ','
LINES
TERMINATED BY '\n'
IGNORE 1 ROWS (url_path,caption,space_type,is_main,listing_id)
SET id=NULL;
COMMIT;
ALTER TABLE photo ENABLE KEYS;
You might want to check the column types as well (for example, use VARCHAR instead). Good luck.
Have you tried putting the path in quotes?
As shown below...
url_path,caption,space_type,is_main,listing_id
"JFK/9184bd57e9f80.jpg","Ready for you, YASS",,false,10000000
I have a CSV file with one column with the next data:
"2015-01-01",
...
...
"2015-03-27"
I created mysql table that way:
CREATE TABLE `my_tbl` ( `id` int(11) NOT NULL AUTO_INCREMENT, `date` DATE NOT NULL, PRIMARY KEY (`id`) );
I am trying to insert data using the next command:
LOAD DATA INFILE '/tmp/myFile.csv' INTO TABLE my_tbl FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' (#col1) set date=#col1;
The problem: When checking my_tbl I see that all records are '0000-00-00'
What am I doing wrong and how can I fix it?
Did I define the table as it should be (e.g. Maybe it was better to define timestamp etc.)
Spot the difference:
[..snip..] BY '\n' (#col1) set date=#co1;
^---------------^
It appears I should have added FIELDS ENCLOSED BY '\"'. So the LOAD query should be this way:
LOAD DATA INFILE '/tmp/myFile.csv' INTO TABLE my_tbl FIELDS ENCLOSED BY '\"' LINES TERMINATED BY '\n' (#col1) set date=#col1;
Tnx for user #Marc B for the hint
I looked for a similar question but I could not find a magic answer....so I'm hoping I'll find one now as this is driving me crazy!
I was going great guns with importing CSV data into MySQL...and now suddenly I'm getting nothing but skipped records. I ought to mention that I have been doing lots of deleting records (just experimenting really as I'm learning about MySQL and building my first database) so I've been chopping and changing the data including lots of copy/pasting in Excel, just in case any of this may be causing the trouble.
An example here, where I chopped the CSV down to just 2 records:
GENRE_NAME
Classic Rock
Electronica
The query to load the data:
LOAD DATA INFILE 'Music_All_2.csv' IGNORE INTO TABLE genres COLUMNS TERMINATED BY ',' ESCAPED BY '\\' LINES TERMINATED BY '\r' IGNORE 1 LINES (genre_name);
Query OK, 0 rows affected (0.00 sec)
Records: 2 Deleted: 0 Skipped: 2 Warnings: 0
And the table is empty (so it's not skipping them due to them already existing):
select * from genres;
Empty set (0.00 sec)
Finally, here's the SHOW CREATE TABLE genres output:
genres | CREATE TABLE `genres` (
`genre_pk` tinyint(2) unsigned NOT NULL AUTO_INCREMENT,
`genre_name` varchar(90) NOT NULL,
PRIMARY KEY (`genre_pk`),
UNIQUE KEY `genre_name` (`genre_name`)
) ENGINE=InnoDB AUTO_INCREMENT=255 DEFAULT CHARSET=latin1 |
Thank you in advance to the solver of my problem, there is sooooo much I don't know - it's fun learning but frustrating at the same time!
I think you have 2 problems.
1. AUTO_INCREMENT value
In your DDL,
genre_pk is TINYINT. So genre_pk can hold 0~255.
in end of CREATE STATEMENT, AUTO_INCREMENT=255
This sets initial value for genre_pk to 255
2. LINES TERMINATED BY
You need to change LINES TERMINATED BY from '\r' to '\n'
3. Summary
I've tested following SQL, and it worked well.
DDL
CREATE TABLE `genres` (
`genre_pk` tinyint(2) unsigned NOT NULL AUTO_INCREMENT,
`genre_name` varchar(90) NOT NULL,
PRIMARY KEY (`genre_pk`),
UNIQUE KEY `genre_name` (`genre_name`)
) ENGINE=InnoDB AUTO_INCREMENT=1 DEFAULT CHARSET=latin1;
LOAD DATA
LOAD DATA INFILE '/path/to/Music_All_2.csv'
INTO TABLE genres FIELDS TERMINATED BY ',' ESCAPED BY '\\'
LINES TERMINATED BY '\n' IGNORE 1 LINES (genre_name);
Reply to OP's comment
1) How to reset AUTO_INCREMENTed value
When table has data which shouldn't be deleted.
ALTER TABLE table_name AUTO_INCREMENT = n;
When existing data could be deleted. (this is what you've already tried)
TRUNCATE TABLE table_name;
2) '\n' v.s '\r'
That's weird. Generally speaking,
'\n' for Unix system
'\r\n' for Windows system
I never heard about '\r' for carriage return. MySQL manaul(http://dev.mysql.com/doc/refman/5.5/en/load-data.html) states that
If you have generated the text file on a Windows system, you might have to use LINES TERMINATED BY '\r\n' to read the file properly, because Windows programs typically use two characters as a line terminator. Some programs, such as WordPad, might use \r as a line terminator when writing files. To read such files, use LINES TERMINATED BY '\r'.
There is a CSV file with the following data:
1;8-25-2010;0:05;210;4
2;8-25-2010;2:45;412;5
3;8-25-2010;3:40;300;3
4;8-25-2010;4:45;226;6
5;8-25-2010;5:20;206;4
6;8-25-2010;5:25;216;3
And there is MySQL Table:
CREATE TABLE IF NOT EXISTS `Schedule` (
`ID` SMALLINT NOT NULL AUTO_INCREMENT,
`Num` INT(10),
`PlannedDate` DATE,
`PlannedTime` TIME NOT NULL,
`resQty` INT(3) NOT NULL,
`stID` VARCHAR(10) NOT NULL,
PRIMARY KEY (`ID`),
FOREIGN KEY `stID` (`stID`) REFERENCES Stands (`stID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
Now I need to fill this table with the data from CSV file. For this I'm using the following code:
TRUNCATE TABLE testDB.Schedule;
LOAD DATA LOCAL INFILE 'C:\\temp\\Input.csv'
INTO TABLE testDB.Schedule FIELDS TERMINATED BY ';' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n' (Num,PlannedDate,PlannedTime,stID,resQty);
But the error message says that "Data truncated for column PlannedDate at row1", ErrorNr. 1265. The same error message for all rows.
The date has the wrong format, it should be 2012-01-19
If you are stuck with that date format (which is not the MySQL default), you can load those dates into a temporary variable and then convert it to a date, like this:
TRUNCATE TABLE testDB.Schedule;
LOAD DATA LOCAL INFILE 'C:\\temp\\Input.csv'
INTO TABLE testDB.Schedule FIELDS TERMINATED BY ';' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
(Num,#PlannedDate,PlannedTime,stID,resQty)
SET PlannedDate = STR_TO_DATE(#PlannedDate,'%m-%d-%Y');
in this line:
(Num,PlannedDate,PlannedTime,stID,resQty);
the two last parameters are reversed: stID,resQty
it SHOULD be:
resQty, stID
This is why you are getting a truncation error.