I have a csv file which I have uploaded here
https://drive.google.com/file/d/1JfYc-7840utoa3k5iamEC-sScPlzsVVK/view
I have created a Table Titanic it has following structure.
mysql> desc Titanic;
+----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| last | varchar(255) | NO | | NULL | |
| first | varchar(255) | NO | | NULL | |
| gender | char(2) | NO | | NULL | |
| age | decimal(3,0) | YES | | NULL | |
| class | int(3) | NO | | NULL | |
| fare | decimal(5,0) | NO | | NULL | |
| embarked | varchar(255) | NO | | NULL | |
| survived | char(3) | NO | | NULL | |
+----------+--------------+------+-----+---------+-------+
8 rows in set (1.89 sec)
I have been asked to use LOAD DATA INFILE Statement to populate this table,
and as per my assignment
A blank entry for age means that the age is unknown
Fare can have more than two digits because money was not base-10 at that time
I try to execute the statement as follows
mysql> load data infile '/var/lib/mysql-files/Titanic.csv' into table Titanic,fields terminated by ',' optinally enclosed by '"' lines terminated by '\n' ignore1 lines;
I get error
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ',fields terminated by ',' optinally enclosed by '"' lines terminated by '\n' ign' at line 1
I try following
mysql> load data infile '/var/lib/mysql-files/Titanic.csv' into table Titanic,fields terminated by ',' optinally enclosed by '"' lines terminated by '\n' ignore
1 lines;
error I get is
ERROR 1064 (40000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ',fields terminated by ',' optinally enclosed by '"' lines terminated by '\n' ign' at line 1
If you look at the csv file in the link I gave row 7 has entry
Moran Mr. James M 3 8.4583 Queenstown no
there is no age mentioned in above row.So age has been assumed to be NULL hence while creating table I used NULL in create table for age.
row 24 has following entry
McGowan Miss Anna "Annie" F 15 3 8.0292 Queenstown yes
decimal value in fare.
row 87 has following entry
Backstrom Mrs. Karl Alfred (Maria Mathilda Gustafsson) F 33 3 15.85 Southampton yes
has a bracket in column whose title is first.
row 150 has following
Navratil Mr. Michel ("Louis M Hoffman") M 36.5 2 26 Southampton no
is having " " which I have in some fields not always.
I am not able to understand how to use LOAD DATA INFILE statement to use this csv which I have.
I am doing it for learning so I do not want to use any GUI tool.
What is the mistake in above LOAD DATA statement which I am trying to execute?
How can I use load data in this kind of csv where some values like double quotes " " and brackets () appear in some fields and some fields do not have any thing in them they are blank or NULL.
I am using mysql on Ubuntu 19.10.
Server version: 8.0.18-0ubuntu0.19.10.1 (Ubuntu)
update 1
as per the discussion in comments here I am pasting the csv file I have as text
last,first,gender,age,class,fare,embarked,survived
Braund,Mr. Owen Harris,M,22,3,7.25,Southampton,no
Cumings,Mrs. John Bradley (Florence Briggs Thayer),F,38,1,71.2833,Cherbourg,yes
Heikkinen,Miss Laina,F,26,3,7.925,Southampton,yes
Futrelle,Mrs. Jacques Heath (Lily May Peel),F,35,1,53.1,Southampton,yes Allen,Mr. William Henry,M,35,3,8.05,Southampton,no
Moran,Mr. James,M,,3,8.4583,Queenstown,no
McGowan," Miss Anna ""Annie""",F,15,3,8.0292,Queenstown,yes
Backstrom,Mrs. Karl Alfred (Maria Mathilda Gustafsson),F,33,3,15.85,Southampton,yes
Ford," Miss Robina Maggie ""Ruby""",F,9,3,34.375,Southampton,no
Navratil," Mr. Michel (""Louis M Hoffman"")",M,36.5,2,26,Southampton,no Byles,Rev. Thomas Roussel Davids,M,42,2,13,Southampton,no
the full csv in text form can be seen here https://pastebin.com/1B1mVYhJ
apart from this here is a screenshot of how it looks at my system when I issue a load data query
load data infile query
update 2
I have done this assignment by changing the definition of table created rather than taking all values as different different data types I took all of them as varchar
the questions which I was tried to do are here
http://arshahuja.blogspot.com/2018/01/deit-14610-big-data-analytics-laboratory.html
solution is also there the only problem was using this kind of csv file.
However I am not very convinced by creating a table like this and Loading the data as mentioned in above problem scenarios following table definition solved my problem. But what if I need to use these values like age and class ,fares in some mathematical calculations then how will I go for writing a query which has every thing as varchar?
mysql> desc Titanic;
+----------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+--------------+------+-----+---------+-------+
| last | varchar(255) | NO | | NULL | |
| first | varchar(255) | NO | | NULL | |
| gender | varchar(255) | NO | | NULL | |
| age | varchar(255) | YES | | NULL | |
| class | varchar(255) | NO | | NULL | |
| fare | varchar(255) | NO | | NULL | |
| embarked | varchar(255) | NO | | NULL | |
| survived | varchar(3) | NO | | NULL | |
+----------+--------------+------+-----+---------+-------+
8 rows in set (0.06 sec)
You have a comma after your table name.
load data infile '/var/lib/mysql-files/Titanic.csv' into table Titanic,fields terminated by ...
If you look at the syntax documentation at https://dev.mysql.com/doc/refman/8.0/en/load-data.html and the example of a complete statement, there is no comma after the table name.
LOAD DATA INFILE '/tmp/test.txt' INTO TABLE test
FIELDS TERMINATED BY ',' LINES STARTING BY 'xxx';
In general, when MySQL reports a syntax error, it tells you exactly at which point in the statement it found something it didn't think matched the syntax rules.
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ',fields terminated by ',' optinally enclosed by '"' lines terminated by '\n' ign' at line 1
The error above tells you it got confused at the comma right before "fields terminated by..."
That's where you should double-check your statement against the syntax reference documentation, or other examples of working statements.
You've misspelled optionally:
mysql> load data infile '/var/lib/mysql-files/Titanic.csv' into table Titanic,fields terminated by ',' optinally enclosed by '"' lines terminated by '\n' ignore 1 lines;
When I try to import my table named "Table.csv" to a MYSQl database using the linux console, it shows me the following message, ERROR 1265 (01000): Data truncated for column 'factor' at row 1
This is the description of the contents of the table that I had previously created:
+---------------------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+---------------------+---------+------+-----+---------+-------+
| degrees | int(11) | YES | | NULL | |
| percentage | double | YES | | NULL | |
| factor | double | YES | | NULL | |
+---------------------+---------+------+-----+---------+-------+
This is how the data delimited only by commas "," is displayed as it is displayed from a plain text file:
1,0.35,1
2,0.10,3.0787
1,0.55,4.32
This is the sentence that I currently enter:
LOAD DATA INFILE '/var/lib/mysql-files/tabla.csv' INTO TABLE Student FIELDS
TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n';
In this case, there should be no problem because the table starts with the values from its header, since I have omitted the IGNORE 1 LINES statement;
I enclose the information where I check more questions:
https://dev.mysql.com/doc/refman/8.0/en/load-data.html
After verifying the way in which the data are structured and how the CSV file is stored, the sentence that should be used corresponds to:
LOAD DATA LOCAL INFILE '/var/lib/mysql-files/table.csv' INTO TABLE Student FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\ n';
Care should be taken with the type of data, its length and if it has some content, in this case all fields are filled with NULL, so they must be replaced by NOT NULL fields.
Query:
LOAD DATA LOCAL INFILE 'actors.csv'
INTO TABLE Actors
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(ACTOR_ID, FNAME, LNAME);
CSV File:
ACTOR_ID, FNAME, LNAME
"66666","Billy","Lou"
"77777","Sally","Lou"
"88888","Hilly","Lou"
mysql> describe Actors;
+----------+-------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+----------+-------------+------+-----+---------+-------+
| ACTOR_ID | char(5) | NO | PRI | | |
| FNAME | varchar(20) | NO | | NULL | |
| LNAME | varchar(20) | NO | | NULL | |
+----------+-------------+------+-----+---------+-------+
> The output after running query:
| 10047 | Shirley | Jones |
| 10048 | Andre | Vippolis |
| 66666 | Billy | Lou"
"77777 |
| 88888 | Hilly | "Lou"
|
+----------+-------------+---------------+
I am trying to put a CSV file into my database. I've gotten the query
from a MySQL tutorial (except put the values I have in there). When I
run the query, My data is not properly inserted. I already have 2 rows
inserted (10047, 10048) and then I try to put the data from the CSV
file in, but it does not go in properly. It seems that the quotations
are not being read properly. But the statement ENCLOSED BY '"'
should handle the quotations. What am I doing wrong here?
It seems there is \r between
"Lou"
"77777"
and not \n
Use text editor to correct this.
Found a related so post
CSV files frequently have a carriage return/line feed as the line terminator. If the file was generated using Excel, for example, you will almost definitely have that.
A way to correct that is to modify your code as follows:
LOAD DATA LOCAL INFILE 'actors.csv'
INTO TABLE Actors
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(ACTOR_ID, FNAME, LNAME);
I do most of my CSV importing that way.
I had brought this up before earlier, but after doing some research, I realized I was looking in the wrong place. Here is the situation. I create this table:
CREATE TABLE PC_Contacts
(
POC VARCHAR(255) PRIMARY KEY NOT NULL,
Phone_1 VARCHAR(255),
Phone_2 VARCHAR(255)
);
I import a CSV file into MySQL which has the values for my table PC_Contacts:
USE Network
LOAD DATA INFILE 'C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Uploads\\PC_Contacts.csv'
INTO Table PC_Contacts
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 ROWS;
My output after importing looks like this:
+------------------+--------------+---------------+
| POC | Phone_1 | Phone_2 |
+------------------+--------------+---------------+
|April Wilson | 123-456-5000 | 123-456-5006
| | 123-456-2222 |
| | 123-456-5331 |
| | 123-456-7772 |
|Anton Watson | 123-456-1258 | 123-456-6005
|Elisa Kerring | 123-456-1075 | 123-456-4475
Now as you may recall, based on my code input, that POC is the PK. I had, in the original CSV file, a value for every line. However, as you see, anything that has no value on the right affects the left column's values. However, if I looked in the GUI and pulled up the table there, it showed the cell as populated with the value, so the data is there. If I were to put in xxx-xxx-xxxx, it would fix the issue:
+------------------+--------------+---------------+
| POC | Phone_1 | Phone_2 |
+------------------+--------------+---------------+
|April Wilson | 123-456-5000 | 123-456-5006
|Nicky Nite | 123-456-2222 | xxx-xxx-xxxx
|Nicole | 123-456-5331 | xxx-xxx-xxxx
|Becky | 123-456-7772 | xxx-xxx-xxxx
|Anton Watson | 123-456-1258 | 123-456-6005
|Elisa Kerring | 123-456-1075 | 123-456-4475
Obviously my intentions are so that I can see the value without having to apply special formatting in the command line. Is there a special SELECT command for that maybe?
Here is a link to a portion of the .CSV, as requested:
https://drive.google.com/file/d/0B0MMqHN75RpGdkZhcGp0SWtmams/view?usp=sharing
Your CSV file contains a carriage return with newline at the end of row, which breaks formatting. Use:
SELECT POC, Phone_1, REPLACE(Phone_2, '\r', '') AS Phone_2 FROM PC_Contacts;
Or change your import query as follows:
USE Network
LOAD DATA INFILE 'C:\\ProgramData\\MySQL\\MySQL Server 5.7\\Uploads\\PC_Contacts.csv'
INTO Table PC_Contacts
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS;
And use simple SELECT:
SELECT * FROM PC_Contacts;
Trying to load a .csv this simple:
"Id";"Values"
"1";"Value1"
"2";"Value2"
...
...
"n";"Valuen"
Into a table with 2 fields
id int(11) primary
values varchar(255)
like this:
LOAD DATA LOCAL INFILE 'file_name.csv'
REPLACE
INTO TABLE tbl_name
FIELDS
TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
but when i load it, I get something like
+----+--------------+
| id | value |
+----+--------------+
| 1 | value1" "2 |
| 2 | value3" "4 |
| 3 | value5" "6 |
| 4 | value7" "8 |
| 5 | value9" "10 |
| 6 | value11" "12 |
+----+--------------+
as if it didn't understand the linebreak rule \n
pulling my hair off already. what am I doind wrong?
in other words, what is the correct query for loading this file?
Try below one-
LOAD DATA LOCAL INFILE 'file_name.csv' REPLACE INTO TABLE tbl_name FIELDS ESCAPED BY '\\' TERMINATED BY ';' ENCLOSED BY '"' LINES TERMINATED BY '\r\n' IGNORE 1 LINES;