mysql table formatting and infile issue - mysql

Each time I go and create my table and then add the .csv file, it keeps not fully loading in all of the data. Below is my code:
CREATE TABLE Parts_Maintenance (
Vehicle_ID BIGINT(20),
State VARCHAR(255),
Repair VARCHAR(255),
Reason VARCHAR(255),
Year YEAR,
Make VARCHAR(255),
Body_Type VARCHAR(255),
PRIMARY KEY (Vehicle_ID)
);
LOAD DATA INFILE '/home/codio/workspace/FleetMaintenanceRecords.csv'
INTO TABLE Parts_Maintenance
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\\n'
IGNORE 1 ROWS;
SELECT * FROM Parts_Maintenance;
Here is a photo of what it looks like in Codio:
And here is a photo of some of the data being brought in:
Could someone please help me pinpoint what I am doing wrong?
Tried to create table and bring in a .csv file. Table was created but the data is not all there and the table looks messed up

I agree with #barmer Your LINES TERMINATED BY is probably wrong, it's probably \r
You May use
LOAD DATA INFILE '/home/codio/workspace/FleetMaintenanceRecords.csv'
INTO TABLE Parts_Maintenance
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 ROWS;
Alternatively, you can use another method with workbench.
Right Click on Table
- Table data import wizard
- Choose the file path
Select the destination table
- Check and configure import settings
- Click on Next to execute.
This is the best and simple solution.

Related

MySQL: Load Data Infile from CSV that consist of comma in VARCHAR field

Below is the code that I use to import CSV file to MySQL database. It works well to divide all the field and its record.
LOAD DATA INFILE 'file.csv'
INTO TABLE customer FIELDS TERMINATED BY ',' LINES TERMINATED BY '\r\n'
(
ID, name, salary, address, status
);
However, when there is a VARCHAR or TEXT field which consist of comma (','), it works improperly. It is because I use FIELDS TERMINATED BY ',' that used to separate each field record.
So, for example, if a customer with salary 50,000 (double), it split the field normally. But, if the customer address is Java Road 15, Hong Kong (varchar/text), Java Road 15 will be saved in address field, while the Hong Kong will be saved to status field. This basically remove any record inside the status field. Any clue for this problem? Thanks in advance.
Are the fields enclosed by double quotes or something else? If so, you can add the "ENCLOSED BY" in your query.
FIELDS TERMINATED BY ',' ENCLOSED BY '\"' LINES TERMINATED BY '\r\n';
"Enclosed by" specifies the character to identify the start and end of a field. In your case, field is enclosed by a double quote such as "Java Road 15, Hong Kong". It helps MYSQL to extract the field correctly even if there is a field delimiter in the field.
MYSQL manual: https://dev.mysql.com/doc/refman/5.7/en/load-data.html
LOAD DATA INFILE 'file.csv'
INTO TABLE customer FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
(
ID, name, salary, address, status
);
Try this one.
If anyone happens to stumble upon this answer, I wanted to share b/c this took me way too long to figure out - using mac, datagrip, mysql 5.7+, survey response questions:
DROP TABLE IF EXISTS surveyQuestion_ID;
#create table
CREATE TABLE surveyQuestion_ID
(
surveyQuestion_ID INT(11) NOT NULL,
surveyDescription TEXT,
surveyResponse VARCHAR(25) DEFAULT NULL,
PRIMARY KEY (surveyQuestion_ID)
);
#load query
LOAD DATA LOCAL INFILE 'file_location/file.csv' INTO TABLE surveyQuestion_ID
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(surveyQuestion_ID, surveyDescription, surveyResponse);
Hope this helps.

Tell sqlldr control file to load missing values as NULL

I have a CSV file. How can I tell the sqlldr control file to load missing values as NULL. (ie the table schema allows NULL for certain column)
Example of CSV
1,Name1
2,Name2
3,
4,Name3
Could you help me to edit my control file here so that a line 3 , the missing value is inserted as NULL in my table
Table
Create table test
( id Number(2), name Varchar(10), primary key (id) );
Control file
LOAD DATA INFILE '{path}\CSVfile.txt'
INSERT INTO test
FIELDS TERMINATED BY ','
(id CHAR,
name CHAR
)
I believe all you should have to do is this:
name CHAR(10) NULLIF(name=BLANKS)
You would have to hint to SQL*Loader that there might be nulls in your data.
2 ways to give that hint to SQL*Loader.
Use TRAILING NULLCOLS option.
LOAD DATA INFILE '{path}\CSVfile.txt'
INSERT INTO test<br>
FIELDS TERMINATED BY ','
TRAILING NULLCOLS
(id CHAR,
name CHAR
)
Recreate your CSV files with enclosed fields and then use OPTIONALLY ENCLOSED BY '"' which lets SQL*Loader clearly see the nulls in your data (nothing between quotes) that looks like "abcd",""
LOAD DATA INFILE '{path}\CSVfile.txt'
INSERT INTO test<br>
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
(id CHAR,
name CHAR
)
I found that using TRAILING NULLCOLS will do the job BUT it has to be for "blanks" at the end of the record line.
LOAD DATA INFILE {path}\Your_File
INSERT INTO TABLE Your_Table
TRAILING NULLCOLS
FIELDS TERMINATED BY ","
(
... your fields
)

Load csv file into mysql database

I'm trying to load a big CSV file into mysql but couldn't find out why it fails.
My CSV file looks like this:
_id,"event","unit","created","r1","r2","r3","r4","space_id","owner_id","name","display_name","users__email"
565ce313819709476d7eaf0e,"create",3066,"2015-12-01T00:00:19.604Z","563f592dd6f47ae719be8b38","3","13","7","55ecdd4ea970e6665f7e3911","55e6e3f0a856461404a60fc1","household","household","foo.bar#ace.com"
565ce350819709476d7eaf0f,"complete",3067,"2015-12-01T00:01:19.988Z","21","","","","55e6df3ba856461404a5fdc9","55e6e3f0a856461404a60fc1","Ace","Base","foo.bar#ace.com"
565ce350819709476d7eaf0f,"delete",3067,"2015-12-01T00:01:19.988Z","21","","","","55e6df3ba856461404a5fdc9","55e6e3f0a856461404a60fc1","Ace","Base","foo.bar#ace.com"
565ce350819709476d7eaf0f,"update",3067,"2015-12-01T00:01:19.988Z","21","","","","55e6df3ba856461404a5fdc9","55e6e3f0a856461404a60fc1","Ace","Base","foo.bar#ace.com"
And my code to load the file into mysql is this one:
CREATE DATABASE IF NOT EXISTS analys;
USE analys;
CREATE TABLE IF NOT EXISTS event_log (
_id CHAR(24) NOT NULL,
event_log VARCHAR(255),
unit CHAR(4),
created VARCHAR(255),
r1 VARCHAR(255),
r2 VARCHAR(255),
r3 VARCHAR(255),
r4 VARCHAR(255),
space_id VARCHAR(255),
owner_id VARCHAR(255),
name VARCHAR(255),
display_name VARCHAR(255),
users__email VARCHAR(255),
PRIMARY KEY (_id)
)
LOAD DATA INFILE 'audits.export.csv'
INTO TABLE event_log
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n\r'
IGNORE 1 ROWS;
Everything is fine, including the query but I get NULL in every column (only one row).
Here is the Action Output:
22:31:21 LOAD DATA INFILE 'audits.export.csv' INTO TABLE event_log FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n\r' IGNORE 1 ROWS 0 row(s) affected Records: 0 Deleted: 0 Skipped: 0 Warnings: 0 0.156 sec
I tried to tweak the table and the load query but it doesn't work
I'm on Windows 7, using Mysql 5.6 and Workbench.
I heard about GUI solution or Excel connectors here (Excel Connector) but I prefer to do it programmaticaly as I need to reuse the code.
Any help? I couldn't solve the problem with similar posts on Stackoverflow.
This doens't seem a valid newline:
LINES TERMINATED BY '\n\r'
change to:
LINES TERMINATED BY '\r\n'
Or only one of them (could be single \n or \r depending on the system, or the software that created the csv file).
Because some might come here wanting to know how to do this with MySQL Workbench:
Save CSV data to a file, for example foo.csv
Open MySQL Workbench
Choose/Open a MySQL Connection
Right-click on a schema in the Object Browser (left Navigator)
Choose "Table Data Import Wizard"
Select the CSV file, which is foo.csv in our example
Follow the wizard; many configuration options are available, including the separator
When finished, the CSV data will be in a new or existing table (your choice)
For additional information, see the documentation titled Table Data Export and Import Wizard.
I just tested this in the example data provided in the question, and it worked.

Adding a string to a field during import with LOAD DATA LOCAL INFILE

Im importing a csv file to a mysql table with the following query;
"LOAD DATA INFILE 'myfielname.csv'
INTO table customers
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '\"'
LINES TERMINATED BY '\r'
IGNORE 3 LINES
(sales,regional,accounts)
";
Is there any way to insert a string of characters before a field that is to be imported?
For example: The field 'sales' above refers to account id numbers, which are being used in the application. Id like to append a URL before account number during import so the final record in the table will be as follows:
String I want to come before 'sales', but within the same record: http://www.url.com?id=
If a given sales id was 1234 the final record in the table would be http://www.url.com?id=1234
Thanks in advance for your help.
Try someting like this
LOAD DATA LOCAL INFILE 'C:/test.csv'
INTO TABLE test.test1
FIELDS TERMINATED BY ';'
(#test1col,#test2col)
set test1col=CONCAT('http://url.com?id=',#test1col),test2col=#test2col;
The test csv has 2 columns. I created a test table like this
CREATE TABLE `test1` (
`test1col` varchar(200) DEFAULT NULL,
`test2col` varchar(2000) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
You could try immediatley with your own, just make sure you name the columns correctly!
Give it a try it worked for me.

load data infile import not working and none of the other topics helped

This is the table i want to import in:
create table if not exists Medici(
m_id int unsigned AUTO_INCREMENT PRIMARY KEY,
m_nume VARCHAR(50),
m_prenume VARCHAR(50),
Statut ENUM('primar', 'specialist'),
Specialitate VARCHAR(50)
) ENGINE = InnoDB DEFAULT CHARSET = utf8;
And the CSV starts with:
Nume,Prenume,Tip,Specialitate
Bunica,Mihai-Daniel,primar,reumatologie
Donca,Cornelia-Ana,primar,chirurgie
Achiriloaie,Lorand-Levente,specialist,neurologie
The code I wrote is:
load data infile 'D:/xxxxxxxxxxxxx/xxxxxxx/xxxxxxxxxx/xxxxxx/xxxxxxxxxx/medici.txt'
into table Medici
character set utf8
fields terminated by ','
lines starting by 'Nume,Prenume,Tip,Specialitate\n'
terminated by '\n'
(m_nume,m_prenume,Statut,Specialitate);
Note: I changed terminated with \n, \r, \r\n, and i still get 1 row. Even with removing lines terminated by and it's not working.
Note: I pasted quickly and messed up a bit, there are 4 rows there. And ye the csv i assume it's formated correctly
Can't wait for advice, Thanks in advance.
more of the code here:
Nume,Prenume,Tip,Specialitate
Bunica,Mihai-Daniel,primar,reumatologie
Donca,Cornelia-Ana,primar,chirurgie
Achiriloaie,Lorand-Levente,specialist,neurologie
Papuc,Raducu-Liviu,primar,homeopatie
Cucuiu,Nutu,primar,ortopedie
Buia,Tache,specialist,ginecologie
Dragomanu,Mitrut,specialist,ecografie
Ticu,Simona,specialist,psihiatrie
Ene,Adrian-Stefan,specialist,pediatrie
Copae,Toma,primar,neurologie
Hotoi,Dragos Alin,specialist,pediatrie
Ceafalau,Vincenţiu Mihail,primar,pediatrie
Briceag,Anca Stefana,primar,imagistica
Condrea,Nutu,primar,fizioterapie
Cruceru,Ioana-Loredana,primar,dermatologie
Soarece,Dan-Cristian,primar,o.r.l.
Tatasel,Alexandru-Ovidiu,specialist,psihologie
Sterian,Gologaneanu,primar,chirurgie
Postelnicu,Habib,primar,chirurgie
Silviu ,Adrian Ionut,primar,dermatologie
Paius,Ioana,specialist,ortopedie
Borza,Marius Florian,specialist,fizioterapie
Tamas,Ciprian Costel,primar,chirurgie
Ograzeanu,Cristina Alexandra,primar,endocrinologie
Rildo,Alex,specialist,ecografie
In the csv these lines are merged ( one row to another )
For example:
After reumatologie it starts with the name but when i paste the code you can see it's actually an \n
Bunica,Mihai-Daniel,primar,reumatologieDonca,(.... here is 2nd row and so on)
Try this command:
load data local infile
'D:/xxxxxxxxxxxxx/xxxxxxx/xxxxxxxxxx/xxxxxx/xxxxxxxxxx/medici.txt'
into table Medici character set utf8 fields terminated by ','
lines terminated by '\n' IGNORE 1 LINES (m_nume,m_prenume,Statut,Specialitate);