I am using LOAD DATA local INFILE command to upload big csv file in mysql db. The field is an integer in the mysql table and so it will accept only integer.
Now if the csv file contains numbers like below then the sql doesn't work.The format of my csv file is
ID
"322"
"445"
"211"
So it is inserting 0 in the table. But if the csv file contains no double quotes then it's fine.
How can I tell "load data infile" command to ignore double quotes( as sometimes it is too difficult to remove double quotes from csv file which has got about 1 million records).My Sql is
$insert_query = "LOAD DATA local INFILE '".$target_path."'
INTO table master_huts
IGNORE 1 LINES
(hids)
";
Try adding this:
FIELDS OPTIONALLY ENCLOSED BY '"'
before:
IGNORE 1 LINES
Related
I am having a hard time loading my data to MySQL from a text file. I have been attempting to choose the correct delimiters but my file contains column names with each value.
The data is structured like this
{"id":"15","name":"greg","age":"32"}
{"id":"16","name":"jim","age":"42"}
the sql statement I am working on looks something like this currently
LOAD DATA LOCAL INFILE '/xxx.txt' INTO TABLE t1 FIELDS
TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' LINES TERMINATED BY '\r\n'(id, name, age);
results are being stored like this
{"id":"16", "name","greg"}
I need to do away with the column name and store the value.
any tips?
Writing a script as suggested by #Shadow will be easier to wrap around but you can check the section on json on this page how to import json-text-xml and csv data into mysql
or
Import JSON to MySQL made easy with the MySQL Shell if you are using MySQL Shell 8.0.13 (GA)
I want to load csv file into mysql table.The query works fine but in csv row is like :
1000002,Kabul,"Kabul,Afghanistan",2004,AF,City,Active
Kabul and Afghanistan goes into 2 separate columns.below is my query:
LOAD DATA LOCAL INFILE "'.$file.'"
INTO TABLE '.$table.'
FIELDS TERMINATED by \',\'
LINES TERMINATED BY \'\n\'
I want "Kabul,Afghanistan" in one column.
The problem is that you need to add
OPTIONALLY ENCLOSED BY '"'
so that the loader knows to treat the quoted string as a single field.
I am trying to import a CSV file into MySQL via the command line with following command :
USE database
LOAD DATA LOCAL INFILE 'C:/TEMP/filename.csv' INTO TABLE tablename
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n';
But when I do that, the system will truncate all the data of some columns to a length of 10 characters. I have tried using the CHARACTER SET option (UTF8, LATIN1), but that did not help. I've also tried to save the initial CSV file differently.
An example of such data item in such column would be "+00000000000#abc.de.fg;abdc=apcde,abcdefghijklmnop#qrs.tu.vw" which will be truncated to +000000000 , sometimes when there is no number it will be truncated to "abc#vgtw.c" or similar.
How do I get the data to be imported without truncating anything, and have the data imported as is ?
I can import CSV file with CSV option but doesnt work for CSV with LOAD DATA. I get the error
Error: 1083 - Field separator argument is not what is expected
...although I am doing everything same,uploading the same file to same table.
You are probably forgetting to set the field separator to a comma. Here's an example command for loading a CSV file from the MySQL documentation.
LOAD DATA INFILE can be used to read files obtained from external sources. For example, many programs can export data in comma-separated values (CSV) format, such that lines have fields separated by commas and enclosed within double quotation marks, with an initial line of column names. If the lines in such a file are terminated by carriage return/newline pairs, the statement shown here illustrates the field- and line-handling options you would use to load the file:
LOAD DATA INFILE 'data.txt' INTO TABLE tbl_name
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES;
Some notes:
If your first row doesn't contain headers, you should omit the IGNORE 1 LINES.
If your fields aren't enclosed by quotation marks, you can omit that too.
Lastly, if your file is
local, you should be using LOAD DATA LOCAL INFILE.
I've been asked to create an excel file saved as a .csv file to import into a mysql db. How do you format the excel file if the data goes to multiple tables in the db?
You need to create separate csv file for each table & then you can use command something like below:
LOAD DATA LOCAL INFILE './csv_data/employees.csv'
INTO TABLE employees
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(first_name,last_name,email_id );
This sample code is under presumption that csv file is comma separated , escape charater is double quote & first line contain header.
You need a CSV file for each table, or you need to import all of your data into an intermediate table and then split that out somehow using either a series of queries, a stored procedure, or a TRIGGER on the import table.
This is because the LOAD DATA INFILE system is limited to one table at a time.