I am using a LOAD DATA INFILE to read from a CSV and insert into a MySQL table. The problem is that one column in the CSV is in scientific notation and when it is read and loaded into the table it won't go into a column of type DOUBLE, FLOAT, or REAL. Is there a solution to this?
114,2015-05-11 00:00:00.000,0,100,14,2.576927E+10,1.730594E+10,1.904524E+10,1.788426E+10,69.40149
In this one row of the CSV are the scientific notation values that when loaded via LOAD DATA INFILE just become 100000
load DATA local infile yourfile
INTO TABLE Direction_Prix_proxi.tmp_prix_conc FIELDS TERMINATED BY ';' LINES TERMINATED BY '\r\n'
(#yourcolumn)
set yourcolumn = REPLACE(#yourcolumn, ',', '.')
Related
i'm unable to load amount using LOAD DATA LOCAL INFILE
I have a text file where there is one column called 'Amount'. My problem is this field has values as 20,200.00 and -61,066.11 etc in the text file. When I try to load these values in my MySQL database, it loads only 20 and -61.
What should I do to load the full amount in my table?
I am using Decimal(10,2) data type.
I am using following query.
LOAD DATA LOCAL INFILE 'C:/Users/Dataction123/Desktop/nordic.txt' INTO TABLE spend_nordic LINES TERMINATED BY '\r\n';
LOAD DATA LOCAL INFILE 'C:/Users/Dataction123/Desktop/nordic.txt' IGNORE
INTO TABLE spend_nordic
FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\r\n'
(int_col, #float_col)
SET float_col = replace(#float_col, ',', '.');
Explanation :
MySQL casts the value into the column type when it reads it from the file, before applying the transformations described in the SET clause. If you instruct it to read it into a variable the value is handled as string from the beginning. float_col is name of field.
For reference Click here
I have imported a CSV file where a specific column has a decimal number.
In the original excel file (before saving it to a CSV), the first number of the column shows up as 218,790. When I choose the cell, the number shows up as 218790.243077911.
In the CSV file the number shows up as 218790 and when I choose the cell it is 218,790.
When I import the file on mySQL and show the table I created, the number shows up as 218.000000000.
Here is the code I used:
create table Apolo_Test(
Leads decimal (15,9)
);
LOAD DATA LOCAL INFILE 'C:/Users/SCRIPTS/file.csv'
INTO TABLE Apolo_Test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 7 ROWS
;
I tried updating the format with this :
update Apolo_Test set Leads = format(Leads, 10, 'de_DE');
but it did not work. I have never had a case where files had a comma before. I guess it is the UK version of numerical fields.
How is it possible to make it work on mySQL without using any MACROS in excel?
UPD:
It works but I get some warnings although I double checked the csv file and the fields :
create table Apolo_Test(
Ad_Group varchar(50),
Impacts int,
Leads decimal (10,3)
);
LOAD DATA LOCAL INFILE 'C:/Users/me/Desktop/SCRIPTS/11/Adalyser.csv'
INTO TABLE Apolo_Test
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 7 ROWS
(Ad_Group, Impacts, #Leads)
SET Leads = replace(#Leads, ',', '');
;
alter table Apolo_Test ADD IPL decimal (10,6) after Leads;
update Apolo_Test set IPL=Impacts/Leads;
select * from Apolo_Test;
You have to use this syntax:
LOAD DATA LOCAL INFILE 'C:/path/to/mytable.txt' IGNORE
INTO TABLE mytable
FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\r\n'
(int_col, #float_col)
SET float_col = replace(#float_col, ',', '.');
For more information read here
The thousands-separator should not matter when moving data around -- Excel internal values and CSV files and MySQL internal values do not include it. Only "formatted" output includes it. And you should not use formatted output for moving numbers around.
Be careful with locale, such as de_DE.
The German "218.790" is the same as English "218,790".
"218790.243077911" is likely to be what Excel had internally for the number.
"218,790" is likely to be the English representation on the screen; note the English thousands separator.
In the CSV file the number shows up as 218790 and when I choose the cell it is 218,790.
What do you mean? Perhaps that there no comma or dot in the file, itself? But what you mean by "choose the cell"?
I can't see how to get "218.000000000" without truncation going on somewhere.
I am importing a .csv into a MySql table using LOAD DATA INFILE and would like to find a way around columns containing formatting like "6.10111E+11" -- this should import as "610111447853" but is instead 610111000000. The table col is VARCHAR as sometimes there are letters and those are necessary. Formatting the .csv column as numeric before saving it in the shared location does not seem to work. Can I specify some form of "Set" on that column upon import?
here is my code:
$query = <<<eof
LOAD DATA LOCAL INFILE '/home/c/Documents/UTC/UTC.csv'
INTO TABLE UTC_import
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
eof;
I want to import large CSV file with following columns
SegmentId;Lengthmm;RoadClass;OptimalSpeedKph;SpeedLimitKph;WKT
1;7329;1;120;70;LINESTRING (28.97845 41.067332,28.978537 41.067332)
...
I am using LOAD DATA INFILE. Problem is that WKT is LINESTRING.
Actually if I add one entry with following query:
INSERT INTO FCD
VALUES (3,1,1,1,1,LineStringFromText( 'LINESTRING (28.947971 41.050192,28.94754 41.049887)'));
It is inserted into table.
However I didn't know to access the WKT parameter in LOAD DATA INFILE command and transform it.
LOAD DATA LOCAL INFILE '/home/username/userfile.csv'
INTO TABLE `FCD`
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
It gives me the following error:
ERROR 1416 (22003): Cannot get geometry object from data you send to the GEOMETRY field
With regards
You can use a variable to load the value from the WKT column and use the SET clause of the LOAD DATA INFILE statement to use the value of the variable in an expression and assign it to a column:
LOAD DATA LOCAL INFILE '/home/username/userfile.csv'
INTO TABLE `FCD`
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(SegmentId, Lengthmm, RoadClass, OptimalSpeedKph, SpeedLimitKph, #WKT)
SET WKT = LineStringFromText(#WKT)
I assumed the first line of the CSV file contains the names of the table columns. Remark the last field in the list is #WKT which is a variable then the SET clause that uses the value of variable #WKT into an expression then assign it to column WKT.
Create a staging table where WKT is a string. Then run
INSERT INTO FCD
SELECT SegmentId, Lengthmm, RoadClass, OptimalSpeedKph, SpeedLimitKph, LineStringFromText (WKT)
FROM FCD_staging;
I've been asked to create an excel file saved as a .csv file to import into a mysql db. How do you format the excel file if the data goes to multiple tables in the db?
You need to create separate csv file for each table & then you can use command something like below:
LOAD DATA LOCAL INFILE './csv_data/employees.csv'
INTO TABLE employees
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
ESCAPED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(first_name,last_name,email_id );
This sample code is under presumption that csv file is comma separated , escape charater is double quote & first line contain header.
You need a CSV file for each table, or you need to import all of your data into an intermediate table and then split that out somehow using either a series of queries, a stored procedure, or a TRIGGER on the import table.
This is because the LOAD DATA INFILE system is limited to one table at a time.