I want to import large CSV file with following columns
SegmentId;Lengthmm;RoadClass;OptimalSpeedKph;SpeedLimitKph;WKT
1;7329;1;120;70;LINESTRING (28.97845 41.067332,28.978537 41.067332)
...
I am using LOAD DATA INFILE. Problem is that WKT is LINESTRING.
Actually if I add one entry with following query:
INSERT INTO FCD
VALUES (3,1,1,1,1,LineStringFromText( 'LINESTRING (28.947971 41.050192,28.94754 41.049887)'));
It is inserted into table.
However I didn't know to access the WKT parameter in LOAD DATA INFILE command and transform it.
LOAD DATA LOCAL INFILE '/home/username/userfile.csv'
INTO TABLE `FCD`
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
It gives me the following error:
ERROR 1416 (22003): Cannot get geometry object from data you send to the GEOMETRY field
With regards
You can use a variable to load the value from the WKT column and use the SET clause of the LOAD DATA INFILE statement to use the value of the variable in an expression and assign it to a column:
LOAD DATA LOCAL INFILE '/home/username/userfile.csv'
INTO TABLE `FCD`
FIELDS TERMINATED BY ';'
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(SegmentId, Lengthmm, RoadClass, OptimalSpeedKph, SpeedLimitKph, #WKT)
SET WKT = LineStringFromText(#WKT)
I assumed the first line of the CSV file contains the names of the table columns. Remark the last field in the list is #WKT which is a variable then the SET clause that uses the value of variable #WKT into an expression then assign it to column WKT.
Create a staging table where WKT is a string. Then run
INSERT INTO FCD
SELECT SegmentId, Lengthmm, RoadClass, OptimalSpeedKph, SpeedLimitKph, LineStringFromText (WKT)
FROM FCD_staging;
Related
I have a file with some MySQL commands for loading a single column into a database table:
TRUNCATE TABLE datamap;
LOAD DATA LOCAL INFILE 'datamap.txt'
INTO TABLE datamap
Fields terminated by ','
enclosed by '"'
escaped by '^'
Lines terminated by '\r\n'
(datapath);
The datapath file contains a list of files and their directories, i.e. x:\files\filename1.txt
All files are now in a single directory and I would like to hard-code the path into the command above, such that I can load only the filename but have the full path saved to the column in my table. I am aware that I could add an additional UPDATE statement above to update every column in the table after I've loaded it but that doesn't seem very efficient.
Does the LOAD DATA INFILE command support concatenating fixed strings with data being read in from a file?
TRUNCATE TABLE datamap;
LOAD DATA LOCAL INFILE 'datamap.txt'
INTO TABLE datamap
Fields terminated by ','
enclosed by '"'
escaped by '^'
Lines terminated by '\r\n'
(col1, col2, #my_variable)
SET col3 = CONCAT('x:\files\', #my_variable);
http://dev.mysql.com/doc/refman/5.6/en/load-data.html
LOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE 'file_name'
...
[(col_name_or_user_var,...)]
[ SET col_name = expr,...]
When processing an input line, LOAD DATA splits it into fields and uses the values according to the column/variable list and the SET clause, if they are present. Then the resulting row is inserted into the table.
i'm unable to load amount using LOAD DATA LOCAL INFILE
I have a text file where there is one column called 'Amount'. My problem is this field has values as 20,200.00 and -61,066.11 etc in the text file. When I try to load these values in my MySQL database, it loads only 20 and -61.
What should I do to load the full amount in my table?
I am using Decimal(10,2) data type.
I am using following query.
LOAD DATA LOCAL INFILE 'C:/Users/Dataction123/Desktop/nordic.txt' INTO TABLE spend_nordic LINES TERMINATED BY '\r\n';
LOAD DATA LOCAL INFILE 'C:/Users/Dataction123/Desktop/nordic.txt' IGNORE
INTO TABLE spend_nordic
FIELDS TERMINATED BY '\t' LINES TERMINATED BY '\r\n'
(int_col, #float_col)
SET float_col = replace(#float_col, ',', '.');
Explanation :
MySQL casts the value into the column type when it reads it from the file, before applying the transformations described in the SET clause. If you instruct it to read it into a variable the value is handled as string from the beginning. float_col is name of field.
For reference Click here
Sorry for the wordy title. I'm running the following query:
LOAD DATA LOCAL INFILE 'file.csv' INTO TABLE tablename
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY ',\r\n'
I have a CSV with data in the following format:
1111, 2222, "32", "1,234.304", 1023.53,
All lines are terminated with an extra comma.
When attempting to load, I get an error when trying to put the "1,234.304" type field into a NUMERIC(12,4) column. I end up with 1.0000 and a warning that the data was truncated. I had expected 1234.304.
Edit: It seems that doing a regular insert for a value which has comma thousand-separators upsets MySQL. Is there a way to modify the load command for the desired behavior?
Break the problem down into two steps:
Get the data into the database
Get the data into the table/format you want it in
If you get all the data into a table with VARCHAR fields, you can easily manipulate it and explicitly convert the values into the types you want them in, rather than relying on whatever implicit conversion you get with LOAD DATA LOCAL INFILE.
I am importing a .csv into a MySql table using LOAD DATA INFILE and would like to find a way around columns containing formatting like "6.10111E+11" -- this should import as "610111447853" but is instead 610111000000. The table col is VARCHAR as sometimes there are letters and those are necessary. Formatting the .csv column as numeric before saving it in the shared location does not seem to work. Can I specify some form of "Set" on that column upon import?
here is my code:
$query = <<<eof
LOAD DATA LOCAL INFILE '/home/c/Documents/UTC/UTC.csv'
INTO TABLE UTC_import
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
eof;
I have a file with some MySQL commands for loading a single column into a database table:
TRUNCATE TABLE datamap;
LOAD DATA LOCAL INFILE 'datamap.txt'
INTO TABLE datamap
Fields terminated by ','
enclosed by '"'
escaped by '^'
Lines terminated by '\r\n'
(datapath);
The datapath file contains a list of files and their directories, i.e. x:\files\filename1.txt
All files are now in a single directory and I would like to hard-code the path into the command above, such that I can load only the filename but have the full path saved to the column in my table. I am aware that I could add an additional UPDATE statement above to update every column in the table after I've loaded it but that doesn't seem very efficient.
Does the LOAD DATA INFILE command support concatenating fixed strings with data being read in from a file?
TRUNCATE TABLE datamap;
LOAD DATA LOCAL INFILE 'datamap.txt'
INTO TABLE datamap
Fields terminated by ','
enclosed by '"'
escaped by '^'
Lines terminated by '\r\n'
(col1, col2, #my_variable)
SET col3 = CONCAT('x:\files\', #my_variable);
http://dev.mysql.com/doc/refman/5.6/en/load-data.html
LOAD DATA [LOW_PRIORITY | CONCURRENT] [LOCAL] INFILE 'file_name'
...
[(col_name_or_user_var,...)]
[ SET col_name = expr,...]
When processing an input line, LOAD DATA splits it into fields and uses the values according to the column/variable list and the SET clause, if they are present. Then the resulting row is inserted into the table.