I want to use LOAD DATA INFILE command of MYSQL to insert about 900,000 row to my table called INFO of a database called A.
the INFO tables is defined like below:
create table INFO (id varchar(10) PRIMARY KEY, receive_time datetime,
check_time datetime, ver varchar(10) NOT NULL)
before inserting in this table, just for testing, I create another database B, and create a table with the same definition of INFO called TMP_INFO. I run this query on the temporary database B and table TMP_INFO and it works:
LOAD DATA INFILE '/home/develop/list.txt' IGNORE INTO TABLE TMP_INFO
LINES TERMINATED BY '\n' IGNORE 1 LINES (id, #receive_time, #check_time, #ver)
set receive_time=NOW(), check_time=NOW(), ver='8754';
the list.txt file is similar to this :
8754
0087653565
0000986759
0000654327
0000453219
but the problem is that when I run the same query on the database A and INFO table, with the same list.txt file, it doesn't work. I get no error, but N warnings (N = number of lines in line.txt - 1 )!!! all the warnings are the same showed bellow:
warning code 1265:Data truncated for column 'id' at row ...
I am going to use this query within a C program.
I am confused, and don't have any idea, Any Idea please?
Related
I am still very new to MySQL and am having some trouble. I keep getting error code 1262 when trying to import a .csv file to my table and cannot figure out what I am doing wrong.
I have tried allocating more space per column as well as checking my .csv in notepad to see if there was a delimiter missing with no avail.
This is my import query:
LOAD DATA INFILE "C:\\ProgramData\\MySQL\\MySQL Server 8.0\\Uploads\\DLSubsidized.csv"
INTO TABLE studentaid.directloans_subsidized
FIELDS TERMINATED BY '|'
LINES TERMINATED BY '\r\n'
IGNORE 6 LINES (DLSubsidizedId, SchoolCode);
The create table query:
CREATE TABLE directloans_subsidized (DLSubsidizedId int NOT NULL PRIMARY KEY,
Recipients VARCHAR(50), NoOfLoans VARCHAR(50), AmountOfLoans DECIMAL(65,2), NoOfDisbursements VARCHAR(50), AmountOfDisbursments DECIMAL(65,2), SchoolCode int);
On MySQL 5.7, I used to populate the "point" column of a table with LOAD DATA INFILE and a SET clause, i.e.
LOAD DATA INFILE 'myfile.txt'
REPLACE INTO TABLE mytable
(#x, #y)
SET geom = Point(#x, #y);
This worked just fine.
I upgraded to MySQL 8.0.12, and I now get an error ERROR 1364 (HY000): Field 'geom' doesn't have a default value.
Has something changed in the way LOAD DATA INFILE handles SET clauses ?
Thanks !
This bug has been confirmed in 8.0.12.
The only workaround I found was :
Drop the geom column
Load data into the (x, y) columns
Recreate the geom column without a not null constraint
Update the geom column
Alter the column to add the not null constraint
Simply method is to create table without spatial (point/geo) fields and load data as text, next load data from this (temporary) table to target table like below:
load data to temporary table with simply types of columns
insert to target table from temporary table converting needed columns to spatial.
I have many files, for example: 20170319, 20170320 ...
For each file I have 2 columns, one for username and the other was data.
I've created a table
create table A(user varchar(35), date date, data varchar(35), primary key(user, date));
Then, I want to load those files into database, and use filename as specific date in date field.
Can I still use sth like:
Load data infile '20170320' into table A
The answer is that you cannot do this in MySQL alone, you need to use an external program or script that builds the load data infile statements with the appropriate SET clause derived from the name of the file:
The SET clause can be used to supply values not derived from the input
file. The following statement sets column3 to the current date and
time:
LOAD DATA INFILE 'file.txt'
INTO TABLE t1
(column1, column2)
SET column3 = CURRENT_TIMESTAMP;
The reason for this is that:
load data infile cannot use the file name as an input variable
neither MySQL prepared statements using the prepare statement, nor stored procedures are allowed to use the load data infile statement.
I am trying to insert some data using Load data infile into a mysql table that already has some data. The table contains id and name. My csv file contains three fields: id, name and code. The table schema also has these three fields, but currently, the table has NULL for the code field. I am trying to insert the code from csv to an existing row in the table if it matches the name, else I am trying to insert a complete new row.
The code I have tried is as follows:
LOAD DATA INFILE 'table1.csv'
INTO TABLE table1
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(#code, #name, #other_columns)
IF EXISTS (SELECT * FROM table1 where name=#name);
BEGIN
set code=#Code;
END
ELSE
BEGIN
set code=#Code, name=#name;
END
By doing so, I am getting a mysql syntax error, but am unable to figure it out. Can anyone point me in the right direction, or suggest to me another approach? I have thousands of new rows to insert and thousands of existing rows to modify based on the certain field, (name in this case).
MySQL does not allow the LOAD DATA INFILE statement inside a stored program, which is where the IF statement appears. Break up your task into two parts. First, LOAD DATA INFILE into a temporary table. Then create a stored program that replaces the loaded data into your table1 from the temporary table.
I created mytable table in mydb, and when I do show tables on that db I see mytable.
However, when I do:
LOAD DATA INPATH '/my/hdfs/path/myfile.csv' OVERWRITE INTO mydb.mytable;
I get
ParseException line 1:73 missing TABLE at 'mydb' near '<EOF>'
I can do a describe on the table, and I see my headers.
But how do I get the data from myfile.csv into the Hive table?
In your query TABLE key is missing.So,instead of
LOAD DATA INPATH '/my/hdfs/path/myfile.csv' OVERWRITE INTO mydb.mytable;
use this---->
use mydb;
LOAD DATA INPATH '/my/hdfs/path/myfile.csv' OVERWRITE INTO TABLE mytable;