I am using the following query to load into mysql:
LOAD DATA INFILE '/var/www/vhosts/httpdocs/xml/insert_feeds.csv'
INTO TABLE `deals_items`
FIELDS TERMINATED BY '###!##'
OPTIONALLY ENCLOSED BY ''
ESCAPED BY ''
LINES TERMINATED BY '###%##'
IGNORE 1 LINES
(#id,dealID,shopid,categoryid,title,permalink,url,startDate,endDate,description,extract,price,previous_price,discount,purchases,image,#location,#lat,#lng,locationText,type,settings,active)
SET id = '', lat=#lat, lng=#lng, locationText=#location, location = GeomFromText(CONCAT(POINT(#lat,#lng)))
Now this used to work just fine, but ever since an mysql was upgraded from 5.0 to 5.1 it stopped working. It now works only if I add the LOCAL to the statement.
The error I'm getting is Can't get stat of /var/www/vhosts/httpdocs/xml/insert_feeds.csv
The user was granted with full permissions to test it and the file was given 0777.
It won't work from any client (mysql,mysqli,pdo and console). Adding the local solves all problems but there is a lot of queries and I cannot go on and change them all. Further more, as I understand from the manual there are security issues with the LOCAL command.
The exact mysql version is 5.1.58-1~dotdeb.0 and the client version 5.0.51a
Related
i´m trying to upload registers in mysql using a procedure in genexus with SQL[!&Query!]. The command works to create the table, but not with the load.
&Query = 'CREATE TABLE tmpent (registro varchar(500));'
SQL [!&Query!]
Commit
&Query = "LOAD DATA LOCAL INFILE 'C:/Program Files/Apache Software Foundation/Tomcat 9.0/txtfiles/Maeent.txt' INTO TABLE tmpent LINES TERMINATED BY '\r\n' (registro);"
SQL [!&Query!]
Commit
i´m doing something wrong?
I've to import some data from a CSV file into a table of db on my Aruba server.
I use the following query:
LOAD DATA LOCAL INFILE 'test.csv' INTO TABLE dailycoppergg
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
(
ddmmyy,
lmedollton,
changedolleuro,
euroton,
lmesterton,
delnotiz,
girm,
sgm
)
I tested this query on other Aruba server and it worked correctly but here, I've the following error:
#1148 - Il comando utilizzato non e` supportato in questa versione di MariaDB
How can I modify my query to import csv file data into dailycoppergg table? Can you help me, please? Thanks!
The query is fine, but MySQL client (mysql) disables local infile by default, you need to run it as mysql --local-infile ..., and then the same query should work.
The error message is a legacy and it's confusing.
Since you're using phpMyAdmin, I highly recommend you just use the Import tab instead of manually entering the import query in the SQL tab. phpMyAdmin can easily import CSV files and I don't see any advantage to entering the query manually.
In MySQL Workbench Add the line below. in the Advanced Tab, check the test connection, and close.
OPT_LOCAL_INFILE=1
I have a console to load the sample.txt file into Table in database but it's failed when the sample.txt file size is more the 4000kb. How to incerase the limit? I think the default limit is 4096kb.
When I run the SQL query in MYSQL its successful insert all the data into Table.
strSql = "LOAD DATA LOCAL INFILE 'D:\\sample.txt'" & _
" INTO TABLE TABLENAME" & _
" FIELDS TERMINATED BY '|'" & _
" LINES TERMINATED BY '\n'" & _
" IGNORE 1 LINES" & _
" (NO,MESSAGE);"
Func: ExecSQL| at MySql.Data.MySqlClient.NativeDriver.SendFileToServer(String filename)
at MySql.Data.MySqlClient.NativeDriver.ReadResult(UInt64& affectedRows, Int64& lastInsertId)
at MySql.Data.MySqlClient.MySqlDataReader.GetResultSet()
at MySql.Data.MySqlClient.MySqlDataReader.NextResult()
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader()
at MySql.Data.MySqlClient.MySqlCommand.ExecuteNonQuery()
at MNC.COM.DBConn.ExecSQL(String strSQL)|MySql.Data|Error during LOAD DATA LOCAL INFILE|LOAD DATA LOCAL INFILE 'D:\\sample.txt' INTO TABLE FILE_RECORD_LOG_TEST FIELDS TERMINATED BY '|' LINES TERMINATED BY '\n' IGNORE 1 LINES (NO,MESSAGE);
I recommend you check the setting of max_allowed_packet on the server and on the client, to make sure they match.
The fact you report the "limit" as 4096K, and the fact that the stack trace seems to report the issue at "SendFileToServer" makes it sound like a possible mismatch in max_allowed_packet.
Also, I'd compare that to read_buffer_size.
SELECT ##global.max_allowed_packet
, ##session.max_allowed_packet
, ##global.read_buffer_size
, ##session.read_buffer_size
The other thing to verify is that there is sufficient space in the temp directory on the server. (With the LOCAL, the file will be transferred to the server, and that will go into the temp directory of the Operating System (not the directory specified in the tmpdir variable. If there's insufficient space, the LOAD DATA will fail with an error.
The fact you report that the statement is working with a small (<4MB) file makes it seem like your syntax is correct, if you are getting the rows added that you expect.
I'd also check that the line separator in the larger file is the same as it is in the smaller file.
What we don't see is the actual error message from MySQL server.
I'm not familiar with VB.NET. But I believe that the MySQLException class provides "Number" and "Error" members that will return the MySQL error number and message, respectively.
Problem solve
I use old version MySql.Data.dll, after used the latest version 6.7.4.0 its working fine, can load huge data into database.
thanks all
I'm trying to run this query from a .Net application
LOAD DATA LOCAL INFILE 'testsFile.txt'
INTO TABLE Test
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES
(idTest, SampleID, Analyst, #Analysed, Device, Comments, #TotalRUL, #RULOne, #RULTwo, #RULThree, #RULFour, Uploaded)
SET
Analysed = nullif(#Analysed,''),
TotalRUL = nullif(#TotalRUL,''),
RULOne = nullif(#RULOne,''),
RULTwo = nullif(#RULTwo,''),
RULThree = nullif(#RULThree,''),
RULFour = nullif(#RULFour,'')
When I run this query from MySQL Workbench everything works fine, but when I use my .net application to run the query I get the following exception:
Parameter '#Analysed' must be defined.
I don't think I can use a declare statement outside of a stored procedure and I cant use a stored procedure due to my use of the LOAD DATA statement
What to do? Is this checkmate?
Sure you can't. If your query works with Workbench, this sounds like a .net bug.
I suggest you try "stupid" solutions like using backticks (after the # and after Analyzed... sorry, Stack Overflows autoformatting doesnt allow me to show you what I mean) or changing the variable's name.
How to use MySQL user-variables with ADO.NET
seems to have the answer to this
I needed to add "allow user variables" to the connection string
Relatively new to Ruby, running: Ruby 1.9.2 and MySQL 5.5.19
I have a csv file full of data that I'd like to add to an existing table on a mysql database. The csv file does not have headers. Yes, I'm aware that there are multiple questions on this topic already. Here's how it breaks down:
Answer #1: Use LOAD DATA INFILE
Unfortunately, LOAD DATA INFILE gives me the following error: "Can't get stat of 'filename.csv' (Errcode: 2)"
This appears to be some kind of permissions issue. I've tried this both directly at the mysql prompt (as root), and through a Ruby script. I've tried various chmod options on the csv file, as well as moving the csv file to various directories where other users have said it works for them. No luck. Regardless, most people at this point recommend...
Answer #2: Use LOAD DATA local INFILE
Unfortunately this also returns an error. Apparently local infile is a mysql option turned off by default, because its a security risk. I've tried turning it on, but still get nothing but errors, such as:
ERROR 1148 (42000): The used command is not allowed with this MySQL version
and also
undefined method `execute' for # (NoMethodError)
Answer #3: I can find various answers involving Rails, which don't fit the situation. This isn't for a web application (although a web app might access it later), I'm just trying to add the data to the database for right now as a one-time thing to do some data analysis.
The Ruby file should be incredibly simple:
require 'rubygems'
require 'csv' (or fastercsv?)
require 'mysql'
db = mysql.connect('localhost','root','','databasename')
CSV.foreach('filename.csv') do |row|
?????
db.execute("INSERT INTO tablename ?????")
end
P.S. Much thanks in advance. Please no answers that point to using LOAD DATA INFILE or LOAD DATA LOCAL INFILE. Already wasted enough hours trying to get that to work...
ad mysql:
LOAD DATA INFILE '/complete/path/csvdata.csv' INTO TABLE mytable(column1,column2,...);
ad ruby
require 'csv'
require 'mysql'
db = mysql.real_connect('localhost','root','password','database')
file=CSV::Reader.parse('filename.csv')
file.each do |row|
values = row.inject([]){|k,v| k<<"'#{v}'";k}.join(',')
db.query("insert into table(column, column ...) values(#{values})")
end
db.close
it assumes csv file contains ALL the columns required..