I'm trying to upload a csv file to a local database using MySQL with mybatis. I created a mysql container to act as the server and use datagrip to access the data. I want to use:
<insert id="batchInsertCsv" parameterType="string">
LOAD DATA LOCAL INFILE #{filePath}
INTO TABLE DATA
COLUMNS TERMINATED BY ','
IGNORE 1 LINES
</insert>
to upload the data but my current error is: Loading local data is disabled; this must be enabled on both the client and server sides. Using:
SET GLOBAL local_infile = true;
I have set local_infile on the server side and checked it with:
SHOW GLOBAL VARIABLES LIKE 'local_infile';
Can anyone offer help or advice? Thanks.
LOAD DATA LOCAL INFILE must be allowed both in server and client and you have already done the server part.
As MyBatis is built on top of JDBC API, you should add a connection property to the JDBC URL to enable the feature in client. e.g.
jdbc:mysql://127.0.0.1:3309/test?allowLoadLocalInfile=true
Note that there are several properties related to this feature.
allowLoadLocalInfile
allowLoadLocalInfileInPath
allowUrlInLocalInfile
The first one is a simple TRUE/FALSE switch.
The latter two allow you to set some restriction on the file being uploaded.
Please see the documentation for the details.
https://dev.mysql.com/doc/connector-j/8.0/en/connector-j-connp-props-security.html
Related
I have a node.js script which is connecting to a local MariaDB server, creating some tables, and then attempting to load some data into them by executing a SQL query like this:
LOAD XML LOCAL INFILE 'C:\\path\\to\\some_data.xml'
INTO TABLE some_data
ROWS IDENTIFIED BY '<data>';
The script fails with an error like:
SqlError: (conn=60, no: 45034, SQLState: 45034) LOCAL INFILE wrong filename. 'C:\path\to\some_data.xml' doesn't correspond to query LOAD XML LOCAL INFILE 'C:\\path\\to\\some_data.xml'
INTO TABLE some_data
ROWS IDENTIFIED BY '<data>';. Query cancelled. Check for malicious server / proxy
The node.js script code is like:
const pathToXmlFile = path.resolve(__dirname, '../some_data.xml')
.replace(/\\/g, '\\\\');
// .replace(/\\/g, '/');
// none of the above work, and neither does no escaping at all
query = `LOAD XML LOCAL INFILE '${pathToXmlFile}'
INTO TABLE some_data
ROWS IDENTIFIED BY '<data>';`;
I tried escaping the path with forward slashes (I'm running the script through Git Bash), the error remains the same, only the way the path is displayed changes. I have set permitLocalInfile: 'true' on the MariaDB client connection and local_infile=1 on both the [mysqld] and [client] sections of my.ini. The node.js connection user has the FILE privilege.
Node version is 16.13.0, MariaDB npm module version is 2.5.5, all running on the same machine with Windows 10 (MariaDB server too, on localhost).
The above query works perfectly fine when executed from HeidiSQL.
Any ideas on what I'm doing wrong?
I've create a mariadb bug for that : https://jira.mariadb.org/browse/CONJS-181
As a workaround, you might use "/" in place of "\": using "C:/path/to/some_data.xml" won't be escaped avoiding this bug
It seems that it's a bug with the mariadb node connector, I have opened an issue here:
https://github.com/mariadb-corporation/mariadb-connector-nodejs/issues/183
In the meantime, I have tried the mysql2 and mysql connectors, the first doesn't seem to support the permitLocalInfile: 'true' connection option, so I switch to the standard mysql connector and everything works fine (with a few changes in the code because of slight differences in method calling).
If the bug in the mariadb connector is fixed, I will update the answer here.
I'm kinda new to SQL. I am trying to add data to an alredy created table, the csv looks like these, it is 132645 lines long:
iso_code;continent;location;date;population;total_cases;new_cases;new_cases_smoothed;total_deaths;new_deaths;new_deaths_smoothed;total_cases_per_million;new_cases_per_million;new_cases_smoothed_per_million;total_deaths_per_million;new_deaths_per_million;new_deaths_smoothed_per_million;reproduction_rate;icu_patients;icu_patients_per_million;hosp_patients;hosp_patients_per_million;weekly_icu_admissions;weekly_icu_admissions_per_million;weekly_hosp_admissions;weekly_hosp_admissions_per_million
AFG;Asia;Afghanistan;24/02/2020;398354280;50;50;NULL;NULL;NULL;NULL;126;126;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;
AFG;Asia;Afghanistan;25/02/2020;398354280;50;0;NULL;NULL;NULL;NULL;126;0;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;
AFG;Asia;Afghanistan;26/02/2020;398354280;50;0;NULL;NULL;NULL;NULL;126;0;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;
AFG;Asia;Afghanistan;27/02/2020;398354280;50;0;NULL;NULL;NULL;NULL;126;0;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;
AFG;Asia;Afghanistan;28/02/2020;398354280;50;0;NULL;NULL;NULL;NULL;126;0;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;NULL;
And my sql query is this:
LOAD DATA LOCAL INFILE 'C:\\Users\\Usuario\\Desktop\\CovidDeaths.csv' INTO TABLE coviddeaths
columns terminated by ';'
LINES TERMINATED BY '\r\n';
I have some alredy uploaded data to the coviddeaths table using the import wizards but the upload was really slow, so tried to finish importing using the LOAD DATA INFILE query.
To see how many lines have been alredy uploaded I used
select count(*) from coviddeaths;
And threw:
count(*)
11312
When I try to run the LOAD DATA LOCAL INFILE query, nothing happens. The mouse pointer spins for a second like it tries to run it but it doesnt add any additional row. It doesnt throw any errors like the ones I have read about Error Code: 1148. The used command is not allowed with this MySQL version. I also tried to place the csv file in the Uploads folder, where my Secure File Priv option pointed to in my.ini:
# Secure File Priv.
secure-file-priv="C:/ProgramData/MySQL/MySQL Server 8.0/Uploads"
It just does nothing. Any ideas?
Thanks in advance!
EDIT 2: Database Schema:
The error comes from the fact that both client and server must enable an option to allow local datafiles to be imported. If you use the command-line client, you would need to open the client like this:
mysql --local-infile -u root -p
In addition, the MySQL Server would need to be configured with the local-infile option.
The secure-file-priv is not needed, because that controls importing or exporting files on the server, not from the client.
See also ERROR 1148: The used command is not allowed with this MySQL version or my answer to "mysql 8.0" local infile fails I tried and display the settings. Could it be permission issues?
The error message is unusually obscure. MySQL error messages are usually more accurate and informative. I logged a bug in 2019 to request an improved error message, and they did fix it in MySQL 8.0.
I'm trying to insert data via a CSV file to a MySQL Table over PowerShell.
$SQL="Driver={MySQL ODBC 8.0 Unicode Driver};Server=theServer;Database=import;UID=Monty;PWD=some_Password;AllowLoadLocalInfile=true;"
$WEBDBConnection=New-Object System.Data.Odbc.OdbcConnection
$WEBDBConnection.ConnectionString=$SQL
$WEBDBConnection.Open()
$WEBDBCommand=New-Object System.Data.Odbc.OdbcCommand
$WEBDBCommand.Connection=$WEBDBConnection
$WEBDBCommand.CommandText="LOAD DATA LOCAL INFILE 'C:\Users\admin\Desktop\Test.csv' INTO TABLE import LINES TERMINATED BY ';';"
$WEBDBCommand.ExecuteReader()
$WEBDBConnection.Close()
Executing the Script I'll get an error that 'loading local data is disabled; this must be enabled on both the client and server sides'
On the Server side I activated 'local_infile' and I thought adding 'AllowLoadLocalInfile=true;' to the connection should activate it on the client side. So I don't get why I get this error message. Any ideas?
Thanks,
Connor
I tried to upload a .txt file into MySQL Workbench, but I have the following issue:
Error Code: 3948 Loading Local data is disable; this must be enable on both the client and server sides
Workbench uses a MySQL feature called LOAD DATA LOCAL for this .txt file import operation. Because that feature exposes some security problems in the server, the operator of the server needs to enable that feature, by running the MySQL server software (mysqld, it's called) with a specific system variable called local_infile. Your error message means that flag is not enabled.
You can try enabling it at runtime before you do your upload operation. Try this SQL statement.
SET ##GLOBAL.local_infile = 1;
If that doesn't work you need to ask the person who runs your server to enable it.
I have been running into a problem with MySQL's Load Data Local Infile command. I know that they disabled it by default due to a security threat. I have a SQL Script that I would like to run in MySql Workbench. I keep getting the error "1148. The used command is not allowed with this MySQL version." I have been able to get this to work from the command line using command line arguements. However I havent had any success in the Workbench. Any help would be appreciated. Here is my script.
DROP DATABASE IF EXISTS a2004TS;
CREATE DATABASE IF NOT EXISTS a2004TS;
USE a2004TS;
DROP TABLE IF EXISTS a2004TS_1;
CREATE TABLE a2004TS_1(
Version CHARACTER(25),
Dsetid CHARACTER(6),
V040001 DOUBLE,
..
.
);
SET GLOBAL local_infile = 1;
LOAD DATA LOCAL INFILE '/home/output/sqlCSV/a2004TS.sqlscripts.table1.csv'
INTO TABLE a2004TS_1
FIELDS TERMINATED BY ',' ENCLOSED BY'"'
LINES TERMINATED BY'
'(columns......);
If the server is started with the option to disable LOAD DATA then a client has to explicitly call mysql_options(...) to enable it (http://dev.mysql.com/doc/refman/5.5/en/load-data-local.html). This does not happen in WB, so you cannot use it to load a file currently. Instead use the command line client for this.
You could file a feature request at http://bugs.mysql.com to have handling implemented that allows the import (e.g. a preference setting).