I'm having problems using 'LOAD DATA INFILE' via a db.executesql() command: I'm getting "InternalError: (1148, u'The used command is not allowed with this MySQL version')"
A bit of digging, and I find I need to set --local-infile as a mysql option (MySQL: Enable LOAD DATA LOCAL INFILE). I've modified my.cnf, but that doesn't seem to be being picked up by web2py. How can I pass options to the mysql client which is used by web2py?
Related
I have a node.js script which is connecting to a local MariaDB server, creating some tables, and then attempting to load some data into them by executing a SQL query like this:
LOAD XML LOCAL INFILE 'C:\\path\\to\\some_data.xml'
INTO TABLE some_data
ROWS IDENTIFIED BY '<data>';
The script fails with an error like:
SqlError: (conn=60, no: 45034, SQLState: 45034) LOCAL INFILE wrong filename. 'C:\path\to\some_data.xml' doesn't correspond to query LOAD XML LOCAL INFILE 'C:\\path\\to\\some_data.xml'
INTO TABLE some_data
ROWS IDENTIFIED BY '<data>';. Query cancelled. Check for malicious server / proxy
The node.js script code is like:
const pathToXmlFile = path.resolve(__dirname, '../some_data.xml')
.replace(/\\/g, '\\\\');
// .replace(/\\/g, '/');
// none of the above work, and neither does no escaping at all
query = `LOAD XML LOCAL INFILE '${pathToXmlFile}'
INTO TABLE some_data
ROWS IDENTIFIED BY '<data>';`;
I tried escaping the path with forward slashes (I'm running the script through Git Bash), the error remains the same, only the way the path is displayed changes. I have set permitLocalInfile: 'true' on the MariaDB client connection and local_infile=1 on both the [mysqld] and [client] sections of my.ini. The node.js connection user has the FILE privilege.
Node version is 16.13.0, MariaDB npm module version is 2.5.5, all running on the same machine with Windows 10 (MariaDB server too, on localhost).
The above query works perfectly fine when executed from HeidiSQL.
Any ideas on what I'm doing wrong?
I've create a mariadb bug for that : https://jira.mariadb.org/browse/CONJS-181
As a workaround, you might use "/" in place of "\": using "C:/path/to/some_data.xml" won't be escaped avoiding this bug
It seems that it's a bug with the mariadb node connector, I have opened an issue here:
https://github.com/mariadb-corporation/mariadb-connector-nodejs/issues/183
In the meantime, I have tried the mysql2 and mysql connectors, the first doesn't seem to support the permitLocalInfile: 'true' connection option, so I switch to the standard mysql connector and everything works fine (with a few changes in the code because of slight differences in method calling).
If the bug in the mariadb connector is fixed, I will update the answer here.
I would like to facilitate opening a database UI for development projects (usually docker containers, bound to arbitrary ports on the host machine) by a generic command.
I wonder if it is possible to open MySQL Workbench and let it connect automatically from the command line.
Similar to giving connection parameters with the mysql console:
mysql --host=127.0.0.1 --port=$port --user=db --password=db db
I haven't found that specifically in the supported arguments, so either it is hidden or maybe possible with any of the other options?
EDIT:
Probably the way is to generate a file to pass to --query dynamically?
Here's the format for the mysqlworkbench --query parameter:
--query="$user:$password#$host:$port"
This feature already exists as an example in ddev - look in the ~/.ddev/commands/host/mysqlworkbench.example file. (See on github).
For ddev, the query is set up as query="root:root#127.0.0.1:${DDEV_HOST_DB_PORT}", which uses the root/root credentials, accesses the 'db' container via localhost on the port provided by ddev at $DDEV_HOST_DB_PORT.
I want to automate the installation and configuration of a mysql server using azure cli.
The installation works well using azure mysql server create, however the configuration using azure mysql server configuration set -n time_zone --value Europe/Paris fails due to the following error:
Deployment failed. Correlation ID: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx. The value 'Europe/Paris' for configuration 'time_zone' is not valid. The allowed values are '[+|-][0]{0,1}[0-9]:[0-5][0-9]|[+|-][1][0-2]:[0-5][0-9]|SYSTEM'.
As I read in the mysql docs I could enable named time zones executing the following sql SET GLOBAL time_zone = timezone;, but unfortunately my user would need super privilege for this to succeed and this is impossible in azure.
The other approach would be to run mysql_tzinfo_to_sql but this is not available using azure cli.
Is there any other way to activate named time zones?
From the Azure DB for MySQL documentation:
Populating the time zone tables
The time zone tables on your server can be populated by calling the mysql.az_load_timezone stored procedure from a tool like the MySQL command line or MySQL Workbench.
CALL mysql.az_load_timezone();
Also, in this doc (you linked to in your question):
Upon initial deployment, an Azure for MySQL server includes systems tables for time zone information, but these tables are not populated. The time zone tables can be populated by calling the mysql.az_load_timezone stored procedure from a tool like the MySQL command line or MySQL Workbench.
According to Error message the format should be one of these three:
[+|-][0]{0,1}[0-9]:[0-5][0-9]
eg. -04:30
or
[+|-][1][0-2]:[0-5][0-9]
e.g -12:00
or
SYSTEM
So have you tried doing it with quotes?
azure mysql server configuration set -n time_zone --value
"Europe/Paris"
CALL mysql.az_load_timezone();
Call this stored procedure from your session on the server,
If you are running the mysql.az_load_timezone command from MySQL Workbench, you may need to turn off safe update mode first using SET SQL_SAFE_UPDATES=0 or in Preferences->SQL Editor->Safe Updates(), and back to connect to your mysql server.
Does JetBrains DataGrip have any way to allow me to set the SQL_MODE automatically for my connection to a MySQL database?
For example, MySQL Workbench has a specific field for the connection that would set the SQL_MODE. Whilst HeidiSQL provide a way to execute a Startup Script.
In DataGrip, follow these steps.
Go to File > Data Sources and select your connection from the list of Project Data Source.
Go to the Advanced tab which lists a lot of variables.
Locate the the variable named sessionVariables. (Tip: You can search for a particular variable by selecting any variable name, then start typing to seek.)
Set the value for the variable with your SQL mode. For example:
sql_mode='STRICT_TRANS_TABLES,NO_ENGINE_SUBSTITUTION,ONLY_FULL_GROUP_BY'
I am scanning 5000 csv files into a database. Regrettably the files have '', for 0. Thus when ever I run my script it crashes. I heard that this error is possible to avoid by simply disabling strict mode. So I attempted to disable strict mode to allow me to read in an empty string as a 0 for my numeric fields. However the error persisted.
So does disabling strict mode allow '' to be read into a int field? (the '' is two qoutes i.e. empty string)
If so why did setting
sql_mode=''
in the my.ini config file not fix the problem.
Thank you!
I guess you import the CSV file using LOAD DATA INFILE command. Before you execute this command, type:
SET sql_mode = '';
More information about various SQL modes can be found in the documentation.
I need do it this, and work correctly:
To disable strict SQL mode, SSH in to your server as root
Create this file: /etc/mysql/conf.d/disable_strict_mode.cnf
Open the file and enter these two lines:
[mysqld]
sql_mode=IGNORE_SPACE,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION
Restart MySQL with this command: sudo service mysql restart
You can modify you LOAD DATA INFILE Statement to correct the values. Something along these lines should work.
LOAD DATA INFILE 'filepath.csv'
INTO TABLE MyTable(Column1,#Col2,#Col3)
SET Column2=CASE WHEN #Col2 = '' THEN 0 ELSE #Col2 END
,Column3=CASE WHEN #Col3 = '' THEN 0 ELSE #Col3 END;
This query imports the value as is into column1, and the fixes the values for columns 2 and 3. Using this, you don't have to disable strict mode, and you are actually in control of what data goes into your database, are are able to fix it in a reliable way. You can also use this feature to change date formats, or import hex encoded blob values. Very useful feature.
I'm putting this in an answer as I am yet to have the rep to comment on your follow up question about the settings not "holding". It has been my experience with MySQL that the settings in the configuration file are only read on starting the server so that you would have to restart the server for new settings to take effect.
The server is the daemon that the command-line client communicates with when you run commands so that restarting the command line client does not actually restart the server itself.
A more elaborate explanation is provided in this answer: https://serverfault.com/a/79051.