I tried to connect to remote mysql server using mysqlsh, however the current username containers # so
mysqlsh mysql://userid#salesdeparment:#123.45.678.123:3306/ ....
reported an error
Invalid URI: Illegal character [#] found at position xxx
Because the first # is actually part of the username userid#salesdeparment, only second # denote the server ip.
I'm encountering problem with MySQL. I'm getting error Lost connection to MySQL server during query. On StackOverflow they say that very likely max_allowed_packet in MySQL config is too low.
I've tried to change it in /etc/mysql/my.cnf but the file isn't editable. sudo can't be used on PA.
I'm using Sqlalchemy to handle interaction with MySQL server.
What can I do ?
Edit:
I've passed it into config variable:
class ProductionConfig:
SQLALCHEMY_DATABASE_URI = 'mysql://myconnection#server$db?max_allowed_packet=32M'
and then passed it into app and db initialization:
app.config.from_object(ProductionConfig)
db.init_app(app) # db = Sqlalchemy()
output:
TypeError: 'max_allowed_packet' is an invalid keyword argument for connect()
Problem went away after I increased pool recycling:
with Sqlalchemy:
class Config:
SQLALCHEMY_DATABASE_URI = 'mysql://<your-connection-string>'
SQLALCHEMY_POOL_RECYCLE = 280
this wasn't my idea. A member of staff at PythonAnywhere suggested it.
You can change global variables if you have SUPER privilege. But you can't make those changes persist, so if the MySQL Server restarts, the change will revert to the original value.
mysql> set global max_allowed_packet=1024*1024*32;
Query OK, 0 rows affected (0.00 sec)
I'm not familiar with Python Anywhere, but I would guess if you don't have permissions to edit the config file, you might not have SUPER privilege either.
The only other option is that you limit yourself to queries and results that fit in the max_allowed_packet that has been configured for you by the people who run Python Anywhere.
If you have need for more power, then it's time to learn to run your own server.
You can specify this flag on client side and on server side, but the server side are the limit. You can follow Bill Karwin answer or try to change on client side and whatch the effects.
On client, shell:
mysql --max_allowed_packet=32M mydb -e 'SELECT * FROM table1'
On client, PyMySQL example:
g.db = pymysql.connect(
host = 'localhost',
port = 3306,
user = 'user',
password = 'password',
database = 'mydb',
max_allowed_packet = '32M'
)
On client, SQLAlchemy, but I'm not sure if this will work as expected:
engine = create_engine("mysql+pymsql://user:pass#some_mariadb/dbname?charset=utf8mb4&max_allowed_packet=32M")
I'm developing a script in RStudio which connects to local MySQL Server using the R package RMariaDB (not RMySQL - for other reasons though the outcome is the same).
I can both connect via storing the password in the script like:
localuserpassword <- "password"
all_projectsDb <- dbConnect(RMariaDB::MariaDB(), user='user', password=localuserpassword, dbname='projects', host='localhost')
or by way of a .my.cnf using credentials:
[client]
[mygroup]
host=127.0.0.1
user=user
password=password
port=3306
database=projects
and R code as
settingsfile = '/Users/oscar_w/.my.cnf'
all_projectsDb <- dbConnect(RMariaDB::MariaDB(), default.file = settingsfile, group="mygroup", dbname = 'projects')
The above work just fine but if I want to connect with .mylogin.cnf created in mysql_config_editor and looks like
[client]
[mygroup]
user = user
password = *****
host = 127.0.0.1
port = 3306
with the R script code like
# define location of config file
settingsfile = '/Users/oscar_w/.mylogin.cnf'
all_projectsDb <- dbConnect(RMariaDB::MariaDB(), default.file = settingsfile, group="mygroup", dbname = 'projects', password = NULL, user = NULL)
I get the error
Error: Failed to connect: Access denied for user 'root'#'localhost' (using password: NO)
I have tried various combinations of arguments expressing null or otherwise. And have entered my password with mysql_config_editor with double quotes around it. In https://cran.r-project.org/web/packages/RMariaDB/RMariaDB.pdf it specifies the use of .mylogin.cnf but I cannot find a way to make it work. Does anyone know a solution to this or has the same issue? Thanks
It looks like you're trying to log in both with and without a password, which isn't allowed. The RMariaDB documentation says that if the password argument is NULL or omitted, only users without a password can log in.
I have a linux server with mysql with symfony framework installed.
Now i'm trying to get data from sql server 2012 and all works great except when i try to insert value in mysql database.
SQLSTATE[HY000]: General error: 1366 Incorrect string value: '\x80 IN P...' for column 'note_anagrafica' at row 1
Seems that the problem is on data conversion.
The data type of column inside mysql where i'm trying to insert the data is:
text
And the collation is:
utf8_unicode_ci
FreeTds configuration is set on UTF-8 and driver is odbc.
Someone have a solution for this problem?
EDIT:
odbc.ini
[server]
Driver = FreeTDS
Server = x.x.x.x
Port = 1234
Database = databasename
TDS_Version = 8
client charseg = UTF-8
server charset = CP1252
freetds.conf
[server]
host = x.x.x.x
port = 1234
tds version = 8.0
client charset = UFT-8
text size = 20971520
use uft-16 = true
doctrine config for mysql.
charset: utf8mb4
default_table_options:
charset: utf8mb4
collate: utf8mb4_unicode_ci
doctrine config for external mssql database:
server:
driver_class: Lsw\DoctrinePdoDblib\Doctrine\DBAL\Driver\PDODblib\Driver
host: "%a_database_host%"
port: "%a_database_port%"
dbname: "%a_database_name%"
user: "%a_database_user%"
password: "%a_database_password%"
charset: utf8mb4
from now i'm getting error when i'm trying to insert data to mysql comes from sql server 2012 with doctrine.
Till now i have no problem on reading execpt that if i out the result on the web the special charaters like ø comes with � symbol.
There is a conversion that i have to made before call
$entityManager->persist($object);
$entityManager->flush();
Set the charset to utf8mb4 for doctrine config and will do the trick.
How would I connect to my VPS based MySQL database remotely (from a cloud based app) using the Ruby Net::SSH or Net::SSH::Gateway gems and key, not password, authentication?
And then connect to the database with Sequel or DataMapper. I'm assuming that after I manage to get the SSH connection working, I would just setup a Sequel/DM connection to 'sql_user#localhost:3306/database'.
I did locate a couple of similar question here, but they all use password authentication, not keys, and only demonstrate executing raw commands to query the database.
UPDATE: I just cannot seem to get this (Net::SSH with key manager) to work.
UPDATE2: Alright I have managed to get authorization when logging in from a computer that has authorized keys stored in the users local .ssh folder, with the following (port is my custom SQL port on the VPS):
sql_gate = Net::SSH::Gateway.new('192.xxx.xxx.xx','sqluser', port: 26000)
However, I will not be able to create a .ssh folder in the app's VM, so I need to somehow pass the path and filename (I will be creating a public key just for SQL access for specified user) as an option ... but haven't been able to figure out how.
UPDATE: Just need to figure out DataMapper access now. Current code being tested (remote_user_sql is my Ubuntu user, sql_user is the MySQL database user with localhost/127.0.0.1 privileges):
require 'net/ssh/gateway'
require 'data_mapper'
require 'dm-mysql-adapter'
class User
include DataMapp......
.
.
end
ssh_gate = Net::SSH::Gateway.new('192.n.n.n','remote_user_sql', {port: 25000, keys: ["sql_rsa"], keys_only: true})
port = ssh_gate.open('localhost',3306,3307)
child = fork do
DataMapper.setup(:default, {
adapter: 'mysql',
database: 'sql_test',
username: 'sql_user',
password: 'passwd',
host: 'localhost',
port: port})
DataMapper.auto_upgrade!
exit
end
puts "child: #{child}"
Process.wait
ssh_gate.close(port)
My solution, in two parts:
Well I have figured how to make the Net::SSH::Gateway gem using a specified keyfile, and then connect to the VPS through ssh via a port other than 22:
Part 1: Net::SSH::Gateway key authentication
First you must generate the keyfiles you want to use, copy the .pub to the remove server and append it to the ~/.ssh/authorized_keys file (cat sql_rsa.pub >> authorized_keys), and then make sure user_sql (the user I created on the VPS to be used only for this purpose) has been added to AllowUsers list in sshd_config. Make note of port used for ssh (25000 for this example) and use the following code to establish the connection:
ssh_gate = Net::SSH::Gateway.new('192.n.n.n','user_sql', {port: 25000, keys: ["sql_rsa"], keys_only: true})
That will read the keyfile sql_rsa in the same directory as script file, then create a new ssh gateway for 'user_sql'#'192.n.n.n' on port 25000.
I can successfully execute raw shell commands on the remove VPS with:
ssh_gate.exec("ls -la")
To close:
ssh_gate.shutdown!
Unfortunately I am still having problems using DataMapper (do-mysql-adapter) to use the gateway. I will update this answer if I figure that part out, but at least the first half of the problem has been solved.
These are the errors that DataMapper::Logger has reported:
When 127.0.0.1 was used:
Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2) (code: 2002, sql state: HY000, query: , uri: )
When localhost was used:
Access denied for user 'user_sql'#'localhost' (using password: YES) (code: 1045, sql state: 28000, query: , uri: )
When the VPS hostname was used:
Unknown MySQL server host 'hostname' (25) (code: 2005, sql state: HY000, query: , uri: )
UPDATE (No success yet): So far the only way I can access the remote MySQL database is by using Net::SSH::Gateway to establish a gateway, and then use the .sshmethod to open a new Net::SSH connection over that gateway, like so:
ssh_gate.ssh('192.n.n.n','user_sql',{port: 25000, keys: ["sql_rsa"], keys_only: true}) do |ssh|
ssh.exec("mysql -u sql_user -p'passwd' -h localhost -P 3306 -e 'SELECT DATABASE();'")
end
In other words, I can only execute SQL commands using the mysql command line. I cannot figure out how to get Sequel or DataMapper to use the gateway to connect.
Part 2: DataMapper/Sequel/mysql2 connection through Net::SSH::Gateway
Make sure your MySQL server is bound to 127.0.0.1 in /etc/mysql/my.cnf, setup your connection - DataMapper example:
DataMapper.setup(:default, {
adapter: 'mysql',
database: 'DATABASE',
username: 'username',
password: 'passwd',
host: '127.0.0.1',
port: 3307}) # local port being forwarded via Net::SSH:Gateway
Followed by any class table definitions and DataMapper.finalize if required. Note that DataMapper doesn't actually connect to the remote MySQL server until either an auto_upgrade!, auto_migrate!, or query is executed, so no need to create the forwarded port yet.
Then create a new Net::SSH::Gateway, and then whenever you need DataMapper/Sequel to access the remote database, just open a port for the process, like so:
port = ssh_gate.open('127.0.0.1',3306,3307)
child = fork do
DataMapper.auto_upgrade! # DM call that accesses MySQL server
exit
end
Process.wait
ssh_gate.close(port)
You may want to put the Net::SSH::Gateway/.open code in a begin..ensure..end block, ensure'ing the port closure and gateway shutdown.
I had to use a fork and Process.wait to establish the connection, without it the method just hangs.