Salt-Stack mysql_grants.present Underscore Wildcard - mysql

I'm using Salt to provision cloud servers but I'm having problems with MySQLdb producing the correct permissions for MySQL. If I was executing the SQL directly it would be:
GRANT ALL ON `install\_%`.* TO 'installer'#'localhost';
The sls file contains:
installer_local_install_grants:
mysql_grants.present:
- grant: all privileges
- database: install\_%.*
- user: installer
- host: localhost
- escape: False
Which produces this error:
Function: mysql_grants.present
Result: False
Comment: An exception occurred in this state: Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/salt/state.py", line 1560, in call
**cdata['kwargs'])
File "/usr/lib/python2.7/dist-packages/salt/states/mysql_grants.py", line 187, in present
grant, database, user, host, grant_option, escape, ssl_option, **connection_args
File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 1666, in grant_add
_execute(cur, qry['qry'], qry['args'])
File "/usr/lib/python2.7/dist-packages/salt/modules/mysql.py", line 505, in _execute
return cur.execute(qry, args)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 159, in execute
query = query % db.literal(args)
TypeError: * wants int
With debug in salt turned on the relevant line prior to submitting to MySQLdb is:
Doing query: GRANT ALL PRIVILEGES ON install\_%.* TO %(user)s#%(host)s args: {'host': 'localhost', 'user': 'installer'}
So it seems Salt is outputting the correct code but MySQLdb is not handling some part of the query correctly. The query is missing the back ticks but I'm really not sure how to get those in.
With the escape removed or set to True the grants look like:
+-----------+-----------------+------------------+
| Host | Db | User |
+-----------+-----------------+------------------+
| localhost | install\\_\% | installer |
+-----------+-----------------+------------------+
When it should look like:
+-----------+-----------------+------------------+
| Host | Db | User |
+-----------+-----------------+------------------+
| localhost | install\_% | installer |
+-----------+-----------------+------------------+

OK, would you open an issue referencing this SO post? Thanks!
https://github.com/saltstack/salt/issues/new

Related

ETIMEDOUT connecting to MySql from Lambda

I am trying to connect to a MySql RDS instance from a Lambda function and getting an ETIMEDOUT error
The Lambda is not part of a VPC
The RDS instance is available publicly, I can connect to it from my laptop using MySqlWorkbench
The RDS instance's security group has inbound rules configured for all ports and 0.0.0.0/0
The Lambda's execution role has many policies (probably too many) including RDSFullAccess, LambdaVPCAccessExecutionRole, ec2:*, even AdministratorAccess!)
Again, the code executes locally, connects to and queries the RDS instance just fine. Executing the same code in Lambda throws the ETIMEDOUT error
Similar posts are resolved by adding the Lambda to the RDS instance's VPC, or by configuring the inbound rules on the database's security group. Nothing seems to work.
Since I can connect to the database from my laptop just fine, my hunch is that it's a problem with the Lambda
Are there additional policies I should attach to the Lambda's execution role?
Is there any other reason the Lambda would time out connecting to a publicly available database?
Additional info:
The Lambda is not running in a VPC. It runs on Node and connects to MySql using the mysql package v2.18.1 and is deployed using Serverless with the following config:
foo:
handler: functions/handlers.foo
timeout: 20
events:
- http:
path: /path/{pathParameter}/foo
method: get
cors: true
caching:
enabled: true
ttlInSeconds: 3600
cacheKeyParameters:
- name: request.path.pathParameter
In the Lambda I try connecting with this function (which, again, works fine when I execute the function on my laptop):
function openDbConnection() {
let connection = mysql.createConnection({
host: 'db-name.cgwxrjuo6oyd.us-east-1.rds.amazonaws.com',
user: process.env.DB_USER,
password: process.env.DB_PASS,
database: 'db-name'
});
try {
connection.connect(function(err) {
console.log("Database is ", connection.state)
if (err) {
return console.error('error: ' + err.message);
}
console.log('Connected to the MySQL server.');
});
} catch (error) {
console.log("Database is ", connection.state)
console.log("Error connecting to MySql: ", error);
}
return connection;
}
The database username and password are retrieved from environment variables that are published to the Lambda with Serverless using a .env.yml file:
provider:
name: aws
runtime: nodejs12.x
lambdaHashingVersion: '20201221'
environment: ${file(.env.yml):}
Below are the Cloudwatch logs for a single execution, which I'm having trouble making sense of. Entries appear out of sequence:
| timestamp | message |
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|
| 1626209636788 | START RequestId: e3e3ceb7-bb55-4c3e-8392-38e08401f679 Version: $LATEST |
| 1626209636791 | 2021-07-13T20:53:56.790Z adfc1cd5-e4be-40b7-970c-d38acabeb199 INFO Database is disconnected |
| 1626209636791 | 2021-07-13T20:53:56.791Z adfc1cd5-e4be-40b7-970c-d38acabeb199 ERROR error: connect ETIMEDOUT |
| 1626209636791 | 2021-07-13T20:53:56.791Z adfc1cd5-e4be-40b7-970c-d38acabeb199 INFO An error occured querying MySql: connect ETIMEDOUT |
| 1626209636792 | 2021-07-13T20:53:56.792Z adfc1cd5-e4be-40b7-970c-d38acabeb199 INFO Database is disconnected |
| 1626209636792 | 2021-07-13T20:53:56.792Z adfc1cd5-e4be-40b7-970c-d38acabeb199 ERROR error: Connection lost: The server closed the connection. |
| 1626209636793 | 2021-07-13T20:53:56.792Z adfc1cd5-e4be-40b7-970c-d38acabeb199 INFO An error occured querying MySql: Connection lost: The server closed the connection. | 1626209636803 | END RequestId: e3e3ceb7-bb55-4c3e-8392-38e08401f679 |
| 1626209636803 | REPORT RequestId: e3e3ceb7-bb55-4c3e-8392-38e08401f679 Duration: 9.32 ms Billed Duration: 10 ms Memory Size: 1024 MB Max Memory Used: 79 MB |
----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Access denied for user 'root'#'external ip' (using password: YES)

i'm working on a google cloud project and i get this error when i run node index.js / try to access mysql database remotely.
this is the complete error message:
Error: ER_ACCESS_DENIED_ERROR: Access denied for user 'root'#'external_ip' (using password: YES)
at Handshake.Sequence._packetToError (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/sequences/Sequence.js:52:14)
at Handshake.ErrorPacket (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/sequences/Handshake.js:130:18)
at Protocol._parsePacket (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/Protocol.js:279:23)
at Parser.write (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/Parser.js:76:12)
at Protocol.write (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/Protocol.js:39:16)
at Socket.<anonymous> (/home/it21695/nodeproject/node_modules/mysql/lib/Connection.js:103:28)
at Socket.emit (events.js:182:13)
at addChunk (_stream_readable.js:277:12)
at readableAddChunk (_stream_readable.js:262:11)
at Socket.Readable.push (_stream_readable.js:217:10)
--------------------
at Protocol._enqueue (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/Protocol.js:145:48)
at Protocol.handshake (/home/it21695/nodeproject/node_modules/mysql/lib/protocol/Protocol.js:52:23)
at Connection.connect (/home/it21695/nodeproject/node_modules/mysql/lib/Connection.js:130:18)
at Object.<anonymous> (/home/it21695/nodeproject/index.js:10:12)
at Module._compile (internal/modules/cjs/loader.js:678:30)
at Object.Module._extensions..js (internal/modules/cjs/loader.js:689:10)
at Module.load (internal/modules/cjs/loader.js:589:32)
at tryModuleLoad (internal/modules/cjs/loader.js:528:12)
at Function.Module._load (internal/modules/cjs/loader.js:520:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:719:10)
this is my index.js code (the test DB and books table both exist):
var mysql = require('mysql');
var connection = mysql.createConnection({
host : 'external_ip',
user : 'root',
password : 'password',
database : 'test',
});
connection.connect();
connection.query('SELECT * from books', function (error, results, fields) {
if (error) throw error;
console.log(results);
});
connection.end();
i have added the mysql port (3306) to the firewall exceptions and i've granted privileges for root user. i turned the mysql and node.js external ips to static. i use the passwords that google cloud has assigned.
+------------------+----------------+------------+
| user | host | grant_priv |
+------------------+----------------+------------+
| root | % | Y |
| root | external_ip | Y |
| mysql.infoschema | localhost | N |
| mysql.session | localhost | N |
| mysql.sys | localhost | N |
| root | localhost | Y |
| stats | localhost | N |
+------------------+----------------+------------+
mysql v8.0.11node.js v10.1.0npm v5.6.0
Before checking the application any further make sure to check the user credentials and access privileges on the database host server.
This problem is related to permissions not with nodejs.
First of all i recommend you to install MySQL Workbech and perform your test from that tool.
Also you need to make some configurations into your MySQL database to allow those remote root connections, take a look at this post:
How to allow remote connection to mysql

No blog table in my first Mezzanine project on shared host

I'm trying to use Mezzanine cms (3.1.4) in a shared hosting at Dreamhost. I successfully created my first blog entry but strangely enough, I don't find the table associated with this entry. Phpadmin page at Dreamhost does show my mezzanine database (named hamlet) but there are only 10 tables in it :
auth_group, auth_group_permissions,auth_permission, auth_user, auth_user_groups, auth_user_user_permissions
django_admin_log, django_content_type, django_migrations, django_session
No blog table. Does someone have an explanation for this?
EDIT: Steps I did:
-Entered in my virtual environment env.mez
-Created the passenger_wsgi.py file:
import sys, os
INTERP = "/home/geantbrun/opt/python-2.7.7/bin/python"
#INTERP is present twice so that the new python interpreter knows the actual executable path
if sys.executable != INTERP: os.execl(INTERP, INTERP, *sys.argv)
cwd = os.getcwd()
sys.path.append(cwd)
sys.path.append(cwd + '/mez') #You must add your project here
sys.path.insert(0,cwd+'/env.mez/bin')
sys.path.insert(0,cwd+'/env.mez/lib/python2.7/site-packages/mezzanine')
sys.path.insert(0,cwd+'/env.mez/lib/python2.7/site-packages')
os.environ['DJANGO_SETTINGS_MODULE'] = "mez.settings"
from django.core.wsgi import get_wsgi_application
application = get_wsgi_application()
-Created on Dreamhost panel the database hamlet
-Created my mezzanine-project (mez)
-Parameters set in settings.py:
DATABASES = {
"default": {
"ENGINE": "django.db.backends.mysql",
"NAME": "hamlet",
"USER": "(myusername)",
"PASSWORD": "(mypass)",
"HOST": "mysql.geantbrun.com",
"PORT": "3306",
}
}
-Static files collected
-Command syncdb :
python manage.py syncdb
Syncing...
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_redirect
Creating table django_session
Creating table django_site
Creating table south_migrationhistory
Creating table django_admin_log
Creating table django_comments
Creating table django_comment_flags
You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'geantbrun'):
Email address:
Password:
Password (again):
Superuser created successfully.
A site record is required.
Please enter the domain and optional port in the format 'domain:port'.
For example 'localhost:8000' or 'www.example.com'.
Hit enter to use the default (127.0.0.1:8000): (mysite.com)
Creating default site record: mysite.com ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
Synced:
> mezzanine.boot
> moderna
> django.contrib.auth
> django.contrib.contenttypes
> django.contrib.redirects
> django.contrib.sessions
> django.contrib.sites
> django.contrib.sitemaps
> django.contrib.staticfiles
> compressor
> filebrowser_safe
> south
> django.contrib.admin
> django.contrib.comments
Not synced (use migrations):
- mezzanine.conf
- mezzanine.core
- mezzanine.generic
- mezzanine.blog
- mezzanine.forms
- mezzanine.pages
- mezzanine.galleries
- mezzanine.twitter
(use ./manage.py migrate to migrate these)
-At this point, on Dreamhost panel, I see that 14 tables are now created in hamlet database (auth_, django_ and south_migrationhistory tables).
-Migrate command: python manage.py migrate
Running migrations for conf:
- Migrating forwards to 0004_ssl_account_settings_rename.
> conf:0001_initial
> conf:0002_auto__add_field_setting_site
> conf:0003_update_site_setting
- Migration 'conf:0003_update_site_setting' is marked for no-dry-run.
> conf:0004_ssl_account_settings_rename
- Migration 'conf:0004_ssl_account_settings_rename' is marked for no-dry-run.
- Loading initial data for conf.
Installed 0 object(s) from 0 fixture(s)
Running migrations for core:
- Migrating forwards to 0006_initial.
> core:0001_initial
(...other migrations)
> core:0006_initial
FATAL ERROR - The following SQL query failed: CREATE TABLE "core_sitepermission" ("id" integer NOT NULL PRIMARY KEY, "user_id" integer NOT NULL)
The error was: table "core_sitepermission" already exists
! Error found during real run of migration! Aborting.
Error in migration: core:0006_initial
Traceback (most recent call last):
File "manage.py", line 29, in <module>
execute_from_command_line(sys.argv)
(...other traces)
File "/home/geantbrun/mysite.com/env.mez/local/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 452, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: table "core_sitepermission" already exists
Note that file referred to at last line is .../backends/sqlite3/base.py. I realize that DATABASES parameter is maybe overridden by local_settings file. I removed the DATABASES parameter from local_settings and rerun the migrate command. Result:
Error in migration: core:0006_initial
Traceback (most recent call last):
File "manage.py", line 29, in <module>
execute_from_command_line(sys.argv)
File "/home/geantbrun/mysite.com/env.mez/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/home/geantbrun/mysite.com/env.mez/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/home/geantbrun/mysite.com/env.mez/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1050, "Table 'core_sitepermission' already exists")
No more mention to sqlite3 but same kind of error at the end ('core_sitepermission already exists'). Does the migrate command should be run with --fake parameter because the database is already created?

I get error when I try to check my MySQLdb connector in Django shell

60 DATABASE_ENGINE = 'django.db.backends.mysql'
61 DATABASE_NAME = 'mysite'
62 DATABASE_USER = 'root'
63 DATABASE_PASSWORD = 'password'
64 DATABASE_HOST = '127.0.0.1'
65 DATABASE_PORT = '3306'
66
67 DATABASES = {
68 'default': {
69 'ENGINE': DATABASE_ENGINE,
70 'NAME': DATABASE_NAME,
71 'USER': DATABASE_USER,
72 'PASSWORD': DATABASE_PASSWORD,
73 'HOST': DATABASE_HOST,
74 'PORT': DATABASE_PORT,
75 }
76 }
Under the settings.py above in my django project,I tried to see if DB adapter(MySQLdb : http://sourceforge.net/projects/mysql-python/) is working OK with my django configuration.
But I ended up with meeting some kind of error I have never seen before like this.
python manage.py shell
>>> from django.db import connection
>>> a = connection()
Traceback (most recent call last):
File "<console>", line 1, in <module>
TypeError: 'DefaultConnectionProxy' object is not callable
I installed mysql on my machine, of course, and since I've done this configuration before and it worked at that time, I am confused what did I have done it wrong this time.
I know what error messages would be like when the error is something due to django's settings.py configuration. But I totally don't get what's going on with this error.
Please help me.
"The object django.db.connection represents the default database connection. To use the database connection, call connection.cursor() to get a cursor object. Then, call cursor.execute(sql, [params]) to execute the SQL and cursor.fetchone() or cursor.fetchall() to return the resulting rows."
https://docs.djangoproject.com/en/1.7/topics/db/sql/#executing-custom-sql-directly
So, if you need connection object you should do:
connection = django.db.connection
Django use persistent connection to database, so you no need to initialize it.

Using Puppetlabs-mysql to allow more than one host

I am using Puppet v3.4.3 and Puppetlabs-mysql module v2.3.1 ( https://forge.puppetlabs.com/puppetlabs/mysql) to create database and I need to allow hosts localhost and % to use it. Both Puppet master and client run on Ubuntu-14.04 servers.
I use code like this:
mysql::db { 'mydb':
user => 'myuser',
password => 'mypass',
host => 'localhost',
grant => ['SELECT', 'UPDATE'],
}
If I try to pass both hosts as an array they get combined together:
host => ['localhost', '%'],
Mysql users:
| user | host |
+------------------+-------------------+
| username | localhost% |
If I try giving host-parameter twice I get error about duplicate declaration:
Error 400 on SERVER: Duplicate parameter 'host' for on Mysql::Db
Is what I want possible with this Puppet-module and if so how is it done?
You create one grant with your mysql::db, as in your first example. To allow the other host, you need to add a dedicated mysql::grant.