I'm trying to connect to my MySQL RDS from a Lambda via AWS.RDS.Signer with the following code and fake credentials:
1 const DB_REGION = 'ap-southeast-2a'
2 const DB_HOST = 'dbinstance.ddtev8utygt.ap-southeast-2.rds.amazonaws.com'
3 const DB_PORT = 3306
4 const DB_USER = 'anyuser'
5 const DB_NAME = 'anydb'
6
7 const signerOptions = {
8 region: DB_REGION,
9 hostname: DB_HOST,
10 port: DB_PORT,
11 username: DB_USER
12 }
13
14 const signer = new AWS.RDS.Signer(signerOptions)
15 const token = await signer.getAuthToken()
16
17 const config = {
18 host: DB_HOST,
19 user: DB_USER,
20 password: token, // "Password123"
21 database: DB_NAME,
22 ssl: 'Amazon RDS',
23 authPlugins: {
24 mysql_clear_password: () => () => token
25 }
26 }
but I always get this error
"Access denied for user 'anyuser'#'172.14.1.12' (using password: NO)"
I'm not entirely sure if that is needed for the AWS.RDS.Signer but I selected this option of my database:
Password and IAM database authentication
Authenticates using the database password and user credentials through AWS IAM users and roles.
NOTE: If I swap the password from token to "Password123" on line 20 I can successfully connect to my RDS.
Am I missing something here or does AWS.RDS.Signer only work with RDS Proxy?
By the way: the getAuthToken function gives me something like that (token truncated)
"dbinstance.ddtev8utygt.apsoutheast2.rds.amazonaws.com:3306/Action=connect&DBUser=anyuser&&X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=ASIAU7VGXF6UCWYZCFEG%2F20210318%2Fap-southeast-2a%2Frds-db%2Faws4_request&X-Amz-Date=20210318T105145Z&X-Amz-Expires=900&X-Amz-Security-Token=IQoJb3JpZ2luX2VjEAsaDmFwLXNvdXRoZWFzdC0yIkgwRgIhAKg8ibwNJ4E3hSOuq7HtDFvqHxmTlpOUk3I6EH2%2B9VdOV3RQ%2F03xiVdvjhEBkHqEXHQ%3D&X-Amz-Signature=749d931f74873e6c2c0d4fec94f0743f42efd5aa95ca0ac0f05c4bef30e3bd4d&X-Amz-SignedHeaders=host"
I finally can connect to my RDS via Lamdba using IAM (aka AWS.RDS.Signer)
So what was the problem?
Short story:
I used the wrong region in my Policy and in the props for the AWS.RDS.Signer.
I assumed that the Availability zone (ap-southeast-2a) equals the region, but that's not true. The correct region is actually described when the AWS command line creds get created. To find out the region invoke cat ~/.aws/config from the terminal. My default region was actually region=ap-southeast-2.
Long story:
When I started with AWS.RDS.Signer I followed the instructions here
When it came to extracting the DB info for the Policy, I used this command
aws rds describe-db-instances --db-instance-identifier <MY INSTANCE NAME> --query "DBInstances[*].DbiResourceId" --region ap-southeast-2a
but I got this error Could not connect to the endpoint URL: "https://rds.ap-southeast-2a.amazonaws.com/"
After a bit of Googling, I realised that the region is not right. That gave me a hint to change the region in the policy and in the code of my Lambda function.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"rds-db:connect"
],
"Resource": [
"arn:aws:rds-db:<my-region>:<my-account-id>:dbuser:<my-db-resource-id>/<my-db-username>"
]
}
]
}
Then I created an extra user (ssluser) in the DB to connect with the AWS.RDS.Signer token
CREATE USER 'ssluser' IDENTIFIED WITH AWSAuthenticationPlugin as 'RDS';
GRANT ALL PRIVILEGES ON <MY DB NAME>.* TO 'ssluser'#'%';
GRANT USAGE ON <MY DB NAME>.* TO 'ssluser'#'%' REQUIRE SSL;
The next step was to add the above policy to the EC2 instance, ssh into the instance, install the MySQL client yum install mysql and try to connect to the instance using the token
mysql --host=dbinstance.chteb5kjtggo.ap-southeast-2.rds.amazonaws.com --port=3306 --ssl-ca=rds-combined-ca-bundle.pem --enable-cleartext-plugin --user=ssluser --password=`aws rds generate-db-auth-token --hostname dbinstance.chteb5kjtggo.ap-southeast-2.rds.amazonaws.com --port 3306 --region ap-southeast-2 --username ssluser`
After successfully connect to the RDS without providing a password I only had to attach the policy to my Lambda and change the username and region in my Lambda code to
1 const DB_REGION = 'ap-southeast-2'
2 const DB_HOST = 'dbinstance.ddtev8utygt.ap-southeast-2.rds.amazonaws.com'
3 const DB_PORT = 3306
4 const DB_USER = 'ssluser'
5 const DB_NAME = 'anydb'
I hope that will help someone in the future.
Related
Okay.. I'm having a really strange issue and not able to figure out why it's happening. I have an AWS lambda function (Node.js) which connects to a MySQL database.
When DB parameters are hardcoded in Node.js, it works fine. Please see below.
const con = mysql.createConnection({
host : "myapp.something.us-west-2.rds.amazonaws.com",
user : "someuser",
password : "somepassword",
port : "3306",
database : "somedatabse"
});
const query = util.promisify(con.query).bind(con);
When I try to connect to the DB by reading these parameters from a config file or lambda environment variables, I get this error.
"ERROR Error: connect ECONNREFUSED 127.0.0.1:3306
at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1159:16)"
const con = mysql.createConnection({
host : dbHost,
user : dbUser,
password : dbPassword,
port : dbPort,
database : dbName
});
const query = util.promisify(con.query).bind(con);
// Retrieving DB parameters from my config file like this. Values are being retrieved successfully.
dbHost = dbConfig.dev_db_host;
dbUser = dbConfig.dev_db_user;
dbPassword = dbConfig.dev_db_password;
dbPort = dbConfig.dev_db_port;
dbName = dbConfig.dev_db_name;
// Able to log all these values correctly.
console.log(dbHost); // myapp.something.us-west-2.rds.amazonaws.com
console.log(dbUser);// someuser
console.log(dbPassword); // somepassword
console.log(dbPort); // 3306
console.log(dbName); // somedatabase
Note - I am able to retrieve the values successfully from config file. I verified by logging the info. Not sure why the DB connection is failing.
Also, I run into the exact same issue when I read the DB parameters from lambda environment variables.
Any help is appreciated. Thanks in advance!
This should be really basic and simple to do but I can seriously not find any understandable information on how to create a simple database for my nodejs typescript project.
I have installed the following packages with npm:
mysql2
sequelize
sequelize-cli
sequelize-typescript
I have attempted the following commands at the terminal
C:\repos\NodeNew>mysql2 -u root -p
'mysql2' is not recognized as an internal or external command,
operable program or batch file.
C:\repos\NodeNew>mysql -u root -p
'mysql' is not recognized as an internal or external command,
operable program or batch file.
C:\repos\NodeNew>node mysql2 -u root -p
module.js:538
throw err;
^
Error: Cannot find module 'C:\repos\NodeNew\mysql2'
at Function.Module._resolveFilename (module.js:536:15)
at Function.Module._load (module.js:466:25)
at Function.Module.runMain (module.js:676:10)
at startup (bootstrap_node.js:187:16)
at bootstrap_node.js:608:3
C:\repos\NodeNew>
So how do I CREATE my database so I can connect to it with sequelize etc?
The tools you have installed are only for connecting a Node.js app to MySQL and do not include command-line tools to manage the MySQL server.
I will assume you have installed MySQL and it's running – you should then be able to find its mysql.exe command line client in the bin/ directory in the server's installation directory. If you haven't fiddled with authentication, just running it might work.
When you get to a MySQL prompt, you can follow any old instructions for creating a database; CREATE DATABASE foo; is the gist of it (authentication and permissions being a different story).
Since you're on Windows, you might want to look into HeidiSQL – it's been a while since I've used it, but it's a decent graphical MySQL management tool.
You can also use mysql2 to create the database – illustrated below – but I recommend getting a management tool.
const mysql = require('mysql2');
const connection = mysql.createConnection({
host: 'localhost',
user: 'root',
database: 'mysql',
});
connection.query('CREATE DATABASE foo');
You should have MySQL installed on your computer, to install mysql on Windows see the following page: MySql
Once you have MySQL up and running on your computer. Open the Command Terminal and execute the following:
npm install mysql
Now you have downloaded and installed a mysql database driver. Node.js can use this module to manipulate the MySQL database:
var mysql = require('mysql');
To create a conecction create a file connection.js:
var con = mysql.createConnection({
host: "localhost",
user: "yourusername",
password: "yourpassword"
});
con.connect(function(err) {
if (err) throw err;
console.log("Connected!");
});
to test it save the file and run:
node connection.js
Which will give you this result:
Connected!
It can be solved using beforeConnect hook of sequelize as below:
const env = process.env.NODE_ENV || 'development';
const config = require(__dirname + '/../config/config.json')[env];
const { host, port, username, password } = config;
# create sequelize instance without providing db name in config
sequelize = new Sequelize('', username, password, config);
sequelize.beforeConnect(async (config) => {
const connection = await mysql.createConnection({ host: host, port: port, user: username, password: password });
await connection.query(`CREATE DATABASE IF NOT EXISTS \`${process.env.DB_NAME}\`;`);
config.database = process.env.DB_NAME;
});
Config.json file contains configurations for different dbs like
{
"development": {
"username": "root",
"password": "init#123",
"host": "mysqldb",
"dialect": "mysql"
},
"test": {
"username": "root",
"password": null,
"host": "127.0.0.1",
"dialect": "mysql"
},
"production": {
"username": "root",
"password": null,
"host": "127.0.0.1",
"dialect": "mysql",
}
}
Database name is provided after db connection and creation(if required) in sequelize beforeConnect hook
I'm trying to connect to a MAMP MySQL database from Wakanda 11.
I tried connecting to the localhost database using Connect to Remote Datastore but I keep getting a Connection failed response. I've also tried using port 127.0.0.1:8081 which it connects to but not when attempting to use port:3306 which is where I have the MySQL database configured on the MAMP server. What am I doing wrong?
I've tried the following script...
model.addSQLCatalog("mysqldb", {
hostname: '127.0.0.1',
port: 3306,
user: 'root',
password: 'xxxxxxxx',
database: 'my_database_name',
ssl: false,
dbType: 'mysql'
});
...but this gives me the following error:
TypeError: JSON.stringify cannot serialize cyclic structures.
How can I fix and make it work on port:3306 so I can see my database structures?
If you are using Wakanda Enterprise Edition 11 or higher there is a MySQL Connector Pro.
Here is an example of connecting with this:
model.mergeSQLCatalog(localName, {
hostname: string 'host name',
port: number remote_port_number,
user: string 'userName',
password: string 'password',
database: string 'SQL database name',
jsFile: string 'configuration JavaScript file',
ssl: boolean true or false,
dbType: string 'mysql' } )
If you are on an older version of Wakanda Enterprise Edition but still using version 7 or higher then there is a Wakanda/MySQL connector.
Here is an example of connecting to MySQL with the connector:
var sql = require('waf-sql');
//use port 3306 and do not use SSL
var dbconn = sql.connect('mysql','192.168.0.21', 'john', 'x54?hsf5x!','arts',3306,false);
var rs = dbconn.update("people", {
name: "smith",
age: 42
}, {
id: 1
});
var rs = dbconn.select("*", "people", {
id: 1
});
var row = rs.getNextRow(); // get the first row
dbcon.close(); // close connection
Here is an example of the available parameters:
var params = {
hostname: [your host name or IP address],
user: [the user name of your DB],
password: [the user password],
database: [the DB name],
port: [the port number of the MySQL Server, by default 3306],
ssl: false,
dbType : 'mysql'
};
If I'm correct your issue has been solved? Does it work for both Windows & Mac ?
To resume , to use ProCOnnector you'll need :
The Enterprise Version of Wakanda
For the current v11 version of Wakanda, you need to use in the model.js file the addSQLCatalog() API.
The parameters accepted are described in the doc
Please note in the next version of Wakanda we 'll provide to wizzard to connect to other DBs instead of the addSQLCatalog(). This will help and ease the process.
I have a node.js server that works but needs to be set up for ssh connections:
var mysql = require('mysql')
var io = require('socket.io').listen(3000)
var db = mysql.createConnection({
host: 'hostname',
user: 'username',
password: '12345',
database: '12345',
port: 3306,
socket: '/var/run/mysqld/mysqld.sock'
})
db.connect(function(err){
if (err) console.log(err)
})
I'm aware that there are ssh npm libraries for this purpose, however the options available (ssh2, node-sshclient, etc) appear to deal with pretty intricate features that may overcomplicate things. I'm looking for the simplest way to connect to my mysql db through ssh. What would be the best way to accomplish this?
If you are running a linux/unix system do the following:
Connect to your mysql server via ssh and proxy the mysql port (default is 3306) via this ssh tunnel.
This works as follows:
1 Type in screen (to start a screen session which is permanent even if the shell gets closed).
2 Type into screen shell:
ssh -L 3306:127.0.0.1:3306 your_servers_domain_or_ip -lyour_login_name
3 Enter your ssh password / or use a PKI auth to avoid manual steps
4 Done... now it’s possible to connect MySQL like you would do when it’s installed on the same machine as your application.
Connect to MySQL from node.js like below:
var db = mysql.createConnection({
host: '127.0.0.1', // Important to connect to localhost after connecting via ssh in screen
user: 'username',
password: '12345',
database: '12345',
port: 3306
});
Sometimes it's preferrable to instantiate the SSH tunnel connection dynamically (in code) rather than separately using OS libraries. For example, it makes it easier to automatically close the connection, share the environment with other developers, or conditionally use an SSH tunnel depending on the environment.
With packages such as tunnel-ssh, this is easy. Building on the example provided, the connection code would look like:
import { createSSHTunnel } from "./sshTunnel";
const { srcAddr, srcPort } = await createSSHTunnel();
var db = mysql.createConnection({
host: srcAddr,
port: srcPort,
user: 'username',
password: '12345',
database: '12345'
});
With all logic cleanly abstracted away in the sshTunnel module, that could look like:
// sshTunnel.js
import { createTunnel } from "tunnel-ssh";
export async function createSSHTunnel(srcAddr = "127.0.0.1", srcPort = 12345) {
const tunnelOptions = {
autoClose: true,
};
const serverOptions = {
port: srcPort,
};
const sshOptions = {
host: process.env.SSH_HOST,
port: parseInt(process.env.SSH_PORT),
username: process.env.SSH_TUNNEL_USER,
password: process.env.SSH_TUNNEL_PASSWORD,
};
const forwardOptions = {
srcAddr: srcAddr,
srcPort: srcPort,
dstAddr: process.env.DB_HOST,
dstPort: parseInt(process.env.DB_PORT),
};
try {
await createTunnel(
tunnelOptions,
serverOptions,
sshOptions,
forwardOptions
);
} catch (error) {
if (error.code === "EADDRINUSE") {
// Assume port is uniquely used by SSH tunnel, so existing connection can be reused
console.log(`Returning existing SSH tunnel on ${srcAddr}:${srcPort}.`);
return { srcAddr, srcPort };
} else {
throw error;
}
}
console.log(`SSH tunnel successfully created on ${srcAddr}:${srcPort}.`);
return { srcAddr, srcPort };
}
Remarks:
The SSH tunnel arbitrarily uses local port 12345
The environment variables involved are:
DB_HOST: the database hostname
DB_PORT: the database port, 3306 in the original MySQL example, 5432 for Postgres etc.
SSH_HOST: the hostname of the machine serving the SSH tunnel
SSH_PORT: the port of the machine serving the SSH tunnel
SSH_TUNNEL_USER: the username for the SSH tunnel
SSH_TUNNEL_PASSWORD: the password for the SSH tunnel
I'm trying to connect to a database using a node.js server both hosted on appfog.com using these settings:
var client = mysql.createConnection({
host: 'mysql-node01.eu-west-1.aws.af.cm',
user: '****#gmail.com',
password: '*******',
database: 'd2dc10d6a450048b587114fa9b11756ed',
port: 3306
});
I'm pretty sure the host, database and maybe port are incorrect but I don't know which values to use, i.e. should I use localhost?
Use the db credentials from the VCAP_SERVICES environment var and do not hard code them. The credentials are provided when the app is started on AppFog. See the Appfog Node Docs
var env = JSON.parse(process.env.VCAP_SERVICES);
var creds = env['mysql-5.1'][0]['credentials']; # grabs the creds for the first mysql database
var client = mysql.createConnection({
host: creds.hostname || 'localhost',
user: creds.username,
password: creds.password,
database: creds.name,
port: creds.port || 3306
});
I created a npm package to help facilitate getting production and development credentials. See AppFog Env
How to use it:
First set a local env var with your local dev database credentials:
export DEV_DB_CREDS='{ "username": "root", "name": "dev-db-name" }'
Then get the creds in the app:
var service = appfog.getService('mysql-db-name', process.env.DEV_DB_CREDS)
var creds = service.credentials
This method allows the same code to work locally and in production.