How to renew connection of database for iam authentication - aws-sdk

Using IAM database authentication the password expires after about 15 minutes. So I ideally have to renew the database connection before the password expires. I set up a timer on the first initialization of the database and query the passed time on each query. How can i refresh the password of the connection before the password expires? Or how can i destroy the old database object and renew the object if necessary?
The error message is: "PAM authentication failed for user iam_user".
code for getting IAM Password:
const pgp = require('pg-promise')();
const AWS = require('aws-sdk');
const ca =
'-----BEGIN CERTIFICATE-----\nMIID9DCCAtyg...
...wZfTUU=\n-----END CERTIFICATE-----\n';
const signer = new AWS.RDS.Signer({
region: process.env.REGION,
username: process.env.DATABASE_USER,
hostname: process.env.DATABASE_HOST,
port: parseInt(`${process.env.DATABASE_PORT}`, 10),
});
module.exports = pgp({
host: process.env.DATABASE_HOST,
port: process.env.DATABASE_PORT,
database: process.env.DATABASE_NAME,
user: process.env.DATABASE_USER,
ssl: { ca },
dialectOptions: { ssl: { require: true } },
password: signer.getAuthToken(),
});
injecting db object to graphql:
const db = require('../db/init');
server.use(
mount(
'/graphql',
graphqlHTTP( () => ({
schema: schema,
context: { startTime: Date.now(), db },
graphiql: true
})),
),
);
Using the database in the resolvers.
I could query the time of the creation of the database connection. Is there a possibility to renew the password if necessary? Or what is the best way to destroy the old database object and create a new database object?
const resolvers = {
Query: {
Post: (root, args, {db}) => {
console.log(args.id);
console.log(db.$config.options)
const postQuery = new PQ({
text:
'SELECT post_id as id FROM tbl_post where post_id = $1',
values: [parseInt(args.id, 10)],
});
return db.one(postQuery).catch((err) => console.log(err));
}
}

As suggested by vitaly-t i used a password function. To avoid adding latency this functions renews the password only every 15 minutes. If the pool gets continuously used in intervals lower than 10 seconds, than the connection stays open without calling the password function at all. According to my tests there are no new connections to the database opened at all.
const ca = '-----BEGIN CERTIFICATE-----\9DC...-----END CERTIFICATE-----\n';
const signer = new AWS.RDS.Signer({
region: process.env.REGION,
username: process.env.DATABASE_USER,
hostname: process.env.DATABASE_HOST,
port: parseInt(`${process.env.DATABASE_PORT}`, 10),
});
const SIGNER = { time: 0, password: undefined};
function getSignedPassword() {
const time = Date.now();
if (time - SIGNER.time > 900000) {
SIGNER.time = new Date().getTime();
SIGNER.password = signer.getAuthToken();
return SIGNER.password;
}
return SIGNER.password;
}
module.exports = pgp({
host: process.env.DATABASE_HOST,
port: process.env.DATABASE_PORT,
database: process.env.DATABASE_NAME,
user: process.env.DATABASE_USER,
ssl: { ca },
password: getSignedPassword,
});

Related

Dynamically change the name of mySql database in pool in node.js

connection.js
var mysql = require("mysql");
const { createPool } = require("mysql");
const pool = createPool({
port: process.env.DB_PORT,
host: process.env.DB_HOST,
user: process.env.DB_USER,
password: process.env.DB_PASS,
database: process.env.MYSQL_DB,
connectionLimit: 10,
});
module.exports = pool;
product.service.js
const pool = require("../../../config/database");
module.exports = {
viewVariate: (callBack) => {
pool.query(
`SELECT varId , varName FROM variations WHERE varIsActive='1'`,
(errors, results, fields) => {
if (errors) {
return callBack(errors);
}
return callBack(null, results);
}
);
},
}
I have to maintain more than 100 users. so each person has own database in that node app. each person database name call via API , so I need to set given database name to above pool dynamically , I referred may ways. it did not work. connection.js file is my database connection with pool. then it call to service files to make connection queries. for that pool data obtains form env file. it doesn't matter when config mentioned requirement will be done.

Query a Google cloud SQL instance in Node.js from a GKE pod with cloud sql proxy running as sidecar

I am tasked with adding a MySql database to a microservice application for work. I am the only person on this task and don't really have anyone to turn too for advice so I am reaching out to the internets for help. I have succesfully deployed a pod that is running a small test application and the cloud-sql-proxy. I have scoured the documentation trying to figure out how to connect to the db and perform a query and this is what I have come up with (but it doesn't work).
const express = require('express');
const mysql = require('mysql');
const bodyParser = require('body-parser')
const cors = require('cors');
const axios = require('axios');
const app = express();
app.use(bodyParser.json());
app.use(cors())
app.enable('trust proxy');
// Automatically parse request body as form data.
app.use(express.urlencoded({extended: false}));
// This middleware is available in Express v4.16.0 onwards
app.use(express.json());
// [START cloud_sql_mysql_mysql_create_tcp]
const createTcpPool = async config => {
// Extract host and port from socket address
const dbSocketAddr = process.env.DB_HOST.split(':');
// Establish a connection to the database
return mysql.createPool({
user: process.env.DB_USER, // e.g. 'my-db-user'
password: process.env.DB_PASS, // e.g. 'my-db-password'
database: process.env.DB_NAME, // e.g. 'my-database'
host: dbSocketAddr[0], // e.g. '127.0.0.1'
port: dbSocketAddr[1], // e.g. '3306'
// ... Specify additional properties here.
...config,
});
};
// [END cloud_sql_mysql_mysql_create_tcp]
var pool = createTcpPool();
const stuff = pool.query('SELECT * FROM entries');
function getQuery() {
console.log(stuff);
}
getQuery()
Here is a picture of the error I get when I deploy the pod and the logs from the proxy to verify it's running
I'm pretty new to MySql and GKE and trying to figure this out has been a huge struggle. I just want to know how I can actually query the db and would greatly appreciate some assistance or code sample to point me in the right direction, thanks internets.
As mentioned in the thread1 ,
Handling such functions can be done through following example :
const mysql = require('mysql');
const pool = mysql.createPool({ connectionLimit : 1, socketPath: '/cloudsql/' + '$PROJECT_ID:$REGION:$SPANNER_INSTANCE_NAME',
user: '$USER', p
assword: '$PASS',
database: '$DATABASE' });
exports.handler = function handler(req, res)
{ //using pool instead of creating connection with function call
pool.query(`SELECT * FROM table where id = ?`,
req.body.id, function (e, results) {
//made reply here
}); };
For more information you can refer to the documentation related to TCP connection when using Node js.
const createTcpPool = async config => {
// Extract host and port from socket address
const dbSocketAddr = process.env.DB_HOST.split(':');
// Establish a connection to the database
return mysql.createPool({
user: process.env.DB_USER, // e.g. 'my-db-user'
password: process.env.DB_PASS, // e.g. 'my-db-password'
database: process.env.DB_NAME, // e.g. 'my-database'
host: dbSocketAddr[0], // e.g. '127.0.0.1'
port: dbSocketAddr[1], // e.g. '3306'
// ... Specify additional properties here.
...config,
});
};
So trying to create a pool by calling createTcpPool seems to have been the issue. I changed it to
let pool = mysql.createPool({
user: process.env.DB_USER, // e.g. 'my-db-user'
password: process.env.DB_PASS, // e.g. 'my-db-password'
database: process.env.DB_NAME, // e.g. 'my-database'
host: '127.0.0.1', // e.g. '127.0.0.1'
port: '3306'
});
and got a succesful return from my db.

Table '[database-name].sessions' doesn't exist - using express-session

Here is some sample code that outlines my issue. I'm trying to get express-session / connect-session-sequelize to work for a website with login functionalities.
However, when I try to call my POST request, I get the following error:
I can only assume it's trying to store session data onto my database, but cannot find a table. I can bypass this by going in and creating the table manually with all the columns it wants, but I'm wondering if there's an issue in my code preventing the package from working properly (or if this is how it's supposed to work.)
require('dotenv').config({ path: './config/.env' })
const express = require('express')
const session = require('express-session')
const mysql = require('mysql2')
const Sequelize = require('sequelize')
const path = require('path')
const SequelizeStore = require('connect-session-sequelize')(session.Store)
const app = express()
const PORT = 9999
app.use(express.json())
app.use(express.static(path.join(__dirname, 'public')))
// Allowing connection to database in Workbench
const db = new Sequelize('somedatabase', 'root', process.env.PASSWORD, {
host: 'localhost',
dialect: 'mysql'
})
db.authenticate()
.then(() => {
console.log('Connected...')
}).catch(error => {
console.log('Failed to connect...', error)
})
// Setting up session
app.use(session({
secret: 'shhh',
store: new SequelizeStore({
db: db
}),
resave: false,
saveUninitialized: true,
cookie: {
maxAge: 1000000
}
}))
// Sample model
const User = db.define('user', {
id: {
type: Sequelize.INTEGER,
primaryKey: true,
autoIncrement: true
},
username: Sequelize.STRING,
password: Sequelize.STRING
})
// Sample request
app.post('/api/create', async (req, res) => {
const newUser = {
username: john,
password: verysecurepassword
}
await User.create(newUser)
})
app.listen(PORT, () => {
console.log(`Listening on localhost:${PORT}...`)
})
In this code, you are using several packages: express-session, which manages the session itself but delegates how the session is saved to connect-session-sequelize.
So the problem is that connect-session-sequelize is trying to save session data in the database, but it cannot because there is no table for sessions.
As written in the documentation of this package (https://www.npmjs.com/package/connect-session-sequelize):
If you want SequelizeStore to create/sync the database table for you, you can call sync() against an instance of SequelizeStore along with options if needed.
So try creating the store, attaching it to the session manager, and then
initializing it (I did not test this code):
// Setting up session
var myStore = new SequelizeStore({
db: db
});
app.use(
session({
secret: "shhh",
store: myStore,
resave: false,
saveUninitialized: true,
cookie: {
maxAge: 1000000
}
})
);
myStore.sync();

AWS Lambda and RDS working example (need it to work with Sequelize)

Here's a working example of AWS Lambda and MySQL, but I'd like it to work with Sequelize. How do I initialize Sequelize to work with AWS Lambda? I have the authenticated IAM role working too.
https://dzone.com/articles/passwordless-database-authentication-for-aws-lambd
'use strict';
const mysql = require('mysql2');
const AWS = require('aws-sdk');
// TODO use the details of your database connection
const region = 'eu-west-1';
const dbPort = 3306;
const dbUsername = 'lambda'; // the name of the database user you created in step 2
const dbName = 'lambda_test'; // the name of the database your database user is granted access to
const dbEndpoint = 'lambdatest-cluster-1.cluster-c8o7oze6xoxs.eu-west-1.rds.amazonaws.com';
module.exports.handler = (event, context, cb) => {
var signer = new AWS.RDS.Signer();
signer.getAuthToken({ // uses the IAM role access keys to create an authentication token
region: region,
hostname: dbEndpoint,
port: dbPort,
username: dbUsername
}, function(err, token) {
if (err) {
console.log(`could not get auth token: ${err}`);
cb(err);
} else {
var connection = mysql.createConnection({
host: dbEndpoint,
port: dbPort,
user: dbUsername,
password: token,
database: dbName,
ssl: 'Amazon RDS',
authSwitchHandler: function (data, cb) { // modifies the authentication handler
if (data.pluginName === 'mysql_clear_password') { // authentication token is sent in clear text but connection uses SSL encryption
cb(null, Buffer.from(token + '\0'));
}
}
});
connection.connect();
// TODO replace with your SQL query
connection.query('SELECT * FROM lambda_test.test', function (err, results, fields) {
connection.end();
if (err) {
console.log(`could not execute query: ${err}`);
cb(err);
} else {
cb(undefined, results);
}
});
}
});
};
Instead of using mysql.createConnection() and use your RDS Signer token:
var sequelize = require('sequelize')
const Sequelize = new sequelize(
process.env.database_name,
process.env.databse_user,
token,
{
dialect: 'mysql',
dialectOptions: {
ssl: 'Amazon RDS',
authPlugins: { // authSwitchHandler is deprecated
mysql_clear_password: () => () => {
return token
}
}
},
host: process.env.db_proxy_endpoint,
port: process.env.db_port,
pool: {
min: 0, //default
max: 5, // default
idle: 3600000
},
define: {
charset: 'utf8mb4'
}
}
// then return your models (defined in separate files usually)
await Sequelize.authenticate() // this just does a SELECT 1+1 as result;
await Sequelize.sync() // DO NOT use this in production, this tries to create tables defined by your models. Consider using sequelize migrations instead of using sync()
Also it's a good idea to keep your database connection parameters in a config file so no one can see them. (process.env)
We are working with Sequelize and Lambda, but you will need to reserve more resources, in our case we need at least 1GB to run a lambda with Sequelize. Without it, just with mysql2 it runs just with 128MB.
But if you really wanna use Sequelize just replace your createConnection for something like what you will find in sequelize doc
Probably you will use the context.callbackWaitsForEmptyEventLoop=true because you may have some issues when you call the callback function and you get nothing because your Event Loop probably will never be empty.

Error: Access denied for user ''#'localhost' (using password: NO)

I'm trying out db migrations with MySQL and Knex.
When I run the command knex migrate:latest, I get
ER_ACCESS_DENIED_ERROR: Access denied for user ''#'localhost' (using password: NO)
I've tried adding a password on the codebase (to '123' and 'NO'), though what confuses me most is that even as I have user: "root" in my database file, the error gives an empty string as the user...
I share what I imagine are the pertinent files:
// mysql_db.js
const knex = require('knex')({
client: 'mysql',
connection: {
host: 'localhost',
user: 'root',
password: '',
database: 'SQL_Data',
},
});
module.exports = knex;
// knexfile.js
const path = require('path');
module.exports = {
development: {
client: 'mysql',
connection: {
filename: '/server/SQL/mysql_db',
},
migrations: {
directory: path.join(__dirname, '/server/SQL/migrations'),
},
seeds: {
directory: path.join(__dirname, '/server/SQL/seeds'),
},
},
};
//knex.js
const environment = proces.env.NODE_ENV || 'development';
const config = require('../../knexfile.js')[environment];
module.exports = require(knex)('config');
// "migration definition"
exports.up = (knex, Promise) => knex.schema.createTable('sql_table', ((table) => {
table.increments();
table.string('name').notNullable();
table.string('email').notNullable();
table.string('description').notNullable();
table.string('url').otNullable();
}));
exports.down = (knex, Promise) => knex.schema.dropTable('sql_table');
As error message say you are trying to login with invalid credentials user whose name is empty string doesn't exist in DB.
This means your configuration is wrong. you have some strange segment in your node-mysql driver configuration, which tries to refer other file, which exports initialized knex instance
client: 'mysql',
connection: {
filename: '/server/SQL/mysql_db'
}
That is just plain wrong. Correct format for knexfile is pretty much the same that is used to create knex instance, except that knexfile supports also selecting the profile according to NODE_ENV environment variable.
const path = require('path');
module.exports = {
development: {
client: 'mysql',
connection: {
host: 'localhost',
user: 'root',
password: '',
database: 'SQL_Data',
},
migrations: {
directory: path.join(__dirname, '/server/SQL/migrations'),
},
seeds: {
directory: path.join(__dirname, '/server/SQL/seeds'),
},
},
};
In your mysql_db you might like to do something like this to init knex to
be able to use the same config:
const knex = require('knex')(
require('knexfile')[process.env.NODE_ENV || 'development']
);