Slaves not used in Symfony 4 with Raw Queries - mysql

I'm using Symfony 4 to interface with an existing Master/Slave MySQL setup and am executing queries against the server using raw sql. Raw SQL is the only option at the moment.
I'm using show full processlist; on the DB server to monitor which DB is used, and I am only seeing connections to the master server. It doesn't appear that any of the slaves are ever used.
For reference, I have two dbal connections setup, the default is NOT master/slave, and uses orm mapping. The second is the master/slave which I'm having issues with, and this is the server I'm executing raw sql queries against.
Below is my doctrine.yml:
doctrine:
dbal:
default_connection: default
connections:
default:
driver: pdo_mysql
host: "%env(DATABASE_HOST)%"
dbname: "db1"
user: "%env(DATABASE_USER)%"
password: "%env(DATABASE_PASS)%"
charset: UTF8
ds:
driver: pdo_mysql
host: "%env(DS_DATABASE_HOST)%"
dbname: "db2"
user: "%env(DS_DATABASE_USER)%"
password: "%env(DS_DATABASE_PASS)%"
slaves:
slave1:
host: "%env(DS_DATABASE_SLAVE1_HOST)%"
user: "%env(DS_DATABASE_USER)%"
password: "%env(DS_DATABASE_PASS)%"
dbname: "db2"
slave2:
host: "%env(DS_DATABASE_SLAVE2_HOST)%"
user: "%env(DS_DATABASE_USER)%"
password: "%env(DS_DATABASE_PASS)%"
dbname: "db2"
orm:
default_entity_manager: default
entity_managers:
default:
connection: default
mappings:
Main:
is_bundle: false
type: annotation
dir: '%kernel.project_dir%/src/Entity/Main'
prefix: 'App\Entity\Main'
alias: Main
ds:
connection: ds
I have configured my entity managers in my services.yml as follows:
# Entity managers
App\Service\Database\MainEntityManager:
arguments:
$wrapped: '#doctrine.orm.default_entity_manager'
App\Service\Database\DSEntityManager:
arguments:
$wrapped: '#doctrine.orm.ds_entity_manager'
The entity manager (in this case DSEntityManager) is injected into the constructor of a class, then the query is executed as such:
$result = $this->em->getConnection()->prepare($sql);
$result->execute($args);
Please let me know if I'm missing any helpful configuration.
Thanks a lot for the help.

Thanks #Cerad for the tip, that got me in the correct direction. Since I was no longer trying to use an entity manager for raw queries that were not mapped to entities, I could work with the connection directly.
I Created a wrapper class which extended MasterSlaveConnection. That worked as long as I was using executeQuery(). Per the docs, that must be used to query the slaves. However, my query required the use of prepare() and query() which both force the master connection.
So inside my new wrapper class I created two new methods, prepareSlave() and querySlave() which do the same as the original; however, they do $this->connect('slave'); instead of $this->connect('master');
Now all my read queries hit slave and everything else hits master.
So here are the following updates I've made to the configuration above to achieve this:
doctrine.yml
ds:
driver: pdo_mysql
host: "%env(DS_DATABASE_HOST)%"
dbname: "db2"
user: "%env(DS_DATABASE_USER)%"
password: "%env(DS_DATABASE_PASS)%"
wrapper_class: "%env(DS_DATABASE_PASS)%"
slaves: App\Service\Database\DSWrapper
slave1: ...
services.yml
# DBAL connections
App\Service\Database\DSWrapper: '#doctrine.dbal.ds_connection'
My new wrapper class
class DSWrapper extends MasterSlaveConnection
{
public function prepareSlave($statement)
{
$this->connect('slave');
try {
$stmt = new Statement($statement, $this);
} catch (\Exception $ex) {
throw DBALException::driverExceptionDuringQuery($this->_driver, $ex, $statement);
}
$stmt->setFetchMode($this->defaultFetchMode);
return $stmt;
}
public function querySlave()
{
$this->connect('slave');
$args = func_get_args();
$logger = $this->getConfiguration()->getSQLLogger();
if ($logger) {
$logger->startQuery($args[0]);
}
$statement = $this->_conn->query(...$args);
if ($logger) {
$logger->stopQuery();
}
return $statement;
}
}
So now if I need to execute a query which would normally require the use of prepare() and query(), I instead use prepareSlave() and querySlave().

Related

sequelize dialectoptions , UTC and dates, for mysql

I am still learning how to use sequelize.
I am using sequelize 5 along with node 10 and mysql2 .
I am setting my connection like so
const sequelize = new Sequelize({
database: dbs,
username: usr,
host: hst,
password: pwd,
port: prt,
dialect: 'mysql',
dialectOptions: {
useUTC: true,
dateFirst: 1
},
define:{
timestamps:false,
paranoid:false,
freezeTableName: true
}
});
and I am getting the following warnings
Ignoring invalid configuration option passed to Connection: useUTC.
This is currently a warning, but in future versions of MySQL2, an
error will be thrown if you pass an invalid configuration option to a
Connection
Ignoring invalid configuration option passed to Connection: dateFirst.
This is currently a warning, but in future versions of MySQL2, an
error will be thrown if you pass an invalid configuration option to a
Connection
If I remove the useUTC: true and dateFirst: 1 from the dialectOptions, the errors obviously go away.
What am I missing and how can I set those options for mysql

Sails database configuration through environment variables

I used to configure Sails connection to my database in config/env/development.js and config/env/production.js like that:
module.exports = {
connections: {
'postgres': {
host: 'localhost',
user: 'myUser',
password: 'myPassword',
database: 'myDatabase'
}
}
};
What if I would like to replace my environment config files by environment variables as explained here?
I expected to use those variables but it doesn't work:
sails__connections_postgres_host
sails__connections_postgres_user
sails__connections_postgres_password
sails__connections_postgres_database
Your underscores are backwards. Looking at the docs you linked, your variables should look like:
sails_connections__postgres__host
etc. (one underscore after 'sails', two underscores in between each key after that).
Additionally, it is worth noting that you can reference environment variables in your code, so the option exists to refer to an environment variable, 'dbHostname' in your config/env/development.js (or production) as such:
module.exports = {
connections: {
'postgres': {
host: process.env.dbHostname,
user: process.env.dbUser,
password: process.env.dbPassword,
database: process.env.db
}
}
}
and then create these environment variables upon lifting your server, i.e.
dbHostname="http://something" dbUser="Your_User" dbPassword="password" db="database" sails lift

Unable to connect to MAMP MySQL database using Wakanda's remote connection

I'm trying to connect to a MAMP MySQL database from Wakanda 11.
I tried connecting to the localhost database using Connect to Remote Datastore but I keep getting a Connection failed response. I've also tried using port 127.0.0.1:8081 which it connects to but not when attempting to use port:3306 which is where I have the MySQL database configured on the MAMP server. What am I doing wrong?
I've tried the following script...
model.addSQLCatalog("mysqldb", {
hostname: '127.0.0.1',
port: 3306,
user: 'root',
password: 'xxxxxxxx',
database: 'my_database_name',
ssl: false,
dbType: 'mysql'
});
...but this gives me the following error:
TypeError: JSON.stringify cannot serialize cyclic structures.
How can I fix and make it work on port:3306 so I can see my database structures?
If you are using Wakanda Enterprise Edition 11 or higher there is a MySQL Connector Pro.
Here is an example of connecting with this:
model.mergeSQLCatalog(localName, {
hostname: string 'host name',
port: number remote_port_number,
user: string 'userName',
password: string 'password',
database: string 'SQL database name',
jsFile: string 'configuration JavaScript file',
ssl: boolean true or false,
dbType: string 'mysql' } )
If you are on an older version of Wakanda Enterprise Edition but still using version 7 or higher then there is a Wakanda/MySQL connector.
Here is an example of connecting to MySQL with the connector:
var sql = require('waf-sql');
//use port 3306 and do not use SSL
var dbconn = sql.connect('mysql','192.168.0.21', 'john', 'x54?hsf5x!','arts',3306,false);
var rs = dbconn.update("people", {
name: "smith",
age: 42
}, {
id: 1
});
var rs = dbconn.select("*", "people", {
id: 1
});
var row = rs.getNextRow(); // get the first row
dbcon.close(); // close connection
Here is an example of the available parameters:
var params = {
hostname: [your host name or IP address],
user: [the user name of your DB],
password: [the user password],
database: [the DB name],
port: [the port number of the MySQL Server, by default 3306],
ssl: false,
dbType : 'mysql'
};
If I'm correct your issue has been solved? Does it work for both Windows & Mac ?
To resume , to use ProCOnnector you'll need :
The Enterprise Version of Wakanda
For the current v11 version of Wakanda, you need to use in the model.js file the addSQLCatalog() API.
The parameters accepted are described in the doc
Please note in the next version of Wakanda we 'll provide to wizzard to connect to other DBs instead of the addSQLCatalog(). This will help and ease the process.

Deprecated: `config.adapters.default` Sails.js

When I lift my sails project I get these warnings on terminal
debug: Deprecated: config.adapters.default debug: (see
http://links.sailsjs.org/docs/config/connections) debug: For now, I'll
pretend you set config.models.connection.
debug: Deprecated: config.adapters debug: (see
http://links.sailsjs.org/docs/config/connections) debug: For now, I'll
pretend you set config.connections.
debug: Deprecated: config.adapters.*.module debug: (see
http://links.sailsjs.org/docs/config/connections) debug: For now, I'll
pretend you set config.connections["disk"].adapter.
And my data is not stored in mysql DB.
This is my api model code
module.exports = {
schema:true,
tableName: 'BusStop',
adapters: 'mysql-adapter',
migrate: 'safe',
attributes: {
stopID:{
type: 'int'
},
stopName:{
type: 'string'
},
latitude:{
type: 'float'
},
longitude:{
type: 'float'
}
}
};
This is my local.js code
module.exports = {
port: process.env.PORT || 1337,
environment: process.env.NODE_ENV || 'development',
adapters:{
'default': 'disk',
disk:{
module:'sails-disk'
},
'mysql-adapter': {
module : 'mysql-sails',
host : '127.0.0.1',
user : 'root',
password : '',
database : 'PublicTransport',
schema : true
}
}
};
And this is my connections.js code
module.exports.connections = {
localDiskDb: {
adapter: 'sails-disk'
},
'mysql-adapter': {
module : 'sials-mysql',
host : '127.0.0.1',
port : 3306,
user : 'root',
password : '',
database : 'PublicTransport'
},
someMongodbServer: {
adapter: 'sails-mongo',
host: 'localhost',
port: 27017,
},
somePostgresqlServer: {
adapter: 'sails-postgresql',
host: 'YOUR_POSTGRES_SERVER_HOSTNAME_OR_IP_ADDRESS',
user: 'YOUR_POSTGRES_USER',
password: 'YOUR_POSTGRES_PASSWORD',
database: 'YOUR_POSTGRES_DB'
}
};
I want to know how can I remove these Deprecated warnings and why my data is not being stored in the Database.
Looks like the links in those deprecation warnings are incorrect. For information on how to configure Sails connections, see http://sailsjs.org/#/documentation/reference/sails.config/sails.config.connections.html.
You should be able to make those messages go away by following their instructions regarding what config keys to set.
In order to set a connection for a specific model, you now set the connection property:
module.exports = {
schema:true,
tableName: 'BusStop',
connection: 'mysql-adapter', // "adapter" is now "connection
migrate: 'safe',
attributes: {...}
}
In order to specify the adapter to use for a connection, use the adapter key, not module; you're already doing this in most of your connections, you just need to update mysql-adapter.
In your config/local.js you're using the deprecated adapters key to set up connections; use connections instead.
Lastly, in order to set a default connection for all of your models, you do as the deprecation message says and set sails.config.models.connection rather than sails.config.adapters.default; you can do so easily in the config/models.js file, or in your config/local.js like so:
module.exports = {
models: {
connection: 'mysql-adapter'
},
...more config...
}
This warning can also be generated if your project has an config/adapters.js file. If the file is empty it can be deleted and Zap the warning is gone.If it contains code then you will need to migrate it to the correct path config/models.js and config/connections.js

Using local.js to store sails-mysql password

I'm trying to use mysql as my database, but I can't figure out how to get my config/adapters.js to use mysql information in config/local.js to connect. What's the correct way to store the connection information so that sails-mysql can connect?
config/local.js is merged on top of all other configuration. So you can just put your own adapters key in there:
{
adapters: {
default: 'myLocalAdapter',
myLocalAdapter: {
module: 'sails-mysql',
host: 'localhost',
user: 'root',
password: 'password',
database: 'database'
}
}
}
and it'll be picked up and used by Sails.
EDIT In sails v0.10.x, use the key connections instead of adapters as the two are now differentiated and connections now hold authentication info. The original answer was for an earlier version of sails.
As of latest version of sails (i.e 0.12), while writing this answer, there is no Adapters.js. The list of connections are specified in the config/connections.js and default connection to use for all models are specified under config/models.js. So, going by this change the way to override the database configuration for local is specifying as below in config.local.js.
module.exports = {
connections: {
localMongoServer: {
adapter: 'sails-mongo',
host: 'localhost',
port: 27017,
user: 'YOUR_DB_USER',
password: 'YOUR_DB_PASSWORD',
database: 'YOUR_DB_NAME'
}
},
models: {
connection: 'localMongoServer'
}
}
NOTE: As in accepted answer, just changing adapters to connections won't work, since there is no default attribute under connections.