Using MySQL on Openshift with Symfony 2 - mysql

I added MySQL, and PHPMyAdmin cartridges to my openshift php app.
After mysql cartridge was added I saw the page which says:
Connection URL: mysql://$OPENSHIFT_MYSQL_DB_HOST:$OPENSHIFT_MYSQL_DB_PORT/
but I have no idea what does it mean.
When I access mysql database through PHPMyAdmin,
I see 127.8.111.1 as db host, so I configured my symfony 2 app (parameters.yml):
parameters:
database_driver: pdo_mysql
database_host: 127.8.111.1
database_port: 3306
database_name: <some_database>
database_user: admin
database_password: <some_password>
Now when I access my web page it throws an error, which I believe related to mysql connection. Can someone show me proper way of doing the above?
EDIT: It seems mysql connection works fine, but somehow
Error 101 (net::ERR_CONNECTION_RESET): Unknown error
is thrown.

The code I use and works very well to make my apps working both on localhost and openshift without changing database config parameters every time I move between them is this:
<?php
# app/config/params.php
if (getEnv("OPENSHIFT_APP_NAME")!='') {
$container->setParameter('database_host', getEnv("OPENSHIFT_MYSQL_DB_HOST"));
$container->setParameter('database_port', getEnv("OPENSHIFT_MYSQL_DB_PORT"));
$container->setParameter('database_name', getEnv("OPENSHIFT_APP_NAME"));
$container->setParameter('database_user', getEnv("OPENSHIFT_MYSQL_DB_USERNAME"));
$container->setParameter('database_password', getEnv("OPENSHIFT_MYSQL_DB_PASSWORD"));
}?>
This will tell the app that if is openshift environment it needs to load different username, host, database, etc.
Then you have to import this file (params.php) from your app/config/config.yml file:
imports:
- { resource: parameters.yml }
- { resource: security.yml }
- { resource: params.php }
...
And that's it. You will never have to touch this file or parameters.yml when you move on openshift or localhost.

Connection URL: mysql://$OPENSHIFT_MYSQL_DB_HOST:$OPENSHIFT_MYSQL_DB_PORT/
OpenShift exposes environment variables to your application containing the host and port information for your database. You should reference these environment variables in your configuration instead of hard-coding values. I am not a Symfony expert, but it looks to me like you would need to do the following in order to use this information in your app:
Create a pre-start hook for your application and export variables in Symfony's expected format. Add the following to the .openshift/action_hooks/pre_start_php-5.3 file in your application's git repo:
export SYMFONY__DATABASE__HOST=$OPENSHIFT_MYSQL_DB_HOST
export SYMFONY__DATABASE__PORT=$OPENSHIFT_MYSQL_DB_PORT
Symphony uses this pattern to identify external configuration in the environment, and will make the this configuration available for use in your YAML configuration:
parameters:
database_driver: pdo_mysql
database_host: "%database.host%"
database_port: "%database.port%"
EDIT:
Another option to expose this information for use in the YAML configuration is to import a php file in your app/config/config.yml:
imports:
- { resource: parameters.php }
In app/config/parameters.php:
$container->setParameter('database.host', getEnv("OPENSHIFT_MYSQL_DB_HOST"));
$container->setParameter('database.port', getEnv("OPENSHIFT_MYSQL_DB_PORT"));

Related

Environment variable not found: DATABASE_URL. Prisma and mysql

I've developped an API with Node.Js, Express, Prisma and Mysql in local firstly. After that it works, I have deployed my API on Heroku and I took the ClearDB add-on to have a Mysql DB on Heroku.
So the deployment is OKAY when I go on my root root URI I have the "Cannot GET /" message, and when I try to connect to my ClearDB with MysqlWorkbench I have my tables, columns etc...
The main problem is from Prisma.
When I go to the "Run console" of my Heroku's project, the command npx prisma init works perfectly BUT when I type npx prisma migrate deploy || dev or also if I try to npx prisma db push I have this error =>
Error: Get Config: Schema parsing - Error while interacting with query-engine-node-api library
Error code: P1012
error: Environment variable not found: DATABASE_URL.
--> schema.prisma:10
|
9 | provider = "mysql"
10 | url = env("DATABASE_URL")
|
All my code is in a GitHub repo, I've configured my .env (which is in the root folder of my server) like this :
DATABASE_URL="mysql://<username>:<my-password>#eu-cdbr-west-30.cleardb.net/heroku_36d295ebb6686a2"
NODE_ENV="development"
APP_SECRET="jwtsecret12"
NODE_PATH="./src"
ACCESS_TOKEN_SECRET="651651651848754cdfce9fz8ef4ef54se8f4sef48s69ef84e"
I hope you have all the informations that you need to help me :)
PS : Locally my project works perfectly
Waiting for your answers, thank you very much !
Your .env file is irrelevant. It should not be used on Heroku (and should not be tracked in your repository).
ClearDB provides an environment variable called CLEARDB_DATABASE_URL, not DATABASE_URL. You can either change your code to use this variable instead of DATABASE_URL, or you can set DATABASE_URL to the same value:
Retrieve your database URL by issuing the following command:
heroku config | grep CLEARDB_DATABASE_URL
CLEARDB_DATABASE_URL => mysql://adffdadf2341:adf4234#us-cdbr-east.cleardb.com/heroku_db?reconnect=true
Copy the value of the CLEARDB_DATABASE_URL config variable.
If you’re using Ruby on Rails and the mysql2 gem, you will need to change the mysql:// scheme in the CLEARDB_DATABASE_URL to mysql2://
heroku config:set DATABASE_URL='mysql://adffdadf2341:adf4234#us-cdbr-east.cleardb.com/heroku_db?reconnect=true'
Adding config vars:
DATABASE_URL => mysql2://adffd...b?reconnect=true
Restarting app... done, v61.
The connection information for Heroku Postgres can change at any time, but since the ClearDB documentation provides the preceding guidance I would hope that it does not do so.

How to prisma with local existing database? When i tried i got the following error

This is the error :
reverie-pc#reveriepc-Latitude-3400:~/VasanthkumarV/prisma$ sudo npm install -g prisma
[sudo] password for reverie-pc:
npm WARN deprecated request#2.88.2: request has been deprecated, see
https://github.com/request/request/issues/3142
/usr/bin/prisma -> /usr/lib/node_modules/prisma/dist/index.js
+ prisma#1.34.10
updated 1 package in 29.734s
(base) reverie-pc#reveriepc-Latitude-3400:~/VasanthkumarV/prisma$ prisma init test
? Set up a new Prisma server or deploy to an existing server? Use existing database
? What kind of database do you want to deploy to? MySQL
? Does your database contain existing data? Yes
? Enter database host localhost
? Enter database port 3306
? Enter database user root
? Enter database password [hidden]
? Please select the schema you want to introspect database_test
Introspecting database database_test 435ms
Created datamodel definition based on 24 tables.
? Select the programming language for the generated Prisma client Prisma JavaScript Client
Created 3 new files:
prisma.yml Prisma service definition
datamodel.prisma GraphQL SDL-based datamodel (foundation for database)
docker-compose.yml Docker configuration file
Next steps:
1. Open folder: cd test
2. Start your Prisma server: docker-compose up -d
3. Deploy your Prisma service: prisma deploy
4. Read more about introspection:url
▸ Syntax Error: Expected Name, found Int "1"
Get in touch if you need help: https://slack.prisma.io/
To get more detailed output, run $ export DEBUG="*"
(node:14055) [DEP0066] DeprecationWarning: OutgoingMessage.prototype._headers is deprecated
Generating schema... !
How to resolve this error..and what is the procedure to connect Prisma server with local database (MySQL)?? and what about the prisma deployment??
How to connect prisma with existing db?
It looks like you are using Prisma 1 which is currently in maintenance mode.
Given that this looks like a new project, I'd suggest you take a look at Prisma 2 which includes many improvements and a simpler mental model.

Docker, AspNetCore, DB connection string best practices

I've been spending the last week or so attempting to learn docker and all the things it can do, however one thing I'm struggling to get my head around is the best practice on how to manage secrets, especially around database connection strings and how these should be stored.
I have a plan in my head where I want to have a docker image, which will contain an ASP.NET Core website, MySQL database and a PHPMyAdmin frontend, and deploy this onto a droplet I have at DigitalOcean.
I've been playing around a little bit and I have a docker-compose.yml file which has the MySQL DB and PhpMyAdmin correctly linked together
version: "3"
services:
db:
image: mysql:latest
container_name: mysqlDatabase
environment:
- MYSQL_ROOT_PASSWORD=0001
- MYSQL_DATABASE=atestdb
restart: always
volumes:
- /var/lib/mysql
phpmyadmin:
image: phpmyadmin/phpmyadmin
container_name: db-mgr
ports:
- "3001:80"
environment:
- PMA_HOST=db
restart: always
depends_on:
- db
This is correctly creating a MySQL DB for me and I can connect to it with the running PHPMyAdmin front end using root / 0001 as the username/password combo.
I know I would now need to add my AspNetCore web app to this, but I'm still stumped by the best way to have my DB password.
I have looked at docker swarm/secrets, but I still don't fully understand how this works, especially if I want to check my docker-compose file into GIT/SCM. Other things I have read have suggested using environment variables, but I still don't seem to understand how that is any different to just checking in the connection string in my appsettings.json file, or for that matter, how this would work in a full CI/CD build pipeline.
This question helped my out a little getting to this point, but they still have their DB password in their docker-compose file.
It might be that I'm trying to overthink this
Any help, guidance or suggestions would be gratefully received.
If you are using Docker Swarm then you can take advantage of the secrets feature and store all your sensitive information like passwords or even the whole connection string as docker secret.
For each secret that is created Docker will mount a file inside the container. By default it will mount all the secrets in /run/secrets folder.
You can create a custom configuration provider to read the secret and map it as configuration value
public class SwarmSecretsConfigurationProvider : ConfigurationProvider
{
private readonly IEnumerable<SwarmSecretsPath> _secretsPaths;
public SwarmSecretsConfigurationProvider(
IEnumerable<SwarmSecretsPath> secretsPaths)
{
_secretsPaths = secretsPaths;
}
public override void Load()
{
var data = new Dictionary<string, string>
(StringComparer.OrdinalIgnoreCase);
foreach (var secretsPath in _secretsPaths)
{
if (!Directory.Exists(secretsPath.Path) && !secretsPath.Optional)
{
throw new FileNotFoundException(secretsPath.Path);
}
foreach (var filePath in Directory.GetFiles(secretsPath.Path))
{
var configurationKey = Path.GetFileName(filePath);
if (secretsPath.KeyDelimiter != ":")
{
configurationKey = configurationKey
.Replace(secretsPath.KeyDelimiter, ":");
}
var configurationValue = File.ReadAllText(filePath);
data.Add(configurationKey, configurationValue);
}
}
Data = data;
}
}
then you must add the custom provider to the application configuration
public static IHostBuilder CreateHostBuilder(string[] args)
{
return Host.CreateDefaultBuilder(args)
.ConfigureAppConfiguration((hostingContext, config) =>
{
config.AddSwarmSecrets();
})
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseStartup<Startup>();
});
}
then if you create a secret with name "my_connection_secret"
$ echo "Server=myServerAddress;Database=myDataBase;Uid=myUsername;Pwd=myPassword;" \
| docker secret create my_connection_secret -
and map it to your service as connectionstrings:DatabaseConnection
services:
app:
secrets:
- target: ConnectionStrings:DatabaseConnection
source: my_connection_secret
it will be the same as writing it to the appsettings.config
{
"ConnectionStrings": {
"DatabaseConnection": "Server=myServerAddress;Database=myDataBase;Uid=myUsername;Pwd=myPassword;"
}
}
If you don't want to store all the connection string as secret then you can use a placeholder for the password
Server=myServerAddress;Database=myDataBase;Uid=myUsername;Pwd={{pwd}};
and use another custom configuration provider to replace it with the password stored as secret.
On my blog post How to manage passwords in ASP.NET Core configuration files I explain in detail how to create a custom configuration provider that allows you to keep only the password as a secret and update the configuration string at runtime. Also the the full source code of this article is hosted on github.com/gabihodoroaga/blog-app-secrets.
Secrets are complicated. I will say that pulling them out into environment variables kicks the problem down the road a bit, especially when you are only using docker-compose (and not something fancier like kubernetes or swarm). Your docker-compose.yaml file would look something like this:
environment:
- MYSQL_ROOT_PASSWORD=${MYSQL_ROOT_PASSWORD}
Compose will pull MYSQL_ROOT_PASSWORD from an .env file or a command line/environment variable when you spin up your services. Most CI/CD services provide ways (either through a GUI or through some command line interface) of encrypting secrets that get mapped to environment variables on the CI server.
Not to say that environment variables are necessarily the best way of handling secrets. But if you do move to an orchestration platform, like kubernetes, there will be a straightforward path to mapping kubernetes secrets to those same environment variables.

Setup environment variable for doctrine and elasticsearch

How to setup environment variable for symfony.
Like if i run my project than it should detetched the envirment and do the action, as an example ---
http: //production.com -> prod * environment *
http: //localhost:9200 -> * dev * environment --- for elasticsearch
http: //localhost:8000 -> * dev * environment --- for doctrine/mysql
So if i run a mysql request on localhost it should make the request at
http: //localhost:8000
and if i make a request for elasticsearch it should make the request at
http: //localhost:9200
and if it runs in the production environment it should do the request at
http: //production.com:9200 --- elasticsearch
http: //production.com:8000 --- doctrine/mysql
I think it can be done at parameters.yml but i really did not get how it can be done.
Can someone help me to solve this problem.
Thanks a lot in advanced .
I'm not exactly sure what's the problem here so I'll give you a more general answer.
Symfony has a really great way to configure your project for different situations (or environments). You should have a look at the official documentation which explains things in depth.
By default, Symfony comes with 3 configurations for different environments:
app/config/config_dev.yml for development
app/config/config_prod.yml for production
app/config/config_test.yml for (unit) testing
Each of these config files can override settings from the base configuration file which is app/config/config.yml. You would store your general/common settings there. Whenever you need to override something for a specific environment, you just go to the environment config and change it.
Lets say you have the following base configuration in app/config/config.yml:
# Doctrine Configuration
doctrine:
dbal:
driver: pdo_mysql
host: "%prod_database_host%"
port: "%prod_database_port%"
dbname: "%prod_database_name%"
user: "%prod_database_user%"
password: "%prod_database_password%"
charset: UTF8
Now lets say, you have 3 different databases for each environment - prod, dev and test. The way to do this is to override the configuration in the environment configuration file (lets say app/config/config_dev.yml:
# Doctrine Configuration
doctrine:
dbal:
driver: pdo_mysql
host: "%dev_database_host%"
port: "%dev_database_port%"
dbname: "%dev_database_name%"
user: "%dev_database_user%"
password: "%dev_database_password%"
charset: UTF8
Add the necessary %dev_*% parameters to your app/config/parameters.yml.dist and app/config/parameters.yml. Now, whenever you open your application using the dev environment, it will connect to the specified database in your parameters (%dev_database...%).
This is pretty much it. You can do the same for any configuration you need to be changed in a specific environment. You should definitely have a look at the documentation. It's explained straight-forward with examples.

Symfony2 and GoDaddy error message with incorrect information

I recently uploaded a Symfony2 project to GoDaddy and I'm having trouble accesing it because I get the message:
An exception occured in driver: SQLSTATE[HY000] [1045] Access denied for user 'root'#'127.0.0.1' (using password: NO)
Obviously the message is clear, so I checked and rechecked my parameters.yml, and the message don't even match what I have there, which I have changed several times to try to fix. This is my parameters.yml:
parameters:
database_driver: pdo_mysql
database_host: localhost
database_port: null
database_name: database1
database_user: database1user
database_password: mytestpassword
mailer_transport: smtp
mailer_host: 127.0.0.1
mailer_user: null
mailer_password: null
locale: en
secret: RandomTokenThatWillBeChanged
debug_toolbar: true
debug_redirects: false
use_assetic_controller: true
So, the error message doesn't tell me what is my real problem, or it is loading the parameters from some cached version that I haven't found yet. Any ideas of what else could cause or where could a cached version of this data be?
One of the best practice when developing a Symfony application is to
make it configurable via a parameters.yml file. It contains
information such as the database name, the mailer hostname, and custom
configuration parameters.
As those parameters can be different on your local machine, your
testing environment, your production servers, and even between
developers working on the same project, it is not recommended to store
it in the project repository. Instead, the repository should contain a
paramaters.yml.dist file with sensible defaults that can be used as a
good starting point for everyone.
Then, whenever a developer starts working on the project, the
parameters`.yml file must be created by using the parameters.yml.dist
as a template. That works quite well, but whenever a new value is
added to the template, you must remember to update the main parameter
file accordingly.
As of Symfony 2.3, the Standard Edition comes with a new bundle that
automates the tedious work. Whenever you run composer install, a
script creates the parameters.yml file if it does not exist and allows
you to customize all the values interactively. Moreover, if you use
the --no-interaction flag, it will silently fallback to the default
values.
http://symfony.com/blog/new-in-symfony-2-3-interactive-management-of-the-parameters-yml-file
So, is it not possible that your paramaters.yml is overwritten by paramaters.yml.dist?
You can also try to completely clear the cache
In Dev:
php app/console cache:clear
In Production:
php app/console cache:clear --env=prod --no-debug