Externalizing Grails Datasource configuration - configuration

Grails 1.x allows using external configuration files by setting the grails.config.locations directive. Is there a similar approach available for externalizing the database configuration in Datasource.groovy (without setting up JNDI)?
It would prove helpful to be able to configure DB credentials in a simple configuration file outside the application.
Thanks in advance!

You can use a properties file specified in the grails.config.locations as a way to externalize the datasource configuration. Below is how I typically set up a Grails project:
In my DataSource.groovy I specify this for the production environment:
....
....
production {
dataSource {
dbCreate = "update"
driverClassName = "com.myorg.jdbcDriverNotExists"
url = ""
username = ""
password = ""
}
}
....
....
I specify an external properties file in my Config.groovy:
grails.config.locations = [ "classpath:app-config.properties"]
In the properties file (stored in grails-app/conf/) I specify the actual datasource info:
dataSource.driverClassName=oracle.jdbc.OracleDriver
dataSource.url=jdbc:oracle:thin:#host:port:sid
dataSource.username=sa
dataSource.password=secret
I also use the properties file as a way to override other values that are in Config.groovy. When the app is deployed, if I have to modify the datasource info I just edit the /WEB-INF/classes/app-config.properties file and restart.

The answer above does not really externalize configuration. He is close, but the configuration is still residing in the application.
I would use a JVM environment var on startup of the application/server to point to a location outside the application where the external configuration resides. Read out the environment var in the config.groovy file and use it get the external configuration file. Something like this:
def extConfig = System.properties.getProperty('ENVVAR');
grails.config.locations = [ "file:${extConfig}/${appName}-config.groovy"]

For me this doesn't work. To get an environment variable.
Better use :
System.getenv().get("ENVVAR").toString()

Just put the configuration file location as following in Config.groovy file
grails.config.locations = [
"file:/yourDirectory/${appName}/${Environment.current.name}-datasource.properties",
"file:/yourDirectory/${appName}/${Environment.current.name}-config.groovy",
"classpath:${appName}-${Environment.current.name}-datasource.properties",
"classpath:${appName}-${Environment.current.name}-config.groovy"
]
And put all the details about datasource and other config values in your appropriate file. Hence you can externalize the configuration and need not restart to change values.

Related

nestjs configuration with dotenv

Referring to official NestJS documentation, it is recommended to use ConfigService in order to use environment variables.
So in the code, we access all vars defined in an .env file with something like:
config.get('PORT')
But it is not recommended to use .env in production environment. So how to deploy in that way?
Why not just use the standard method with dotenv and process.env.PORT?
There are two problems that make the ConfigService less useful.
First
When no .env file is present in any environment, readFileSync in
dotenv.parse(fs.readFileSync(filePath))
will fail:
[Nest] 63403 [ExceptionHandler] path must be a string or Buffer
TypeError: path must be a string or Buffer
at Object.fs.openSync (fs.js:646:18)
at Object.fs.readFileSync (fs.js:551:33)
at new ConfigService (../config/config.service.ts:8:38)
Even if e.g. process.env.API_KEY is available
this.configService.get('API_KEY')
will not return anything. So the ConfigService forces you to use a prod.env file, which dotenv advocates against:
No. We strongly recommend against having a "main" .env file and an
"environment" .env file like .env.test. Your config should vary
between deploys, and you should not be sharing values between
environments.
https://github.com/motdotla/dotenv#should-i-have-multiple-env-files
Second
You have to import the config module and inject the service in order to use it. When you use env variables like this
imports: [
MongooseModule.forRoot(process.env.MONGO_URI, { useNewUrlParser: true }),
ConfigModule,
],
the config service is useless.
Read more about config in the environment here: https://12factor.net/config
But this is not recommended to use .env in production environnement. So how to deploy that way ?
Actually, it is not recommended to commit your .env files. It's perfectly fine to use them in production :-).
Why not use the standard method with dotenv and process.env.PORT?
It allows decoupling your core code from the code responsible for providing configuration data. Thus:
The core code is easier to test: doing some manual changes/mocking of process.env is such - a - pain, whereas mocking a "ConfigService" is pretty easy
You can imagine using anything else than environment variables in the future by just replacing a single method (or a few getters) in a dedicated class, instead of replacing all the occurrences of process.env.* in your code // to be fair, this is unlikely to happen, as using env. variables is the most common way to load configuration data, but still.
Using #nestjs/config (a.k.a. ConfigModule) makes environment variables available to your app whether they come from a .env file or set in the environment. Locally you use a .env file and on production use the environment.

Node.JS/Express store DB config for different environment in JSON

I'm running NodeJS Express app,
Currently have dev env, test env, and prod env.
However, the DB connection settings are in the code, is there a secure and best practice way to store DB config and all other configs in JSON file format by declaring them in a module (separately for each env or all in one module to be exported, maybe have a default.JSON, Dev.JSON, Prod.JSON...etc) for each environment then require them accordingly by setting the correct configuration for the correct environment in app.js.
I would like to achieve this without depending on any 3rd party package like .env or ncof.
Most of the main NodeJs hosting providers uses a simple environment variable. You can use this :
process.env.NODE_ENV
For defining it by yourself, for exemple 'development' on your local, you can do :
NODE_ENV=developpment node yourapp.js
With this, I suggest you to use a config tool, like nconf (there are some good competitors). You can do like this for example :
nconf
.argv() // Takes arguments from CLI
.file('./env.' + process.env.NODE_ENV + '.json') // takes from specific env file
.file('package', './package.json'); // takes from package.json
Here priority is from the most important to the least :
1) argv
2) specific environment file
3) package.json
You can require file based on the environment.
const env = 'test'; // This value can be taken from config or .env
const configs = require(`../path/${env}`);
console.log('DB Config', configs.DB_PATH);
Depending on your environment you can load the file. And value for environment can be retrieved from .env or any other config.

Magnolia CMS: reference YAML config from JCR config

I have my page configuration done via JCR configuration.
I have the component configuration using YAML configuration.
I want to make this component available to the a template configured in the JCR.
The component config is under: /project-website-module/src/main/resources/website-module/components/linkList/linkList.yaml
I tried to reference this in the template's component availability in different ways:
website-module:components/linkList/linkList
website-module:components/linkList/linkList.yaml
/website-module/components/linkList/linkList
/website-module/components/linkList/linkList.yaml
src/main/resources/website-module/components/linkList/linkList
src/main/resources/website-module/components/linkList/linkList.yaml
But no luck, I keep getting the error:
"Caused by: info.magnolia.config.registry.Registry$NoSuchDefinitionException: <pathToComponentConfiguration>"
The component config is under:
/project-website-module/src/main/resources/website-module/dialogs/linkList/linkList.yaml
... that's the dialog config, where's the component config yaml?
Path to the component config not to dialog config is the one that you need to use when referring to the component. And that config file needs to be physically at src/main/resources/website-module/templates/components/... and reference is then website-module:components/....
Also if you have specified module descriptor for the module, name in the module descriptor better match the website-module.
I had to move the YAML component configuration under (notice the templates directory added):
/project-website-module/src/main/resources/website-module/templates
In my case, move linkList.yaml under:
/project-website-module/src/main/resources/website-module/templates/components/linkList/linkList.yaml
Then in the JCR config, use the following path:
website-module:components/linkList/linkList
Note: likewise, YAML dialog configurations must be under:
/project-website-module/src/main/resources/website-module/dialogs

How to set aspnetcore_environment in publish file?

I have ASP.NET Core application (Web Api). The documentation has explained working with multiple environments, however it failed to explain how to set aspnetcore_environment when publishing the web site.
So lets say if i have 3 environments Development, Staging and Production
In classic ASP.NET Web Application i used to create 3 build configurations. Development, Staging and Production ( Like shown in picture below). and then 3 .pubxml files, one for each configuration. Do i need to use the same approach for ASP.NET Core application as well?
How do i set aspnetcore_environment in .pubxml file?
If the approach specified in Question 1 is obsolete, then what's the alternate approach? ( I use Jenkins for CI)
Update 1
I understand that I have to set ASPNETCORE_ENVIRONMENT however I am not able to understand where do we set this? During development I can set this in profile in launchSettings.json, however question was how do we set this when publishing to staging or production? do we set environment variable on the target server itself?
Update 2
I found article here that explains different ways of setting environment variable. This partially answered my question. However when I publish the application, the publish process does not honor the environment variable while publishing appsettings.{env.EnvironmentName}.json
I have created separate post for that question
You could pass in the desired ASPNETCORE_ENVIRONMENT into the dotnet publish command as an argument using:
/p:EnvironmentName=Staging
e.g.
dotnet publish /p:Configuration=Release /p:EnvironmentName=Staging
This will generate out the web.config with the correct environment specified for your project:
<environmentVariables>
<environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Staging" />
</environmentVariables>
I had the same requirement, and I came up with the following solutions. This works well with automated deployments and require fewer configuration changes.
1. Modifying the project file (.CsProj) file
MSBuild supports the EnvironmentName Property which can help to set the right environment variable as per the Environment you wish to Deploy. The environment name would be added in the web.config during the Publish phase.
Simply open the project file (*.csProj) and add the following XML.
<!-- Custom Property Group added to add the Environment name during publish
The EnvironmentName property is used during the publish for the Environment variable in web.config
-->
<PropertyGroup Condition=" '$(Configuration)' == '' Or '$(Configuration)' == 'Debug'">
<EnvironmentName>Development</EnvironmentName>
</PropertyGroup>
<PropertyGroup Condition=" '$(Configuration)' != '' AND '$(Configuration)' != 'Debug' ">
<EnvironmentName>'$(Configuration)'</EnvironmentName>
</PropertyGroup>
Above code would add the environment name as Development for empty or Debug configuration. For any other Configuration the Environment name would be picked from the configuration which was selected. This will add the ASPNETCORE_ENVIRONMENT environment with the desired configuration. You can modify the logic for environment name as desired by updating the CsProj file. More details here
2. Adding the EnvironmentName Property in the publish profiles.
We can add the <EnvironmentName> property in the publish profile as well. Open the publish profile file which is located at the Properties/PublishProfiles/{profilename.pubxml} This will set the Environment name in web.config when the project is published. More Details here
<PropertyGroup>
<EnvironmentName>Development</EnvironmentName>
</PropertyGroup>
As shown in above image, environment can be added for each configuration and the name of the EnvironmentName property can be changed in each *.pubxml file.
3. Command line options using dotnet publish
Additionaly, we can pass the property EnvironmentName as a command line option to the dotnet publish command. Following command would include the environment variable as Development in the web.config file.
dotnet publish -c Debug -r win-x64 /p:EnvironmentName=Development
When hosting the application under IIS you can set the environment variable in web.config.
https://learn.microsoft.com/en-us/aspnet/core/hosting/aspnet-core-module
To generate it on publish add a web.config to the root of your project, "dotnet publish" will use this file as the basis for the one that is generated for in the publish folder. Then you can change the value in your deployment system.
<?xml version="1.0" encoding="utf-8" ?>
<!-- Used to overwrite settings web.config generated by "dotnet publish", Only used when hosting under IIS -->
<configuration>
<system.webServer>
<aspNetCore stdoutLogEnabled="true">
<environmentVariables>
<environmentVariable name="ASPNETCORE_ENVIRONMENT" value="Development" />
</environmentVariables>
</aspNetCore>
</system.webServer>
</configuration>
I think you can't do it in the publish profile. You have to set environment variable, e.g. ASPNETCORE_ENVIRONMENT = Staging.
I had to do a similar thing with a aspnet core web app on Azure. I wanted to have dev, staging and production. The way I did it was exactly with env variable.
To setup two or more profiles, you need to create additional profile, as mentioned in a linked article, and your launchSettings.json will contain an array:
"profiles": {
"IIS Express": {
"commandName": "IISExpress",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Development"
}
},
"IIS Express (Staging)": {
"commandName": "IISExpress",
"launchBrowser": true,
"environmentVariables": {
"ASPNETCORE_ENVIRONMENT": "Staging"
}
}
}
To be able to read the environment variable, you need to specify it during startup and call additional method AddEnvironmentVariables to variables take action:
public class Startup
{
public Startup(IHostingEnvironment env)
{
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
// general properties
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
// specify the environment-based properties
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
// do not forget to add environment variables to your config!
.AddEnvironmentVariables();
Configuration = builder.Build();
}
}
Simple way to set it in visual studio IDE.
Project > Properties> Debug > Environment variables
Please do not use environment variables of machine level instead scope
to the application , there is a possibility of other application doing
same, changing may affect other application.

Message Driven bean external configuration for JBoss with IBM MQ

I am working on a Notification Service using IBM MQ messaging provider with JBoss eap 6.1 environment. I am successfully able to send messages via MQ JCA provider rar i.e. wmq.jmsra.rar file. However on consumer part my current configuration looks like this
#MessageDriven(
activationConfig = {
#ActivationConfigProperty(propertyName="destinationType", propertyValue="javax.jms.Queue"),
#ActivationConfigProperty(propertyName="destination", propertyValue="F2.QUEUE"),
#ActivationConfigProperty(propertyName="providerAdapterJNDI", propertyValue="java:jboss/jms/TopicFactory"),
#ActivationConfigProperty(propertyName="queueManager", propertyValue="TOPIC.MANAGER"),
#ActivationConfigProperty(propertyName="hostName", propertyValue="10.239.217.242"),
#ActivationConfigProperty(propertyName="userName", propertyValue="root"),
#ActivationConfigProperty(propertyName = "channel", propertyValue = "TOPIC.CHANNEL"),
#ActivationConfigProperty(propertyName = "port", propertyValue = "1422")
})
My problem is that consumer of this service does not want to add any port numbers, hostName, queueManager properties in these beans. Also they do not want to use ejb-jar.xml to externalize these configs. I have researched and found that we can add a domain IBM Message Driven Bean but with no success. Any suggestions on what I can do here to externalize all these configurations ?
EDIT: Adding --> The JCA resource adapter is deployed at consumer end if it makes it any easier.
Thanks
You can actually externalize an MDBs activation spec properties to the server configuration file.
Create the ejb-jar.xml file, but do not put the actual value in the file, use a property placeholder:
<activation-config-property>
<activation-config-property-name>hostName</activation-config-property-name>
<activation-config-property-value>${wmq.host}</activation-config-property-value>
</activation-config-property>
Do this for all of the desired properties.
Ensure that property replacement for Java EE spec files (ejb-jar.xml, in this case) is enabled in the server configuration file:
<subsystem xmlns="urn:jboss:domain:ee:1.2">
<spec-descriptor-property-replacement>true</spec-descriptor-property-replacement>
Then, in the server configuration file, provide values for your properties:
<system-properties>
<property name="wmq.host" value="10.0.0.150"/>
Once your MDBs are packaged, you will not need to change any of the files in the MDB jar - just provide the properties in the server configuration.
you can avoid to add host name, port number and so on in MDB, you just want to define destinationType in MDB, and rest of the thing u can configure in your application server, like Activation Specification, Queues and Queue Connection Factories.
I have done the same thing but i used IBM Websphere Application Server.