Given a configuration named "Data:ConnectionString" in appsettings.json file (ASP.NET Core application), how do I override this in the build? By overriding it can either be that there is a step which changes the value in appsettings.json before compilation during build, or that I override the parameter when using "dotnet test", or something else.
More info:
I have a ASP.NET Core application with standard configuration in appsettings.json. I do not want any connection string or sensitive data checked in the source control.
I am building my application using Visual Studio Team Service (cloud TFS). There is a step where tests are executed, and I want these tests to run against a remote service for which I do not want to check in the credentials.
There are a number of extensions available on http://marketplace.visualstudio.com that will help you without any complicated ness.
https://marketplace.visualstudio.com/items?itemName=YodLabs.VariableTasks
I like the Variable Tasks Pack that comes with:
Set Variable Set a variable value and optionally apply a transformation to it.
Set Variables with Credential Sets the username/password from an existing service endpoint
Set Variable from JSON Extracts a value from JSON using JSONPath
Set Variable from XML Extracts a value from XML using XPath
Update Build Number Allows you to change a build number
Increment Version Increments a semver version number
Super easy... You can also just search for "json" or "variable" to find other options...
Most popular ways:
Use app secrets
Use scripts section in your project.json. You have 4 events -
precompile, postcompile, prepublish, postpublish
You can set the an environmental variable ASPNETCORE_ENVIRONMENT in the build to something like "Test". Create an appsettings.json file named appsettings.Test.Json. Then when you are setting up your configuration in Startup.cs do something like...
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true);
When the environmental variable is set to TEST, you new appsettings file will be loaded and can set the connection string to whatever you want.
Related
Example:
Assume that there is a string variable called "passcode" .
I have to fetch value for the variable passcode at runtime based on the environments
Environments
Value
DEV
abc123
SIT
def321
QA
S1O8F9
PROD
3a2b1c
In webMethods, Are there any alternate solutions to configure and fetch the value at each environment level apart from Global variable option?
Can configure environment variable in db or extended settings --> based on environment options DEV, SIT...
Logic can be written in service for different values or can read configuration from any properties file...
In ASP.NET Core, the JsonConfigurationProvider will load configuration from appsettings.json, and then will read in the environment version, appsettings.{Environment}.json, based on what IHostingEnvironment.EnvironmentName is. The environment version can override the values of the base appsettings.json.
Is there any reasonable way to preview what the resulting overridden configuration looks like?
Obviously, you could write unit tests that explicitly test that elements are overridden to your expectations, but that would be a very laborious workaround with upkeep for every time you change a setting. It's not a good solution if you just wanted to validate that you didn't misplace a bracket or misspell an element name.
Back in ASP.NET's web.config transforms, you could simply right-click on a transform in Visual Studio and choose "Preview Transform". There are also many other ways to preview an XSLT transform outside of Visual Studio. Even for web.config parameterization with Parameters.xml, you could at least execute Web Deploy and review the resulting web.config to make sure it came out right.
There does not seem to be any built-in way to preview appsettings.{Environment}.json's effects on the base file in Visual Studio. I haven't been able to find anything outside of VS to help with this either. JSON overriding doesn't appear to be all that commonplace, even though it is now an integral part of ASP.NET Core.
I've figured out you can achieve a preview with Json.NET's Merge function after loading the appsettings files into JObjects.
Here's a simple console app demonstrating this. Provide it the path to where your appsettings files are and it will emit previews of how they'll look in each environment.
static void Main(string[] args)
{
string targetPath = #"C:\path\to\my\app";
// Parse appsettings.json
var baseConfig = ParseAppSettings($#"{targetPath}\appsettings.json");
// Find all appsettings.{env}.json's
var regex = new Regex(#"appsettings\..+\.json");
var environmentConfigs = Directory.GetFiles(targetPath, "*.json")
.Where(path => regex.IsMatch(path));
foreach (var env in environmentConfigs)
{
// Parse appsettings.{env}.json
var transform = ParseAppSettings(env);
// Clone baseConfig since Merge is a void operation
var result = (JObject)baseConfig.DeepClone();
// Merge the two, making sure to overwrite arrays
result.Merge(transform, new JsonMergeSettings
{
MergeArrayHandling = MergeArrayHandling.Replace
});
// Write the preview to file
string dest = $#"{targetPath}\preview-{Path.GetFileName(env)}";
File.WriteAllText(dest, result.ToString());
}
}
private static JObject ParseAppSettings(string path)
=> JObject.Load(new JsonTextReader(new StreamReader(path)));
While this is no guarantee there won't be some other config source won't override these once deployed, this will at least let you validate that the interactions between these two files will be handled correctly.
There's not really a way to do that, but I think a bit about how this actually works would help you understand why.
With config transforms, there was literal file modification, so it's easy enough to "preview" that, showing the resulting file. The config system in ASP.NET Core is completely different.
It's basically just a dictionary. During startup, each registered configuration provider is run in the order it was registered. The provider reads its configuration source, whether that be a JSON file, system environment variables, command line arguments, etc. and builds key-value pairs, which are then added to the main configuration "dictionary". An "override", such as appsettings.{environment}.json, is really just another JSON provider registered after the appsettings.json provider, which obviously uses a different source (JSON file). Since it's registered after, when an existing key is encountered, its value is overwritten, as is typical for anything being added to a dictionary.
In other words, the "preview" would be completed configuration object (dictionary), which is composed of a number of different sources, not just these JSON files, and things like environment variables or command line arguments will override even the environment-specific JSON (since they're registered after that), so you still wouldn't technically know the the environment-specific JSON applied or not, because the value could be coming from another source that overrode that.
You can use the GetDebugView extension method on the IConfigurationRoot with something like
app.UseEndpoints(endpoints =>
{
if(env.IsDevelopment())
{
endpoints.MapGet("/config", ctx =>
{
var config = (Configuration as IConfigurationRoot).GetDebugView();
return ctx.Response.WriteAsync(config);
});
}
});
However, doing this can impose security risks, as it'll expose all your configuration like connection strings so you should enable this only in development.
You can refer to this article by Andrew Lock to understand how it works: https://andrewlock.net/debugging-configuration-values-in-aspnetcore/
From Miguel de Icaza:
We use a library profile that is better suited for mobile devices, so we removed features that are not necessary (like the entire System.Configuration stack, just like Silverlight does).
After years of .NET development, I'm accustomed to storing configuration settings in web.config and app.config files.
When using Mono for Android, where should I put my configuration settings?
If it matters, I'd like to store different configuration settings for different build configurations as well.
I would probably recommend using shared preferences and compilation symbols to manage different configurations. Below is an example of how you can use a preferences file to add or change keys based on the compilation symbols. Additionally, you could create a separate preferences file that is only available for a particular configuration. Because these keys are not available on all configurations, make sure to always perform checks for them before using.
var prefs = this.GetSharedPreferences("Config File Name", FileCreationMode.Private);
var editor = prefs.Edit();
#if MonoRelease
editor.PutString("MyKey", "My Release Value");
editor.PutString("ReleaseKey", "My Release Value");
#else
editor.PutString("MyKey", "My Debug Value");
editor.PutString("DebugKey", "My Debug Value");
#endif
editor.PutString("CommonKey", "Common Value");
editor.Commit();
We have had exactly the same problem in our current project.
My first impulse was to put the configuration in a sqlite key-value table but then my internal customer reminded me the main reason for a configuration file - it should support simple editing.
So instead we created an XML file and put it there:
string documentsPath = Environment.GetFolderPath(Environment.SpecialFolder.Personal);
And access it using these properties:
public string this[string key]
{
get
{
var document = XDocument.Load(ConfigurationFilePath);
var values = from n in document.Root.Elements()
where n.Name == key
select n.Value;
if(values.Any())
{
return values.First();
}
return null;
}
set
{
var document = XDocument.Load(ConfigurationFilePath);
var values = from n in document.Root.Elements()
where n.Name == key
select n;
if(values.Any())
{
values.First().Value = value;
}
else
{
document.Root.Add(new XElement(key, value));
}
document.Save(ConfigurationFilePath);
}
}
}
via a singleton class we call Configuration so for .NET developers it is very similar to using the app.config files. Might not be the most efficient solution but it gets the job done.
there's a Xamarin centric AppSetting reader: https://www.nuget.org/packages/PCLAppConfig
pretty useful for continuous delivery (so a deployment server such as octopus allows to alter your config file for each environment with values stored on the cd server)
there's a Xamarin centric AppSetting reader available at https://www.nuget.org/packages/PCLAppConfig
it is pretty useful for continuous delivery;
use as per below:
1) Add the nuget package reference to your pcl and platforms projects.
2) Add a app.config file on your PCL project, then as a linked file on all your platform projects. For android, make sure to set the build action to 'AndroidAsset', for UWP set the build action to 'Content'. Add you settings keys/values: <add key="config.text" value="hello from app.settings!" />
3) Initialize the ConfigurationManager.AppSettings on each of your platform project, just after the 'Xamarin.Forms.Forms.Init' statement, that's on AppDelegate in iOS, MainActivity.cs in Android, App in UWP/Windows 8.1/WP 8.1:
ConfigurationManager.Initialise(PCLAppConfig.FileSystemStream.PortableStream.Current);
3) Read your settings : ConfigurationManager.AppSettings["config.text"];
ITNOA
Maybe PCLAppConfig is help you to create and read from app.config in Xamarin.Forms PCL Project or other Xamarin projects.
For different configuration in different build mode such as release and debug you can use Configuration Transform on app.config.
I am using the EntLib in an environment where database connection strings are retrieved from a separate library call that decrypts a proprietary config file. I have no say over this practice or the format of the config file.
I want to do EntLib exception logging to the database in this setting. I therefore need to set up a EntLib database configuration instance with the name of the database, with the connection string. Since I can't get the connection string until run time, but EntLib does allow run-time configuration, I use the following code, as described in this:
builder.ConfigureData()
.ForDatabaseNamed("Ann")
.ThatIs.ASqlDatabase()
.WithConnectionString(connectionString)
.AsDefault();
The parameter connectionString is the one I've retrieved from the separate library.
The sample code goes on to merge the created configuration info with an empty DictionaryConfigurationSource. I, however, need to merge it with the rest of the configuration code from the app.config. So I do this:
var configSource = new SystemConfigurationSource();
builder.UpdateConfigurationWithReplace(configSource);
EnterpriseLibraryContainer.Current
= EnterpriseLibraryContainer.CreateDefaultContainer(configSource);
... which is based very closely on the sample code.
But: I get an internal error in Microsoft.Practices.EnterpriseLibrary.Common.Configuration.SystemConfigurationSource.Save. The failing code is this:
var fileMap = new ExeConfigurationFileMap { ExeConfigFilename = ConfigurationFilePath };
var config = ConfigurationManager.OpenMappedExeConfiguration(fileMap, ConfigurationUserLevel.None);
config.Sections.Remove(section);
config.Sections.Add(section, configurationSection);
config.Save();
... where 'section' is "connectionStrings". The code fails on the Add method call, saying that you can't add a duplicate section. Inspection shows that the connectionStrings section is still there even after the Remove.
I know from experience that there's always a default entry under connectionStrings when the configuration files are actually read and interpreted, inherited from the machine.config. So perhaps you can never really remove the connectionStrings section.
That would appear to leave me out of luck, though, unless I want to modify the EntLib source, which I do not.
I could perhaps build all the configuration information for the EntLib at run time, using the fluent API. But I'd rather not. The users want their Operations staff to be able to make small changes to the logging without having to involve a developer.
So my question, in several parts: is there a nice simple workaround for this? Does it require a change to the EntLib source? Or have I missed something really simple that would do away with the problem?
I found a workaround, thanks to this post. Rather than taking the system configuration source and attempting to update it from the builder, I copy the sections I set up in app.config into the builder, and then do an UpdateConfigurationWithReplace on an empty dummy configuration source object in order to create a ConfigurationSource that can be used to create the default container.
var builder = new ConfigurationSourceBuilder();
var configSource = new SystemConfigurationSource();
CopyConfigSettings("loggingConfiguration", builder, configSource);
CopyConfigSettings("exceptionHandling", builder, configSource);
// Manually configure the database settings
builder.ConfigureData()
.ForDatabaseNamed("Ann")
.ThatIs.ASqlDatabase()
.WithConnectionString(connectionString)
.AsDefault();
// Update a dummy, empty ConfigSource object with the settings we have built up.
// Remember, this is a config settings object for the EntLib, not for the entire program.
// So it doesn't need all 24 sections or however many you can set in the app.config.
DictionaryConfigurationSource dummySource = new DictionaryConfigurationSource();
builder.UpdateConfigurationWithReplace(dummySource);
// Create the default container using our new ConfigurationSource object.
EnterpriseLibraryContainer.Current
= EnterpriseLibraryContainer.CreateDefaultContainer(dummySource);
The key is this subroutine:
/// <summary>
/// Copies a configuration section from the SystemConfigurationSource to the ConfigurationSourceBuilder.
/// </summary>
/// <param name="sectionName"></param>
/// <param name="builder"></param>
/// <param name="configSource"></param>
private static void CopyConfigSettings(string sectionName, ConfigurationSourceBuilder builder, SystemConfigurationSource configSource)
{
ConfigurationSection section = configSource.GetSection(sectionName);
builder.AddSection(sectionName, section);
}
PROBLEM
Let's say I have a jenkins/hudson job (for example free-style) that takes two parameters PARAM_ONE and PARAM_TWO. Now, I do not know the values of those parameters, but I can run some script (perl/shell) to find values of those parameters and then I want the user to select from a dropdown list after which I can start the build.
Is there any way of doing that?
Sounds like you've found a plug-in that does what you need, that is pretty similar to the built-in Parameterized Builds functionality.
To answer your second question: when you define parameterized builds, the parameters are typically passed to your job as environment variables. So you'd access them however you access environment variables in your language, for instance, if you defined a parameter PARAM_ONE, you'd access it as:
In bash:
$PARAM_ONE
In Windows batch:
%PARAM_ONE%
In Python:
import os
os.getenv('PARAM_ONE')
etc.
I imagine this would be the same for the Extended Choice Parameter plugin you are using.
Just install this, and give the parameter in the build script like:
Windows
"your build script" %PARAMONE% %PARAMTWO%
In Java, you can access these parameters off the run object
EnvVars envVars = new EnvVars();
envVars = run.getEnvironment(listener);
for (String envName2 : envVars.keySet()) {
listener.getLogger().println(envName2 + " = " + envVars.get(envName2));
}