We develop many different web applications that end up in the same public site and use sitecore as cms which merges the different config files to a single web config. Unfortunately there's often duplicate config in the sitecore settings.
Currently: it just joins the config files leaving only the last definition of every duplicate key. Thus updating one of the config files doesn't reflect a change in the merged file, if there's another config file with that setting getting processed later.
Expected: prevent defining 1 config in multiple config files. If there's a duplicate key among multiple config files, it shouldn't silently overwrite but throw an error, so that we can deduplicate our config.
Does anyone know how we could to prevent this? For example a way to override the pipeline that does the merge, to never overwrite anything (or at least when it does it should warn us about it)
EDIT 1:
To illustrate how evil the config-join job is, here are 2 example config files:
aaa.config:
<configuration><sitecore>
<settings>
<setting name="key0" value "a-value">
<setting name="key1" value "a-value">
</settings>
</sitecore></configuration>
zzz.config:
<configuration><sitecore>
<settings>
<setting name="key1" value "z-value">
<setting name="key2" value "z-value">
</settings>
</sitecore></configuration>
gets joined to:
<configuration><sitecore>
<settings>
<setting name="key0" value "a-value">
<setting name="key1" value "z-value">
<setting name="key2" value "z-value">
</settings>
</sitecore></configuration>
but instead of silently joining the files, I'd love it to throw an exception saying "key1" is ambiguous!
There are no settings available OOTB to prevent this from happening, and it is the expected and intended behaviour in Sitecore. You can read more about how config patching works in this document or this blog post.
It may be possible to provide your own implementation of Sitecore.Configuration.ConfigReader to prevent this BUT Sitecore uses config patches itself even in a default install. Preventing this behaviour is likely to cause Sitecore to break (and/or cause other issues). If you do go down this route then make sure you only target a specific custom folder.
The solution a colleague just suggested: a very strict naming policy, for example prefixing every config key with the project name and country: "Docs_NZ_UploadMaxSize", "Loan_NL_ProductsPath", etc... Which would thus also prevent duplicate keys across projects and/or countries.
Downside: renaming/refactoring in existing projects.
#jammykam's answer sounds more fun though. Anyway, I'll raise both possible solutions after our standup to have a vote which route to take (thus what answer to accept).
Related
I thought ivy:info task would do the job by...
<ivy:info file="${local_ivy_file}"/>
<ivy:info organisation="${ivy.organisation}" module="${ivy.module}" revision="${ivy.revision}" property="check"/>
But even when the artifact does not exist on the repo, the properties of "check" (check.organisation, check.module, check.revision) always returns the same values as that of ivy.organisation, etc... when the expected values of "check" properties should be null (since they don't exist on repo)
Am I missing something?
Why this is needed:
I have a large set of third party jars to upload, but i do not have admin rights to overwrite (and should not overwrite) if file exists.
Doing
<ivy:publish overwrite="false"/>
will break the entire process whereas I need the task to continue cycle through the whole set of jars/ivy.xml.
Thanks
i found my problem. my original method does work, except that i have to remove the ivy cache before running the info task, since ivy looks in the cache before looking in the repo.
<delete dir="${ivy_cache_path}"/>
<ivy:info file="${local_ivy_file}"/>
<ivy:info organisation="${ivy.organisation}" module="${ivy.module}" revision="${ivy.revision}" property="check"/>
now ${check.revision} correctly captures the revision of the artifact in the source ivy file if the artifact exists in the online repo, and has no value if it does not exist. I can use this as a condition for the subsequent tasks.
I want to reexecute some targets when the configuration changes.
Consider this example:
I have a configuration variable (that is either read from environment variables or a config.local file):
CONF:=...
Based on this variable CONF, I assemble a header file conf.hpp like this:
conf.hpp:
buildConfHeader $(CONF)
Now, of course, I want to rebuild this header if the configuration variable changes, because otherwise the header would not reflect the new configuration. But how can I track this with make? The configuration variable is not tied to a file, as it may be read from environment variables.
Is there any way to achieve this?
I have figured it out. Hopefully this will help anyone having the same problem:
I build a file name from the configuration itself, so if we have
CONF:=a b c d e
then I create a configuration identifier by replacing the spaces with underscores, i.e.,
null:=
space:= $(null) #
CONFID:= $(subst $(space),_,$(strip $(CONF))).conf
which will result in CONFID=a_b_c_d_e.conf
Now, I use this $(CONFID) as dependency for the conf.hpp target. In addition, I add a rule for $(CONFID) to delete old .conf files and create a new one:
$(CONFID):
rm -f *.conf #remove old .conf files, -f so no error when no .conf files are found
touch $(CONFID) #create a new file with the right name
conf.hpp: $(CONFID)
buildConfHeader $(CONF)
Now everything works fine. The file with name $(CONFID) tracks the configuration used to build the current conf.hpp. If the configuration changes, then $(CONFID) will point to a non-existant .conf file. Thus, the first rule will be executed, the old conf will be deleted and a new one will be created. The header will be updated. Exactly what I want :)
There is no way for make to know what to rebuild if the configuration changed via a macro or environment variable.
You can, however, use a target that simply updates the timestamp of conf.hpp, which will force it to always be rebuilt:
conf.hpp: confupdate
buildConfHeader $(CONF)
confupdate:
#touch conf.hpp
However, as I said, conf.hpp will always be built, meaning any targets that depend upon it will need rebuilt as well. A much more friendly solution is to generate the makefile itself. CMake or the GNU Autotools are good for this, except you sacrifice a lot of control over the makefile. You could also use a build script that creates the makefile, but I'd advise against this since there exist tools that will allow you to build one much more easily.
The matbpotlib documentation describes the order in which "matplotlib looks for matplotlibrc" files, but it is unclear whether the first matplotlibrc file encountered completely determines the rc settings, or whether only those values set in a given file are used.
Does the first matplotlibrc file encountered completely determine matplotlib's configuration, or are settings absent from a given file sought in files later in the search order?
Yes, it only reads the first file it hits. The rest of the values are filled in from the defaults. The relevant functions are in __init__.py.
There is a PR (which will hopefully be merged to master soon) which allows you to load 'style sheet' like rcparam files to set just some of the parameters.
I renamed a folder and updated my namespace declarations, but ReSharper 6 claims that the namespace should be a reflection of how it was before the rename. Where is it storing the file location data?
Check to make sure your assembly name matches your new namespace. If you've changed your folder structure to match your new namespace, you may still see the ReSharper alert until you update the project properties.
As delliottg's comment says, in Visual Studio, go to
Project > [project name] Properties > Application
and change "Assembly name" as well as "Default namespace".
I also had this problem with a folder/namespace and none of the above steps fixed it.
In my case I had to do this in Visual Studio:
Right-click the "problem" folder in the solution explorer to open the properties
Ensure the "Namespace Provider" is set to true
This fixed the ReSharper issue for me and I was able to adjust namespaces as normal.
Root namespace is needed to be changed as following.
I use Resharper 2019.3.2 in VS 2019 vs 16.5.2 and I had similar issues.
When developing, I first work out my namespace hierarchy in a single project, then split the project in seperate class libraries. In the first stage, it is convenient to always let the subdirectory correspond to the namespace.
For example, my prototype MeshTools.dll project currently contains:
Meshtools ........................ 3 cs-files in \MeshTools
MeshTools.HeightField .......... 2 cs-files in \MeshTools\HeightField
MeshTools.VectorTools .......... 3 cs-files in \MeshTools\VectorTools
The above answers all assume one single namespace per project. Renaming directories manually may confuse Resharper and that can be repaired by setting the default assembly in the .csproj file to the proper namespace. Thanks for the tip.
However in my case, I have several namespaces in a single project, with each namespace in a Solution directory corresponding to a real directory. Setting the default assembly "Meshtools" does not affect ReSharper behaviour for HeightField and VectorTools, when things have gone wrong with the renaming.
I googled this issue and came by https://www.jetbrains.com/help/resharper/Refactorings__Adjust_Namespaces.html#
It turns out there is a right-click option on a Solution Directory -> Properties. You will find an option to decide, if the Solution Directory is a NameSpace provider or not. When something has gone wrong, Visual studio will reset the field to False. Set it back to True and Resharper will correctly refactor namespace or file location when needed..
If you're using JetBrains Rider, go to the Solution Explorer and right click on the csproj file, then properties in the context menu. In my case the Assembly Name was already updated but "Root Namespace" wasn't, updating Root Namespace allowed JetBrains to automatically update all namespaces.
I need to distribute some sort of static configuration through my application. What is the best practice to do that?
I see three options:
Call application:get_env directly whenever a module requires to get configuration value.
Plus: simpler than other options.
Minus: how to test such modules without bringing the whole application thing up?
Minus: how to start certain module with different configuration (if required)?
Pass the configuration (retrieved from application:get_env), to application modules during start-up.
Plus: modules are easier to test, you can start them with different configuration.
Minus: lot of boilerplate code. Changing the configuration format requires fixing several places.
Hold the configuration inside separate configuration process.
Plus: more-or-less type-safe approch. Easier to track where certain parameter is used and change those places.
Minus: need to bring up configuration process before running the modules.
Minus: how to start certain module with different configuration (if required)?
Another approach is to transform your configuration data into an Erlang source module that makes the configuration data available through exports. Then you can change the configuration at any time in a running system by simply loading a new version of the configuration module.
For static configuration in my own projects, I like option (1). I'll show you the steps I take to access a configuration parameter called max_widgets in an application called factory.
First, we'll create a module called factory_env which contains the following:
-define(APPLICATION, factory).
get_env(Key, Default) ->
case application:get_env(?APPLICATION, Key) of
{ok, Value} -> Value;
undefined -> Default
end.
set_env(Key, Value) ->
application:set_env(?APPLICATION, Key, Value).
Next, in a module that needs to read max_widgets we'll define a macro like the following:
-define(MAX_WIDGETS, factory_env:get_env(max_widgets, 1000)).
There are a few nice things about this approach:
Because we used application:set_env/3 and application:get_env/2, we don't actually need to start the factory application in order to have our tests pass.
max_widgets gets a default value, so our code will still work even if the parameter isn't defined.
A second module could use a different default value for max_widgets.
Finally, when we are ready to deploy, we'll put a sys.config file in our priv directory and load it with -config priv/sys.config during startup. This allows us to change configuration parameters on a per-node basis if desired. This cleanly separates configuration from code - e.g. we don't need to make another commit in order to change max_widgets to 500.
You could use a process (a gen_server maybe?) to store your configuration parameters in its state. It should expose a get/set interface. If a value hasn't been explicitly set, it should retrieve a default value.
-export([get/1, set/2]).
...
get(Param) ->
gen_server:call(?MODULE, {get, Param}).
...
handle_call({get, Param}, _From, State) ->
case lookup(Param, State#state.params) of
undefined ->
application:get_env(...);
Value ->
{ok, Value}
end.
...
You could then easily mockup this module in your tests. It will also be easy to update the process with some new configuration at run-time.
You could use pattern matching and tuples to associate different configuration parameters to different modules:
set({ModuleName, ParamName}, Value) ->
...
get({ModuleName, ParamName}) ->
...
Put the process under a supervision tree, so it's started before all the other processes which are going to need the configuration.
Oh, I'm glad nobody suggested parametrized modules so far :)
I'd do option 1 for static configuration. You can always test by setting options via application:set_env/3,4. The reason you want to do this is that your tests of the application will need to run the whole application anyway at some time. And the ability to set test-specific configuration at that point is really neat.
The application controller runs by default, so it is not a problem that you need to go the application-way (you need to do that anyway too!)
Finally, if a process needs specific configuration, say so in the configuration data! You can store any Erlang-term, in particular, you can store a term which makes you able to override configuration parameters for a specific node.
For dynamic configuration, you are probably better off by using a gen_server or using the newest gproc features that lets you store such dynamic configuration.
I've also seen people use a .hrl (erlang header file) where all the configuration is defined and include it at the start of any file that needs configuration.
It makes for very concise configuration lookups, and you get configuration of arbitrary complexity.
I believe you can also reload configuration at runtime by performing hot code reloading of the module. The disadvantage is that if you use configuration in several modules and reload only one of them, only that one module will get its configuration updated.
However, I haven't actually checked if it works like that, and I couldn't find definitive documentation on how .hrl and hot code reloading interact, so make sure to double-check this before you actually use it.