It is possible for System administrators to define certain properties as final, so that client daemons may not override them.
However properties set using System.setProperty() method take preference over those set using Hadoop's configuration API, in that case is it not possible for a client daemon to override a final property by setting it by calling System.setProperty()?
Thanks.
Note that while configuration properties can be defined in terms of system properties,
unless system properties are redefined using configuration properties, they are not accessible
through the configuration API
Internally, the hadoop system will access the configuration properties only through the Configuration interface. In case the user was unable to modify a configuration property through the Configuration interface (and he will be unable if the property is defined as final by the admin) the system will not see the modification the user made in the System's properties
Related
We are upgrading to solr 4.4 and would like to be able to use the new core.properties for the new core discovery.
Currently, we have a couple of properties defined in the solr.xml for most of the cores, and I am not able to create these same properties in the new core.properties file.
Here is a core as currently defined in solr.xml:
<core name="core1003" instanceDir="core1003">
<property name="xmlDataDir" value="D:\Solrfiles\ImportFiles\core1003.xml" />
</core>
This is then used in DIH-XPathEntityProcessor.xml with <str name="xmlDataDir">${xmlDataDir}</str>.
How can I define core specific property variables like this? It doesn't necessarily have to be in core.properties, but that would make it a little easier to manage.
In your core instance directories that contain your core.properties file create a conf/ subdirectory and a solrcore.properties file in it. Define your core specific properties there and it will be picked up automatically.
Alternatively, you can add a value for "properties" to your core.properties file to point to any other file:
properties=/your/path/here.properties
I believe but have not confirmed that for this to work your cores must load on startup and must not be transient. Any non-standard properties you add to core.properties will be ignored because the CoreDescriptor copy constructor only persists the following properties from that file in memory: instanceDir, config, schema, name and dataDir (see CoreDescriptor:91 in the 4.4.0 source). I believe this is a bug?
I need to get all configuration settings (current role or all roles not matter) from Azure cscfg file. I want to do this because i dont want to get all values one by one via RoleEnvironment.GetConfigurationSettingValue(key) method.
Is there any way to do this?
Regards
The short answer is 'no' the RoleEnvironment does not support getting all the configuration setting values.
A slightly longer answer is that getting configuration settings from the role environment in the current implementation is done through a call to native code. The separation of Windows Azure Application from Windows Azure Configuration and the ability to swap settings on a running application is at the root of this somehow. This is done inside of msshrtmi.dll (which should mean something like Microsoft Shared Runtime Managed Interop). This is the only reference Microsoft.WindowsAzure.ServiceRuntime.dll has apart from standard references to .NET.
Here is the method call to native code (I have not gone further than this):
[MethodImpl(MethodImplOptions.Unmanaged, MethodCodeType=MethodCodeType.Native), SuppressUnmanagedCodeSecurity, DllImport("", EntryPoint="", CallingConvention=CallingConvention.StdCall, SetLastError=true)]
internal static extern unsafe int modopt(IsLong) modopt(CallConvStdcall) RdGetApplicationConfigurationSetting(ushort modopt(IsConst), ushort*);
It might seem like a slightly round-about way of doing it, but if you want to get the configurations for all the roles in a deployment you can use the management api.
Kudu has an API for this.
You get to Kudu like via App Services > Advanced Details > Go
https://{app-service-name}.scm.azurewebsites.net/
or
https://{app-service-name}-{slot-name}.scm.azurewebsites.net/
The Url for the settings API is:
https://{app-service-name}.azurewebsites.net/api/settings or https://{app-service-name}-{slot-name}.scm.azurewebsites.net/
Packaging a log4j configuration file in a NetBeans Platform application apparently requires some thinking through. This is what I tried...
I put log4j.xml in src/main/resources/my/package/log4j.xml of some_netbeans_module. The package is a public module package (i.e. classes from this package are used from other packages). I rebuilt the module and confirmed that the file does, in fact, get packaged into the module.
In my classes I get an instance of the logger the way I always do:
static final Logger log = Logger.getLogger(ThisClass.class);
Every NetBeans Platform application has a my_app.conf file which makes it possible to set certain properties. This is where I set log4j.conf:
log4j.configuration="/my/package/log4j.xml"
Now, when I run the application, I see the following output:
[INFO] /home/me/my_app/application/target/my_app/bin/../etc/my_app.conf: 5:
log4j.configuration=/my/package/log4j.xml: not found
What is wrong with the above configuration?
In the my_app.conf file if you append the log4j.configuration property to the default_options property, like so:
default_options="...<other options> -J-Dlog4j.configuration=my/package/log4j.xml"
then this option will get passed to the JVM. Notice that the log4j property has -J-D appended to it. The -J is used by NetBeans to delineate JVM properties and the -D is used by the JVM to delineate a system property.
Also you can/should drop the quotes and the initial / as the quotes are not necessary and NetBeans will complain if you have the initial /
The other way to do this, and the way that I prefer since it doesn't require editing the .conf file, is to put the log4j.xml file into the default package. If you have other requirements that prevents you from doing this then remember that you must put the log4j.configuration property in the app's platform.properties file while your in dev mode and running the app inside of the IDE. Like so:
run.args.extra=-J-Dlog4j.configuration=my/package/log4j.xml
Edit: For questions regarding NetBeans Platform you might have better luck posting to the NetBeans Platform Users forum.
I need to distribute some sort of static configuration through my application. What is the best practice to do that?
I see three options:
Call application:get_env directly whenever a module requires to get configuration value.
Plus: simpler than other options.
Minus: how to test such modules without bringing the whole application thing up?
Minus: how to start certain module with different configuration (if required)?
Pass the configuration (retrieved from application:get_env), to application modules during start-up.
Plus: modules are easier to test, you can start them with different configuration.
Minus: lot of boilerplate code. Changing the configuration format requires fixing several places.
Hold the configuration inside separate configuration process.
Plus: more-or-less type-safe approch. Easier to track where certain parameter is used and change those places.
Minus: need to bring up configuration process before running the modules.
Minus: how to start certain module with different configuration (if required)?
Another approach is to transform your configuration data into an Erlang source module that makes the configuration data available through exports. Then you can change the configuration at any time in a running system by simply loading a new version of the configuration module.
For static configuration in my own projects, I like option (1). I'll show you the steps I take to access a configuration parameter called max_widgets in an application called factory.
First, we'll create a module called factory_env which contains the following:
-define(APPLICATION, factory).
get_env(Key, Default) ->
case application:get_env(?APPLICATION, Key) of
{ok, Value} -> Value;
undefined -> Default
end.
set_env(Key, Value) ->
application:set_env(?APPLICATION, Key, Value).
Next, in a module that needs to read max_widgets we'll define a macro like the following:
-define(MAX_WIDGETS, factory_env:get_env(max_widgets, 1000)).
There are a few nice things about this approach:
Because we used application:set_env/3 and application:get_env/2, we don't actually need to start the factory application in order to have our tests pass.
max_widgets gets a default value, so our code will still work even if the parameter isn't defined.
A second module could use a different default value for max_widgets.
Finally, when we are ready to deploy, we'll put a sys.config file in our priv directory and load it with -config priv/sys.config during startup. This allows us to change configuration parameters on a per-node basis if desired. This cleanly separates configuration from code - e.g. we don't need to make another commit in order to change max_widgets to 500.
You could use a process (a gen_server maybe?) to store your configuration parameters in its state. It should expose a get/set interface. If a value hasn't been explicitly set, it should retrieve a default value.
-export([get/1, set/2]).
...
get(Param) ->
gen_server:call(?MODULE, {get, Param}).
...
handle_call({get, Param}, _From, State) ->
case lookup(Param, State#state.params) of
undefined ->
application:get_env(...);
Value ->
{ok, Value}
end.
...
You could then easily mockup this module in your tests. It will also be easy to update the process with some new configuration at run-time.
You could use pattern matching and tuples to associate different configuration parameters to different modules:
set({ModuleName, ParamName}, Value) ->
...
get({ModuleName, ParamName}) ->
...
Put the process under a supervision tree, so it's started before all the other processes which are going to need the configuration.
Oh, I'm glad nobody suggested parametrized modules so far :)
I'd do option 1 for static configuration. You can always test by setting options via application:set_env/3,4. The reason you want to do this is that your tests of the application will need to run the whole application anyway at some time. And the ability to set test-specific configuration at that point is really neat.
The application controller runs by default, so it is not a problem that you need to go the application-way (you need to do that anyway too!)
Finally, if a process needs specific configuration, say so in the configuration data! You can store any Erlang-term, in particular, you can store a term which makes you able to override configuration parameters for a specific node.
For dynamic configuration, you are probably better off by using a gen_server or using the newest gproc features that lets you store such dynamic configuration.
I've also seen people use a .hrl (erlang header file) where all the configuration is defined and include it at the start of any file that needs configuration.
It makes for very concise configuration lookups, and you get configuration of arbitrary complexity.
I believe you can also reload configuration at runtime by performing hot code reloading of the module. The disadvantage is that if you use configuration in several modules and reload only one of them, only that one module will get its configuration updated.
However, I haven't actually checked if it works like that, and I couldn't find definitive documentation on how .hrl and hot code reloading interact, so make sure to double-check this before you actually use it.
I've created a custom binding and want to make it configurable over App.config.
The binding has no special options at the moment, so it would be sufficient to support just
<endpoint address="http://myAddress"
binding="myBinding"
contract="myContract">
After checking some sites, I found out that I have to enable configuration support through a <BindingExtension>. However, the MSDN site didn't help much, since when I try to add
<extensions>
<bindingExtensions>
<add name="myBinding"
type="MyNamespace.MyHttpBinding, NameOfMyDll, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null"/>
</bindingExtensions>
</extensions>
, I only receive the following error message when trying to launch the program:
Configuration binding extension 'system.serviceModel/bindings/myBinding' could not be found. Verify that this binding extension is properly registered in system.serviceModel/extensions/bindingExtensions and that it is spelled correctly.
The type mentioned in the bindingExtension points to the type which inherits from Binding.
What do I have to add to enable configuration support for my binding?
My goal is just to be able to export my binding to the config file. I don't want to allow any special settings for the binding. It should just be usable over the config file's <endpoint> tag.
You're on the right track. The key point, however, is that the bindingExtension element should not point directly to your binding class itself.
Instead, you need to have several class that support the configuration model.
For starting, the bindingExtension you register is really a class that inherits from StandardBindingCollectionElement. This represents a collection of StandardBindingElement, which is the configuration class that has all the configuration properties that your binding will support in the .config file and would be responsible for creating your Binding instance and setting any properties on it that were set in the .config file.
Also, notice that, normally, you'd follow a similar pattern for creating a configuration view of your TransportBindingElement (if you're doing a transport channel) so that you can create custom bindings using it though configuration. In that case, you'd have a class inheriting TransportElement.
P.S. If you're thinking this is an awful lot of repetitive code if you've got lots of settings, then I agree.
Update: Found what your problem was: You need at least an empty <bindings/> section in your config file. Just add it and the binding will be recognized now.