How to programatically access appenders from logback-access config? - logback

I have enabled logback-access log for Tomcat 7.0, using LogbackValve.
My logback-access config looks like this.
<configuration>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>access.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>access.%d{yyyy-MM-dd}.log.zip</fileNamePattern>
</rollingPolicy>
<encoder>
<pattern>combined</pattern>
</encoder>
</appender>
<appender-ref ref="FILE" />
</configuration>
Now this works perfect and I can see access logs. But in some other code I want to retrieve the appender configured in logback-access.xml programmatically. Is there a way to do it. Since there are no Loggers defined , you cannot use LoggerContext .

You can use LogbackAccessContext to retrieve the appenders configured in logback-access.xml. This is my solution:
TomcatServletWebServerFactory sc = (TomcatServletWebServerFactory) context.getBean(TomcatServletWebServerFactory.class);
List<Valve> values = (List<Valve>) sc.getEngineValves();
LogbackAccessTomcatValve accessTomcatValve = null;
for(Valve value : values) {
if(value.getClass() == LogbackAccessTomcatValve.class) {
accessTomcatValve = (LogbackAccessTomcatValve) value;
}
}
//get logbackAccessContext from the logbackAccessTomcatValve by invoking.
LogbackAccessContext logbackAccessContext = null;
Field field = null;
try {
field = accessTomcatValve.getClass().getDeclaredField("logbackAccessContext");
field.setAccessible(true);
logbackAccessContext = (LogbackAccessContext) field.get(accessTomcatValve);
} catch (Exception e) {
log.error("Exception happened when fetching the logback access context ." +e.getMessage());
}

Related

How to configure different levels for different appenders but under same logger in logback

We are writing 2 appenders to write logs in different formats to 2 different files.
But we want to enable these logs based on some configuration.
So if the user wants to enable both of the formats, then both of logs will be printed. But if the user wants to disable one, that log should not be created.
Below is my logger configuration :
<logger name="package.name" additivity="false" level="DEBUG">
<appender-ref ref="STDOUT" />
<appender-ref ref="json_logs"/>
<appender-ref ref="text_logs"/>
</logger>
Now I want to put separate levels for this appender-ref. and value of these levels either should come from some property file which will be edited by user or user could simply update logback.xml file only.
I am not able to find a way to put separate levels for these appenders.
And since I have to write logs from same classes, I cant create 2 separate loggers too.
Also, if the user does not want to see txt logs, then the corresponding log.txt file should not be created.
What you are after is the ThresholdFilter which can be fine tuned for each appender:
http://logback.qos.ch/manual/filters.html#thresholdFilter
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<!-- deny all events with a level below INFO, that is TRACE and DEBUG -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
...
</appender>
As for not writing the file at all based on some flag... you could write your own appender. Something like this ultra-simplistic example that could be refined in a number of ways but should hopefully get you started:
public class ConditionalAppender<E> extends OutputStreamAppender<E> {
private boolean masterSwitch;
private String file;
#Override
public void start() {
if(masterSwitch)
setOutputStream(new FileOutputStream(file));
else
setOutputStream(new NullOutputStream());
super.start();
}
public void setMasterSwitch(final boolean enabled) {
this.masterSwitch = enabled;
}
public void setFile(final String file) {
this.file = file;
}
}
<appender name="CONDITIONAL" class="mypackage.ConditionalAppender">
<!-- switch this appender on or off -->
<masterSwitch>true</masterSwitch>
<!-- set the output file -->
<file>/var/log/app/app.log</file>
<!-- deny all events with a level below INFO, that is TRACE and DEBUG -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
...
</appender>
Hope this helps.

Output Spark application id in the logs with Log4j

I have a custom Log4j file for the Spark application. I would like to output Spark app id along with other attributes like message and date so the JSON string structure would look like this:
{"name":,"time":,"date":,"level":,"thread":,"message":,"app_id":}
Now, this structure looks like this:
{"name":,"time":,"date":,"level":,"thread":,"message":}
How can I define such layout for the Spark driver logs?
My log4j file looks like this:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="Json" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.hadoop.log.Log4Json">
<param name="ConversionLayout" value=""/>
</layout>
</appender>
<root>
<level value="INFO"/>
<appender-ref ref="Json"/>
</root>
</log4j:configuration>
I doubt that org.apache.hadoop.log.Log4Json can be adjusted for this purpose. According to its javadoc and source code it might be rather cumbersome.
Although it looks like you are using Log4j 1x, its API is quite flexible and we can easily define our own layout by extending org.apache.log4j.Layout.
We'll need a case class that will be transformed into JSON according to the target structure:
case class LoggedMessage(name: String,
appId: String,
thread: String,
time: Long,
level: String,
message: String)
And Layout might be extended as follows. To access the value of "app_id", we'll use Log4j's Mapped Diagnostic Context
import org.apache.log4j.Layout
import org.apache.log4j.spi.LoggingEvent
import org.json4s.DefaultFormats
import org.json4s.native.Serialization.write
class JsonLoggingLayout extends Layout {
// required by the API
override def ignoresThrowable(): Boolean = false
// required by the API
override def activateOptions(): Unit = { /* nothing */ }
override def format(event: LoggingEvent): String = {
// we are using json4s for JSON serialization
implicit val formats = DefaultFormats
// retrieve app_id from Mapped Diagnostic Context
val appId = event.getMDC("app_id") match {
case null => "[no_app]" // logged messages outside our app
case defined: AnyRef => defined.toString
}
val message = LoggedMessage("TODO",
appId,
Thread.currentThread().getName,
event.getTimeStamp,
event.getLevel.toString,
event.getMessage.toString)
write(message) + "\n"
}
}
Finally, when the Spark session is created, we put the app_id value into MDC:
import org.apache.log4j.{Logger, MDC}
// create Spark session
MDC.put("app_id", session.sparkContext.applicationId)
logger.info("-------- this is info --------")
logger.warn("-------- THIS IS A WARNING --------")
logger.error("-------- !!! ERROR !!! --------")
This produces following logs:
{"name":"TODO","appId":"local-1550247707920","thread":"main","time":1550247708149,"level":"INFO","message":"-------- this is info --------"}
{"name":"TODO","appId":"local-1550247707920","thread":"main","time":1550247708150,"level":"WARN","message":"-------- THIS IS A WARNING --------"}
{"name":"TODO","appId":"local-1550247707920","thread":"main","time":1550247708150,"level":"ERROR","message":"-------- !!! ERROR !!! --------"}
And, of course, do not forget to refer the implementation in log4j config xml:
<appender name="Json" class="org.apache.log4j.ConsoleAppender">
<layout class="stackoverflow.q54706582.JsonLoggingLayout" />
</appender>

logback RollingFileAppender rolling at fixed time

A typical logback appender configuration is
<appender name="NAME"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>FILEPATH.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>FILEPATH.%d{yyyy-MM-dd-HH}.log</fileNamePattern>
<maxHistory>24</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
</appender>
Usually it works fine to me. But I found that it will not split log file as the rollingPolicy specified when there's no log appended to the logger respected.
Please tell me How I can configure it to split log file for every hours even there's no log record for some hours. If there's no log record for any hour, I need logback to create an empty file for that hour.
I was with the same problem and I guess logback doesn't able to do it per default.
I found this issue at logback backlog to handle exactly with this: https://jira.qos.ch/browse/LOGBACK-554 and it`s still open.
What I did for now was implement my own rolling appender policy, based on it: https://ronanquillevere.github.io/2015/08/04/rolling-log-15-min.html
You can extend the RollingFileAppender class and override the rollover method, like this:
public class CustomAppender<E> extends RollingFileAppender<E>
{
private static long start = System.currentTimeMillis();
private int rollOverTimeInMinutes = 60;
#Override
public void rollover()
{
long currentTime = System.currentTimeMillis();
int maxIntervalSinceLastLoggingInMillis = rollOverTimeInMinutes * 60 * 1000;
if ((currentTime - start) >= maxIntervalSinceLastLoggingInMillis)
{
super.rollover();
start = System.currentTimeMillis();
}
}
}
And on logback file xml config, you just set your custom pollicy on appender property:
<appender name="FILE" class="<your-package>.CustomAppender" >

MEF plugins with their own configuration files?

I'm trying to load plugins at runtime and access their configuration files. The configuration sections in their config files are mapped to classes derived from ConfigurationElementCollection, ConfigurationElement and ConfigurationSection. The plugins and their configuration files are location in a subfolder called "Plugins".
The problem is that I can't seem to load the plugin configuration data and deserialize it into their respective classes correctly.
Here is an example of a plugin config for the plugin EmailPlugin.dll:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="EmailConfigurationSection" type="Foo.Plugins.EmailConfigurationSection, EmailPlugin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" allowDefinition="Everywhere" allowExeDefinition="MachineToApplication" restartOnExternalChanges="true"/>
</configSections>
<EmailConfigurationSection server="192.168.0.10">
<EmailSettings>
<add keyword="ERROR"
sender="error#error.com"
recipients="foo#bar.com, wiki#waki.com"
subject = "Error occurred"
body = "An error was detected"
/>
</EmailSettings>
</EmailConfigurationSection>
</configuration>
I load this using this code:
private static System.Configuration.Configuration config = null;
public static System.Configuration.Configuration CurrentConfiguration
{
get
{
if (config == null)
{
Assembly assembly = Assembly.GetAssembly(typeof(EmailPlugin));
string directory = Path.GetDirectoryName(assembly.CodeBase);
string filename = Path.GetFileName(assembly.CodeBase);
string assemblyPath = Path.Combine(directory, filename);
config = ConfigurationManager.OpenExeConfiguration(new Uri(assemblyPath).LocalPath);
}
return config;
}
}
This results in the error:
An error occurred creating the configuration section handler for EmailConfigurationSection: Could not load file or assembly 'EmailPlugin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. The system cannot find the file specified.
I added this to the top of the config file:
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="Plugins"/>
</assemblyBinding>
</runtime>
So the DLL is found, but it doesn't not cast to the proper class when I try to retrieve it:
EmailConfigurationSection defaults = CurrentConfiguration.Sections["EmailConfigurationSection"] as EmailConfigurationSection;
It always returns null. I know it's looking at the correct location and configuration file because I can retrieve the XML using this code:
var section = CurrentConfiguration.Sections["EmailConfigurationSection"];
string configXml = section.SectionInformation.GetRawXml();
However, when I try to deserialize it with this code:
var serializer = new XmlSerializer(typeof(EmailConfigurationSection));
object result;
EmailConfigurationSection defaults;
using (TextReader reader = new StringReader(configXml))
{
defaults = (EmailConfigurationSection)serializer.Deserialize(reader);
}
... I get an exception:
There was an error reflecting type 'Foo.Plugins.EmailConfigurationSection'.
This is the contents of the InnerException:
You must implement a default accessor on System.Configuration.ConfigurationLockCollection because it inherits from ICollection.
I assume it's referring to the class EmailConfigElementCollection, but then the message does not make sense because this class does have a default accessor:
public EmailConfigElement this[int index]
{
get
{
return (EmailConfigElement)BaseGet(index);
}
set
{
if (BaseGet(index) != null)
{
BaseRemoveAt(index);
}
BaseAdd(index, value);
}
}
I've used this code successfully in other projects (even with separate DLLs/configs), but this is the first time I'm trying to use it with MEF. Does anyone know what the problem is, or a suitable workaround?
I'm using .NET 4.5
I fixed this with the following modification:
public static System.Configuration.Configuration CurrentConfiguration
{
get
{
if (config == null)
{
// Added the next bit
AppDomain.CurrentDomain.AssemblyResolve += (o, args) =>
{
var loadedAssemblies = AppDomain.CurrentDomain.GetAssemblies();
return loadedAssemblies.Where(asm => asm.FullName == args.Name)
.FirstOrDefault();
};
Assembly assembly = Assembly.GetAssembly(typeof(EmailPlugin));
string directory = Path.GetDirectoryName(assembly.CodeBase);
string filename = Path.GetFileName(assembly.CodeBase);
string assemblyPath = Path.Combine(directory, filename);
config = ConfigurationManager.OpenExeConfiguration(new Uri(assemblyPath).LocalPath);
}
return config;
}
}
I got this from this question:
Custom configuration sections in MEF exporting assemblies. I had actually tried earlier it with no success.
The trick was that I had to move the runtime tag to the bottom of the XML configuration:
<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<configSections>
<section name="EmailConfigurationSection" type="Foo.Plugins.EmailConfigurationSection, EmailPlugin, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null" allowDefinition="Everywhere" allowExeDefinition="MachineToApplication" restartOnExternalChanges="true"/>
</configSections>
<EmailConfigurationSection server="255.255.255.1">
<EmailSettings>
<clear />
<add keyword="FOO"
sender="foo#foo.com"
recipients="me#you.com"
subject = "Foo occurred"
body = "Hello"
/>
</EmailSettings>
</EmailConfigurationSection>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="Plugins"/>
</assemblyBinding>
</runtime>
</configuration>

How do I Log the Class file without the path in log4net

I want to be able to log the class file and line number in my log file so I am using the following config...
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<!--etc-->
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level (%file:%line) %logger => %message%newline" />
</layout>
</appender>
However the %file attribute is making my log file entries waaaay too long to read comfortably....
2009-08-07 16:41:55,271 [7] INFO (O:\mystream\aevpallsrv\DotNet\com.mycompany.au\myapp\Myappp\Controller.cs:75) MyApp.Controller => Controller.EnqueueWorkerThreads() - START
Is there a way to show just the class file ('Controller.cs') instead of the full path to the file also???
Michael
Although you're asking about the file name, I believe you can use %type to get the fully qualified type name (a.b.className). If you just want the class name, use %type{1}
Note that any method that generates caller information (%file and %type) have a performance cost associated with them.
As an aside, you can bypass the performance hit by naming the Logger using the Type name.
namespace MyNamespace
{
public class Foo
{
private static ILog log = LogManager.GetLogger(typeof(Foo));
}
}
Your conversion pattern would look like:
"%date [%thread] %-5level %logger %message"
Where your logger would be "MyNameSpace.Foo". Likewise, if you only want the class name, use "%logger{1}, which will resolve as "Foo".
One of the best advantages to this approach is that you can leverage log4net's hierarchical repository system to adjust logging levels per type:
<!-- all classes in MyNamespace are warn -->
<logger name="MyNamespace">
<level value="WARN" />
</logger>
<!-- only Foo is in debug -->
<logger name="MyNamespace.Foo">
<level value="DEBUG" />
</logger>
Out of the box, PatternLayout supports only the %file token. What you can do is subclass PatternLayout and add your own pattern, say %filename, and for this token only output the name of the file.