Logback - how to get each logger logging to a separate log file? - logback

My application has lots of EJBs. The current bespoke Logger implementation creates a logger like this;
private static Logger logger = Logger.getInstance("SERVICE_NAME");
, and the logging will go into a file;
(path)/SERVICE_NAME/SERVICE_NAME.log
I want to replicate this behaviour with logback, but having real trouble grabbing the 'logger' name in the logback.xml configuration. It can be seen in the log encoder.pattern, i.e. "%d %-5level %logger{35} - %msg %n".
Any ideas how I can get this into a property/variable and then use it in the element?

I have a partial solution. If I create my own Discriminator, I can then use the Discriminator in the logback.xml to implement seperate-log-files-per-EJB.
Discriminator;
public class LoggerNameBasedDiscriminator implements Discriminator<ILoggingEvent> {
private static final String KEY = "loggerName";
private boolean started;
#Override
public String getDiscriminatingValue(ILoggingEvent iLoggingEvent) {
return iLoggingEvent.getLoggerName();
}
#Override
public String getKey() {
return KEY;
}
public void start() {
started = true;
}
public void stop() {
started = false;
}
public boolean isStarted() {
return started;
}
}
Then my logback.xml;
<configuration debug="true" scan="true" scanPeriod="30 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg</pattern>
</encoder>
</appender>
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="package.to.LoggerNameBasedDiscriminator"/>
<sift>
<appender name="FILE-${loggerName}" class="ch.qos.logback.core.FileAppender">
<FILE>path/to/logs/${loggerName}/${loggerName}.log</FILE>
<encoder>
<pattern>%d{HH:mm:ss.SSS} %-50(%level %logger{35}) %msg%n</pattern>
</encoder>
</appender>
</sift>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
<appender-ref ref="SIFT" />
</root>
</configuration>
This solution seems to work, but now I have no time or size based log rotation!

Thanks to your example I implemented a solution for a loggername-based discriminator which routes different logger output to different files. Although the documentation of logback is so verbose, I couldn't find this essential information. You've surely found the solution mentioned by yayitswei already.
logback.xml:
[...]
<timestamp key="startTimestamp" datePattern="yyyy-MM-dd"/>
<timestamp key="folderTimestamp" datePattern="MM-yyyy"/>
<property name="LOGDIR" value="/var/log/spock" />
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="com.enterprise.spock.LoggerNameBasedDiscriminator" />
<sift>
<appender name="FILE-${loggerName}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOGDIR}/${loggerName}-${startTimestamp}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOGDIR}/${folderTimestamp}/${loggerName}-%d{yyyy-MM-dd}-%i.log.gz</fileNamePattern>
<maxFileSize>500KB</maxFileSize>
<maxHistory>100</maxHistory>
<totalSizeCap>50MB</totalSizeCap>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>%level %date{HH:mm:ss.SSS}: %msg %n</pattern>
</encoder>
</appender>
</sift>
</appender>
[...]
Edit:
I replaced TimeBasedRollingPolicy with SizeAndTimeBasedRollingPolicy as proposed here. You'll need at least logback 1.1.7 for that.
What you get with this, is a logfile for each Logger created.
Every logfile will look like this: /var/log/loggername-2017-08-03.log.
When about 500KB was written to the file, it will be archived as a gz-zipfile into /var/log/loggername/08-2017/loggername-2017-08-03-0.log.gz.
The 0 at the end of the gz-zipfile-name is the %i from the <fileNamePattern> above. Without the %i it won't work. Remember to use <configuration debug=true> in logback.xml if something won't work.

Related

Logback AsyncAppender not writing logs to underlying appenders

In logback.xml of my application I have an AsyncAppender defined as below.
<appender name="socketAppender" class="ch.qos.logback.classic.net.SocketAppender">
<param name="RemoteHost" value="127.0.0.1" />
<param name="Port" value="15000" />
<param name="ReconnectionDelay" value="10" />
<param name="Threshold" value="DEBUG" />
</appender>
<appender name="appLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logs/myApp.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>logs/myApp.%d{yyyyMMdd.HH}00.log</fileNamePattern>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>[%d{yyyy/MM/dd HH:mm:ss.SSS}][%p][%c{0}] %m%n]</pattern>
</encoder>
</appender>
<appender name="AsyncLog" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="appLog" />
<appender-ref ref="socketAppender" />
</appender>
When I deploy the war in Tomcat on Windows 10.
The log server listening on port 15000 wasn't receiving logs.
So I moved up the socketAppender to first position like below.
<appender name="AsyncLog" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="socketAppender" />
<appender-ref ref="appLog" />
</appender>
With this change log server started receiving logs, but the log file stopped writing/appending.
3) I also tried setting the queueSize and discardingThreshold properties on the appender, to no avail.
<discardingThreshold>0</discardingThreshold>
<queueSize>500</queueSize>
Can any logback experts please tell me what I am doing wrong here? Are there any other properties of AsyncAppender that may help fix this behaviour?
A colleague pointed me to this method in the class ch.qos.logback.core.AsyncAppenderBase. AsyncAppender can attach only one appender and ignores any more appenders added beyond that.
public void addAppender(Appender<E> newAppender) {
if (appenderCount == 0) {
appenderCount++;
addInfo("Attaching appender named [" + newAppender.getName() + "] to AsyncAppender.");
aai.addAppender(newAppender);
} else {
addWarn("One and only one appender may be attached to AsyncAppender.");
addWarn("Ignoring additional appender named [" + newAppender.getName() + "]");
}
}

logback smtpAppender mailing on both error and info log level

I am trying to use logback SMTP appender to send email alerts.
I am getting emails only when log level is 'error' (I know this is by default).
How do I get emails when log level is being set to 'info'?
<property resource="application.properties"/>
<appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<smtpHost>${spring.mail.host}</smtpHost>
<username>${spring.mail.username}</username>
<password>${spring.mail.password}</password>
<smtpPort>${spring.mail.port}</smtpPort>
<STARTTLS>false</STARTTLS>
<SSL>true</SSL>
<subject>Exception: Registraion App %m</subject>
<to>${spring.mail.to}</to>
<from>${spring.mail.from}</from>
<asynchronousSending>true</asynchronousSending>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date %-5level %logger{35} - %message%n</pattern>
</layout>
</appender>
<logger name="errorLogger" level="error" additivity="false">
<appender-ref ref="EMAIL"/>
</logger>
In the java class I am invoking this as below
public static final Logger emailExceptionLOGGER = LoggerFactory.getLogger("errorLogger");
try{
.....
}catch(Exception e){
emailExceptionLOGGER.error("To send exception email")
}
I would also like to send success emails when log level is being set to 'info'
<appender name="SUCCESS-EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<smtpHost>${spring.mail.host}</smtpHost>
<username>${spring.mail.username}</username>
<password>${spring.mail.password}</password>
<smtpPort>${spring.mail.port}</smtpPort>
<STARTTLS>false</STARTTLS>
<SSL>true</SSL>
<subject>Registraion App %m</subject>
<to>${spring.mail.to}</to>
<from>${spring.mail.from}</from>
<asynchronousSending>true</asynchronousSending>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date %-5level %logger{35} - %message%n</pattern>
</layout>
</appender>
<logger name="successLogger" level="info" additivity="false">
<appender-ref ref="SUCCESS-EMAIL"/>
</logger>
In the java class I am invoking this something like below
public static final Logger emailSuccessLOGGER = LoggerFactory.getLogger("successLogger");
if(success){
emailSuccessLOGGER.info("To send success email")
}
I would like to handle both together. Thanks in advance.
include below after layout in your email appender. It should filter out info logs.
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>

Logback adding appender at runtime

Version data:
slf4j-api - 1.7.25
logback-core - 1.2.3
logback-classic - 1.2.3
I have a pretty simple appender I'm testing with:
public class MyAppender extends AppenderBase<ILoggingEvent> {
#Override
protected String getName() {
return "Test Instance";
}
#Override
protected void append(ILoggingEvent event) {
System.err.println("Hey, it worked!");
}
}
It doesn't get called when I do this:
Logger logger = (Logger) LoggerFactory.getLogger(MyTest.class);
logger.addAppender(new MyAppender());
Assert.assertNotNull(logger.getAppender("Test Instance"));
logger.info("Some message");
My logback-text.xml looks like this:
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Is there something I am missing in initialization of my appender?
As always I answer this as soon as I post it.
For anyone else hitting this you need to add:
MyAppender appender = new MyAppender();
appender.start();

How to log specific HTTP header using logback.xml

I would like to create a console appender that displays some log info, and also prints out a particular http header, similar to this:
> [INFO] { "time": "2017-08-31 12:14:32,583", "app-id": "my-app", "my-header": "my-header-value" } -- "Hello, World"
I have created a logback-spring.xml file like below, but "my-header" just prints out blank.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<springProperty name="appId" source="spring.app.application_id"/>
<!-- Appender to log to console -->
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- Minimum logging level to be presented in the console logs-->
<level>INFO</level>
</filter>
<encoder>
<pattern>
%clr(%5p) %clr({ "time": "%date{ISO8601}", "app-id": "${appId}", "my-header": "%X{my-header}"}){faint} -- %msg%n
</pattern>
<charset>utf8</charset>
</encoder>
</appender>
​
<root level="INFO">
<appender-ref ref="console"/>
</root>
</configuration>
I have read that using logback-access gives you access to HTTP request/response properties, but when I try setting the encoder class I cannot use any of the classic logback conversion words:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<!-- Appender to log to console -->
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- Minimum logging level to be presented in the console logs-->
<level>INFO</level>
</filter>
<encoder class="ch.qos.logback.access.PatternLayoutEncoder">
<pattern class="ch.qos.logback.access.PatternLayoutEncoder">
%clr(%5p) %clr({ "time": "%date{ISO8601}", "app-id": "${appId}", "my-header": "%header{my-header}"}){faint} -- %msg%n
</pattern>
<charset>utf8</charset>
</encoder>
</appender>
​
<root level="INFO">
<appender-ref ref="console"/>
</root>
</configuration>
The logback above gives these errors:
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:1.4.0.RELEASE:run (default-cli) on project pd-thundera-server: An exception occurred while running. null: InvocationTargetException: Logback configuration error detected:
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#4782f0a4 - There is no conversion class registered for conversion word [p]
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#4782f0a4 - [p] is not a valid conversion word
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#6071227e - There is no conversion class registered for conversion word [msg]
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#6071227e - [msg] is not a valid conversion word
How can I access a request header?
This configuration worked for me:
<encoder>
<charset>utf-8</charset>
<pattern>%t{yyyy-MM-dd HH:mm:ss,SSS} %h X-Forwarded-For: %header{X-Forwarded-For} "%r", Response status:%s, Bytes sent:%b, Response time:%D</pattern>
</encoder>
Name of the header goes in between curly braces, e.g. %header{Content-type} or %header{My-header}, or just %header if you want all headers to be logged.
Source:
https://logback.qos.ch/access.html#configuration
https://logback.qos.ch/manual/layouts.html#AccessPatternLayout
Please find below example with Spring
If you want to log http headers then you need to use MDC feature: http://logback.qos.ch/manual/mdc.html
create Filter:
#Component
class RequestHeaderFilterConfig : Filter {
private val xRequestId = "X-Request-Id"
override fun doFilter(request: ServletRequest?, response: ServletResponse?, chain: FilterChain?) {
val httpRequest = request as HttpServletRequest
MDC.put(xRequestId, httpRequest.getHeader(xRequestId))
chain?.doFilter(request, response)
}
override fun destroy() = MDC.remove(xRequestId)
}
Controller:
#PostMapping
fun operation(#RequestHeader(value = "X-Request-Id", required = false) xRequestId: String? = null): ResponseEntity<Output> {
....
}
logback.xml
<Pattern>%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level [X-Request-Id: %X{X-Request-Id}] %logger{36}.%M:%line - %msg%n</Pattern>

Logback AyncAppender not printing File and Line number

I have the following configuration file that is very similar to the standard example in the Logback manual. The only difference is the addition of [%F:%L]. while everything works, %F and %L do not work. If I remove the async appender and log directly using the file appender, everything works just great.
can somebody explain what is going on? And how to print the file name and line number as these two parameters are supposed to?
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>myapp.log</file>
<encoder><pattern>%logger{35} - [%F:%L] - %msg%n</pattern></encoder>
</appender>
<appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE" />
</appender>
<root level="DEBUG"><appender-ref ref="ASYNC" /></root>
</configuration>
You need to set AsyncAppender's includeCallerData property to true. Here is the modified config file:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>myapp.log</file>
<encoder><pattern>%logger{35} - [%F:%L] - %msg%n</pattern></encoder>
</appender>
<appender name="ASYNC" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE" />
<!-- add the following line -->
<includeCallerData>true</includeCallerData>
</appender>
<root level="DEBUG"><appender-ref ref="ASYNC" /></root>
</configuration>
I post same answer in groovy format for someone who want groovy style like me.
appender('FILE', ch.qos.logback.core.FileAppender) {
file = 'myapp.log'
encoder(PatternLayoutEncoder) {
pattern = '%logger{35} - [%F:%L] - %msg%n'
}
}
appender('ASYNC', ch.qos.logback.classic.AsyncAppender) {
appenderRef('FILE')
//add the following line
includeCallerData = true
}
root(DEBUG, ['ASYNC'])