Version data:
slf4j-api - 1.7.25
logback-core - 1.2.3
logback-classic - 1.2.3
I have a pretty simple appender I'm testing with:
public class MyAppender extends AppenderBase<ILoggingEvent> {
#Override
protected String getName() {
return "Test Instance";
}
#Override
protected void append(ILoggingEvent event) {
System.err.println("Hey, it worked!");
}
}
It doesn't get called when I do this:
Logger logger = (Logger) LoggerFactory.getLogger(MyTest.class);
logger.addAppender(new MyAppender());
Assert.assertNotNull(logger.getAppender("Test Instance"));
logger.info("Some message");
My logback-text.xml looks like this:
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%-5level] - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
</configuration>
Is there something I am missing in initialization of my appender?
As always I answer this as soon as I post it.
For anyone else hitting this you need to add:
MyAppender appender = new MyAppender();
appender.start();
Related
In logback.xml of my application I have an AsyncAppender defined as below.
<appender name="socketAppender" class="ch.qos.logback.classic.net.SocketAppender">
<param name="RemoteHost" value="127.0.0.1" />
<param name="Port" value="15000" />
<param name="ReconnectionDelay" value="10" />
<param name="Threshold" value="DEBUG" />
</appender>
<appender name="appLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logs/myApp.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>logs/myApp.%d{yyyyMMdd.HH}00.log</fileNamePattern>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>[%d{yyyy/MM/dd HH:mm:ss.SSS}][%p][%c{0}] %m%n]</pattern>
</encoder>
</appender>
<appender name="AsyncLog" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="appLog" />
<appender-ref ref="socketAppender" />
</appender>
When I deploy the war in Tomcat on Windows 10.
The log server listening on port 15000 wasn't receiving logs.
So I moved up the socketAppender to first position like below.
<appender name="AsyncLog" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="socketAppender" />
<appender-ref ref="appLog" />
</appender>
With this change log server started receiving logs, but the log file stopped writing/appending.
3) I also tried setting the queueSize and discardingThreshold properties on the appender, to no avail.
<discardingThreshold>0</discardingThreshold>
<queueSize>500</queueSize>
Can any logback experts please tell me what I am doing wrong here? Are there any other properties of AsyncAppender that may help fix this behaviour?
A colleague pointed me to this method in the class ch.qos.logback.core.AsyncAppenderBase. AsyncAppender can attach only one appender and ignores any more appenders added beyond that.
public void addAppender(Appender<E> newAppender) {
if (appenderCount == 0) {
appenderCount++;
addInfo("Attaching appender named [" + newAppender.getName() + "] to AsyncAppender.");
aai.addAppender(newAppender);
} else {
addWarn("One and only one appender may be attached to AsyncAppender.");
addWarn("Ignoring additional appender named [" + newAppender.getName() + "]");
}
}
I need to be able to log an Exception as a single record in my logs, which will make it much easier to investigate issues in Kibana / Elasticsearch. From what I can tell from the documentation for slf4j, the Logger interface requires messages to be Strings. Is my only option to remove newline characters from the Exception message before passing it to the Logger?
For context, I am using the following:
.m2/repository/org/slf4j/slf4j-api/1.7.28/slf4j-api-1.7.28.jar
Java 11
Sprint Boot version 2.1.8.RELEASE
This is a trimmed down version of my custom exception handler :
import my.error.Error; // custom Error class
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.servlet.mvc.method.annotation.ResponseEntityExceptionHandler;
#ControllerAdvice
public class CustomExceptionHandler extends ResponseEntityExceptionHandler {
private final Logger logger = LoggerFactory.getLogger(this.getClass());
private void logError(Error error, Exception ex){
logger.error(String.format("id: %s, message: %s", error.getId(), ex.getMessage()), ex);
}
}
Initially I had attempted to alter the logging behavior with changes to my logback.xml file, within src/main/java/resources. Unfortunately, this appears to do nothing, so my assumption now is that the CustomExceptionHandler that I am creating is overruling the specification set in the logback.xml file. Specifically, the <pattern> of the <encoder> has been changed based on other research. It's attempting to replace all newline characters.
<?xml version="1.0" encoding="UTF-8"?>
<configuration scan="true" debug="true">
<include resource="org/springframework/boot/logging/logback/base.xml" />
<appender name="FILE-ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/gateway.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/archived/gateway/gateway.%d{yyyy-MM-dd}.%i.log.gz</fileNamePattern>
<!-- each archived file, size max 5MB -->
<maxFileSize>5MB</maxFileSize>
<!-- total size of all archive files, if total size > 10GB, it will delete old archived file -->
<totalSizeCap>10GB</totalSizeCap>
<!-- 30 days to keep -->
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d %p %c{1.} [%t] %m MULTIEXCEPTION %replace(%xException){'\n','\u2028'}%nopex%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="FILE-ROLLING" level="DEBUG" additivity="false"/>
</root>
<springProfile name="local">
<logger name="my.gateway" level="TRACE" additivity="false">
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE-ROLLING" />
</logger>
<logger name="com.netflix" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
<logger name="org.springframework" level="DEBUG" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
<logger name="com" level="INFO" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
<logger name="gov" level="INFO" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
<logger name="org" level="INFO" additivity="false">
<appender-ref ref="CONSOLE" />
</logger>
</springProfile>
</configuration>
Links
Logback Docs
SLF4J Docs
SLF4J Manual
Combine Logback and SLF4J
Logback and SLF4J - StructuredArguments
JSON Logging w/ Logback
Collapse Logs - Multiple Lines into One
Apache Log4j Layouts
SO : Override Logback 1
SO : Override Logback 2
SO : Make Logback Output JSON
Format SLF4J
Collapse multi-line logs into one with Logback or Log4j2
Baeldung Log4j2 JSON Logging
Java Logging Guide
I changed my logError method to this:
import org.slf4j.MDC; // new import to add property
// ... other imports from before
private void logError(Error error, Exception exception){
MDC.put("error id", error.getId().toString()); // add a new property to the thread context
LOGGER.error(exception.getMessage(), exception);
MDC.clear(); // remove all new properties after logging the error
}
My new pom.xml File Dependencies:
<dependencies>
<!-- logging -->
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>5.2</version>
</dependency>
<dependency>
<groupId>ch.qos.logback.contrib</groupId>
<artifactId>logback-jackson</artifactId>
<version>0.1.5</version>
</dependency>
<!-- ch.qos.logback.contrib.json.classic.JsonLayout -->
<dependency>
<groupId>ch.qos.logback.contrib</groupId>
<artifactId>logback-json-classic</artifactId>
<version>0.1.5</version>
</dependency>
<!-- Other dependencies -->
<!-- ... -->
<!-- ... -->
</dependencies>
New logback.xml file:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender class="ch.qos.logback.core.rolling.RollingFileAppender" name="FILE-ROLLING">
<!-- Old encoder is removed:
<encoder>
<pattern>%d %p %c{1.} [%t] %m%n</pattern>
</encoder>
-->
<file>${LOG_PATH}/data.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/archived/data/data.%d{yyyy-MM-dd}.%i.log.gz
</fileNamePattern>
<!-- each archived file, size max 10MB -->
<maxFileSize>10MB</maxFileSize>
<!-- total size of all archive files, if total size > 20GB, it will delete old archived file -->
<maxHistory>60</maxHistory>
<!-- 60 days to keep -->
<totalSizeCap>20GB</totalSizeCap>
</rollingPolicy>
<!-- TODO : configure a pretty printer -->
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider"/>
</encoder>
</appender>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<!-- TODO : configure a pretty printer -->
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<provider class="net.logstash.logback.composite.loggingevent.ArgumentsJsonProvider"/>
</encoder>
</appender>
<!-- The base logback configuration is removed, but note that this will also remove the default Spring start-up messages, so those need to be added back into the <root>
<include resource="org/springframework/boot/logging/logback/base.xml"/>
-->
<root level="INFO">
<appender-ref ref="FILE-ROLLING"/>
<appender-ref ref="CONSOLE"/> <!-- This adds Spring start-up messages back into the logs -->
</root>
<springProfile name="local">
<logger additivity="false" level="TRACE" name="org.springframework.web">
<appender-ref ref="CONSOLE"/>
</logger>
</springProfile>
<springProfile name="dev">
<logger additivity="false" level="DEBUG" name="org.springframework">
<appender-ref ref="FILE-ROLLING"/>
</logger>
</springProfile>
<springProfile name="prod">
<logger additivity="false" level="DEBUG" name="my.app.path">
<appender-ref ref="FILE-ROLLING"/>
</logger>
<logger additivity="false" level="DEBUG" name="org.jooq">
<appender-ref ref="FILE-ROLLING"/>
</logger>
<logger additivity="false" level="DEBUG" name="org.springframework">
<appender-ref ref="FILE-ROLLING"/>
</logger>
</springProfile>
</configuration>
SLF4J Docs - Mapped Diagnostic Context (MDC)
Baeldung Example - MDC + Log4J2
I am trying to use logback SMTP appender to send email alerts.
I am getting emails only when log level is 'error' (I know this is by default).
How do I get emails when log level is being set to 'info'?
<property resource="application.properties"/>
<appender name="EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<smtpHost>${spring.mail.host}</smtpHost>
<username>${spring.mail.username}</username>
<password>${spring.mail.password}</password>
<smtpPort>${spring.mail.port}</smtpPort>
<STARTTLS>false</STARTTLS>
<SSL>true</SSL>
<subject>Exception: Registraion App %m</subject>
<to>${spring.mail.to}</to>
<from>${spring.mail.from}</from>
<asynchronousSending>true</asynchronousSending>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date %-5level %logger{35} - %message%n</pattern>
</layout>
</appender>
<logger name="errorLogger" level="error" additivity="false">
<appender-ref ref="EMAIL"/>
</logger>
In the java class I am invoking this as below
public static final Logger emailExceptionLOGGER = LoggerFactory.getLogger("errorLogger");
try{
.....
}catch(Exception e){
emailExceptionLOGGER.error("To send exception email")
}
I would also like to send success emails when log level is being set to 'info'
<appender name="SUCCESS-EMAIL" class="ch.qos.logback.classic.net.SMTPAppender">
<smtpHost>${spring.mail.host}</smtpHost>
<username>${spring.mail.username}</username>
<password>${spring.mail.password}</password>
<smtpPort>${spring.mail.port}</smtpPort>
<STARTTLS>false</STARTTLS>
<SSL>true</SSL>
<subject>Registraion App %m</subject>
<to>${spring.mail.to}</to>
<from>${spring.mail.from}</from>
<asynchronousSending>true</asynchronousSending>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date %-5level %logger{35} - %message%n</pattern>
</layout>
</appender>
<logger name="successLogger" level="info" additivity="false">
<appender-ref ref="SUCCESS-EMAIL"/>
</logger>
In the java class I am invoking this something like below
public static final Logger emailSuccessLOGGER = LoggerFactory.getLogger("successLogger");
if(success){
emailSuccessLOGGER.info("To send success email")
}
I would like to handle both together. Thanks in advance.
include below after layout in your email appender. It should filter out info logs.
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
I would like to create a console appender that displays some log info, and also prints out a particular http header, similar to this:
> [INFO] { "time": "2017-08-31 12:14:32,583", "app-id": "my-app", "my-header": "my-header-value" } -- "Hello, World"
I have created a logback-spring.xml file like below, but "my-header" just prints out blank.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<springProperty name="appId" source="spring.app.application_id"/>
<!-- Appender to log to console -->
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- Minimum logging level to be presented in the console logs-->
<level>INFO</level>
</filter>
<encoder>
<pattern>
%clr(%5p) %clr({ "time": "%date{ISO8601}", "app-id": "${appId}", "my-header": "%X{my-header}"}){faint} -- %msg%n
</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="console"/>
</root>
</configuration>
I have read that using logback-access gives you access to HTTP request/response properties, but when I try setting the encoder class I cannot use any of the classic logback conversion words:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<include resource="org/springframework/boot/logging/logback/defaults.xml"/>
<!-- Appender to log to console -->
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<!-- Minimum logging level to be presented in the console logs-->
<level>INFO</level>
</filter>
<encoder class="ch.qos.logback.access.PatternLayoutEncoder">
<pattern class="ch.qos.logback.access.PatternLayoutEncoder">
%clr(%5p) %clr({ "time": "%date{ISO8601}", "app-id": "${appId}", "my-header": "%header{my-header}"}){faint} -- %msg%n
</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="console"/>
</root>
</configuration>
The logback above gives these errors:
[ERROR] Failed to execute goal org.springframework.boot:spring-boot-maven-plugin:1.4.0.RELEASE:run (default-cli) on project pd-thundera-server: An exception occurred while running. null: InvocationTargetException: Logback configuration error detected:
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#4782f0a4 - There is no conversion class registered for conversion word [p]
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#4782f0a4 - [p] is not a valid conversion word
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#6071227e - There is no conversion class registered for conversion word [msg]
[ERROR] ERROR in ch.qos.logback.core.pattern.parser.Compiler#6071227e - [msg] is not a valid conversion word
How can I access a request header?
This configuration worked for me:
<encoder>
<charset>utf-8</charset>
<pattern>%t{yyyy-MM-dd HH:mm:ss,SSS} %h X-Forwarded-For: %header{X-Forwarded-For} "%r", Response status:%s, Bytes sent:%b, Response time:%D</pattern>
</encoder>
Name of the header goes in between curly braces, e.g. %header{Content-type} or %header{My-header}, or just %header if you want all headers to be logged.
Source:
https://logback.qos.ch/access.html#configuration
https://logback.qos.ch/manual/layouts.html#AccessPatternLayout
Please find below example with Spring
If you want to log http headers then you need to use MDC feature: http://logback.qos.ch/manual/mdc.html
create Filter:
#Component
class RequestHeaderFilterConfig : Filter {
private val xRequestId = "X-Request-Id"
override fun doFilter(request: ServletRequest?, response: ServletResponse?, chain: FilterChain?) {
val httpRequest = request as HttpServletRequest
MDC.put(xRequestId, httpRequest.getHeader(xRequestId))
chain?.doFilter(request, response)
}
override fun destroy() = MDC.remove(xRequestId)
}
Controller:
#PostMapping
fun operation(#RequestHeader(value = "X-Request-Id", required = false) xRequestId: String? = null): ResponseEntity<Output> {
....
}
logback.xml
<Pattern>%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level [X-Request-Id: %X{X-Request-Id}] %logger{36}.%M:%line - %msg%n</Pattern>
My application has lots of EJBs. The current bespoke Logger implementation creates a logger like this;
private static Logger logger = Logger.getInstance("SERVICE_NAME");
, and the logging will go into a file;
(path)/SERVICE_NAME/SERVICE_NAME.log
I want to replicate this behaviour with logback, but having real trouble grabbing the 'logger' name in the logback.xml configuration. It can be seen in the log encoder.pattern, i.e. "%d %-5level %logger{35} - %msg %n".
Any ideas how I can get this into a property/variable and then use it in the element?
I have a partial solution. If I create my own Discriminator, I can then use the Discriminator in the logback.xml to implement seperate-log-files-per-EJB.
Discriminator;
public class LoggerNameBasedDiscriminator implements Discriminator<ILoggingEvent> {
private static final String KEY = "loggerName";
private boolean started;
#Override
public String getDiscriminatingValue(ILoggingEvent iLoggingEvent) {
return iLoggingEvent.getLoggerName();
}
#Override
public String getKey() {
return KEY;
}
public void start() {
started = true;
}
public void stop() {
started = false;
}
public boolean isStarted() {
return started;
}
}
Then my logback.xml;
<configuration debug="true" scan="true" scanPeriod="30 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg</pattern>
</encoder>
</appender>
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="package.to.LoggerNameBasedDiscriminator"/>
<sift>
<appender name="FILE-${loggerName}" class="ch.qos.logback.core.FileAppender">
<FILE>path/to/logs/${loggerName}/${loggerName}.log</FILE>
<encoder>
<pattern>%d{HH:mm:ss.SSS} %-50(%level %logger{35}) %msg%n</pattern>
</encoder>
</appender>
</sift>
</appender>
<root level="debug">
<appender-ref ref="STDOUT" />
<appender-ref ref="SIFT" />
</root>
</configuration>
This solution seems to work, but now I have no time or size based log rotation!
Thanks to your example I implemented a solution for a loggername-based discriminator which routes different logger output to different files. Although the documentation of logback is so verbose, I couldn't find this essential information. You've surely found the solution mentioned by yayitswei already.
logback.xml:
[...]
<timestamp key="startTimestamp" datePattern="yyyy-MM-dd"/>
<timestamp key="folderTimestamp" datePattern="MM-yyyy"/>
<property name="LOGDIR" value="/var/log/spock" />
<appender name="SIFT" class="ch.qos.logback.classic.sift.SiftingAppender">
<discriminator class="com.enterprise.spock.LoggerNameBasedDiscriminator" />
<sift>
<appender name="FILE-${loggerName}" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOGDIR}/${loggerName}-${startTimestamp}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOGDIR}/${folderTimestamp}/${loggerName}-%d{yyyy-MM-dd}-%i.log.gz</fileNamePattern>
<maxFileSize>500KB</maxFileSize>
<maxHistory>100</maxHistory>
<totalSizeCap>50MB</totalSizeCap>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>%level %date{HH:mm:ss.SSS}: %msg %n</pattern>
</encoder>
</appender>
</sift>
</appender>
[...]
Edit:
I replaced TimeBasedRollingPolicy with SizeAndTimeBasedRollingPolicy as proposed here. You'll need at least logback 1.1.7 for that.
What you get with this, is a logfile for each Logger created.
Every logfile will look like this: /var/log/loggername-2017-08-03.log.
When about 500KB was written to the file, it will be archived as a gz-zipfile into /var/log/loggername/08-2017/loggername-2017-08-03-0.log.gz.
The 0 at the end of the gz-zipfile-name is the %i from the <fileNamePattern> above. Without the %i it won't work. Remember to use <configuration debug=true> in logback.xml if something won't work.