Backlog multi appender with different level - logback

How can I log everything to WARN to a file and the console, but my.package to DEBUG only for file?
I tried:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
<File>target/log/test-log</File>
<append>false</append>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
</appender>
<logger name="my.package" level="WARN">
<appender-ref ref="STDOUT"/>
</logger>
<logger name="my.package" level="DEBUG">
<appender-ref ref="FILE" />
</logger>
<root level="WARN">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE" />
</root>
</configuration>
But it would log everything to DEBUG in the console too.
This also doesn't work:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
<File>target/log/test-log</File>
<append>false</append>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
</appender>
<logger name="io.chumps">
<appender-ref ref="STDOUT" level="WARN"/>
<appender-ref ref="FILE" level="DEBUG" />
</logger>
<root>
<appender-ref ref="STDOUT" level="WARN" />
<appender-ref ref="FILE" level="WARN" />
</root>
</configuration>

Solved it using GEventEvaluator with custom Groovy scripting:
http://logback.qos.ch/manual/filters.html#GEventEvaluator
<configuration>
<property name="LAYOUT" value="%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n" />
<root>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.GEventEvaluator">
<expression>e.level.toInt() >= WARN.toInt()</expression>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
<encoder>
<pattern>${LAYOUT}</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.GEventEvaluator">
<expression>
e.level.toInt() >= WARN.toInt() ||
(e.level.toInt() >= TRACE.toInt() && e.getLoggerName().startsWith("my.package"))
</expression>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
<encoder>
<pattern>${LAYOUT}</pattern>
</encoder>
<File>target/log/test-log</File>
<append>false</append>
</appender>
</root>
</configuration>
I can't believe this so simple usecase is not supported by default.

Related

Logback.xml is not generating the logs separately for error and debug in separate rollingfile

I want to generate logs in separate file for different level like error or debug. Means when root level is enable for Debug the logs should be generated in debug file only. And when root level is enable for error the logs should be generated in error file. But for my logback.xml configuration the logs are generating in both the files as the same time.
Please help.
=========Below is the logback.xml========
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="DEV_HOME" value="logs" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%-5level | %d{yyyy-MM-dd HH:mm:ss,SSS} [%thread] %logger{36} - %msg%n
</Pattern>
</layout>
<!-- filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>FATAL</level>
<onMatch>DENY</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>WARN</level>
<onMatch>DENY</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>INFO</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter-->
</appender>
<appender name="Recipe_DEBUG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/Recipe_DEBUG.log</file>
<filter class="ch.qos.logback.core.filter.ThresholdFilter">
<level>DEBUG</level>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%-5level | %d{yyyy-MM-dd HH:mm:ss,SSS} [%thread] %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/Recipe_DEBUG.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<appender name="Recipe_ERROR" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/Recipe_ERROR.log</file>
<filter class="ch.qos.logback.core.filter.ThresholdFilter">
<level>ERROR</level>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%-5level | %d{yyyy-MM-dd HH:mm:ss,SSS} [%thread] %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/Recipe_ERROR.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<root level="ERROR">
<appender-ref ref="Recipe_ERROR" level="ERROR" additivity="true"/>
<appender-ref ref="Recipe_DEBUG" level="DEBUG" additivity="true"/>
</root>
</configuration>
There were some issues in filter and root tag, correcting and attaching the corrected file.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="DEV_HOME" value="logs" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>
%-5level | %d{yyyy-MM-dd HH:mm:ss,SSS} [%thread] %logger{36} - %msg%n
</Pattern>
</layout>
<!-- filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>FATAL</level>
<onMatch>DENY</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>WARN</level>
<onMatch>DENY</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter>
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>INFO</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>NEUTRAL</onMismatch>
</filter-->
</appender>
<appender name="Recipe_DEBUG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/Recipe_DEBUG.log</file>
<append>true</append>
<!--filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>DEBUG</level>
</filter-->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>DEBUG</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%-5level | %d{yyyy-MM-dd HH:mm:ss,SSS} [%thread] %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/Recipe_DEBUG.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<appender name="Recipe_ERROR" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${DEV_HOME}/Recipe_ERROR.log</file>
<!--filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>ERROR</level>
</filter-->
<filter class="ch.qos.logback.classic.filter.LevelFilter">
<level>ERROR</level>
<onMatch>ACCEPT</onMatch>
<onMismatch>DENY</onMismatch>
</filter>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%-5level | %d{yyyy-MM-dd HH:mm:ss,SSS} [%thread] %logger{36} - %msg%n
</Pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${DEV_HOME}/archived/Recipe_ERROR.%d{yyyy-MM-dd}.%i.log
</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
</appender>
<!--additivity=false ensures this log data only goes to the this log, and no one more -->
<logger name="root" level="ERROR" additivity="false">
<appender-ref ref="Recipe_ERROR" />
<appender-ref ref="Recipe_DEBUG"/>
</logger>
</configuration>

How to use JSON appender in Logback?

I am trying to output all my log events in JSON format.
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<pattern>%d{yyyy-MM-dd'T'HH:mm:ss.SSSZ} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<!--<appender-ref ref="kafkaAppender" />-->
<appender-ref ref="STDOUT" />
</root>
Maven Dependency:
<dependency>
<groupId>net.logstash.logback</groupId>
<artifactId>logstash-logback-encoder</artifactId>
<version>4.7</version>
</dependency>
I see an error while running this configuration. Did i miss anything?
here is the stacktrace:
Exception in thread "main" java.lang.AbstractMethodError: net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder.headerBytes()[B
at ch.qos.logback.core.OutputStreamAppender.encoderInit(OutputStreamAppender.java:180)
at ch.qos.logback.core.OutputStreamAppender.setOutputStream(OutputStreamAppender.java:171)
at ch.qos.logback.core.ConsoleAppender.start(ConsoleAppender.java:81)
at ch.qos.logback.core.joran.action.AppenderAction.end(AppenderAction.java:90)
at ch.qos.logback.core.joran.spi.Interpreter.callEndAction(Interpreter.java:309)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:193)
at ch.qos.logback.core.joran.spi.Interpreter.endElement(Interpreter.java:179)
at ch.qos.logback.core.joran.spi.EventPlayer.play(EventPlayer.java:62)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:165)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:152)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:110)
at ch.qos.logback.core.joran.GenericConfigurator.doConfigure(GenericConfigurator.java:53)
at ch.qos.logback.classic.util.ContextInitializer.configureByResource(ContextInitializer.java:75)
at ch.qos.logback.classic.util.ContextInitializer.autoConfig(ContextInitializer.java:150)
at org.slf4j.impl.StaticLoggerBinder.init(StaticLoggerBinder.java:84)
at org.slf4j.impl.StaticLoggerBinder.<clinit>(StaticLoggerBinder.java:55)
at org.slf4j.LoggerFactory.bind(LoggerFactory.java:150)
at org.slf4j.LoggerFactory.performInitialization(LoggerFactory.java:124)
at org.slf4j.LoggerFactory.getILoggerFactory(LoggerFactory.java:412)
at org.slf4j.LoggerFactory.getLogger(LoggerFactory.java:357)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:155)
at org.apache.commons.logging.impl.SLF4JLogFactory.getInstance(SLF4JLogFactory.java:132)
at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:273)
I had encountered the same error while using 4.7 version of the logstash-logback-encoder, and upgrading to 4.11 resolved the issue.
Ref- https://github.com/thumbtack/becquerel/issues/2

How to switch defined appenders in logback

I have a defined logback in my project, and here is the configuration:
<?xml version="1.0" encoding="UTF-8"?>
<appender name="DAILY" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/logApp/application.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>/logApp/application.log.%d{yyyy-MM-dd}.%i.gz</fileNamePattern>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level %logger{36}:%line - %msg%n</pattern>
</encoder>
</appender>
<appender name="HOURLY" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>/logApp/application.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>/logApp/application.log.%d{yyyy-MM-dd-HH}.%i.gz</fileNamePattern>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36}:%line - %msg%n</pattern>
</encoder>
</appender>
<root level="info">
<appender-ref ref="DAILY" />
</root>
A you can see I only use daily appender in my root configuration. How can I switch between those defined appenders (HOURLY / DAILY) in runtime?
I have tried with the following code, but my newAppender is always null:
String loggingAppender = "HOURLY";
LoggerContext lc = (LoggerContext) LoggerFactory.getILoggerFactory();
ch.qos.logback.classic.Logger root = lc.getLogger(Logger.ROOT_LOGGER_NAME);
root.detachAndStopAllAppenders();
Appender<ILoggingEvent> newAppender = root.getAppender(loggingAppender);
root.addAppender(newAppender);

logback ThresholdFilter how todo the opposite

I need a filter that only passes through everything info and LESS instead of info and more. I have seen this done somewhere before in logback. My full xml is here and I want info and less to go to stdout (without writing a filter as I know that was possible somehow)...
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>info</level>
</filter>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date{ISO8601} %X{sessionid}-%X{user} %caller{1} %-4level: %message%n</pattern>
</layout>
</appender>
<appender name="STDERR" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>warn</level>
</filter>
<layout class="ch.qos.logback.classic.PatternLayout">
<pattern>%date{ISO8601} %X{sessionid}-%X{user} %caller{1} %-4level: %message%n</pattern>
</layout>
<target>System.err</target>
</appender>
figured it out finally....
<?xml version="1.0" encoding="UTF-8" ?>
<configuration scan="true" scanPeriod="30 seconds">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.GEventEvaluator">
<expression>
e.level.toInt() <= INFO.toInt()
</expression>
</evaluator>
<OnMismatch>DENY</OnMismatch>
<OnMatch>NEUTRAL</OnMatch>
</filter>
<encoder>
<pattern>%date{ISO8601} %X{sessionid}-%X{user} %caller{1} %-4level: %message%n</pattern>
</encoder>
</appender>
<appender name="STDERR" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>warn</level>
</filter>
<encoder>
<pattern>%date{ISO8601} %X{sessionid}-%X{user} %caller{1} %-4level: %message%n</pattern>
</encoder>
<target>System.err</target>
</appender>
<appender name="TEMP" class="com.alvazan.play.logging.CassandraAppender">
<appender-ref ref="STDOUT"/>
</appender>
<root>
<level value="INFO" />
<appender-ref ref="STDOUT"/>
<appender-ref ref="STDERR"/>
<!-- appender-ref ref="TEMP"/-->
</root>
</configuration>

SMTPAppender for log4j -- can't get it to work

I'm tying to add the SMTPAppender to an application for error notification (tomcat app). I haven't been able to get it to work up to this point. I have tried it using configuration both in the log4j.properties file as well as through the log4j.xml configuration. I'm not seeing any errors in the log related to a misconfiguration of the appender setup. I'm just not seeing any email attempts at all. Initially I has to bring in javax.mail and javax.activation so I know that it's attempting to set up and use the SMTPAppender, but I'm not seeing anything as far as miconfiguration errors or emails that are sent when errors occur.
Here's the log4j.properties that I tried (changed the private info)
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%-5p %d [%t] %c: %m%n
log4j.appender.email=org.apache.log4j.net.SMTPAppender
log4j.appender.email.Threshold=ERROR
log4j.appender.email.BufferSize=512
log4j.appender.email.To=myemail#mydomain.com
log4j.appender.email.From=from#mydomain.com
log4j.appender.email.SMTPHost=mysmtphost
log4j.appender.email.Subject=MULE -- error
log4j.appender.email.layout=org.apache.log4j.PatternLayout
log4j.appender.email.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
log4j.appender.email.SMTPUsername=mysmtpuser
log4j.appender.email.SMTPPassword=mysmtppassword
Here's the configuation that I tried in the log4j.xml file
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p (%F:%L) - %m%n" />
</layout>
</appender>
<appender name="file" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="/apps/data/pbymuleservices.log"/>
<param name="Append" value="true"/>
<param name="MaxFileSize" value="1000KB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p [%d{ISO8601}] (%F:%L) - %m%n"/>
</layout>
</appender>
<appender name="email" class="org.apache.log4j.net.SMTPAppender">
<param name="BufferSize" value="512" />
<param name="SMTPHost" value="mysmtphost" />
<param name="SMTPUsername" value="mysmtpusername" />
<param name="SMTPPassword" value="mysmtppassword" />
<param name="From" value="fromemail#mydomain.com" />
<param name="To" value="toemail#mydomain.com" />
<param name="Threshold" value="error" />
<param name="Subject" value="MULE -- Error" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="[%d{ISO8601}]%n%n%-5p%n%n%c%n%n%m%n%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
<param name="LevelMin" value="error" />
<param name="LevelMax" value="fatal" />
</filter>
</appender>
<logger name="com.mytoplevel.package" additivity="false">
<level value="INFO"/>
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="org.mule" additivity="false">
<level value="WARN"/>
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="com.mulesource" additivity="false">
<level value="WARN"/>
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<root>
<priority value="error" />
<appender-ref ref="console" />
<appender-ref ref="file"/>
<appender-ref ref="email"/>
</root>
Anyone see anything that should keep this from working? I'm not seeing any configuration related errors in the log, but I'm also not seeing emails when errors occur.
Thanks
Add this param to the CATALINA_OPTS env variable: -Dlog4j.debug It will print where it's picking up the logging configuration from.