how to limit usage of CPU when compress file by logback - logback

versions: logback 1.1.7
I use logback to compress logfiles. But excessive CPU utilization by compressing log files affects the execution efficiency of business code.
How to limit the CPU utilization of logback compressed files
<appender name="SYSTEM_APPENDER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${path}/${name}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${path}/system/${name}.%d{yyyy-MM-dd}.gz</fileNamePattern>
<maxHistory>${maxHistory}</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %logger{36} - %msg%n</pattern>
<charset class="java.nio.charset.Charset">UTF-8</charset>
</encoder>
</appender>
<appender name="APPENDER2"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${path2}/${name2}</file>
<rollingPolicy
class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${path2}/system/${name2}.%d{yyyy-MM-dd}.gz</fileNamePattern>
<maxHistory>${maxHistory}</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} %-5level %logger{36} - %msg%n</pattern>
<charset
class="java.nio.charset.Charset">UTF-8</charset>
</encoder>
</appender>

Related

SLF4J LogBack - multiple loggers and appanders

I need guidance for a logging requirement I am having.
My application got two services - upload and download.
I need to maintain separate logs for both - So far I have done below in logback.xml
<appender name="DownloadAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>download.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>download.log_%d{yyyy-MM-dd}_%i.zip</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator> <!-- defaults to type ch.qos.logback.classic.boolex.JaninoEventEvaluator -->
<expression>logger.equals("DownloadLogger")</expression>
</evaluator>
<OnMismatch>DENY</OnMismatch>
<OnMatch>ACCEPT</OnMatch>
</filter>
</appender>
<appender name="UploadAppender" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>upload.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>upload.log_%d{yyyy-MM-dd}_%i.zip</fileNamePattern>
<maxFileSize>100MB</maxFileSize>
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%date [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator> <!-- defaults to type ch.qos.logback.classic.boolex.JaninoEventEvaluator -->
<expression>logger.equals(UploadLogger")</expression>
</evaluator>
<OnMismatch>DENY</OnMismatch>
<OnMatch>ACCEPT</OnMatch>
</filter>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>
%d{yyyy-MM-dd HH:mm:ss} [%thread] %-5level %logger{36} - %msg%n
</pattern>
</encoder>
</appender>
<logger name="DownloadLogger" level="DEBUG" additivity="true">
<appender-ref ref="DownloadAppender" />
</logger>
<logger name="UploadLogger" level="DEBUG" additivity="true">
<appender-ref ref="UploadAppender" />
</logger>
<root level="DEBUG">
<appender-ref ref="STDOUT"/>
</root>
Usage of defined loggers in application is used by respective service classes.
ie classes of upload service obtain logger by
LoggerFactory.getLogger("UploadLogger")
and vice-versa for DownloadLogger.
Upon running the service I notice that only upload.log is created and filled in.
download.log gets created but no content are logged in it .
I suspect usage of filter is incorrect but I couldn't find any solution yet such that both log files can be maintained properly.
Can someone please help me how to correct the config?
The above config is fit for the purpose. The problem was in the way logback.xml was provided as command line argument.
The problem was resolved after providing logback.xml with -Dlogback.configurationFile

logback printing the logs in random manner

When the load is running the logs are not generated in order as per timestamp.
We are using the below logback configuration:
<appender name="messageResponse" class="ch.qos.logback.core.rolling.RollingFileAppender">
<append>true</append>
<file>message-response.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>message-response-%d{yyyy-MM-dd}.log.%i.gz</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>20MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>10</maxHistory>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>%msg%n</pattern>
</encoder>
</appender>

logback console display not honoring threshold filter

I have a logback.xml file containing the following:
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<param name="Target" value="System.out"/>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<encoder>
<pattern>%date %-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<appender name="debugfile" class="ch.qos.logback.core.rolling.RollingFileAppender" >
<file>debugFile.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>debugFile.%d{yyyy-MM-dd}.log</fileNamePattern>
<MaxHistory>30</MaxHistory>
</rollingPolicy>
<encoder>
<pattern>%date %-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<appender name="infofile" class="ch.qos.logback.core.rolling.RollingFileAppender" >
<file>infoFile.log</file>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>infoFile_%i.log</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>3</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>1MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%date %-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<appender name="warnfile" class="ch.qos.logback.core.rolling.RollingFileAppender" >
<file>infoFile.log</file>
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>WARN</level>
</filter>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>warnFile_%i.log</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>3</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>1MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%date %-4relative [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<!-- Setup the Root category -->
<root>
<appender-ref ref="console"/>
<appender-ref ref="debugfile"/>
<appender-ref ref="infofile"/>
<appender-ref ref="warnfile"/>
</root>
I want it to only show me INFO level logs or greater. So, I added the ThresholdFilter and set its level to INFO. Doing this worked for the RollingFileAppender (where one includes INFO and greater and the other includes DEBUG and greater).
This didn't make a difference as the console statements continue to include DEBUG level statements. Can anyone tell me why?
I can't guarentee that this will work, didn't actually test the setup.
Add element after your .
<logger name="ch.qos.logback" level="INFO">
<appender-ref ref="console" />
</logger>
That should modify the logging level of only the referenced appenders to the desired level.
See Logback docs for some more info on the thinking here.

logback rollingpolicy dont work (no backup file is created)

I try to use logback for a project and want to have a classic daily rollover.
I cant make it work : all logs keep in the default file.
Either I badly understood the rolling policy, or it dont work corectly. In fact I thought that I I run for instance my tomcat one minute after the last log, the "current" log file is backed up in another file...
For test purpose I tried a "minutely" rollover. Someone see something horrible in my config ? :
<appender name="dailyRollingFileAppender"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>C:/myProject/programs/apache-tomcat-8.0.21-presentation/logs/presentation-log.log
</File>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<FileNamePattern>C:/myProject/programs/apache-tomcat-8.0.21-presentation/logs/presentation-log.%d{yyyy-MM-dd_HH-mm}.log</FileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<!-- keep 1 day history -->
<maxHistory>1</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
<encoder>
<Pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{35} - %msg %n
</Pattern>
</encoder>
</appender>
Thanks!
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<appender name="dailyRollingFileAppender"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>${catalina.base}/logs/presentation-log.log
</File>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<FileNamePattern>${catalina.base}/logs/presentation-log.%d{yyyy-MM-dd_HH-mm}.log</FileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>10MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>30</maxHistory>
<cleanHistoryOnStart>true</cleanHistoryOnStart>
</rollingPolicy>
<encoder>
<Pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{35} - %msg %n
</Pattern>
</encoder>
</appender>
<logger name="com.cloudsviews.presentation" additivity="false">
<level value="WARN" />
<appender-ref ref="dailyRollingFileAppender" />
</logger>
<root>
<level value="WARN" />
<appender-ref ref="STDOUT" />
<appender-ref ref="dailyRollingFileAppender" />
</root>
</configuration>

OverlappingFileLockException in logback

I'm hitting the following OverlappingFileLockException in LogBack. Is there any know issues with prudent mode in this version of Logback?
logback-classic 1.0.7
logback-core 1.0.7
slf4j-api 1.7.2
slf4j-ext 1.7.2
11:16:59,252 |-ERROR in
ch.qos.logback.core.rolling.RollingFileAppender[FILE_LOG] - Appender
[FILE_LOG] failed to append.
java.nio.channels.OverlappingFileLockException
at java.nio.channels.OverlappingFileLockException
at at sun.nio.ch.SharedFileLockTable.checkList(FileLockTable.java:255)
at at sun.nio.ch.SharedFileLockTable.add(FileLockTable.java:152)
at at sun.nio.ch.FileChannelImpl.lock(FileChannelImpl.java:972)
at at java.nio.channels.FileChannel.lock(FileChannel.java:1052)
at at ch.qos.logback.core.FileAppender.safeWrite(FileAppender.java:187)
at at ch.qos.logback.core.FileAppender.writeOut(FileAppender.java:204)
at at ch.qos.logback.core.OutputStreamAppender.subAppend(OutputStreamAppender.java:212)
at at ch.qos.logback.core.rolling.RollingFileAppender.subAppend(RollingFileAppender.java:148)
at at ch.qos.logback.core.OutputStreamAppender.append(OutputStreamAppender.java:103)
at at ch.qos.logback.core.UnsynchronizedAppenderBase.doAppend(UnsynchronizedAppenderBase.java:88)
at at ch.qos.logback.core.spi.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:48)
at at ch.qos.logback.classic.Logger.appendLoopOnAppenders(Logger.java:280)
at at ch.qos.logback.classic.Logger.callAppenders(Logger.java:267)
at at ch.qos.logback.classic.Logger.buildLoggingEventAndAppend(Logger.java:449)
at at ch.qos.logback.classic.Logger.filterAndLog_0_Or3Plus(Logger.java:403)
at at ch.qos.logback.classic.Logger.log(Logger.java:803)
at at org.slf4j.ext.LoggerWrapper.info(LoggerWrapper.java:490)
<appender name="FILE_LOG" class="ch.qos.logback.core.rolling.RollingFileAppender">
<prudent>true</prudent>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>C:/MyLog-%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
<maxHistory>10</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%-20.20(%thread)] %-6level %-30.30logger{0} - [%-12.12(S:%mdc{pucid}]) %msg%n</pattern>
</encoder>
</appender>
On http://jira.qos.ch/browse/LOGBACK-864 Mahesh Chimanpure says: "timeBasedFileNamingAndTriggeringPolicy can not be used in prudent mode."