Log4j2 log size accumulates instead of rollover - multiple-instances

I have tried below config and found that the log rotation happens as expected and a new log file created for every rotation.
However when there are multiple instances of servers are running pointing to same log file. Once the log reaches the limit, the log gets backuped up, but instead of new file (0KB), new logs gets written to the same old file thus incresing the file size.
Looks like in multiple instances, where the file is locked by both instances for writing, the rollback is not proper.
Any workaround/suggestion? well having a new file for each instance is not really an option.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn" name="" packages="">
<Appenders>
<RollingRandomAccessFile name="RollingRandomAccessFile"
fileName="../logs/ws_new.log" append="false" filePattern=".../logs/ws_new-%d{MM-dd-yyyy}-%i.log">
<PatternLayout>
<Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
</PatternLayout>
<Policies>
<TimeBasedTriggeringPolicy />
<SizeBasedTriggeringPolicy size="50 KB" />
</Policies>
<DefaultRolloverStrategy max="3"/>
</RollingRandomAccessFile>
</Appenders>
<Loggers>
<root level="error">
<AppenderRef ref="RollingRandomAccessFile" />
</root>
<Logger name="org.exm.test" level="trace" additivity="false">
<AppenderRef ref="RollingRandomAccessFile"/>
</Logger>
</Loggers>
</Configuration>

Log4j is not designed to be used that way. If multiple processes try to write to the same log file the results cannot be guaranteed. You may end up losing log messages.
I strongly recommend you have separate log files for your processes.

Related

conditions in logback configuration

I'm trying to remove redundant logback config files by using janino's conditional processing.
Below is the conditional logic which I had added
<root level="INFO">
<appender-ref ref="ROLLING" />
<!-- use console appender on windows, email appender on linux -->
<if condition='property("os.name").contains("win")'>
<then>
<appender-ref ref="CONSOLE" />
</then>
<else>
<appender-ref ref="EMAIL" />
</else>
</if>
</root>
but this throws the below errors
12:30:34,877 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter#90:55 - no applicable action for [if], current pattern is [[configuration][root][if]]
12:30:34,877 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter#91:10 - no applicable action for [then], current pattern is [[configuration][root][if][then]]
12:30:34,877 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter#92:35 - no applicable action for [appender-ref], current pattern is [[configuration][root][if][then][appender-ref]]
12:30:34,877 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter#94:10 - no applicable action for [else], current pattern is [[configuration][root][if][else]]
12:30:34,877 |-ERROR in ch.qos.logback.core.joran.spi.Interpreter#95:35 - no applicable action for [appender-ref], current pattern is [[configuration][root][if][else][appender-ref]]
The configuration works fine if I remove the conditional logic and use something like
<root level="INFO">
<appender-ref ref="ROLLING" />
<appender-ref ref="CONSOLE" />
<appender-ref ref="EMAIL" />
</root>
How can I configure this correctly in such a way that the CONSOLE appender is used only on Windows and the EMAIL appender everywhere else?
Can you try the latest version? I used the latest version and the above conditional statements work perfectly fine

logback - remapping a log level for a specific logger

I have a logback configuration that has an appender with a threshold filter:
<appender name="SYSLOG" class="ch.qos.logback.classic.net.SyslogAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
...
</appender>
This ensures that only info and higher (warn, error) get logged to syslog. However, one of the 3rd party libraries we use is logging a particular event at DEBUG, and I would like to log this event to syslog. The first approach I had in mind was to try remap the log level in the logger, but am not sure if this is possible? Something like:
<logger name="akka.some.Thing" level="DEBUG" logAs="INFO">
<appender-ref ref="SYSLOG" />
</logger>
obviously, the "logAs" parameter doesn't exist, so I can't do that. What would be the best approach to logging akka.some.Thing to the SYSLOG appender while leaving the filter in place for other loggers?
The other approach would be to create a 2nd appender called SYSLOG2 that doesn't have the filter in place and set the specific logger to use that, but was wondering if there was a way to configure logback with just 1 SYSLOG appender...
Thanks,
I know this is an old question - but it is actually possible to do what the OP wants to do with a single SyslogAppender.
If others are searching for an example of how to remap you can take a look at the org.springframework.boot.logging.logback.LevelRemappingAppender class.
With that appender it is possible to both remap what appender is finally used for the log event, and it is also possible to remap the level that is used for the final log event - e.g. by changing a DEBUG level into an INFO level.
Usage example in logback config file (taken from https://github.com/spring-projects/spring-boot/blob/master/spring-boot/src/main/resources/org/springframework/boot/logging/logback/defaults.xml):
<appender name="DEBUG_LEVEL_REMAPPER" class="org.springframework.boot.logging.logback.LevelRemappingAppender">
<!-- Optional: specify the destination logger the event ends up in -->
<destinationLogger>org.springframework.boot</destinationLogger>
<!-- Optional: specify log level remapping -->
<remapLevels>INFO->DEBUG,ERROR->WARN</remapLevels>
</appender>
<logger name="org.thymeleaf" additivity="false">
<appender-ref ref="DEBUG_LEVEL_REMAPPER"/>
</logger>
Note that remapping to a specific destination logger can make it harder to find the source code of the original log event - so use it with care.
What you can do, is writing a second logger + appender with the same output:
<appender name="SYSLOG-2" class="ch.qos.logback.classic.net.SyslogAppender">
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>DEBUG</level>
</filter>
...
</appender>
<logger name="akka.some.Thing" level="DEBUG">
<appender-ref ref="SYSLOG-2" />
</logger>
This will add your specific DEBUG tasks to the same output.

How to configure logback for package?

Is it possible to configure logback to log at e.g. WARN or INFO level for all packages but x.y? And then separate configuration for package x.y only.
I don't know of any way you can get the "NOT" package aspect of your question, but I routinely log one package at DEBUG and all the rest at INFO and...above is it...WARN and ERROR. This is straight up-the-middle logback. My loggers are all like...
package rekdev.org.service.api;
public class DefaultConfigResource {
// ...
private static final Logger log = LoggerFactory.getLogger( DefaultConfigResource.class );
// ...
}
...on a logback.xml configuration like...
<logger name="rekdev.org.service.api" level="debug" />
...
<root level="info">
<appender-ref ref="STDOUT" />
<appender-ref ref="DAILY_ROLLING" />
<appender-ref ref="SYSLOG" />
</root>
Has the effect of most output popping out at INFO, WARN, ERROR but all the rekdev.org.service.api classes. All the classes in the rekdev.org.service.api package produce output at DEBUG, INFO, WARN, ERROR.
Or did I misunderstand your questions entirely?

How to prevent logback from outputting its own status at the start of every log when using a layout

This seems like a carelessness error, but I can't seem to find the cause. Logging with logback/slf4j (most recent version slf4j-api-1.6.1, logback core/classic 0.9.24). Simplest log configuration for testing is:
<configuration>
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<!-- DONT USE THIS FORMATTER FOR LIVE LOGGING THE %L LINE NUMBER OUTPUTTER IS SLOW -->
<pattern>%le %-1r [%c{1}:%L] %m%n</pattern>
</layout>
</appender>
<root level="DEBUG">
<appender-ref ref="stdout" />
</root>
</configuration>
Every log setup starts with logback's internal status lines:
11:21:27,825 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
11:21:27,826 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback-test.xml] at [file:.../logback-test.xml]
11:21:28,116 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - debug attribute not set
11:21:28,124 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.ConsoleAppender]
11:21:28,129 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [stdout]
11:21:28,180 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Pushing component [layout] on top of the object stack.
11:21:28,206 |-WARN in ch.qos.logback.core.ConsoleAppender[stdout] - This appender no longer admits a layout as a sub-component, set an encoder instead.
11:21:28,206 |-WARN in ch.qos.logback.core.ConsoleAppender[stdout] - To ensure compatibility, wrapping your layout in LayoutWrappingEncoder.
11:21:28,206 |-WARN in ch.qos.logback.core.ConsoleAppender[stdout] - See also http://logback.qos.ch/codes.html#layoutInsteadOfEncoder for details
11:21:28,207 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to DEBUG
11:21:28,207 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [stdout] to Logger[ROOT]
which is, according to the docs, the format logback uses for default. It then finishes reading the config (which is set up to output a different format) and continues with the properly formatted output. There's a config parameter <configuration debug="false"> which does not affect this.
Anyone know how to shut this off?
If you set the debug attribute of the configuration element to true, you will get all status information to the console. If this is your problem, just set it to false or remove it.
If you have any configuration problems of level WARN or above, you will also get all status information logged to the console (including messages of level INFO). The best solution to this problem is to fix the problem (in your case replace the <layout> element with an <encoder> element).
If you for some reason cannot fix the problem, but want to remove the status-information from the console, you can instead configure an alternative StatusListener. Use the NopStatusListener to completely remove the status-information:
<configuration>
<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
<!-- etc -->
</configuration>
As described in the docs, if warnings or errors occur during the parsing of the configuration file, logback will automatically print status data on the console.
Follow http://logback.qos.ch/codes.html#layoutInsteadOfEncoder i.e.
the link mentioned by logback in its warning message. Once you follow the steps mentioned therein, that is, if you replace <layout> element with <encoder>, logback will stop printing messages on the console.
Ceki answer is correct:
(...)if warnings or errors occur during the parsing of the configuration file, logback will automatically print status data on the console.
Once you get it right, there won't be any pollution in the first lines of your log anymore.
As of March 2015, in Logback 1.1.2, you need to use <encoder> sub-component - <layout> is now deprecated and if use it, error messages will appear. You cannot control this, it´s Logback default behavior.
Some internal classes have been renamed too, and even the examples in their manual page are outdated!
Here is the code snippet from their Errors Code Help page, which has the correct way to config the logger. This fixed the issue completely in my project.
http://logback.qos.ch/codes.html#layoutInsteadOfEncoder
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>testFile.log</file>
...
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<pattern>%msg%n</pattern>
</encoder>
</appender>
I had the same problem i added this line
<!-- Stop output INFO at start -->
<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
in the logback and it succefully worked
I realized Steve found the fix but he didn't mention it on the thread. In case if any other person hit the same issue here is the fix.
Replace "<layout>" elements with "<encoder>..</encoder>"
The culprit is:
<layout class="ch.qos.logback.classic.PatternLayout">
I prefer to use status listener in order to switch off own logback logs:
<configuration>
<statusListener class="ch.qos.logback.core.status.NopStatusListener" />
...
</configuration>
But as was mentioned NopStatusListener also prevents showing warning and errors. So you can write your custom status listener and change log level for it manually:
package com.your.package;
import ch.qos.logback.core.status.OnConsoleStatusListener;
import ch.qos.logback.core.status.Status;
import java.util.List;
public class PrintOnlyWarningLogbackStatusListener extends OnConsoleStatusListener {
private static final int LOG_LEVEL = Status.WARN;
#Override
public void addStatusEvent(Status status) {
if (status.getLevel() == LOG_LEVEL) {
super.addStatusEvent(status);
}
}
#Override
public void start() {
final List<Status> statuses = context.getStatusManager().getCopyOfStatusList();
for (Status status : statuses) {
if (status.getLevel() == LOG_LEVEL) {
super.start();
}
}
}
}
Then use it in your logback.xml file:
<configuration>
<statusListener class="com.your.package.PrintOnlyWarningLogbackStatusListener" />
...
</configuration>
Struggled with the same problem myself i.e. there were a bunch of lines logged right at the beginning which were not related to my code. Here is how I fixed it.
<configuration debug="false">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<!-- <encoder> <pattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [%thread] %-5level
%logger{36} - %msg%n</pattern> </encoder> -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} %-5level %logger{10} - %msg%n</pattern>
</encoder>
</appender>
<root level="error">
<appender-ref ref="STDOUT" />
</root>
<logger name="fun.n.games" level="DEBUG" />
This is running with the following entry in the pom.xml
<dependency>
<groupId>ch.qos.logback</groupId>
<artifactId>logback-classic</artifactId>
<version>1.2.3</version>
</dependency>
This seems to be Fixed in 0.9.29. Just made several tests. No Joran INFO anymore. I guess this is the fixing commit.
I've tried everything and nothing worked for me. My problem was due to multiple logback.xml files in my classpath. This is the common case in multi modular projects.
When there is only one logback.xml file in classpath, there is no ambiguity and the problem is solved.
Using the logback.groovy: statusListener(NopStatusListener) (in the src/test/resources/logback.groovy) works.
(A valid use case is e.g. if working with ANT in Eclipse, using logback logging, groovy classes and unit tests where the unit tests take the src/test/resources/logback.groovy, but will also see the src/main/resources/logback.groovy (or similar) you cannot exclude (if ANT's classpath is said to use the projects classpath).)
Just for people landing here because of the status messages logged by ch.qos.logback.access.tomcat.LogbackValve#LogbackValve (recent versions). Just set the quiet flag:
var v = new LogbackValve();
v.setQuiet(true); // disable status messages!
v.setFilename("logback-access.xml");
See also the documentation for XML configuration.

log4net logger configuration

Is it possible to set the logger from configuration. I have a web app
using a framework. The framework is extensible and has the logger.
When I log, currently, the logger is set to the framework class.
Is it possible that I can configure my web app and set the logger for the web app to
loggerForWebApp and the logger for a console app (which is using the
same framework) to loggerForConsoleApp?
In addition to the root logger (which must always be there) you can have named loggers with their own appender-refs and levels.
For instance, you could have something like this:
<root>
....
</root>
<logger name="loggerForWebApp">
<level value="WARN" />
<appender-ref ... />
</logger>
<logger name="loggerForConsoleApp">
<level value="WARN" />
<appender-ref ... />
</logger>
In code, you would summon these loggers by their name:
var log = LogManager.GetLogger("loggerForWebApp");
Most definitely, and this is one of the great things about log4net: it can log out to a wide range of loggers.
For examples of the appenders, see here. Probably the most common one in use is the RollingFileAppender, but the ConsoleAppender can be very handy for console applications. Alternatively the TraceAppender can write out to the standard .NET trace listeners for further redirection (or display in the debug Output window in Visual Studio).
To create your own, implement IAppender.
details to follow