Logback log level change not working - logback

I'm using an application called Apache Drill that uses logback for logging. I'm trying to edit it's shipped logback.xml configuration file to output debugging messages.
Here is the file, untouched
<?xml version="1.0" encoding="UTF-8" ?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<configuration>
<!-- <appender name="SOCKET"
class="de.huxhorn.lilith.logback.appender.ClassicMultiplexSocketAppender">
<Compressing>true</Compressing>
<ReconnectionDelay>10000</ReconnectionDelay>
<IncludeCallerData>true</IncludeCallerData>
<RemoteHosts>${LILITH_HOSTNAME:-localhost}</RemoteHosts>
</appender>
-->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
</pattern>
</encoder>
</appender>
<appender name="QUERY" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.query.path}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${log.query.path}.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>100MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.path}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>${log.path}.%i</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>10</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>100MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%date{ISO8601} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<logger name="org.apache.drill" additivity="false">
<level value="info" />
<appender-ref ref="FILE" />
</logger>
<logger name="query.logger" additivity="false">
<level value="info" />
<appender-ref ref="QUERY" />
<!-- <appender-ref ref="SOCKET" /> -->
</logger>
<!--
<logger name="org.apache.drill" additivity="false">
<level value="debug" />
<appender-ref ref="SOCKET" />
</logger>
-->
<root>
<level value="error" />
<appender-ref ref="STDOUT" />
</root>
</configuration>
All I did afterwards was editing
<logger name="org.apache.drill" additivity="false">
<level value="info" />
<appender-ref ref="FILE" />
</logger>
to be
<logger name="org.apache.drill" additivity="false">
<level value="debug" />
<appender-ref ref="FILE" />
</logger>
And absolutely nothing changed !
This is the java command used to start the application (Search for -cp /home/gelbana/drillv11/conf, the logback.xml file is located in the conf directory)
/usr/java/jdk1.8.0_112/bin/java -Xms10G -Xmx10G -XX:MaxDirectMemorySize=230G -XX:ReservedCodeCacheSize=1G -Ddrill.exec.enable-epoll=false -XX:MaxPermSize=512M -Djava.io.tmpdir=/home/gelbana/drillv11/tmp -Ddrill.memory.debug.allocator=true -XX:+CMSClassUnloadingEnabled -XX:+UseG1GC -Dlog.path=/home/gelbana/drillv11/log/drillbit.log -Dlog.query.path=/home/gelbana/drillv11/log/drillbit_queries.json -cp /home/gelbana/drillv11/conf:/home/gelbana/drillv11/jars/*:/home/gelbana/drillv11/jars/ext/*:/home/gelbana/drillv11/jars/3rdparty/*:/home/gelbana/drillv11/jars/classb/* org.apache.drill.exec.server.Drillbit
The drillbit.out file has the following lines at the beginning
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/gelbana/drillv11/jars/3rdparty/slf4j-simple-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/gelbana/drillv11/jars/classb/logback-classic-1.0.13.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.SimpleLoggerFactory]
I tried changing all levels to DEBUG, ALL but still, nothing changed. I can't find a single DBEUG in the logs.

The changes you made should work. I made the same change and ran a drillbit and I can see debug logs in drillbit.log. For example:
DEBUG o.a.drill.exec.server.StartupOptions - Parsing arguments.
The drillbit.out file should tell you something about how Logback configured itself. Have a look at that file, specifically the entries which relate to LoggerContext for example:
Found resource [logback.xml] at ...
Resource [logback.xml] occurs multiple times on the classpath
And you'll likely find that Logback relveals the answer via it's own logging.

Related

Logback - Different log file path base on the application profile

I am trying to use logback.xml for my project and for development i am using windows env and for deployment i am using unix env so i have created below xml file.
However when my application starts look like it's trying to validate the path specified. and it end up with below error.
how do i achieve this using logback?
java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.core.rolling.RollingFileAppender[UNIX_FILE] - Failed to create parent directories for [C:\as\Users\satishkn\apps\workspaceslogs\carats-api-2019-02-18.log]
ERROR in ch.qos.logback.core.rolling.RollingFileAppender[UNIX_FILE] - openFile(null,true) call failed. java.io.FileNotFoundException: \as\Users\satishkn\apps\workspaceslogs\carats-api-2019-02-18.log (The system cannot find the path specified)
at org.springframework.boot.logging.logback.LogbackLoggingSystem.loadConfiguration(LogbackLoggingSystem.java:169)
logback.xml
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<property name="windows_log_dir" value="./logs/" />
<property name="unix_log_dir" value="/as/Users/satishkn/apps/workspaceslogs/" />
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>
%d{dd-MM-yyyy HH:mm:ss.SSS} %magenta([%thread]) %highlight(%-5level) %logger.%M - %msg%n
</pattern>
</encoder>
</appender>
<appender name="WIN_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${windows_log_dir}carats-api-%d{yyyy-MM-dd}.log</fileNamePattern>
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M - %msg%n
</Pattern>
</encoder>
</appender>
<appender name="UNIX_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${unix_log_dir}carats-api-%d{yyyy-MM-dd}.log</fileNamePattern>
<maxHistory>30</maxHistory>
</rollingPolicy>
<encoder class="ch.qos.logback.classic.encoder.PatternLayoutEncoder">
<Pattern>
%d{dd-MM-yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36}.%M - %msg%n
</Pattern>
</encoder>
</appender>
<springProfile name="default">
<root level="info">
<appender-ref ref="STDOUT" />
<appender-ref ref="WIN_FILE" />
</root>
</springProfile>
<!-- <springProfile name="dev">
<root level="info">
<appender-ref ref="STDOUT" />
<appender-ref ref="UNIX_FILE" />
</root>
</springProfile>
<springProfile name="prod">
<root level="info">
<appender-ref ref="UNIX_FILE" />
</root>
</springProfile> -->
</configuration>
Using your configuration both appenders will be instanciated in both unix and windows environnement. You should use conditional configuration to achive your requirements.
I suggest you define a log_dir param this way :
<if condition='property("os.name").contains("win")'>
<then>
<property name="log_dir" value="./logs/" />
</then>
<else>
<property name="log_dir" value="/as/Users/satishkn/apps/workspaceslogs/" />
</else>
</if>
Then create a single appender that uses the log_dir property.

logback is not logging application log statements to neither console nor file

we are using logback for our logging and we have following jars in our class path
jcl-over-slf4j-1.7.7.jar
logback-classic-1.1.3.jar
logback-core-1.1.3.jar
slf4j-api-1.7.7.jar
janino-2.7.8.jar
In each app, i have minimal config in logback.xml. Like this
<configuration scan="true" scanPeriod="10 seconds">
<statusListener
class="ch.qos.logback.core.status.OnConsoleStatusListener" />
<contextName>myapp- ${HOSTNAME}</contextName>
<include file="${logback.path}/logback.xml"/>
</configuration>
Then in my file system, I have config like this
<included>
<property name="LOG_HOME" value="C:\\tmp" />
<appender name="stdout" class="ch.qos.logback.core.ConsoleAppender">
<layout class="ch.qos.logback.classic.PatternLayout">
<Pattern>%d{yyyy-MM-dd-HH:mm:ss.SSS} cn=%contextName [%thread] %-5level %logger{36} - %msg%n</Pattern>
</layout>
</appender>
<appender name="file" class="ch.qos.logback.core.rolling.RollingFileAppender">
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_HOME}/application.%d{yyyy-MM-dd-HH-mm}_%i.zip</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>250KB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
<encoder>
<pattern>%d{yyyy-MM-dd-HH:mm:ss.SSS} cn=%contextName [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<logger name="org.springframework" level="INFO"/>
<logger name="com" level="INFO"/>
<root level="INFO">
<appender-ref ref="stdout"/>
<appender-ref ref="file" />
</root>
Now when i deploy my app, i do see logs from spring framework in application.xxx.log file but my actual application code which logs some statement are not showing up in this log file
In my code, i m using slf4j logger factory to get the logger and then just log some dummy statements, like this
private static final Logger logger = LoggerFactory.getLogger(GameService.class);
logger.info("Playing cricket game.......");
I think i figured out the issue. Some where in our lib, we were using common logger util class which was modifying logger context object. As soon as i removed that dependency , things worked out fine. Since that was the old way of logging , we no longer needed common logger util.

Package-specific logging levels for different Logback appenders

I have this simple Logback config file, containing two appenders and some custom logging levels based on package name.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE xml>
<configuration>
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<charset>UTF-8</charset>
<pattern>%date{HH:mm}\t%-5level\t%msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>my_logger.log</file>
<encoder>
<charset>UTF-8</charset>
<pattern>%date{dd MMM HH:mm}|%class{0}|%-5level|%msg%n</pattern>
</encoder>
</appender>
<!-- custom logging levels -->
<logger name="myapp.package1" level="INFO" />
<logger name="myapp.package2" level="INFO" />
<root>
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE" />
</root>
</configuration>
With this configuration, the custom logging levels are applied for both appenders.
How can I change it so that only ConsoleAppender uses these custom logging levels, while FileAppender sticks to default levels?
If you don't want to implement a custom filter, you can create a new appender with a fixed threshold (in you case INFO):
<appender name="INFO_CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
...
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
</appender>
Then, for your custom logging levels add the INFO_CONSOLE and your FILE appender. The additivity="false" attribute prevents the logger from logging to the CONSOLE appender inherited from root.
<logger name="myapp.package1" additivity="false">
<appender-ref ref="INFO_CONSOLE" />
<appender-ref ref="FILE" />
</logger>
This should log DEBUG and above to the FILE and CONSOLE appenders, except for myapp.package1 which will only log INFO and above to CONSOLE.

Logback Configuration: Two appenders, logging based on package

I'm trying to configure logback so that certain packages are logged differently depending on the appender. I have a ConsoleAppender and a RollingFileAppender. I want TRACE level logging for the ConsoleAppender and I want only two packages logged to the RollingFileAppender. What ends up happening is my ConsoleAppender and RollingFileAppender behave exactly the same.
Here is my logback.xml:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE configuration>
<configuration>
<property resource="logback.properties" />
<appender name="consoleAppender" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>.%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg %n</pattern>
</encoder>
</appender>
<appender name="dailyRollingFileAppender"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${logback.dir}${logback.filename}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${logback.dir}${logback.filename}.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 90 days' worth of history -->
<maxHistory>90</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{35} - %msg %n</pattern>
</encoder>
</appender>
<logger name="org.springframework" level="INFO">
<appender-ref ref="dailyRollingFileAppender" />
</logger>
<logger name="x.y" level="DEBUG">
<appender-ref ref="dailyRollingFileAppender" />
</logger>
<root>
<level value="TRACE" />
<appender-ref ref="consoleAppender" />
</root>
</configuration>

Root Level Logging is effective in service mix

I am using slf4j & logback for logging and my application is deployed in Apache Service Mix where other modules are using my logging service. Everything works fine when i test it in a standalone environment but i face problem when i deploy it in a service mix container. At that time, only root level logging is effective and other logging levels are ignored.
Below is my logback-test.xml configuration, Kindly help i am stuck in this issue for the last 4 days.
<!-- This property describes the location of the property file. -->
<property
file="C:/Users/evikdew/ccl_code/log.properties" />
<!-- This appender prints on the console. -->
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n
</pattern>
</encoder>
</appender>
<!-- This appender contains the properties for the logs that would be written
to a file. -->
<appender name="FILE"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_DIRECTORY}\${LOG_FILE_NAME}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE_PATTERN}</fileNamePattern>
<maxHistory>${MAX_LOG_HISTORY}</maxHistory>
</rollingPolicy>
<encoder>
<pattern>${LOGGING_PATTERN}
</pattern>
</encoder>
</appender>
<logger name="com.ericsson" level="Error" additivity="false">
<appender-ref ref="FILE" />
</logger>
<logger name="com.ericsson" level="INFO" additivity="false">
<appender-ref ref="FILE" />
</logger>
<logger name="com.ericsson" level="DEBUG" additivity="false">
<appender-ref ref="FILE" />
</logger>
<root level="DEBUG">
<appender-ref ref="STDOUT" />
</root>
As Servicemix 4.x is based on top of Karaf, the standard logging mechanism is Pax Logging. Pax Logging is configured by the org.ops4j.pax.logging.cfg file, as Karaf uses Configuration Admin Service to configure OSGi services. That's why the config file still uses the log4j properties notation.
If you need special appenders, some are already included like a MDC appender for logging bundle wise, or a ZipRollingFileappender.
Support for the external logback configuration was introduced with Pax Loggin 1.7 I think, and I doubt it is already included in ServiceMix 4.5.x
So you have to stick to the log4j notation or exchange the Pax Logging versions yourself.