My app runs in Jetty8 and is leveraging Logback's RequestLogImpl (which is just great, by the way). Recently we discovered that if logback has a problem rolling over files, the logging message that would have alerted us to that appears nowhere because we had not configured Jetty to redirect its stderrout to any log file. My current release corrected that problem, but now I notice an excessive volume of INFO messages from logback in the jetty stderrout file like
06:32:14,893 |-INFO in c.q.l.co.rolling.helper.RenameUtil - Renaming file [/data/logs/md-stage-app4.dev.mgg.request.3.log] to [/data/logs/md-stage-app4.dev.mgg.request.4.log]
I only really care about these messages if the rename failed or something, which these messages come out as WARN. How can I get the logback stuff to just log at WARN and above into the jetty stderrout logfile?
My app itself does, indeed, <root level="info"> the root logger.
etc/jetty.xml has the following excerpt:
<!-- Logback Access Log implementation -->
<Ref id="RequestLog">
<Set name="requestLog">
<New id="requestLogImpl" class="ch.qos.logback.access.jetty.RequestLogImpl">
<Set name="fileName">etc/logbackAccess.xml</Set>
</New>
</Set>
</Ref>
etc/logbackAccess.xml is:
<configuration>
<!-- always a good activate OnConsoleStatusListener -->
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
<appender name="SIZE_ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>/data/logs/md-app3.request.log</File>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>/data/logs/md-app3.request.%i.log</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>5</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>1MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%h %l %u [%t] "%r" %s %b%n%fullRequest%n</pattern>
</encoder>
</appender>
<appender-ref ref="SIZE_ROLLING" />
</configuration>
The status messages are printed because of the line
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
in logback-access.xml (etc/logbackAccess.xml in your case). By adapting OnConsoleStatusListener and OnPrintStreamStatusListenerBase, you should be able able to create a custom StatusListener which prints status messages except INFO messages originating at RenameUtil.
Related
I have a Java web app that I run in Eclipse for my development environment. I use logback for logging to a custom file. The problem is certain logging statements, specifically those that traditionally go to catalina.out, do not end up in my log file. They do show up in my Eclipse console, but not in my custom logback log file.
When I run the same app in tomcat outside of Eclipse (via startup.bat), those catalina logging statements do get captured in a catalina.out file. But when running in Eclipse no catalina.out is created, so those logs don't persist.
Here's my logback-test.xml:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<!-- RollingFileAppender that rolls based on size and time -->
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${catalina.base}/logs/mylog.log</file>
<encoder>
<pattern>%date [%thread] %-5level %logger{10} [%file:%line] - %msg%n</pattern>
</encoder>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<!-- rollover daily -->
<fileNamePattern>${catalina.base}/logs/mylog.%d{yyyy-MM-dd}.%i.log</fileNamePattern>
<!-- each file at most 20MB; keep 10 files worth of history, max total 20GB -->
<maxFileSize>20MB</maxFileSize>
<maxHistory>10</maxHistory>
<totalSizeCap>20GB</totalSizeCap>
</rollingPolicy>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>logback-test %date [%thread] %highlight(%-5level) %logger{10} - %msg%n</pattern>
</encoder>
</appender>
<appender name="STDERR" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>logback-test %date [%thread] %highlight(%-5level) %logger{10} - %msg%n</pattern>
</encoder>
</appender>
<!-- set logging levels for specific packages -->
<logger name="org.apache.catalina.startup" level="INFO"/>
<!-- set level of the root logger and associate it with both appenders -->
<root level="INFO">
<appender-ref ref="FILE"/>
<appender-ref ref="STDOUT"/>
<appender-ref ref="STDERR"/>
</root>
</configuration>
This answer resolved my question: https://stackoverflow.com/a/5045247/3368818
Apparently, if, while running an app on tomcat via Eclipse, you want to capture the catalina logs in a file, you need to specify that via the Eclipse tomcat launch configuration settings.
I guess in a way this makes sense in that the catalina logs are perhaps not application specific, so maybe outside the scope of logback appenders. But on the other hand, shouldn't they get written to a log file by default anyway, just as they do if you launch your tomcat server outside of Eclipse?
I have no idea how to debug since there is no errors. I have this logback file but this made all the logs disappear for my cloudrun instance EXCEPT the system.out stuff that logback logs on startup..
<?xml version="1.0" encoding="UTF-8" ?>
<configuration scan="true" scanPeriod="30 seconds" debug="true">
<contextListener class="ch.qos.logback.classic.jul.LevelChangePropagator">
<resetJUL>true</resetJUL>
</contextListener>
<appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">
<!-- Optional : filter logs at or above a level -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
<flushLevel>WARN</flushLevel> <!-- Optional : default ERROR -->
</appender>
<appender name="ASYNC-SERVERLOG" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="CLOUD" />
</appender>
<!-- TBD for later use for apps that use log4jdbc it doesn't hurt apps that
don't use it -->
<logger name="jdbc.sqlonly" level="INFO" />
<logger name="jdbc.sqltiming" level="WARN" />
<logger name="jdbc.audit" level="WARN" />
<logger name="jdbc.resultset" level="WARN" />
<logger name="jdbc.connection" level="WARN" />
<root>
<level value="INFO" />
<appender-ref ref="ASYNC-SERVERLOG" />
</root>
</configuration>
Any ideas why logging stops working or where the logging is going to? Now that I changed logging to this, logging seems to go to the void :(.
I also tried out this post which logs to ConsoleAppender but NOTHING shows up in the json format although it is doing a line for EACH log statement I do which is really nice
GKE & Stackdriver: Java logback logging format?
I then tried this approach which is using FileAppender
How do I map my java app logging events to corresponding cloud logging event levels in GCP Felexible non-compat App Engine?
My logback for this one was
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>/var/log/app.log</file>
<append>true</append>
<encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
<layout class="com.orderlyhealth.GCPCloudLoggingJSONLayout">
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg</pattern>
</layout>
</encoder>
</appender>
<appender name="ASYNC-SERVERLOG" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="FILE" />
</appender>
Any ideas?
The Java logging instructions in the documentation should help: https://cloud.google.com/run/docs/logging#run_manual_logging-java
In Cloud Run, output to Stdout and Stderr are forwarded on to Stackdriver. Logs which are recorded in files are lost.
I'm not sure the implications of the ASYNC-SERVERLOG appender, but on Cloud Run there is only CPU available to process and send logs while a request is being processed, so if something is asynchronously sending logs it will only work during that request or a future request that reaches the same container instance.
I try configure logging via external file. WLS 10.3.6, startWebLogic.properties include:
-Dproject_name_home=D:\path\to\project_home ^
-Dlogging.config=${project_name_home}/conf/logback.xml ^
-Dlogging.path=${project_name_home}/log ^
-Dlogging.file=${project_name_home}/log/out.log ^
-Dorg.apache.cxf.Logger=org.apache.cxf.common.logging.Slf4jLogger
out.log was created in project home folder, but it is empty.
according to log logback.xml used inside application, not from external file:
19:05:27,621 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
19:05:27,622 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
19:05:27,622 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [zip:D:/Weblogic10.3.6/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/project_name/pjckx7/war/WEB-INF/lib/_wl_cls_gen.jar!/logback.xml]
What's wrong? Please help to configure logging with external file.
My external logback.xml is:
<configuration>
<property name="LOG_PATH" value="${project_home}/log" />
<property name="LOG_FILE" value="${LOG_PATH}/out.log" />
<appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/archive/out/out-%d{yyyy-MM-dd}.%i.zip</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
<encoder>
<pattern>%d{dd/MM/yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n%ex</pattern>
</encoder>
</appender>
<logger name="package.name" level="debug">
<appender-ref ref="ROLLING" />
</logger>
</configuration>
Logging configure inside application, application.yml is:
logging:
level:
package:
name: DEBUG
path: ${project_home}/logs
pattern:
console: "${CONSOLE_LOG_PATTERN:%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %maskedM%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}}"
file: "${FILE_LOG_PATTERN:%d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:%5p} ${PID:- } --- [%t] %-40.40logger{39} : %maskedM%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}}"
So, the next steps for logging configuration via external logback.xml are:
set logging.config (path to external logback.xml) in WLS start file.
also logging levels defined in application.yml need configure in WLS start file (logging.level. parameter, according to question need set logging.level.package.name parameter).
levels for another packages (not defined in application.yml) are configure in logback.xml.
other parameters (logging.path, logging.file) should be deleted.
all other parameters are configure at external logback.xml as before.
I am using tomcat server, maven, spring framework and logback.
I'm trying to use RollingFileAppender but I have a big problem. I need to log the contents of a web service request that is logged as system.out. When I start the server logs everything normally, both logs using org.slf4j.LoggerFactory as system.out, but when you turn the log at midnight, it generates a backup of the old log and created a new log. This precession is ok, but this new log, after the turn does not allow login with system.out and therefore fails to log the webservice request.
Below is my logback:
<configuration scan="true" scanPeriod="30 seconds">
<property name="LOG_PATH" value="${TOMCAT_HOME}/logs" />
<property name="LOG_PATH_BACKUP" value="${TOMCAT_HOME}/logs/backup" />
<appender name="SERVER" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_PATH}/server.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- rollover daily -->
<FileNamePattern>${LOG_PATH_BACKUP}/server/server.%d{yyyy-MM-dd}.log.gz</FileNamePattern>
</rollingPolicy>
<encoder>
<Pattern>%green(%d) %highlight(%-5level) %cyan(%logger{0}) mdc=%X{id}%msg %n</Pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="SERVER" />
</root>
</configuration>
What can I do to solve this problem?
I am looking to use Logback Classic DB Appender (ch.qos.logback.classic.db.DBAppender) in the application. Database is Sybase ASE. I use c3p0 connection pool.
Besides the DB appender, I also have a rolling file appender. Problem is, with the DB Appender I see a drastic decrease in performance. For instance, with only rolling file appender, it takes 13 ms (milli seconds) to log the messages to file. However, with rolling file appender and db appender together, it takes 4510 ms (4.5 seconds) to output the same set of messages to file and database.
Logback DBAppender document mentions that with c3p0 connection pool, it takes around 1ms to insert a single logging statement. Numbers I am seeing are thousand times more than that. Wonder what could be wrong. Following is my logback.xml file:
<configuration debug="true" scan="true">
<property resource="log.properties" />
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<!-- encoders are assigned the type ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${log.file.dir}/${project.artifactId}.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<!-- daily rollover -->
<fileNamePattern>${project.artifactId}.%d{yyyy-MM-dd}.log</fileNamePattern>
<!-- keep 30 days' worth of history -->
<maxHistory>${log.file.rolling.history.days}</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%-4relative [%thread] %-5level %logger{35} - %msg%n</pattern>
</encoder>
</appender>
<appender name="DB" class="ch.qos.logback.classic.db.DBAppender">
<connectionSource class="ch.qos.logback.core.db.DataSourceConnectionSource">
<dataSource class="com.mchange.v2.c3p0.ComboPooledDataSource">
<driverClass>${log.db.driver}</driverClass>
<jdbcUrl>jdbc:sybase:Tds:${log.db.server}:${log.db.port}/${log.db.name}</jdbcUrl>
<serverName>${log.db.server}</serverName>
<databaseName>${log.db.name}</databaseName>
<user>${log.db.user}</user>
<password>${log.db.password}</password>
</dataSource>
</connectionSource>
</appender>
<root level="INFO">
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE" />
</root>
</configuration>
Any pointers appreciated.
Thanks.
I found the following from logback manual:
If your JDBC driver supports the getGeneratedKeys method introduced in JDBC 3.0 specification, assuming you have created the appropriate database tables as mentioned above, then no more steps are required, except for the usual logback configuration.
Otherwise, there must be an SQLDialect appropriate for your database system. Currently, we have dialects for PostgreSQL, MySQL, Oracle and MS SQL Server.
for oracle, it said the supported version is :(10.2.0.1)
link:
http://logback.qos.ch/manual/appenders.html#DBAppender
If you have no a Dialect class in your JDBC driver, you can get from hibernate, and put it in your source, I don't know whether it can work.