SMTPAppender for log4j -- can't get it to work - configuration

I'm tying to add the SMTPAppender to an application for error notification (tomcat app). I haven't been able to get it to work up to this point. I have tried it using configuration both in the log4j.properties file as well as through the log4j.xml configuration. I'm not seeing any errors in the log related to a misconfiguration of the appender setup. I'm just not seeing any email attempts at all. Initially I has to bring in javax.mail and javax.activation so I know that it's attempting to set up and use the SMTPAppender, but I'm not seeing anything as far as miconfiguration errors or emails that are sent when errors occur.
Here's the log4j.properties that I tried (changed the private info)
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%-5p %d [%t] %c: %m%n
log4j.appender.email=org.apache.log4j.net.SMTPAppender
log4j.appender.email.Threshold=ERROR
log4j.appender.email.BufferSize=512
log4j.appender.email.To=myemail#mydomain.com
log4j.appender.email.From=from#mydomain.com
log4j.appender.email.SMTPHost=mysmtphost
log4j.appender.email.Subject=MULE -- error
log4j.appender.email.layout=org.apache.log4j.PatternLayout
log4j.appender.email.layout.ConversionPattern=%5p [%t] (%F:%L) - %m%n
log4j.appender.email.SMTPUsername=mysmtpuser
log4j.appender.email.SMTPPassword=mysmtppassword
Here's the configuation that I tried in the log4j.xml file
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p (%F:%L) - %m%n" />
</layout>
</appender>
<appender name="file" class="org.apache.log4j.RollingFileAppender">
<param name="File" value="/apps/data/pbymuleservices.log"/>
<param name="Append" value="true"/>
<param name="MaxFileSize" value="1000KB"/>
<param name="MaxBackupIndex" value="10"/>
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p [%d{ISO8601}] (%F:%L) - %m%n"/>
</layout>
</appender>
<appender name="email" class="org.apache.log4j.net.SMTPAppender">
<param name="BufferSize" value="512" />
<param name="SMTPHost" value="mysmtphost" />
<param name="SMTPUsername" value="mysmtpusername" />
<param name="SMTPPassword" value="mysmtppassword" />
<param name="From" value="fromemail#mydomain.com" />
<param name="To" value="toemail#mydomain.com" />
<param name="Threshold" value="error" />
<param name="Subject" value="MULE -- Error" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="[%d{ISO8601}]%n%n%-5p%n%n%c%n%n%m%n%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
<param name="LevelMin" value="error" />
<param name="LevelMax" value="fatal" />
</filter>
</appender>
<logger name="com.mytoplevel.package" additivity="false">
<level value="INFO"/>
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="org.mule" additivity="false">
<level value="WARN"/>
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<logger name="com.mulesource" additivity="false">
<level value="WARN"/>
<appender-ref ref="console"/>
<appender-ref ref="file"/>
</logger>
<root>
<priority value="error" />
<appender-ref ref="console" />
<appender-ref ref="file"/>
<appender-ref ref="email"/>
</root>
Anyone see anything that should keep this from working? I'm not seeing any configuration related errors in the log, but I'm also not seeing emails when errors occur.
Thanks

Add this param to the CATALINA_OPTS env variable: -Dlog4j.debug It will print where it's picking up the logging configuration from.

Related

Logback-spring.xml unable to squash multiline stack trace to single line

I tried multiple solutions on SO, but none of them seem to be working for me. I am fairly new to Logback too, probably why I'm unable to find the right approach for my usecase.
Basically, we push logs to a centrally managed log service where we are required to send stack traces in a single line to avoid increasing the rate of ingestion (each line is a new log in the log service) and to make debugging easier.
Sample ERROR log:
14:50:41.856 ERROR [] 8 --- [trace=394a5287-6719-41b3-91e3-47b68b356856,span=394a5287-6719-41b3-91e3-47b68b356856] [jetty-274] c.e.f.s.c.a.controllers.BaseController : Error while processing request:
com.ExceptionClass : An exception occurred
at com.SomeClass.somefunction(SomeClass:237)
and so on, I want to have these all in a single statement
Here's how my logback-spring.xml file looks like
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<!-- Read https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-logging.html -->
<!-- Taken from https://github.com/spring-projects/spring-boot/blob/master/spring-boot/src/main/resources/org/springframework/boot/logging/logback/defaults.xml -->
<conversionRule conversionWord="clr"
converterClass="org.springframework.boot.logging.logback.ColorConverter" />
<conversionRule conversionWord="ex"
converterClass="org.springframework.boot.logging.logback.ThrowableProxyConverter" />
<conversionRule conversionWord="wex"
converterClass="org.springframework.boot.logging.logback.WhitespaceThrowableProxyConverter" />
<conversionRule conversionWord="wEx"
converterClass="org.springframework.boot.logging.logback.ExtendedWhitespaceThrowableProxyConverter" />
<!-- We use an additional new line %n at the end of every line so that read_mode=1
can be enabled for log service This enables us to add stack traces to the corresponding
error line automatically. See https://sites.google.com/a/flipkart.com/logsvc/home/components/ingestion-frontend/file-based-handoff/nuances -->
<property name="CONSOLE_LOG_PATTERN"
value="%n%clr(%d{HH:mm:ss.SSS}){faint} %highlight(${LOG_LEVEL_PATTERN:-%5p}) [%marker] %clr(${PID:- }){magenta} %clr(---){faint} [trace=%X{X-Trace-Id:-},span=%X{X-Span-Id:-}] %magenta([%t]){faint} %yellow(%-40.40logger{39}){cyan}%clr(:){faint} %m${LOG_EXCEPTION_CONVERSION_WORD:-%throwable}%n" />
<appender name="DEBUG_LEVEL_REMAPPER"
class="org.springframework.boot.logging.logback.LevelRemappingAppender">
<destinationLogger>org.springframework.boot</destinationLogger>
</appender>
<logger name="org.apache.catalina.startup.DigesterFactory" level="ERROR" />
<logger name="org.apache.catalina.util.LifecycleBase" level="ERROR" />
<logger name="org.apache.coyote.http11.Http11NioProtocol" level="WARN" />
<logger name="org.apache.sshd.common.util.SecurityUtils" level="WARN" />
<logger name="org.apache.tomcat.util.net.NioSelectorPool" level="WARN" />
<logger name="org.crsh.plugin" level="WARN" />
<logger name="org.crsh.ssh" level="WARN" />
<logger name="org.eclipse.jetty.util.component.AbstractLifeCycle"
level="ERROR" />
<logger name="org.hibernate.validator.internal.util.Version" level="WARN" />
<logger
name="org.springframework.boot.actuate.autoconfigure.CrshAutoConfiguration"
level="WARN" />
<logger name="org.springframework.boot.actuate.endpoint.jmx"
additivity="false">
<appender-ref ref="DEBUG_LEVEL_REMAPPER" />
</logger>
<!-- http://logback.qos.ch/manual/appenders.html -->
<!-- Taken from https://github.com/spring-projects/spring-boot/blob/master/spring-boot/src/main/resources/org/springframework/boot/logging/logback/console-appender.xml -->
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>${CONSOLE_LOG_PATTERN}</pattern>
<charset>utf8</charset>
</encoder>
</appender>
<!-- Log to the console in addition to the log file -->
<root level="INFO">
<appender-ref ref="CONSOLE" />
</root>
</configuration>
I know that I need to update this line
<property name="CONSOLE_LOG_PATTERN"
value="%n%clr(%d{HH:mm:ss.SSS}){faint} %highlight(${LOG_LEVEL_PATTERN:-%5p}) [%marker] %clr(${PID:- }){magenta} %clr(---){faint} [trace=%X{X-Trace-Id:-},span=%X{X-Span-Id:-}] %magenta([%t]){faint} %yellow(%-40.40logger{39}){cyan}%clr(:){faint} %m${LOG_EXCEPTION_CONVERSION_WORD:-%throwable}%n" />
but I can't figure out how.
Here's what I tried
<property name="CONSOLE_LOG_PATTERN"
value="%n%clr(%d{HH:mm:ss.SSS}){faint} %highlight(${LOG_LEVEL_PATTERN:-%5p}) [%marker] %clr(${PID:- }){magenta} %clr(---){faint} [trace=%X{X-Trace-Id:-},span=%X{X-Span-Id:-}] %magenta([%t]){faint} %yellow(%-40.40logger{39}){cyan}%clr(:){faint} %m${LOG_EXCEPTION_CONVERSION_WORD:-%throwable} %replace(%msg){'\n', ''}%n" />
but it did not work as expected
Thanks in advance

Logback AsyncAppender not writing logs to underlying appenders

In logback.xml of my application I have an AsyncAppender defined as below.
<appender name="socketAppender" class="ch.qos.logback.classic.net.SocketAppender">
<param name="RemoteHost" value="127.0.0.1" />
<param name="Port" value="15000" />
<param name="ReconnectionDelay" value="10" />
<param name="Threshold" value="DEBUG" />
</appender>
<appender name="appLog" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>logs/myApp.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>logs/myApp.%d{yyyyMMdd.HH}00.log</fileNamePattern>
</rollingPolicy>
<encoder>
<charset>UTF-8</charset>
<pattern>[%d{yyyy/MM/dd HH:mm:ss.SSS}][%p][%c{0}] %m%n]</pattern>
</encoder>
</appender>
<appender name="AsyncLog" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="appLog" />
<appender-ref ref="socketAppender" />
</appender>
When I deploy the war in Tomcat on Windows 10.
The log server listening on port 15000 wasn't receiving logs.
So I moved up the socketAppender to first position like below.
<appender name="AsyncLog" class="ch.qos.logback.classic.AsyncAppender">
<appender-ref ref="socketAppender" />
<appender-ref ref="appLog" />
</appender>
With this change log server started receiving logs, but the log file stopped writing/appending.
3) I also tried setting the queueSize and discardingThreshold properties on the appender, to no avail.
<discardingThreshold>0</discardingThreshold>
<queueSize>500</queueSize>
Can any logback experts please tell me what I am doing wrong here? Are there any other properties of AsyncAppender that may help fix this behaviour?
A colleague pointed me to this method in the class ch.qos.logback.core.AsyncAppenderBase. AsyncAppender can attach only one appender and ignores any more appenders added beyond that.
public void addAppender(Appender<E> newAppender) {
if (appenderCount == 0) {
appenderCount++;
addInfo("Attaching appender named [" + newAppender.getName() + "] to AsyncAppender.");
aai.addAppender(newAppender);
} else {
addWarn("One and only one appender may be attached to AsyncAppender.");
addWarn("Ignoring additional appender named [" + newAppender.getName() + "]");
}
}

Backlog multi appender with different level

How can I log everything to WARN to a file and the console, but my.package to DEBUG only for file?
I tried:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
<File>target/log/test-log</File>
<append>false</append>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
</appender>
<logger name="my.package" level="WARN">
<appender-ref ref="STDOUT"/>
</logger>
<logger name="my.package" level="DEBUG">
<appender-ref ref="FILE" />
</logger>
<root level="WARN">
<appender-ref ref="STDOUT" />
<appender-ref ref="FILE" />
</root>
</configuration>
But it would log everything to DEBUG in the console too.
This also doesn't work:
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
<File>target/log/test-log</File>
<append>false</append>
</appender>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n</pattern>
</encoder>
</appender>
<logger name="io.chumps">
<appender-ref ref="STDOUT" level="WARN"/>
<appender-ref ref="FILE" level="DEBUG" />
</logger>
<root>
<appender-ref ref="STDOUT" level="WARN" />
<appender-ref ref="FILE" level="WARN" />
</root>
</configuration>
Solved it using GEventEvaluator with custom Groovy scripting:
http://logback.qos.ch/manual/filters.html#GEventEvaluator
<configuration>
<property name="LAYOUT" value="%d{HH:mm:ss.SSS} [%thread] %-5level %logger{5} - %msg%n" />
<root>
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.GEventEvaluator">
<expression>e.level.toInt() >= WARN.toInt()</expression>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
<encoder>
<pattern>${LAYOUT}</pattern>
</encoder>
</appender>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<filter class="ch.qos.logback.core.filter.EvaluatorFilter">
<evaluator class="ch.qos.logback.classic.boolex.GEventEvaluator">
<expression>
e.level.toInt() >= WARN.toInt() ||
(e.level.toInt() >= TRACE.toInt() && e.getLoggerName().startsWith("my.package"))
</expression>
</evaluator>
<OnMatch>ACCEPT</OnMatch>
<OnMismatch>DENY</OnMismatch>
</filter>
<encoder>
<pattern>${LAYOUT}</pattern>
</encoder>
<File>target/log/test-log</File>
<append>false</append>
</appender>
</root>
</configuration>
I can't believe this so simple usecase is not supported by default.

log4net - conversionPattern per level for single FileAppender?

While it is not difficult to setup different log files, each with a different conversionPattern, per level, I would like to have minimal logging for all but errors, where I'd like a detailed log entry. Here's a snippet of my current configuration:
<appender name="WarningsAndBelowFileAppender" type="log4net.Appender.FileAppender">
<file value="log.txt" />
<appendToFile value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%m%n" />
</layout>
<filter type="log4net.Filter.LevelRangeFilter">
<levelMax value="WARN" />
</filter>
</appender>
<appender name="ErrorsFileAppender" type="log4net.Appender.FileAppender">
<file value="errors.txt" />
<appendToFile value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%d [%t] %-5p %c [%x] - %m%n" />
</layout>
<threshold value="ERROR" />
</appender>
I'd love to have both of these configurations go into a single log file, just have different conversionPatterns for each. Can this be done?
you can't do it within the config file but you can make it with pattern layout converter.
See information here
This should be the converted code:
class MyMessageConverter : PatternLayoutConverter
{
protected override void Convert(TextWriter writer, LoggingEvent loggingEvent)
{
if (loggingEvent.Level.Value <= Level.Warn.Value)
{
writer.Write(loggingEvent.RenderedMessage);
}
else
{
writer.Write("{0} [{1}] {2,5} {3} - {4}",
loggingEvent.TimeStamp,
loggingEvent.ThreadName,
loggingEvent.Level.Name,
loggingEvent.LocationInformation.ClassName,
loggingEvent.RenderedMessage);
}
}
}
And here is the config file:
<appender name="FileAppender" type="log4net.Appender.FileAppender">
<file value="log.txt" />
<appendToFile value="false" />
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%MSG%n" />
<converter>
<name value="MSG" />
<type value="yournamespace.MyMessageConverter " />
</converter>
</layout>
</appender>

Logging into the file in Mule ESB

Can logging be done into the file in MULE ESB?.If so,please give me some examples and code snippets.Thanks in advance
Narayanan
You need to put log4j properties in your mule project inside main/resources directory. Here's sample log4j.xml that I am using with mule esb.
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j="http://jakarta.apache.org/log4j/">
<appender name="console" class="org.apache.log4j.ConsoleAppender">
<param name="Target" value="System.out" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd-MM-yyyy HH:mm:ss} %-5p [%t] [%c] - %m%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
<param name="LevelMin" value="ERROR" />
<param name="LevelMax" value="FATAL" />
<param name="AcceptOnMatch" value="true" />
</filter>
</appender>
<appender name="file" class="org.apache.log4j.DailyRollingFileAppender">
<param name="file" value="${mule.home}/logs/test.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd'.log'" />
<param name="Append" value="true" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d{dd-MM-yyyy HH:mm:ss} %-5p [%t] [%c] - %m%n" />
</layout>
<filter class="org.apache.log4j.varia.LevelRangeFilter">
<param name="LevelMin" value="INFO" />
<param name="LevelMax" value="FATAL" />
<param name="AcceptOnMatch" value="true" />
</filter>
</appender>
<root>
<priority value="DEBUG" />
<appender-ref ref="console" />
<appender-ref ref="file" />
</root>
</log4j:configuration>
Use the below log4j.xml file. Name the files as log4j.xml place it in the mule project classpath so that mule will pick this file when starting the app.
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="STDOUT" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%t] %C{2} (%F:%L) - %m%n" />
</layout>
</appender>
<appender name="FILE" class="org.apache.log4j.DailyRollingFileAppender">
<param name="File" value="c:/logs/gid/gid.log" />
<param name="DatePattern" value="'.'yyyy-MM-dd" />
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%d %-5p [%t] %C{2} (%F:%L) - %m%n" />
</layout>
</appender>
<root>
<level value="INFO" />
<appender-ref ref="FILE" />
</root>
</log4j:configuration>
Hope this helps you.
Thanks...
Yes we can save the entire logs in a file in any location. For that change the target(give the path where you want) in te log4j.xml file.
Since Mule 3.6 they changed the Log System from Log4j to Log4j2. Use the following content for your log4j2.xml file. Has been tested with Mule 3.8.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="Console" target="SYSTEM_OUT">
<PatternLayout pattern="%-5p %d [%t] %c: %m%n"/>
</Console>
</Appenders>
<Loggers>
<!-- CXF is used heavily by Mule for web services -->
<AsyncLogger name="org.apache.cxf" level="WARN"/>
<!-- Apache Commons tend to make a lot of noise which can clutter the log-->
<AsyncLogger name="org.apache" level="WARN"/>
<!-- Reduce startup noise -->
<AsyncLogger name="org.springframework.beans.factory" level="WARN"/>
<!-- Mule classes -->
<AsyncLogger name="org.mule" level="INFO"/>
<AsyncLogger name="com.mulesoft" level="INFO"/>
<AsyncRoot level="INFO">
<AppenderRef ref="Console"/>
</AsyncRoot>
</Loggers>
</Configuration>