How do I Log the Class file without the path in log4net - configuration

I want to be able to log the class file and line number in my log file so I am using the following config...
<appender name="RollingLogFileAppender" type="log4net.Appender.RollingFileAppender">
<!--etc-->
<layout type="log4net.Layout.PatternLayout">
<conversionPattern value="%date [%thread] %-5level (%file:%line) %logger => %message%newline" />
</layout>
</appender>
However the %file attribute is making my log file entries waaaay too long to read comfortably....
2009-08-07 16:41:55,271 [7] INFO (O:\mystream\aevpallsrv\DotNet\com.mycompany.au\myapp\Myappp\Controller.cs:75) MyApp.Controller => Controller.EnqueueWorkerThreads() - START
Is there a way to show just the class file ('Controller.cs') instead of the full path to the file also???
Michael

Although you're asking about the file name, I believe you can use %type to get the fully qualified type name (a.b.className). If you just want the class name, use %type{1}
Note that any method that generates caller information (%file and %type) have a performance cost associated with them.
As an aside, you can bypass the performance hit by naming the Logger using the Type name.
namespace MyNamespace
{
public class Foo
{
private static ILog log = LogManager.GetLogger(typeof(Foo));
}
}
Your conversion pattern would look like:
"%date [%thread] %-5level %logger %message"
Where your logger would be "MyNameSpace.Foo". Likewise, if you only want the class name, use "%logger{1}, which will resolve as "Foo".
One of the best advantages to this approach is that you can leverage log4net's hierarchical repository system to adjust logging levels per type:
<!-- all classes in MyNamespace are warn -->
<logger name="MyNamespace">
<level value="WARN" />
</logger>
<!-- only Foo is in debug -->
<logger name="MyNamespace.Foo">
<level value="DEBUG" />
</logger>

Out of the box, PatternLayout supports only the %file token. What you can do is subclass PatternLayout and add your own pattern, say %filename, and for this token only output the name of the file.

Related

How to configure different levels for different appenders but under same logger in logback

We are writing 2 appenders to write logs in different formats to 2 different files.
But we want to enable these logs based on some configuration.
So if the user wants to enable both of the formats, then both of logs will be printed. But if the user wants to disable one, that log should not be created.
Below is my logger configuration :
<logger name="package.name" additivity="false" level="DEBUG">
<appender-ref ref="STDOUT" />
<appender-ref ref="json_logs"/>
<appender-ref ref="text_logs"/>
</logger>
Now I want to put separate levels for this appender-ref. and value of these levels either should come from some property file which will be edited by user or user could simply update logback.xml file only.
I am not able to find a way to put separate levels for these appenders.
And since I have to write logs from same classes, I cant create 2 separate loggers too.
Also, if the user does not want to see txt logs, then the corresponding log.txt file should not be created.
What you are after is the ThresholdFilter which can be fine tuned for each appender:
http://logback.qos.ch/manual/filters.html#thresholdFilter
<appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
<!-- deny all events with a level below INFO, that is TRACE and DEBUG -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
...
</appender>
As for not writing the file at all based on some flag... you could write your own appender. Something like this ultra-simplistic example that could be refined in a number of ways but should hopefully get you started:
public class ConditionalAppender<E> extends OutputStreamAppender<E> {
private boolean masterSwitch;
private String file;
#Override
public void start() {
if(masterSwitch)
setOutputStream(new FileOutputStream(file));
else
setOutputStream(new NullOutputStream());
super.start();
}
public void setMasterSwitch(final boolean enabled) {
this.masterSwitch = enabled;
}
public void setFile(final String file) {
this.file = file;
}
}
<appender name="CONDITIONAL" class="mypackage.ConditionalAppender">
<!-- switch this appender on or off -->
<masterSwitch>true</masterSwitch>
<!-- set the output file -->
<file>/var/log/app/app.log</file>
<!-- deny all events with a level below INFO, that is TRACE and DEBUG -->
<filter class="ch.qos.logback.classic.filter.ThresholdFilter">
<level>INFO</level>
</filter>
...
</appender>
Hope this helps.

Output Spark application id in the logs with Log4j

I have a custom Log4j file for the Spark application. I would like to output Spark app id along with other attributes like message and date so the JSON string structure would look like this:
{"name":,"time":,"date":,"level":,"thread":,"message":,"app_id":}
Now, this structure looks like this:
{"name":,"time":,"date":,"level":,"thread":,"message":}
How can I define such layout for the Spark driver logs?
My log4j file looks like this:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration xmlns:log4j='http://jakarta.apache.org/log4j/'>
<appender name="Json" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.hadoop.log.Log4Json">
<param name="ConversionLayout" value=""/>
</layout>
</appender>
<root>
<level value="INFO"/>
<appender-ref ref="Json"/>
</root>
</log4j:configuration>
I doubt that org.apache.hadoop.log.Log4Json can be adjusted for this purpose. According to its javadoc and source code it might be rather cumbersome.
Although it looks like you are using Log4j 1x, its API is quite flexible and we can easily define our own layout by extending org.apache.log4j.Layout.
We'll need a case class that will be transformed into JSON according to the target structure:
case class LoggedMessage(name: String,
appId: String,
thread: String,
time: Long,
level: String,
message: String)
And Layout might be extended as follows. To access the value of "app_id", we'll use Log4j's Mapped Diagnostic Context
import org.apache.log4j.Layout
import org.apache.log4j.spi.LoggingEvent
import org.json4s.DefaultFormats
import org.json4s.native.Serialization.write
class JsonLoggingLayout extends Layout {
// required by the API
override def ignoresThrowable(): Boolean = false
// required by the API
override def activateOptions(): Unit = { /* nothing */ }
override def format(event: LoggingEvent): String = {
// we are using json4s for JSON serialization
implicit val formats = DefaultFormats
// retrieve app_id from Mapped Diagnostic Context
val appId = event.getMDC("app_id") match {
case null => "[no_app]" // logged messages outside our app
case defined: AnyRef => defined.toString
}
val message = LoggedMessage("TODO",
appId,
Thread.currentThread().getName,
event.getTimeStamp,
event.getLevel.toString,
event.getMessage.toString)
write(message) + "\n"
}
}
Finally, when the Spark session is created, we put the app_id value into MDC:
import org.apache.log4j.{Logger, MDC}
// create Spark session
MDC.put("app_id", session.sparkContext.applicationId)
logger.info("-------- this is info --------")
logger.warn("-------- THIS IS A WARNING --------")
logger.error("-------- !!! ERROR !!! --------")
This produces following logs:
{"name":"TODO","appId":"local-1550247707920","thread":"main","time":1550247708149,"level":"INFO","message":"-------- this is info --------"}
{"name":"TODO","appId":"local-1550247707920","thread":"main","time":1550247708150,"level":"WARN","message":"-------- THIS IS A WARNING --------"}
{"name":"TODO","appId":"local-1550247707920","thread":"main","time":1550247708150,"level":"ERROR","message":"-------- !!! ERROR !!! --------"}
And, of course, do not forget to refer the implementation in log4j config xml:
<appender name="Json" class="org.apache.log4j.ConsoleAppender">
<layout class="stackoverflow.q54706582.JsonLoggingLayout" />
</appender>

Camel way: JSON to Array of Object

Need to deserialize stringls like "[100,200]" into array of Integer to later use from:
<to uri="class:java.awt.Robot?method=mouseMove&multiParameterArray=true"/>
multiParameterArray=true is a key.
<unmarshal ref="json"/>
with
<dataFormats>
<json id="json" library="Jackson" allowJmsType="true"/>
</dataFormats>
and stuck. Working in restricted environment, can't create custom beans that perform the stuff. Need some standard camel way.
UPDATE 1
Seems this construct does exactly what I need. But I'd better use <unmarshal/> rather then it
<setBody>
<constant>[100,200]</constant>
</setBody>
<setBody>
<jsonpath>$.*</jsonpath>
</setBody>

logback RollingFileAppender rolling at fixed time

A typical logback appender configuration is
<appender name="NAME"
class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>FILEPATH.log</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>FILEPATH.%d{yyyy-MM-dd-HH}.log</fileNamePattern>
<maxHistory>24</maxHistory>
</rollingPolicy>
<encoder>
<pattern>%msg%n</pattern>
<charset>UTF-8</charset>
</encoder>
</appender>
Usually it works fine to me. But I found that it will not split log file as the rollingPolicy specified when there's no log appended to the logger respected.
Please tell me How I can configure it to split log file for every hours even there's no log record for some hours. If there's no log record for any hour, I need logback to create an empty file for that hour.
I was with the same problem and I guess logback doesn't able to do it per default.
I found this issue at logback backlog to handle exactly with this: https://jira.qos.ch/browse/LOGBACK-554 and it`s still open.
What I did for now was implement my own rolling appender policy, based on it: https://ronanquillevere.github.io/2015/08/04/rolling-log-15-min.html
You can extend the RollingFileAppender class and override the rollover method, like this:
public class CustomAppender<E> extends RollingFileAppender<E>
{
private static long start = System.currentTimeMillis();
private int rollOverTimeInMinutes = 60;
#Override
public void rollover()
{
long currentTime = System.currentTimeMillis();
int maxIntervalSinceLastLoggingInMillis = rollOverTimeInMinutes * 60 * 1000;
if ((currentTime - start) >= maxIntervalSinceLastLoggingInMillis)
{
super.rollover();
start = System.currentTimeMillis();
}
}
}
And on logback file xml config, you just set your custom pollicy on appender property:
<appender name="FILE" class="<your-package>.CustomAppender" >

How to log exceptions with network targets in NLog

I am using the NLog logging framework and am trying to get exception and stacktrace information showing up in any UDP logger appliaction, such as Sentinel and Log2Console, but can only get the log message part displayed. Outputting to a file works well as most examples do just that, so the problem revolves around using network targets with NLog.
Bonus if a custom format can be applied on inner exceptions and stacktrace, but this is not required. Exception.ToString() would go a long way.
Note on the example code: With Log2Console I found an article on how to send exception as a separate log entry. Although this worked, I was not happy with the solution.
Example exception logging code:
Logger Log = LogManager.GetCurrentClassLogger();
try
{
throw new InvalidOperationException("My ex", new FileNotFoundException("My inner ex1", new AccessViolationException("Innermost ex")));
}
catch (Exception e)
{
Log.ErrorException("TEST", e);
}
Example NLog.config:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<targets async="true">
<!-- Send by UDP to Sentinel with NLogViewer protocol -->
<target name="network" xsi:type="NLogViewer" address="udp://192.168.1.3:9999" layout="${message}${onexception:inner=${newline}${exception:format=tostring}}" />
<!-- Send message by UDP to Log2Console with Chainsaw protocol -->
<target name="network2" xsi:type="Chainsaw" address="udp://192.168.1.3:9998" appinfo="Grocelist"/>
<!-- Send exception/stacktrace by UDP to Log2Console with generic network protocol -->
<target name="network2ex" xsi:type="Network" address="udp4://192.168.1.3:9998" layout="${exception:format=ToString}" />
<target name="logfile" xsi:type="File" layout="${longdate}|${level:uppercase=true}|${logger}|${message}|${exception:format=tostring}"
createDirs="true"
fileName="${basedir}/logs/${shortdate}.log"
/>
</targets>
<rules>
<logger name="*" minlevel="Debug" writeTo="logfile" />
<logger name="*" minlevel="Debug" writeTo="network" />
<logger name="*" minlevel="Debug" writeTo="network2" />
<logger name="*" minlevel="Warn" writeTo="network2ex" />
</rules>
</nlog>
Some links:
http://nlog-project.org
http://nlog-project.org/wiki/Targets
http://nlog-project.org/wiki/Exception_layout_renderer
http://nlog-project.org/2011/04/20/exception-logging-enhancements.html
http://nlog-project.org/wiki/How_to_properly_log_exceptions%3F
How to tell NLog to log exceptions?
https://stackoverflow.com/a/9684111/134761
http://nlog-forum.1685105.n2.nabble.com/How-to-send-stacktrace-of-exceptions-to-Chainsaw-or-Log2Console-td5465045.html
Edit:
After searching some more this seems to be a limitation on NLog's end. A recent patch is apparently out there: log4jxmlevent does not render Exception
Edit2:
I rebuilt NLog with patch, but it did not seem to help in Sentinel or Log2Console apps. I might have to try log4net to make sure those apps really do support what I am trying to achieve.
Edit3:
I currently use string.Format() to join and format message and exception text myself. This works well, but is not what I'm looking for here.
You can also extend NLog to include exceptions for network logging.
Create an extended layout:
[Layout("Log4JXmlEventLayoutEx")]
public class Log4JXmlEventLayoutEx : Log4JXmlEventLayout
{
protected override string GetFormattedMessage(LogEventInfo logEvent)
{
string msg = logEvent.Message + " ${exception:format=Message,Type,ToString,StackTrace}";
msg = SimpleLayout.Evaluate(msg, logEvent);
LogEventInfo updatedInfo;
if (msg == logEvent.Message) {
updatedInfo = logEvent;
} else {
updatedInfo = new LogEventInfo(
logEvent.Level, logEvent.LoggerName,
logEvent.FormatProvider, msg,
logEvent.Parameters, logEvent.Exception);
}
return base.GetFormattedMessage(updatedInfo);
}
}
Create a target that uses that layout
[Target("NLogViewerEx")]
public class NLogViewerTargetEx : NLogViewerTarget
{
private readonly Log4JXmlEventLayoutEx layout = new Log4JXmlEventLayoutEx();
public override Layout Layout { get { return layout; } set {} }
}
Update NLog.config:
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<extensions>
<add assembly="Assembly.Name.That.Contains.Extended.Target"/>
</extensions>
<targets>
<target name="logViewer"
xsi:type="NLogViewerEx"
address="udp://localhost:7071">
</targets>
...
</nlog>
A few years later and this is pretty trivial, try adding
includeSourceInfo="true"
to your target file, so it looks like;
<target name="viewer"
xsi:type="NLogViewer"
includeSourceInfo="true"
address="udp://127.0.0.1:9999" />
Gives you Source File, Line, Class and Method info.
I had this problem, and just updated my NLog nuget package to 2.0.1.2
Now I have exceptions coming through to Log2Console just fine.
Have you tried the latest developer snapshot of Chainsaw? It will display stack traces and supports log4net/UDP appenders, and according to NLog you can use it as well:
http://nlog-project.org/wiki/Chainsaw_target
Try the latest developer snapshot, has a ton of features: http://people.apache.org/~sdeboy
Just download and build the latest (NLog-Build-2.0.0.2007-0-g72f6495) sources from GitHub: https://github.com/jkowalski/NLog/tree/
This issue is fixed there by NLog developer.
In your NLog.config modify the target like the following.
<target name="file" xsi:type="File" fileName="log.txt" layout="${longdate}:${message} ${exception:format=message,stacktrace:separator=*}" />
The part that you are looking for is
${exception:format=message,stacktrace:separator=*}
For more information on this look here.