I am trying to configure logback based log masking for Apache Storm topologies.
When I try to replace logback.xml file inside Apache Storm log4j2- directory and update worker.xml and cluster.xml file, Apache Storm nimbus and supervisors are unable to understand logback based keywords.
Error:
2022-10-02 16:31:51,671 Log4j2-TF-1-ConfiguratonFileWatcher-2 ERROR Unable to locate appender "A1" for logger config "root"
2022-10-02 16:32:51,681 Log4j2-TF-7-ConfiguratonFileWatcher-4 ERROR Error processing element appender ([configuration: null]): CLASS_NOT
Sample cluster.xml file:
<configuration monitorInterval="60" shutdownHook="disable">
<properties>
<property name="pattern">%msg%n</property>
</properties>
<import class="ch.qos.logback.classic.encoder.PatternLayoutEncoder"/>
<import class="ch.qos.logback.core.FileAppender"/>
<FileAppender name="A1">
<file>logfilename.log</file>
<encoder>
<pattern>${pattern}</pattern>
</encoder>
</FileAppender>
<loggers>
<root level="info"> <!-- We log everything -->
<appender-ref ref="A1"/>
</root>
</loggers>
</configuration>
To my best knowledge, Apache Storm uses naturally log4j2, as also your logfile indicates. However, when I used log4j in Storm, I did not need to import any further classes. You also do not seem to use these logback-classes in the rest of your xml-file. So have you tried to simply remove those?
Related
I would like to configure logging appender based on the environment, for example while running in production I would like to configure an appender that would send log to elasticsearch but while on test or development mode this appender would not be enabled.
You can override the default logback config file by using "logback.configurationFile" system variable.
java -Dlogback.configurationFile=logback-prod.xml -jar your.jar
But, if what you need is the ability to use an env variable you can do this
without to use a third-party library:
Override the logback system variable inside the main micronaut class before call main, like following
#Singleton
public class MyMicronautApplication {
public static void main(String[] args) {
var env = System.getenv("MICRONAUT_ENVIRONMENTS");
if (env != null && !env.isEmpty()) {
System.setProperty(ContextInitializer.CONFIG_FILE_PROPERTY, "logback-" + env + ".xml");
}
Micronaut.run(MyMicronautApplication.class);
}
}
create you custom env based logback config file like: logback-dev.xml and put in resources dir.
the set env var MICRONAUT_ENVIRONMENTS=dev according to your deployment logic.
enjoy using logback-dev.xml, logback-prod.xml, logback-stagging.xml, etc
The work around i found was by doing conditional expressions in logback. You will need the following dependency
<!-- https://mvnrepository.com/artifact/org.codehaus.janino/janino -->
<dependency>
<groupId>org.codehaus.janino</groupId>
<artifactId>janino</artifactId>
<version>3.1.2</version>
</dependency>
Then in your logback.xml file, you can do a conditional statement such as following for selecting the appender you want to you use based on a micronaut profile. In my case, I wanted to activate the STDOUT appender if i was running the application locally but i did not want to activate the STDOUT profile if the app was running in any other environment such as dev or prod profiles, instead i wanted the RSYSLOG appender to be used.
<root level="info">
<if condition='property("MICRONAUT_ENVIRONMENTS").contains("local")'>
<then>
<appender-ref ref="STDOUT"/>
</then>
<else>
<appender-ref ref="RSYSLOG"/>
</else>
</if>
</root>
You can use conditional statements to configure other properties in your logback file.
As far I understand, Micronaut doesn't have similar thing like Spring boot ( ) implemented.
I think logback-production.xml (where production is profile
) doesn't work too - only logback.xml and logback-test.xml is suported.
I wasn't crazy about the idea of having multiple logback config files or pulling in another dependency (janino) to support this use-case.
You can also do this using environment variables.
In my logback.xml I defined 2 appenders, one for "DEV" and one for "PROD".
Then I dynamically select which appender to use via the LOG_TARGET variable. If the variable is not set then it defaults to "DEV".
<configuration>
<appender name="DEV" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%date{ISO8601} %-5level [%X{trace_id},%X{span_id}] [%thread] %logger{36} - %msg%n</pattern>
</encoder>
</appender>
<appender name="PROD" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder" />
</appender>
<appender name="OTEL" class="io.opentelemetry.instrumentation.logback.v1_0.OpenTelemetryAppender">
<appender-ref ref="${LOG_TARGET:-DEV}"/>
</appender>
<root level="info">
<appender-ref ref="OTEL"/>
</root>
</configuration>
I'm running Sonatype Nexus 3.15.0-01 and am a little stumped about how to override the default logback configs.
I created a file called 'logback-overrides.xml' in the 'nexus-data/etc/logback' folder containing the following:
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<appender name="FILE" class="ch.qos.logback.core.FileAppender">
<file>/nexus-data/log/myApp.log</file>
<encoder>
<pattern>%date %level [%thread] %logger{10} [%file:%line] %msg%n</pattern>
</encoder>
</appender>
<root level="debug">
<appender-ref ref="FILE" />
</root>
</configuration>
This is essentially just a simple bit of config that should cause logs to be written to '/nexus-data/log/myApp.log'. I restarted the server after adding this file, to confirm it would pick up the new configs.
However, when I check for that file, it's not present. What am I missing here?
I posted this same question on the Sonatype forums here. To sum up the answer I got there, it isn't possible to override the default logback config this way.
Possible workarounds are:
Create your own logback.xml file and build your own Docker image that extends Sonatype’s official image.
Create a volume mount for /opt/sonatype/nexus/etc/logback and customize the logback.xml on your host machine.
I try configure logging via external file. WLS 10.3.6, startWebLogic.properties include:
-Dproject_name_home=D:\path\to\project_home ^
-Dlogging.config=${project_name_home}/conf/logback.xml ^
-Dlogging.path=${project_name_home}/log ^
-Dlogging.file=${project_name_home}/log/out.log ^
-Dorg.apache.cxf.Logger=org.apache.cxf.common.logging.Slf4jLogger
out.log was created in project home folder, but it is empty.
according to log logback.xml used inside application, not from external file:
19:05:27,621 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback.groovy]
19:05:27,622 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Could NOT find resource [logback-test.xml]
19:05:27,622 |-INFO in ch.qos.logback.classic.LoggerContext[default] - Found resource [logback.xml] at [zip:D:/Weblogic10.3.6/user_projects/domains/base_domain/servers/AdminServer/tmp/_WL_user/project_name/pjckx7/war/WEB-INF/lib/_wl_cls_gen.jar!/logback.xml]
What's wrong? Please help to configure logging with external file.
My external logback.xml is:
<configuration>
<property name="LOG_PATH" value="${project_home}/log" />
<property name="LOG_FILE" value="${LOG_PATH}/out.log" />
<appender name="ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
<fileNamePattern>${LOG_PATH}/archive/out/out-%d{yyyy-MM-dd}.%i.zip</fileNamePattern>
<timeBasedFileNamingAndTriggeringPolicy
class="ch.qos.logback.core.rolling.SizeAndTimeBasedFNATP">
<maxFileSize>100MB</maxFileSize>
</timeBasedFileNamingAndTriggeringPolicy>
</rollingPolicy>
<encoder>
<pattern>%d{dd/MM/yyyy HH:mm:ss.SSS} [%thread] %-5level %logger{36} - %msg%n%ex</pattern>
</encoder>
</appender>
<logger name="package.name" level="debug">
<appender-ref ref="ROLLING" />
</logger>
</configuration>
Logging configure inside application, application.yml is:
logging:
level:
package:
name: DEBUG
path: ${project_home}/logs
pattern:
console: "${CONSOLE_LOG_PATTERN:%clr(%d{yyyy-MM-dd HH:mm:ss.SSS}){faint} %clr(${LOG_LEVEL_PATTERN:%5p}) %clr(${PID:- }){magenta} %clr(---){faint} %clr([%15.15t]){faint} %clr(%-40.40logger{39}){cyan} %clr(:){faint} %maskedM%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}}"
file: "${FILE_LOG_PATTERN:%d{yyyy-MM-dd HH:mm:ss.SSS} ${LOG_LEVEL_PATTERN:%5p} ${PID:- } --- [%t] %-40.40logger{39} : %maskedM%n${LOG_EXCEPTION_CONVERSION_WORD:%wEx}}"
So, the next steps for logging configuration via external logback.xml are:
set logging.config (path to external logback.xml) in WLS start file.
also logging levels defined in application.yml need configure in WLS start file (logging.level. parameter, according to question need set logging.level.package.name parameter).
levels for another packages (not defined in application.yml) are configure in logback.xml.
other parameters (logging.path, logging.file) should be deleted.
all other parameters are configure at external logback.xml as before.
I want to get hibernate conncection info (like driver_class, url, username, password) in logback file. (Not manually)
This belows is my logback file.
<appender name="dbAppender" class="ch.qos.logback.classic.db.DBAppender">
<append>false</append> //unfortunately it does not work.
<connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">
<driverClass>oracle.jdbc.driver.OracleDriver</driverClass>
<url>jdbc:oracle:thin:#localhost:1521:aaa</url>
<user>bbb</user>
<password>ccc</password>
</connectionSource>
</appender>
It works. But I want to get information about connection info(like driver class, url, user, password) from hibernate.cfg.xml automatically in logback.xml.
Your cooperation would be appreciated.
Please Help.
Thanks.
Getting the values out of your hibernate.cfg.xml would probably require writing some code. However, if you externalise all your hibernate properties in a properties file you can import them into your logback configuration.
Hibernate supports using a properties file called hibernate.properties in the root of the classpath, or you could pass an instance of java.util.Properties to Configuration.setProperties() when configuring your session factory.
If you can get your hibernate properties into a properties file you can use it from logback like so:
properties file on the classpath
<configuration>
<property resource="hibernate.properties" />
<appender name="dbAppender" class="ch.qos.logback.classic.db.DBAppender">
<append>false</append> //unfortunately it does not work.
<connectionSource class="ch.qos.logback.core.db.DriverManagerConnectionSource">
<driverClass>${hibernate.connection.driver_class}</driverClass>
<url>${hibernate.connection.url}</url>
<user>${hibernate.connection.username}</user>
<password>${hibernate.connection.password}</password>
</connectionSource>
</appender>
</configuration>
If your properties aren't on the classpath you can use <property file="path/to/some.properties" /> instead.
My app runs in Jetty8 and is leveraging Logback's RequestLogImpl (which is just great, by the way). Recently we discovered that if logback has a problem rolling over files, the logging message that would have alerted us to that appears nowhere because we had not configured Jetty to redirect its stderrout to any log file. My current release corrected that problem, but now I notice an excessive volume of INFO messages from logback in the jetty stderrout file like
06:32:14,893 |-INFO in c.q.l.co.rolling.helper.RenameUtil - Renaming file [/data/logs/md-stage-app4.dev.mgg.request.3.log] to [/data/logs/md-stage-app4.dev.mgg.request.4.log]
I only really care about these messages if the rename failed or something, which these messages come out as WARN. How can I get the logback stuff to just log at WARN and above into the jetty stderrout logfile?
My app itself does, indeed, <root level="info"> the root logger.
etc/jetty.xml has the following excerpt:
<!-- Logback Access Log implementation -->
<Ref id="RequestLog">
<Set name="requestLog">
<New id="requestLogImpl" class="ch.qos.logback.access.jetty.RequestLogImpl">
<Set name="fileName">etc/logbackAccess.xml</Set>
</New>
</Set>
</Ref>
etc/logbackAccess.xml is:
<configuration>
<!-- always a good activate OnConsoleStatusListener -->
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
<appender name="SIZE_ROLLING" class="ch.qos.logback.core.rolling.RollingFileAppender">
<File>/data/logs/md-app3.request.log</File>
<rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
<fileNamePattern>/data/logs/md-app3.request.%i.log</fileNamePattern>
<minIndex>1</minIndex>
<maxIndex>5</maxIndex>
</rollingPolicy>
<triggeringPolicy class="ch.qos.logback.core.rolling.SizeBasedTriggeringPolicy">
<maxFileSize>1MB</maxFileSize>
</triggeringPolicy>
<encoder>
<pattern>%h %l %u [%t] "%r" %s %b%n%fullRequest%n</pattern>
</encoder>
</appender>
<appender-ref ref="SIZE_ROLLING" />
</configuration>
The status messages are printed because of the line
<statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" />
in logback-access.xml (etc/logbackAccess.xml in your case). By adapting OnConsoleStatusListener and OnPrintStreamStatusListenerBase, you should be able able to create a custom StatusListener which prints status messages except INFO messages originating at RenameUtil.