Log4J2 Programmatic Configuration from XML byte stream - configuration

I am at the end of my tether with Log4J2, hopefully somebody can help. I have the following code to initialize Log4J2, pretty soon after startup:
try (InputStream configStream = new ByteArrayInputStream(writer.toString().getBytes("UTF-8"))) {
ConfigurationSource configurationSource = new ConfigurationSource(configStream);
Configurator.initialize(null, configurationSource);
}
Where writer is a StringWriter and toString() produces the following config (which I have validated is correct through other means):
<?xml version="1.0" encoding="UTF-8"?>
<Configuration>
<Appenders>
<Console name="C" target="SYSTEM_OUT">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS z} %-5p %m%n"/>
</Console>
<RollingFile name="R" fileName="C:\temp\logfile.log" filePattern="C:\temp\logfile.log.%d{yyyy-MM-dd}">
<PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss,SSS z} %-5p [%t] %m%n"/>
<Policies>
<TimeBasedTriggeringPolicy interval="1" modulate="true"/>
</Policies>
</RollingFile>
</Appenders>
<Loggers>
<Logger name="somename" level="debug">
<appender-ref ref="R"/>
</Logger>
<Root level="debug" additivity="false">
<appender-ref ref="C" level="info"/>
</Root>
</Loggers>
</Configuration>
As you may have guessed, this does not work and I get no error message other than the expected:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
The reason I say this is expected is because I am configuring Log4J2 manually and have not suppressed this message yet.
Unfortunately, I cannot read the config from a file, for legacy reasons.
UPDATE 1 :
After taking Remko's advice, I added the following block before invoking the initialize method:
System.setProperty("log4j2.disable.jmx", "true");
StatusLogger status = StatusLogger.getLogger();
status.clear(); // remove old listeners that may prevent status output
status.setLevel(Level.TRACE);
status.reset(); // I could not see any trace info until I called this
status.trace("Status -- TRACE"); // I added this to prove that trace level logging was working
This gave me the following output:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.
TRACE StatusLogger Status -- TRACE
DEBUG StatusLogger Stopping LoggerContext[name=sun.misc.Launcher$AppClassLoader#647e05, org.apache.logging.log4j.core.LoggerContext#a3defe]
DEBUG StatusLogger Stopping LoggerContext[name=sun.misc.Launcher$AppClassLoader#647e05, org.apache.logging.log4j.core.LoggerContext#a3defe]...
DEBUG StatusLogger Unregistering MBean org.apache.logging.log4j2:type=sun.misc.Launcher$AppClassLoader#647e05
DEBUG StatusLogger Unregistering MBean org.apache.logging.log4j2:type=sun.misc.Launcher$AppClassLoader#647e05,component=StatusLogger
DEBUG StatusLogger Unregistering MBean org.apache.logging.log4j2:type=sun.misc.Launcher$AppClassLoader#647e05,component=ContextSelector
DEBUG StatusLogger Unregistering MBean org.apache.logging.log4j2:type=sun.misc.Launcher$AppClassLoader#647e05,component=Appenders,name=Console
TRACE StatusLogger Stopping org.apache.logging.log4j.core.config.DefaultConfiguration#1a1440e...
TRACE StatusLogger AbstractConfiguration stopped 0 AsyncLoggerConfigs.
TRACE StatusLogger AbstractConfiguration stopped 0 AsyncAppenders.
DEBUG StatusLogger Shutting down OutputStreamManager SYSTEM_OUT
TRACE StatusLogger AbstractConfiguration stopped 1 Appenders.
TRACE StatusLogger AbstractConfiguration stopped 0 Loggers.
DEBUG StatusLogger Stopped org.apache.logging.log4j.core.config.DefaultConfiguration#9a6398 OK
DEBUG StatusLogger Stopped LoggerContext[name=sun.misc.Launcher$AppClassLoader#647e05, org.apache.logging.log4j.core.LoggerContext#9a6398]...
UPDATE 2 :
I decided to figure out a way to work with a file rather than a ByteArrayInputStream and got it working. FWIW, I think there is a bug in the Log4J2 code, when attempting to initialize using an InputStream, my theory:
In Log4jContextFactory the following method:
public LoggerContext getContext(final String fqcn, final ClassLoader loader, final Object externalContext,
final boolean currentContext, final ConfigurationSource source)
Has the following if statement, which always evaluates to false, which means the default config is always returned...
if (ctx.getState() == LifeCycle.State.INITIALIZED) {
if (source != null) {
ContextAnchor.THREAD_CONTEXT.set(ctx);
final Configuration config = ConfigurationFactory.getInstance().getConfiguration(source);
LOGGER.debug("Starting LoggerContext[name={}] from configuration {}", ctx.getName(), source);
ctx.start(config);
ContextAnchor.THREAD_CONTEXT.remove();
} else {
ctx.start();
}
}

At first glance I don't see why your configuration does not work. (You could try using forward slashes in the paths to make absolutely sure, but chances are that the slashes are not the problem.)
Can you try the following to generate more log4j2 debug output to see where the configuration goes wrong? Please post the result in your question.
import org.apache.logging.log4j.Level;
import org.apache.logging.log4j.status.StatusLogger;
// In an XML configuration you can just use <Configuration status="TRACE"...
// Here we use less elegant code to switch on status logging
// since we're not sure where things break down.
System.setProperty("log4j2.disable.jmx", "true");
StatusLogger status = StatusLogger.getLogger();
status.clear(); // remove old listeners that may prevent status output
status.setLevel(Level.TRACE);
// now configure log4j2...
// This should generate trace-level debug output to the console.
try (InputStream configStream = new ByteArrayInputStream(writer.toString().getBytes())) {
ConfigurationSource configurationSource = new ConfigurationSource(configStream);
Configurator.initialize(null, configurationSource);
}

Related

rolling or backing up logfile on startup dropwizard

I need to rotate or backup the current log file on startup but can't work out how to do it.
I have a typical log config:
appenders:
- type: console
threshold: INFO
- type: file
threshold: DEBUG
logFormat: "%-6level [%d{HH:mm:ss.SSS}] [%t] %logger{5} - %X{code} %msg %n"
currentLogFilename: /var/log/dw-service.log
archive: true
archivedLogFilenamePattern: /var/log/tmp/dw-service-%d{yyyy-MM-dd}-%i.log.gz
archivedFileCount: 3
timeZone: UTC
maxFileSize: 10MB
Ideally I would like to cause an archive of the logfile immediately on startup but failing that, I would settle for getting a handle to the appender and copying the file.
I can't work out how to do either. Please could somebody offer any help?
I've got this far:
private void backupLogFile() {
ch.qos.logback.classic.Logger log = (ch.qos.logback.classic.Logger)LoggerFactory.getLogger(LOG.ROOT_LOGGER_NAME);
Iterator<Appender<ILoggingEvent>> itr = log.iteratorForAppenders();
List<Appender<ILoggingEvent>> appenders = new LinkedList<Appender<ILoggingEvent>>();
while (itr.hasNext()) {
appenders.add(itr.next());
}
for(int i=0;i<appenders.size();++i){
if(appenders.get(i).getName().equals("async-file-appender"))
{
LOG.info("FOUND FILE APPENDER");
FileAppender myFile = (FileAppender)appenders.get(i);
String filename = myFile.getFile();
}
}
}
At runtime, Java tells me
Exception in thread "main" java.lang.ClassCastException: ch.qos.logback.classic.AsyncAppender incompatible with ch.qos.logback.core.FileAppender
I can't work out how to get the configured currentLogFilename, anybody know how do this?
Thanks

Rollback in MyBatis using JDBC (no Spring, no containers)

I've seen all sorts of posts on using Spring and MyBatis with transactions, but I'm facing a problem with rollbacks not working with plain old JDBC.
My ( test / throwaway) code is pretty simple : I open a session, insert a rec, throw an error on purpose and rollback the transaction. However, it always commits.
public static void main (String[] args){
//-- omitted for brevity
try {
org.apache.ibatis.logging.LogFactory.useSlf4jLogging();
inputStream = Resources.getResourceAsStream("mybatis-config.xml");
sqlSessionFactory = new SqlSessionFactoryBuilder().build(inputStream);
sess = sqlSessionFactory.openSession(false);
BillsMapper mapper = sess.getMapper(BillsMapper.class);
BillState billState = new BillState();
billState.setBillId(-1);
billState.setLastName("TESTER");
billState.setFirstName("TESTER");
mapper.insert(billState);
logger.info("Post insert: key = {}", billState.getBillId());
if(1 == 1)
throw new RuntimeException("Error Thrown on purpose...testing rollback ");
sess.commit();
}catch(Exception e){
logger.error("Error: {}", e);
sess.rollback();
}finally{
sess.close();
logger.info("Finito!");
}
}
The logs show:
DEBUG | (BaseJdbcLogger.java:145) - ==> Preparing: insert into bills (users_userId, refId, firstName, ...
DEBUG | (BaseJdbcLogger.java:145) - ==> Parameters: 67(Integer), 67-120530180328(String), TESTER(String), ...
DEBUG | (BaseJdbcLogger.java:145) - <== Updates: 1
INFO | (TestAction.java:50) - Post insert: key = 2478
ERROR | (TestAction.java:56) - Error: {} java.lang.RuntimeException: Error Thrown on purpose...testing rollback at com.s2stest.TestAction.main(TestAction.java:53)
DEBUG | (JdbcTransaction.java:79) - Rolling back JDBC Connection [com.mysql.jdbc.JDBC4Connection#371e88fb]
DEBUG | (JdbcTransaction.java:122) - Resetting autocommit to true on JDBC Connection [com.mysql.jdbc.JDBC4Connection#371e88fb]
DEBUG | (JdbcTransaction.java:90) - Closing JDBC Connection [com.mysql.jdbc.JDBC4Connection#371e88fb]
DEBUG | (PooledDataSource.java:344) - Returned connection 924748027 to pool.
Note the resetting of autocommit before closing the connection.... Would resetting autcommit before closing the SqlSession cause my rolled-back transaction to be committed? If so, is this a bug? Has anyone gotten JDBC working with transactions? I need it for testing, and I'd value some help. Right now, no transactions can be rolled back.
I've looked at the MyBatis source, and it indeed calls resetAutocommit before closing the connection. I'm using MySQL 5.6 and mysql-connector-java-5.1.36.jar for the driver if someone has a workaround that they've found.
--- UPDATE ---
mybatis-config.xml is as follows:
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE configuration
PUBLIC "-//mybatis.org//DTD Config 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-config.dtd">
<configuration>
<settings>
<setting name="logImpl" value="SLF4J" />
</settings>
<typeAliases>
<package name="com.ship2storage.domain" />
</typeAliases>
<environments default="development">
<environment id="development">
<transactionManager type="JDBC" />
<dataSource type="POOLED">
<property name="driver" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/mytestDb?zeroDateTimeBehavior=convertToNull" />
<property name="username" value="--shhh!!--" />
<property name="password" value="--shhh!!--" />
</dataSource>
</environment>
</environments>
<mappers>
<mapper resource="com/ship2storage/db/maps/BillsMapper.xml" />
</mappers>
</configuration>
OK, I've found the answer by digging deeper into my setup. It seems that the MySQL storage engine I installed for my test DB is ISAM. ISAM does not support transactions. I switched to InnoDB using the following SQL tidbit, and transactions now work with JDBC:
ALTER TABLE bills ENGINE=InnoDB;
I haven't tried this, but it looks like you can also do this temporarily too:
SET default_storage_engine=InnoDB;
Hopefully this will help someone. The code/config posted above works.

Logging errors in a grails application in the logs

When my Grails application crashes, it shows the error and the stacktrace on the error page because the error.gsp page has the following snippet <g:renderException exception="${exception}" />. However nothing gets logged in the log file.
How can I change this? because for the production application I plan to remove the renderException because I don't want users to see the entire stacktrace.
My log4j settings are as follows:
appenders {
rollingFile name:'catalinaOut', maxFileSize:1024, fileName:"${System.properties.getProperty('catalina.home')}/logs/mylog.log"
}
root {
error 'catalinaOut'
debug 'catalinaOut'
additivity = true
}
error 'org.codehaus.groovy.grails.web.servlet', // controllers
'org.codehaus.groovy.grails.web.pages', // GSP
'org.codehaus.groovy.grails.web.sitemesh', // layouts
'org.codehaus.groovy.grails.web.mapping.filter', // URL mapping
'org.codehaus.groovy.grails.web.mapping', // URL mapping
'org.codehaus.groovy.grails.commons', // core / classloading
'org.codehaus.groovy.grails.plugins', // plugins
'org.codehaus.groovy.grails.orm.hibernate', // hibernate integration
'org.springframework',
'org.hibernate',
'net.sf.ehcache.hibernate',
'grails.app'
debug 'grails.app'
}
I'm running the app in development as grails run-app
I use these settings for console and file based logging. You can remove stdout if you don't want/need console. Just copy all your error classes in the corresponding list.
log4j = {
def loggerPattern = '%d %-5p >> %m%n'
def errorClasses = [] // add more classes if needed
def infoClasses = ['grails.app.controllers.myController'] // add more classes if needed
def debugClasses = [] // add more classes if needed
appenders {
console name:'stdout', layout:pattern(conversionPattern: loggerPattern)
rollingFile name: "file", maxFileSize: 1024, file: "./tmp/logs/logger.log", layout:pattern(conversionPattern: loggerPattern)
}
error stdout: errorClasses, file: errorClasses
info stdout: infoClasses, file: infoClasses
debug stdout: debugClasses, file: debugClasses
}

ESB Mule Client staring with xml-properties fails

I use Mule 3.x
I have a code that tests MuleClient connectivity.
This test is ok and works proper way:
public void testHello() throws Exception
{
MuleClient client = new MuleClient(muleContext);
MuleMessage result = client.send("http://127.0.0.1:8080/hello", "some data", null);
assertNotNull(result);
assertNull(result.getExceptionPayload());
assertFalse(result.getPayload() instanceof NullPayload);
//TODO Assert the correct data has been received
assertEquals("hello", result.getPayloadAsString());
}
But this tes is not ok - it fail with an connection exceptions:
public void testHello_with_Spring() throws Exception {
MuleClient client = new MuleClient("mule-config-test.xml");
client.getMuleContext().start();
//it fails here
MuleMessage result = client.send("http://127.0.0.1:8080/hello", "some data", null);
assertNotNull(result);
assertNull(result.getExceptionPayload());
assertFalse(result.getPayload() instanceof NullPayload);
//TODO Assert the correct data has been received
assertEquals("hello", result.getPayloadAsString());
}
The 'mule-config-test.xml' is used in both tests, the path for this file is ok, i checked.
This is error message I have in the end:
Exception stack is:
1. Address already in use (java.net.BindException) java.net.PlainSocketImpl:-2 (null)
2. Failed to bind to uri "http://127.0.0.1:8080/hello" (org.mule.transport.ConnectException)
org.mule.transport.tcp.TcpMessageReceiver:81
(http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/transport/ConnectException.html)
-------------------------------------------------------------------------------- Root Exception stack trace: java.net.BindException: Address already in
use at java.net.PlainSocketImpl.socketBind(Native Method) at
java.net.PlainSocketImpl.bind(PlainSocketImpl.java:383) at
java.net.ServerSocket.bind(ServerSocket.java:328)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
[10-05 16:33:37] ERROR HttpConnector [main]:
org.mule.transport.ConnectException: Failed to bind to uri
"http://127.0.0.1:8080/hello" [10-05 16:33:37] ERROR ConnectNotifier
[main]: Failed to connect/reconnect: HttpConnector {
name=connector.http.mule.default lifecycle=stop this=7578a7d9
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true connected=false
supportedProtocols=[http] serviceOverrides= } . Root Exception
was: Address already in use. Type: class java.net.BindException [10-05
16:33:37] ERROR DefaultSystemExceptionStrategy [main]: Failed to bind
to uri "http://127.0.0.1:8080/hello"
org.mule.api.lifecycle.LifecycleException: Cannot process event as
"connector.http.mule.default" is stopped
I think the problem is in what you're not showing: testHello_with_Spring() is probably executing while Mule is already running. The second Mule you're starting in it then port-conflicts with the other one.
Are testHello() and testHello_with_Spring() in the same test suite? If yes, seeing that testHello() relies on an already running Mule, I'd say that would be the cause of port conflict for testHello_with_Spring().

Why is log4j not behaving as expected?

I have a co-worker who is trying to get log4j to behave as follows:
Log to Stdout
By default, disable most output
Show only messages from java.sql.PrepareStatement at level debug and up
He's getting caught up in the 'level' vs 'priority'. Here is his config file:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE log4j:configuration SYSTEM "D:/Java/apache-log4j-1.2.15/src/main/resources/org/apache/log4j/xml/log4j.dtd" >
<log4j:configuration>
<!-- Appenders -->
<appender name="stdout" class="org.apache.log4j.ConsoleAppender">
<layout class="org.apache.log4j.PatternLayout">
<param name="ConversionPattern" value="%5p %d{ISO8601} [%t][%x] %c - %m%n" />
</layout>
</appender>
<!-- Loggers for ibatus and JDBC database -->
<logger name="java.sql.PreparedStatement">
<level value="debug"/>
</logger>
<!-- The Root Logger -->
<root>
<level value="error"/>
<appender-ref ref="stdout"/>
</root>
</log4j:configuration>
Simpler configuration as shown (Root Log Level = ERROR):
log4j: reset attribute= "false".
log4j: Threshold ="null".
log4j: Retreiving an instance of org.apache.log4j.Logger.
log4j: Setting [java.sql.PreparedStatement] additivity to [true].
log4j: Level value for java.sql.PreparedStatement is [debug].
log4j: java.sql.PreparedStatement level set to DEBUG
log4j: Level value for root is [error].
log4j: root level set to ERROR
log4j: Class name: [org.apache.log4j.ConsoleAppender]
log4j: Parsing layout of class: "org.apache.log4j.PatternLayout"
log4j: Setting property [conversionPattern] to [%5p %d{ISO8601} [%t][%x] %c - %m%n].
log4j: Adding appender named [stdout] to category [root].
Configuration with Root Log Level changed to debug (replace queries with …)
log4j: reset attribute= "false".
log4j: Threshold ="null".
log4j: Retreiving an instance of org.apache.log4j.Logger.
log4j: Setting [java.sql.PreparedStatement] additivity to [true].
log4j: Level value for java.sql.PreparedStatement is [debug].
log4j: java.sql.PreparedStatement level set to DEBUG
log4j: Level value for root is [debug].
log4j: root level set to DEBUG
log4j: Class name: [org.apache.log4j.ConsoleAppender]
log4j: Parsing layout of class: "org.apache.log4j.PatternLayout"
log4j: Setting property [conversionPattern] to [%5p %d{ISO8601} [%t][%x] %c - %m%n].
log4j: Adding appender named [stdout] to category [root].
DEBUG 2010-03-19 12:59:58,256 [main][] com.ibatis.common.jdbc.SimpleDataSource - Created connection 1309601.
DEBUG 2010-03-19 12:59:58,256 [main][] java.sql.Connection - {conn-100000} Connection
DEBUG 2010-03-19 12:59:58,256 [main][] java.sql.Connection - {conn-100000} Preparing Statement: …
DEBUG 2010-03-19 12:59:58,287 [main][] java.sql.PreparedStatement - {pstm-100001} Executing Statement: …
DEBUG 2010-03-19 12:59:58,287 [main][] java.sql.PreparedStatement - {pstm-100001} Parameters: [%ATL]
DEBUG 2010-03-19 12:59:58,287 [main][] java.sql.PreparedStatement - {pstm-100001} Types: [java.lang.String]
DEBUG 2010-03-19 12:59:58,366 [main][] java.sql.ResultSet - {rset-100002} ResultSet
DEBUG 2010-03-19 12:59:58,381 [main][] java.sql.ResultSet - {rset-100002} Header: …
DEBUG 2010-03-19 12:59:58,381 [main][] java.sql.ResultSet - {rset-100002} Result: …
DEBUG 2010-03-19 12:59:58,381 [main][] java.sql.ResultSet - {rset-100002} Result: …
DEBUG 2010-03-19 12:59:58,381 [main][] java.sql.ResultSet - {rset-100002} Result: …
DEBUG 2010-03-19 12:59:58,397 [main][] com.ibatis.common.jdbc.SimpleDataSource - Returned connection 1309601 to pool.
How does he need to change his log4j.xml config file to make it behave as he's expecting?
I looked at the source code for mybatis, shown below. You have to enable DEBUG on java.sql.Connection in order for it execute the logging for java.sql.PreparedStatement. I struggled with this all day!
private Connection wrapConnection(Connection connection) {
if (log.isDebugEnabled()) {
return ConnectionLogger.newInstance(connection);
} else {
return connection;
}
}
Insert the following addition into your log4j.xml:
<!-- *******************************************************
WARNING: iBatus 2.3.3 (only ver. tested) is a little weird.
YOU MUST SET
java.sql.Connection to debug to get any
java.sql.PreparedStatement debug logs.
************************************************************** -->
<logger name="java.sql.Connection" additivity="false">
<level value="debug"/>
</logger>
DEBUG 2010-03-19 14:27:08,425 [main][][java.sql.PreparedStatement] - {pstm-100001} Executing Statement:
...
I think the reason there are no log messages is that code you want to see logs from, doesn't use java.sql.PrepareStatement logger, but different loggers. Loggers are usually (although not necessarilly) named after classes that use them. I.e. com.ibatis.SomeClass would typically not use java.sql.PrepareStatement logger.
Set your root logger to DEBUG, and check out names of loggers that give you messages you want. Then configure these loggers with DEBUG and let root log at ERROR level only.
Btw, it's Prepare_d_Statement, it is interface (i.e. no logging code there), and it definitely doesn't use log4j since it is in JDK.