How to deal with logging and unit test coverage? - junit

Any idea how to approach this issue:
I have classes with a lot of unit tests
Those classes use Log4j with following levels: DEBUG, INFO, ERROR
When I simply run unit test and calculate coverage depending on the configuration I will not get through all the logging statements. I use if (logger.isDebugEnabled()) ...
I have already had problems in the past when application started to fail on production, and we changed the log level do DEBUG to get more insight, but this caused NullPointerExceptions (bad coding of course)
In theory we can run all the tests with different levels of logging.
Any smart way to do it or any tools that help to solve this issue?

As suggested in comments:
use a test-specific log4j configuration that sets the level to DEBUG.
You also mention:
we changed the log level do DEBUG to get more insight, but this caused NullPointerExceptions
That sounds like a bug. Identifying bugs like this is the whole point of tests: now you can fix them.

Related

How to reduce log level for specific classes/packages in logback.xml

In our application, the information useful for me to analyze performance issues is reported at DEBUG level.
However, it can be very verbose for some classes, and I'd like to change the log level for those specific classes only to INFO level instead of DEBUG.
I'm able to do the opposite in the logback by setting the global log level to INFO, and setting a specific class/package to DEBUG, but I can't seem to make the opposite work.
Does anyone have any suggestions?
Thanks in advance

Sikuli, Java, and java.lang.ThreadDeath exception

we are using Sikuli with Java (Sikuli 1.1.1), but we are running into java.lang.ThreadDeath exception for a new client. In Java Configuration, we have selected mix code of Enable - hide warning and run with protections. Has anyone run into this issue before and what is the reason and possible fix?
Somewhere in the code Thread.stop() is being called.
According to the documentation don't do this! It releases all locks held by that thread may cause locked objects to be accessed in inconsistent state.

How to take screenshot on test failure with junit 5

Can someone tell me please: how to take a screenshot when test method fails (jUnit 5). I have a base test class with BeforeEach and AfterEach methods. Any other classes with #Test methods extends base class.
Well, it is possible to write java code that takes screenshots, see here for example.
But I am very much wondering about the real problem you are trying to solve this way. I am not sure if you figured that yet, but the main intention of JUnit is to provide you a framework that runs your tests in various environments.
Of course it is nice that you can run JUnit within your IDE, and maybe you would find it helpful to get a screenshot. But: "normally" unit tests also run during nightly builds and such - in environments where "taking a screenshot" might not make any sense!
Beyond that: screenshorts are an extremely ineffective way of collecting information! When you have a fail, you should be locking for textual log files, html/xml reports, whatever. You want that failing tests generate information that can be easily digested.
So, the real answer here is: step back from what you are doing right now, and re-consider non-screenshot solutions to the problem you actually want to solve!
You don't need to take screen shots for JUnit test failes/passes, rather the recommended way is to generate various reports (Tests Passed/Failed Report, Code coverage Report, Code complexity Report etc..) automatically using the below tools/plugins.
You can use Cobertura maven plugin or Sonarqube code quality tool so that these will automatically generate the reports for you.
You can look here for Cobertura-maven-plugin and here for Sonarqube for more details.
You need to integrate these tools with your CI (Continuous Integration) environments and ensure that if the code is NOT passing certain quality (in terms of tests coverage, code complexity, etc..) then the project build (war/ear) should fail automatically.

Unexplainable ClassCastException thrown in Play Framework

I've been experiencing a strange error while working on my Play Framework project. While my project is running, I will sometimes receive a ClassCastException, but the error is this:
ClassCastException occured : models.Person cannot be cast to models.Person
This occurs usually when I'm calling a find method such as:
Person p = Person.find("name=?","Joe").first();
If I restart the project, the problem goes away, but only temporarily. It makes testing my project a major pain. How do I fix this?
I've experienced this error while in Dev mode in Play, in two scenarios (as far as I can remember):
Modify an entity and try to recover values from cache that are objects of that entity class.
A compilation error while reloading the code of the page/application
In both scenarios fixing compilation errors or cleaning the cache solved the issue.
Not saying that those are the only possibilities, it may be that you are having some other issue.
This most likely occurs because you've somehow loaded the Person class under two different class loaders. When a class is loaded twice in two different class loaders it's effectively two distinct classes.
(Unfortunately, I can't tell you where/how you might have done this.)
(And it is a bit curious to have the problem pop up on the statement you list. Are you certain that's where it's occurring? Perhaps you should show the exception traceback.)
In my case, this is related to applying evolutions from the web interface. Someone raised a bug for this, but so far it hasn't received any attention from the dev team. There is a patch attached to the ticket, but I haven't tried it, so YMMV.

What are the best practices to log an error?

Many times I saw logging of errors like these:
System.out.println("Method aMethod with parameters a:"+a+" b: "+b);
print("Error in line 88");
so.. What are the best practices to log an error?
EDIT:
This is java but could be C/C++, basic, etc.
Logging directly to the console is horrendous and frankly, the mark of an inexperienced developer. The only reason to do this sort of thing is 1) he or she is unaware of other approaches, and/or 2) the developer has not thought one bit about what will happen when his/her code is deployed to a production site, and how the application will be maintained at that point. Dealing with an application that is logging 1GB/day or more of completely unneeded debug logging is maddening.
The generally accepted best practice is to use a Logging framework that has concepts of:
Different log objects - Different classes/modules/etc can log to different loggers, so you can choose to apply different log configurations to different portions of the application.
Different log levels - so you can tweak the logging configuration to only log errors in production, to log all sorts of debug and trace info in a development environment, etc.
Different log outputs - the framework should allow you to configure where the log output is sent to without requiring any changes in the codebase. Some examples of different places you might want to send log output to are files, files that roll over based on date/size, databases, email, remoting sinks, etc.
The log framework should never never never throw any Exceptions or errors from the logging code. Your application should not fail to load or fail to start because the log framework cannot create it's log file or obtain a lock on the file (unless this is a critical requirement, maybe for legal reasons, for your app).
The eventual log framework you will use will of course depend on your platform. Some common options:
Java:
Apache Commons Logging
log4j
logback
Built-in java.util.logging
.NET:
log4net
C++:
log4cxx
Apache Commons Logging is not intended for applications general logging. It's intended to be used by libraries or APIs that don't want to force a logging implementation on the API's user.
There are also classloading issues with Commons Logging.
Pick one of the [many] logging api's, the most widely used probably being log4j or the Java Logging API.
If you want implementation independence, you might want to consider SLF4J, by the original author of log4j.
Having picked an implementation, then use the logging levels/severity within that implementation consistently, so that searching/filtering logs is easier.
The easiest way to log errors in a consistent format is to use a logging framework such as Log4j (assuming you're using Java). It is useful to include a logging section in your code standards to make sure all developers know what needs to be logged. The nice thing about most logging frameworks is they have different logging levels so you can control how verbose the logging is between development, test, and production.
A best practice is to use the java.util.logging framework
Then you can log messages in either of these formats
log.warning("..");
log.fine("..");
log.finer("..");
log.finest("..");
Or
log.log(Level.WARNING, "blah blah blah", e);
Then you can use a logging.properties (example below) to switch between levels of logging, and do all sorts of clever stuff like logging to files, with rotation etc.
handlers = java.util.logging.ConsoleHandler
.level = WARNING
java.util.logging.ConsoleHandler.level = ALL
com.example.blah = FINE
com.example.testcomponents = FINEST
Frameworks like log4j and others should be avoided in my opinion, Java has everything you need already.
EDIT
This can apply as a general practice for any programming language. Being able to control all levels of logging from a single property file is often very important in enterprise applications.
Some suggested best-practices
Use a logging framework. This will allow you to:
Easily change the destination of your log messages
Filter log messages based on severity
Support internationalised log messages
If you are using java, then slf4j is now preferred to Jakarta commons logging as the logging facade.
As stated slf4j is a facade, and you have to then pick an underlying implementation. Either log4j, java.util.logging, or 'simple'.
Follow your framework's advice to ensuring expensive logging operations are not needlessly carried out
The apache common logging API as mentioned above is a great resource. Referring back to java, there is also a standard error output stream (System.err).
Directly from the Java API:
This stream is already open and ready
to accept output data.
Typically this stream corresponds to
display output or another output
destination specified by the host
environment or user. By convention,
this output stream is used to display
error messages or other information
that should come to the immediate
attention of a user even if the
principal output stream, the value of
the variable out, has been redirected
to a file or other destination that is
typically not continuously monitored.
Aside from technical considerations from other answers it is advisable to log a meaningful message and perhaps some steps to avoid the error in the future. Depending on the errors, of course.
You could get more out of a I/O-Error when the message states something like "Could not read from file X, you don't have the appropriate permission."
See more examples on SO or search the web.
There really is no best practice for logging an error. It basically just needs to follow a consistent pattern (within the software/company/etc) that provides enough information to track the problem down. For Example, you might want to keep track of the time, the method, parameters, calling method, etc.
So long as you dont just print "Error in "