Eclipse MAT thread attributes - heap-dump

I have Liferay 6.1.2 CE application deployed to Jboss EAP 6.4
I am trying to analyze heap dump using Eclipse MAT after server crashed by OOM.
In dominator tree I see couple of threads that occupying a lot of memory.
My question what do parkBlocker and other Treads Attributes mean ?

'parkBlocker' is a field name of java.lang.Thread. Attribute here means field name.
The actual use of a field depends on the code, but my guess is that is the object used to block on when a thread is parked(). See the park*() methods on java.lang.Thread.

Related

Configuring Weld's threadpool in Wildfly 19

How can I configure the size of the executor service the Weld subsystem of Wildfly uses to execute asynchronous event observer methods? Specifically I want to increase the size of the thread pool.
The Weld documentation has some config parameters but points out that those can be ignored by integrators and that Wildfly is one that does. The Wildfly documentation on the other hand contains configuration option for nearly every subsystem except the Weld subsystem.
I'm using Wildfly 19.
The actual executor service that WFLY uses for Weld purposes is WeldExecutorServices and even more precisely for async obsever notification, this method returns the executor.
With a little bit of digging I could find that this is set in WeldSubsystemAdd, here. So it has some defaults but it is pulling the config from somewhere before using the default.
Therefore, you should be able to adjust this by configuring the given WildFly subsystem, in this case Weld.
I have found out that documentation mentions certain options for Weld subsystem, one of which is thread-pool-size. See https://docs.wildfly.org/19/wildscribe/subsystem/weld/index.html
I don't know exactly how to pass in these options to WFLY because it has been a long time since I last used it. However, it is some generic way in which you can pass in options for any subsystem. Once you figure that out, you should be good to go.

SonarQube, no data for depth inheritance tree and other metrics

Hi this is my first post on legendary Stackoverflow!
I am currently trying to find a platform in order to create an inheritance plugin (in order to calculate metrics) for my thesis and I am trying SonarQube for this purpose. So far, I am learning the basics. The problem is that although everything goes fine, I have a problem when it's time to add a "custom measures" widget. As you can see, the widget shows the message "no data" on configure widget menu, when I add measures like Number of Children, Depth in Tree.
I have installed SonarQube 3.7 for Windows 7 64bit and SonarQube Runner version 2.3. I have also installed SonarQube on eclipse. My .properties file is this :
Thanks in advance :-)
The computing of LCOM4 and RFC metrics is based on byte-code, so if you want to have them, you need to:
compile your code
add the "sonar.binaries" property in your file (it should point to the folder where the code was compiled, e.g. "target/classes" or "bin")
As for the 2 other metrics, they are not fed by SonarQube. So we'll eventually remove them (https://jira.codehaus.org/browse/SONAR-4643).

Glassfish 3.1.2 - Difference Between default-config and server-config

What is the difference between default-config and server-config? If I want to adjust a thread pool size for example, do I need to do it in both places? If just one, which one?
It appears that the default-config is used by Glassfish as a template to create other configs, so anything you actually want to configure on a server should go in the server-config.
Also see Glassfish 3 has two configurations

How to configure the pool-size for a Message driven Bean in JBoss 6?

I want to configure a bigger pool size for my message consumer in my java ee application, as the consumption of a message requires some time (mostly waiting).
How can i do this?
I would prefer a solution that uses annotation.
Is this task dependant on the used resource adapter? (mine is activeMQ)
it seems like the jboss doesn't handle the mdb pool size by itself. The resourceadapter implementation of your jms broker does that.
For activemq the activeation-config-property is "maxSessions"
for hornetq (default) the property seems to be "MaxPoolSize"
another suggestion i found is to change this section in ejb3-interceptors-aop.xml
<annotation expr="!class(#org.jboss.ejb3.annotation.Pool)">
#org.jboss.ejb3.annotation.Pool (value="StrictMaxPool", maxSize=15, timeout=10000)
</annotation>
although this would apply to every mdb (and also the #Pool annotation had no effect on my mdbs)

Dealing with "java.lang.OutOfMemoryError: PermGen space" error

Recently I ran into this error in my web application:
java.lang.OutOfMemoryError: PermGen space
It's a typical Hibernate/JPA + IceFaces/JSF application running on Tomcat 6 and JDK 1.6.
Apparently this can occur after redeploying an application a few times.
What causes it and what can be done to avoid it?
How do I fix the problem?
The solution was to add these flags to JVM command line when Tomcat is started:
-XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
You can do that by shutting down the tomcat service, then going into the Tomcat/bin directory and running tomcat6w.exe. Under the "Java" tab, add the arguments to the "Java Options" box. Click "OK" and then restart the service.
If you get an error the specified service does not exist as an installed service you should run:
tomcat6w //ES//servicename
where servicename is the name of the server as viewed in services.msc
Source: orx's comment on Eric's Agile Answers.
You better try -XX:MaxPermSize=128M rather than -XX:MaxPermGen=128M.
I can not tell the precise use of this memory pool, but it have to do with the number of classes loaded into the JVM. (Thus enabling class unloading for tomcat can resolve the problem.) If your applications generates and compiles classes on the run it is more likely to need a memory pool bigger than the default.
App server PermGen errors that happen after multiple deployments are most likely caused by references held by the container into your old apps' classloaders. For example, using a custom log level class will cause references to be held by the app server's classloader. You can detect these inter-classloader leaks by using modern (JDK6+) JVM analysis tools such as jmap and jhat to look at which classes continue to be held in your app, and redesigning or eliminating their use. Usual suspects are databases, loggers, and other base-framework-level libraries.
See Classloader leaks: the dreaded "java.lang.OutOfMemoryError: PermGen space" exception, and especially its followup post.
Common mistakes people make is thinking that heap space and permgen space are same, which is not at all true. You could have lot of space remaining in the heap but still can run out of memory in permgen.
Common causes of OutofMemory in PermGen is ClassLoader. Whenever a class is loaded into JVM, all its meta data, along with Classloader, is kept on PermGen area and they will be garbage collected when the Classloader which loaded them is ready for garbage collection. In Case Classloader has a memory leak than all classes loaded by it will remain in memory and cause permGen outofmemory once you repeat it a couple of times. The classical example is Java.lang.OutOfMemoryError:PermGen Space in Tomcat.
Now there are two ways to solve this:
1. Find the cause of Memory Leak or if there is any memory leak.
2. Increase size of PermGen Space by using JVM param -XX:MaxPermSize and -XX:PermSize.
You can also check 2 Solution of Java.lang.OutOfMemoryError in Java for more details.
Use the command line parameter -XX:MaxPermSize=128m for a Sun JVM (obviously substituting 128 for whatever size you need).
Try -XX:MaxPermSize=256m and if it persists, try -XX:MaxPermSize=512m
I added -XX: MaxPermSize = 128m (you can experiment which works best) to VM Arguments as I'm using eclipse ide. In most of JVM, default PermSize is around 64MB which runs out of memory if there are too many classes or huge number of Strings in the project.
For eclipse, it is also described at answer.
STEP 1 : Double Click on the tomcat server at Servers Tab
STEP 2 : Open launch Conf and add -XX: MaxPermSize = 128m to the end of existing VM arguements.
I've been butting my head against this problem while deploying and undeploying a complex web application too, and thought I'd add an explanation and my solution.
When I deploy an application on Apache Tomcat, a new ClassLoader is created for that app. The ClassLoader is then used to load all the application's classes, and on undeploy, everything's supposed to go away nicely. However, in reality it's not quite as simple.
One or more of the classes created during the web application's life holds a static reference which, somewhere along the line, references the ClassLoader. As the reference is originally static, no amount of garbage collecting will clean this reference up - the ClassLoader, and all the classes it's loaded, are here to stay.
And after a couple of redeploys, we encounter the OutOfMemoryError.
Now this has become a fairly serious problem. I could make sure that Tomcat is restarted after each redeploy, but that takes down the entire server, rather than just the application being redeployed, which is often not feasible.
So instead I've put together a solution in code, which works on Apache Tomcat 6.0. I've not tested on any other application servers, and must stress that this is very likely not to work without modification on any other application server.
I'd also like to say that personally I hate this code, and that nobody should be using this as a "quick fix" if the existing code can be changed to use proper shutdown and cleanup methods. The only time this should be used is if there's an external library your code is dependent on (In my case, it was a RADIUS client) that doesn't provide a means to clean up its own static references.
Anyway, on with the code. This should be called at the point where the application is undeploying - such as a servlet's destroy method or (the better approach) a ServletContextListener's contextDestroyed method.
//Get a list of all classes loaded by the current webapp classloader
WebappClassLoader classLoader = (WebappClassLoader) getClass().getClassLoader();
Field classLoaderClassesField = null;
Class clazz = WebappClassLoader.class;
while (classLoaderClassesField == null && clazz != null) {
try {
classLoaderClassesField = clazz.getDeclaredField("classes");
} catch (Exception exception) {
//do nothing
}
clazz = clazz.getSuperclass();
}
classLoaderClassesField.setAccessible(true);
List classes = new ArrayList((Vector)classLoaderClassesField.get(classLoader));
for (Object o : classes) {
Class c = (Class)o;
//Make sure you identify only the packages that are holding references to the classloader.
//Allowing this code to clear all static references will result in all sorts
//of horrible things (like java segfaulting).
if (c.getName().startsWith("com.whatever")) {
//Kill any static references within all these classes.
for (Field f : c.getDeclaredFields()) {
if (Modifier.isStatic(f.getModifiers())
&& !Modifier.isFinal(f.getModifiers())
&& !f.getType().isPrimitive()) {
try {
f.setAccessible(true);
f.set(null, null);
} catch (Exception exception) {
//Log the exception
}
}
}
}
}
classes.clear();
The java.lang.OutOfMemoryError: PermGen space message indicates that the Permanent Generation’s area in memory is exhausted.
Any Java applications is allowed to use a limited amount of memory. The exact amount of memory your particular application can use is specified during application startup.
Java memory is separated into different regions which can be seen in the following image:
Metaspace: A new memory space is born
The JDK 8 HotSpot JVM is now using native memory for the representation of class metadata and is called Metaspace; similar to the Oracle JRockit and IBM JVM's.
The good news is that it means no more java.lang.OutOfMemoryError: PermGen space problems and no need for you to tune and monitor this memory space anymore using Java_8_Download or higher.
Alternatively, you can switch to JRockit which handling permgen differently then sun's jvm. It generally has better performance as well.
http://www.oracle.com/technetwork/middleware/jrockit/overview/index.html
1) Increasing the PermGen Memory Size
The first thing one can do is to make the size of the permanent generation heap space bigger. This cannot be done with the usual –Xms(set initial heap size) and –Xmx(set maximum heap size) JVM arguments, since as mentioned, the permanent generation heap space is entirely separate from the regular Java Heap space,
and these arguments set the space for this regular Java heap space. However, there are similar arguments which can be used(at least with the Sun/OpenJDK jvms) to make the size of the permanent generation heap bigger:
-XX:MaxPermSize=128m
Default is 64m.
2) Enable Sweeping
Another way to take care of that for good is to allow classes to be unloaded so your PermGen never runs out:
-XX:+CMSClassUnloadingEnabled -XX:+CMSPermGenSweepingEnabled
Stuff like that worked magic for me in the past. One thing though, there’s a significant performance trade off in using those, since permgen sweeps will make like an extra 2 requests for every request you make or something along those lines. You’ll need to balance your use with the tradeoffs.
You can find the details of this error.
http://faisalbhagat.blogspot.com/2014/09/java-outofmemoryerror-permgen.html
I had the problem we are talking about here, my scenario is eclipse-helios + tomcat + jsf and what you were doing is making a deploy a simple application to tomcat. I was showing the same problem here, solved it as follows.
In eclipse go to servers tab double click on the registered server in my case tomcat 7.0, it opens my file server General registration information. On the section "General Information" click on the link "Open launch configuration" , this opens the execution of server options in the Arguments tab in VM arguments added in the end these two entries
-XX: MaxPermSize = 512m
-XX: PermSize = 512m
and ready.
The simplest answer these days is to use Java 8.
It no longer reserves memory exclusively for PermGen space, allowing the PermGen memory to co-mingle with the regular memory pool.
Keep in mind that you will have to remove all non-standard -XXPermGen...=... JVM startup parameters if you don't want Java 8 to complain that they don't do anything.
Open tomcat7w from Tomcat's bin directory or type Monitor Tomcat in start menu
(a tabbed window opens with various service information).
In the Java Options text area append this line:
-XX:MaxPermSize=128m
Set Initial Memory Pool to 1024 (optional).
Set Maximum Memory Pool to 1024 (optional).
Click Ok.
Restart the Tomcat service.
Perm gen space error occurs due to the use of large space rather then jvm provided space to executed the code.
The best solution for this problem in UNIX operating systems is to change some configuration on the bash file. The following steps solve the problem.
Run command gedit .bashrc on terminal.
Create JAVA_OTPS variable with following value:
export JAVA_OPTS="-XX:PermSize=256m -XX:MaxPermSize=512m"
Save the bash file. Run command exec bash on the terminal. Restart the server.
I hope this approach will work on your problem. If you use a Java version lower than 8 this issue occurs sometimes. But if you use Java 8 the problem never occurs.
Increasing Permanent Generation size or tweaking GC parameters will NOT help if you have a real memory leak. If your application or some 3rd party library it uses, leaks class loaders the only real and permanent solution is to find this leak and fix it. There are number of tools that can help you, one of the recent is Plumbr, which has just released a new version with the required capabilities.
Also if you are using log4j in your webapp, check this paragraph in log4j documentation.
It seems that if you are using PropertyConfigurator.configureAndWatch("log4j.properties"), you cause memory leaks when you undeploy your webapp.
I have a combination of Hibernate+Eclipse RCP, tried using -XX:MaxPermSize=512m and -XX:PermSize=512m and it seems to be working for me.
Set -XX:PermSize=64m -XX:MaxPermSize=128m. Later on you may also try increasing MaxPermSize. Hope it'll work. The same works for me. Setting only MaxPermSize didn't worked for me.
I tried several answers and the only thing what finally did the job was this configuration for the compiler plugin in the pom:
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.3.2</version>
<configuration>
<fork>true</fork>
<meminitial>128m</meminitial>
<maxmem>512m</maxmem>
<source>1.6</source>
<target>1.6</target>
<!-- prevent PermGen space out of memory exception -->
<!-- <argLine>-Xmx512m -XX:MaxPermSize=512m</argLine> -->
</configuration>
</plugin>
hope this one helps.
jrockit resolved this for me as well; however, I noticed that the servlet restart times were much worse, so while it was better in production, it was kind of a drag in development.
The configuration of the memory depends on the nature of your app.
What are you doing?
What's the amount of transactions precessed?
How much data are you loading?
etc.
etc.
etc
Probably you could profile your app and start cleaning up some modules from your app.
Apparently this can occur after redeploying an application a few times
Tomcat has hot deploy but it consumes memory. Try restarting your container once in a while. Also you will need to know the amount of memory needed to run in production mode, this seems a good time for that research.
They Say that the latest rev of Tomcat (6.0.28 or 6.0.29) handles the task of redeploying servlets much better.
I run into exactly the same problem, but unfortunately none of the suggested solutions really worked for me. The problem did not happen during deployment, and I was neither doing any hot deployments.
In my case the problem occurred every time at the same point during the execution of my web-application, while connecting (via hibernate) to the database.
This link (also mentioned earlier) did provide enough insides to resolve the problem. Moving the jdbc-(mysql)-driver out of the WEB-INF and into the jre/lib/ext/ folder seems to have solved the problem. This is not the ideal solution, since upgrading to a newer JRE would require you to reinstall the driver.
Another candidate that could cause similar problems is log4j, so you might want to move that one as well
First step in such case is to check whether the GC is allowed to unload classes from PermGen. The standard JVM is rather conservative in this regard – classes are born to live forever. So once loaded, classes stay in memory even if no code is using them anymore. This can become a problem when the application creates lots of classes dynamically and the generated classes are not needed for longer periods. In such a case, allowing the JVM to unload class definitions can be helpful. This can be achieved by adding just one configuration parameter to your startup scripts:
-XX:+CMSClassUnloadingEnabled
By default this is set to false and so to enable this you need to explicitly set the following option in Java options. If you enable CMSClassUnloadingEnabled, GC will sweep PermGen too and remove classes which are no longer used. Keep in mind that this option will work only when UseConcMarkSweepGC is also enabled using the below option. So when running ParallelGC or, God forbid, Serial GC, make sure you have set your GC to CMS by specifying:
-XX:+UseConcMarkSweepGC
Assigning Tomcat more memory is NOT the proper solution.
The correct solution is to do a cleanup after the context is destroyed and recreated (the hot deploy). The solution is to stop the memory leaks.
If your Tomcat/Webapp Server is telling you that failed to unregister drivers (JDBC), then unregister them. This will stop the memory leaks.
You can create a ServletContextListener and configure it in your web.xml. Here is a sample ServletContextListener:
import java.sql.Driver;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.util.Enumeration;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.apache.log4j.Logger;
import com.mysql.jdbc.AbandonedConnectionCleanupThread;
/**
*
* #author alejandro.tkachuk / calculistik.com
*
*/
public class AppContextListener implements ServletContextListener {
private static final Logger logger = Logger.getLogger(AppContextListener.class);
#Override
public void contextInitialized(ServletContextEvent arg0) {
logger.info("AppContextListener started");
}
#Override
public void contextDestroyed(ServletContextEvent arg0) {
logger.info("AppContextListener destroyed");
// manually unregister the JDBC drivers
Enumeration<Driver> drivers = DriverManager.getDrivers();
while (drivers.hasMoreElements()) {
Driver driver = drivers.nextElement();
try {
DriverManager.deregisterDriver(driver);
logger.info(String.format("Unregistering jdbc driver: %s", driver));
} catch (SQLException e) {
logger.info(String.format("Error unregistering driver %s", driver), e);
}
}
// manually shutdown clean up threads
try {
AbandonedConnectionCleanupThread.shutdown();
logger.info("Shutting down AbandonedConnectionCleanupThread");
} catch (InterruptedException e) {
logger.warn("SEVERE problem shutting down AbandonedConnectionCleanupThread: ", e);
e.printStackTrace();
}
}
}
And here you configure it in your web.xml:
<listener>
<listener-class>
com.calculistik.mediweb.context.AppContextListener
</listener-class>
</listener>
"They" are wrong because I'm running 6.0.29 and have the same problem even after setting all of the options. As Tim Howland said above, these options only put off the inevitable. They allow me to redeploy 3 times before hitting the error instead of every time I redeploy.
In case you are getting this in the eclipse IDE, even after setting the parameters
--launcher.XXMaxPermSize, -XX:MaxPermSize, etc, still if you are getting the same error, it most likely is that the eclipse is using a buggy version of JRE which would have been installed by some third party applications and set to default. These buggy versions do not pick up the PermSize parameters and so no matter whatever you set, you still keep getting these memory errors. So, in your eclipse.ini add the following parameters:
-vm <path to the right JRE directory>/<name of javaw executable>
Also make sure you set the default JRE in the preferences in the eclipse to the correct version of java.
The only way that worked for me was with the JRockit JVM. I have MyEclipse 8.6.
The JVM's heap stores all the objects generated by a running Java program. Java uses the new operator to create objects, and memory for new objects is allocated on the heap at run time. Garbage collection is the mechanism of automatically freeing up the memory contained by the objects that are no longer referenced by the program.
I was having similar issue.
Mine is JDK 7 + Maven 3.0.2 + Struts 2.0 + Google GUICE dependency injection based project.
Whenever i tried running mvn clean package command, it was showing following error and "BUILD FAILURE" occured
org.apache.maven.surefire.util.SurefireReflectionException: java.lang.reflect.InvocationTargetException; nested exception is java.lang.reflect.InvocationTargetException: null
java.lang.reflect.InvocationTargetException
Caused by: java.lang.OutOfMemoryError: PermGen space
I tried all the above useful tips and tricks but unfortunately none worked for me.
What worked for me is described step by step below :=>
Go to your pom.xml
Search for <artifactId>maven-surefire-plugin</artifactId>
Add a new <configuration> element and then <argLine> sub element in which pass -Xmx512m -XX:MaxPermSize=256m as shown below =>
<configuration>
<argLine>-Xmx512m -XX:MaxPermSize=256m</argLine>
</configuration>
Hope it helps, happy programming :)