I am new to API Connect, and I have installed npm, node and apic in my local MacBook. Now I have created a loopback directory and trying to run the loopback application using apic Start. However, I am not understanding the error received from the logs:
MacBook-Pro-2:socialreviews Work$ apic start
Error: Service socialreviews started but did not initialize within the timeout period. Dumping log buffer.
[Tue Apr 30 12:02:42 2019] com.ibm.diagnostics.healthcenter.loader INFO: Node Application Metrics 3.1.3.201810251210 (Agent Core 3.2.6)
[Tue Apr 30 12:02:42 2019] com.ibm.diagnostics.healthcenter.mqtt INFO: Connecting to broker localhost:1883
strong-supervisor attaching dashboard at /appmetrics-dash
2019-04-30T09:02:42.694Z pid:44844 worker:0 INFO supervisor starting (pid 44844)
2019-04-30T09:02:42.697Z pid:44844 worker:0 INFO supervisor reporting metrics to internal:
2019-04-30T09:02:42.709Z pid:44844 worker:0 INFO supervisor size set to undefined
2019-04-30T09:02:42.905Z pid:44848 worker:1 [Tue Apr 30 12:02:42 2019] com.ibm.diagnostics.healthcenter.loader INFO: Node Application Metrics 3.1.3.201810251210 (Agent Core 3.2.6)
2019-04-30T09:02:42.959Z pid:44848 worker:1 [Tue Apr 30 12:02:42 2019] com.ibm.diagnostics.healthcenter.mqtt INFO: Connecting to broker localhost:1883
2019-04-30T09:02:42.995Z pid:44848 worker:1 strong-supervisor attaching dashboard at /appmetrics-dash
2019-04-30T09:02:45.017Z pid:44848 worker:1 Web server listening at: http://0.0.0.0:4004
2019-04-30T09:24:29.148Z pid:44844 worker:0 WARN received SIGTERM, shutting down
2019-04-30T09:24:29.148Z pid:44844 worker:0 INFO supervisor size set to undefined
2019-04-30T09:24:29.148Z pid:44844 worker:0 INFO supervisor stopped
Can someone help me with this issue please?
This issue is solved, however I am not 100% sure what was the exact issue. What I have done is ran apic edit command then when clicking on start it suddenly asked me to install docker. I have done so, then I started to have readable errors of some packages not installed. So I used npm commands to install them and my issue is fixed now! :)
Related
Jenkins: 2.89.4-x rolling version
Jenkins was running slow due to memory issues.
Restarted Jenkins (sudo/usual way), I got the following SEVERE issue.
Finally Restarted Jenkins machine thinking that'll help; got memory back (mem looks good now).
Trying to start Jenkins again and getting the same SEVERE issue and installation just HANGS at a given plugin while doing Initializing plugin <somePlugin-here> by pool-6-thread-2.
Running from: /usr/lib/jenkins/jenkins.war
Jun 06, 2019 6:49:14 AM Main deleteWinstoneTempContents
WARNING: Failed to delete the temporary Winstone file /tmp/winstone/jenkins.war
Jun 06, 2019 6:49:14 AM org.eclipse.jetty.util.log.Log initialized
..
...more lines here...
...
..
Jun 06, 2019 6:49:19 AM jenkins.model.Jenkins$5 runTask
INFO: Took 54ms for LogRecorderManager.init by pool-6-thread-7
Jun 06, 2019 6:49:19 AM com.cloudbees.jenkins.plugins.updates.envelope.LoggerLog log
SEVERE: Unable to parse provided JSON
net.sf.json.JSONException: A JSONObject text must begin with '{' at character 0 of
at net.sf.json.util.JSONTokener.syntaxError(JSONTokener.java:499)
at net.sf.json.JSONObject._fromJSONTokener(JSONObject.java:919)
at net.sf.json.JSONObject._fromString(JSONObject.java:1145)
at net.sf.json.JSONObject.fromObject(JSONObject.java:162)
at net.sf.json.JSONObject.fromObject(JSONObject.java:132)
at com.cloudbees.jenkins.plugins.updates.envelope.JSONObjectLoader.from(JSONObjectLoader.java:38)
at com.cloudbees.jenkins.plugins.updates.envelope.JSONObjectLoader$1.apply(JSONObjectLoader.java:51)
at com.cloudbees.jenkins.plugins.updates.envelope.JSONObjectLoader$1.apply(JSONObjectLoader.java:47)
at com.cloudbees.jenkins.plugins.updates.envelope.Validation$Value.flatMap(Validation.java:229)
at com.cloudbees.jenkins.plugins.updates.envelope.JSONObjectLoader.from(JSONObjectLoader.java:47)
at com.cloudbees.jenkins.plugins.updates.envelope.ParsedEnvelope$Loader.fromJSON(ParsedEnvelope.java:95)
at com.cloudbees.jenkins.cjp.installmanager.BaseFolder.loadInstalled(BaseFolder.java:136)
at com.cloudbees.jenkins.cjp.installmanager.WAREnvelope.loadInstalled(WAREnvelope.java:140)
at com.cloudbees.jenkins.cjp.installmanager.CJPPluginManager$StartUp.execute(CJPPluginManager.java:297)
at com.cloudbees.jenkins.cjp.installmanager.CJPPluginManager.loadPlugins(CJPPluginManager.java:231)
at com.cloudbees.jenkins.cjp.installmanager.CJPPluginManager.loadBundledPlugins(CJPPluginManager.java:209)
at hudson.PluginManager$1$1.run(PluginManager.java:379)
at org.jvnet.hudson.reactor.TaskGraphBuilder$TaskImpl.run(TaskGraphBuilder.java:169)
at org.jvnet.hudson.reactor.Reactor.runTask(Reactor.java:282)
at jenkins.model.Jenkins$5.runTask(Jenkins.java:1066)
at org.jvnet.hudson.reactor.Reactor$2.run(Reactor.java:210)
at org.jvnet.hudson.reactor.Reactor$Node.run(Reactor.java:117)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Jun 06, 2019 6:49:19 AM com.cloudbees.jenkins.plugins.updates.envelope.LoggerLog log
SEVERE: Unable to read JSON from [/var/lib/jenkins/cb-envelope/envelope.json]
Jun 06, 2019 6:49:19 AM jenkins.model.Jenkins$5 runTask
..
..
..
..
..
...more lines here...regarding inspecting/installing/loading/extracting plugins .hpi/.jpi files (looks good no ERROR/FATAL/SEVERE lines in this area).
...
..
..
..
..
..
Jun 06, 2019 6:49:34 AM jenkins.model.Jenkins$5 runTask
INFO: Took 0ms for Initializing plugin ColumnPack-plugin by pool-6-thread-2
Jun 06, 2019 6:49:34 AM jenkins.model.Jenkins$5 runTask
INFO: Took 274ms for Discovering plugin initialization tasks by pool-6-thread-10
Jun 06, 2019 6:49:34 AM jenkins.model.Jenkins$5 runTask
INFO: Took 0ms for SupportPlugin.threadDumpStartup by pool-6-thread-15
Jun 06, 2019 6:49:34 AM jenkins.model.Jenkins$5 runTask
INFO: Took 2ms for XStreamDOMCompat.addAliases by pool-6-thread-16
The above log shows only has 1 SEVERE error about the following:
SEVERE: Unable to parse provided JSON - [/var/lib/jenkins/cb-envelope/envelope.json]
Questions:
1. What is this cb-envelop folder in JENKINS_HOME?
2. Why Jenkins is not coming up?
I see this folder has 0 byte envelop.json file and both cje-xx folders contain nothing, example ls output shown below.
[giga123#jenkins_dev_machine ~]$ ls -l /var/lib/jenkins/cb-envelope/
total 8
drwxr-xr-x 2 jenkins_svc_user jenkins_group 4096 Jul 3 2017 cje-2.46.3.2
drwxr-xr-x 2 jenkins_svc_user jenkins_group 4096 Apr 20 2018 cje-2.89.4.2
-rw-r--r-- 1 jenkins_svc_user jenkins_group 0 Mar 27 2018 envelope.json
-rw-r--r-- 1 jenkins_svc_user jenkins_group 0 Jul 5 2017 envelope.json.disabled
[giga123#jenkins_dev_machine ~]$ ls -l /var/lib/jenkins/cb-envelope/cje-2.89.4.2/
total 0
Solution:
Soft fix: Renamed the envelop.json file to envelop.json.disabled.
Try Jenkins start now (ex: sudo /sbin/service jenkins stop/start and after start Jenkins should come up fine. Didn't catch that this issue came sometime back and someone did disable this file for Jenkins to startup.
The issue was a BLANK json file (thus '{' parse SEVERE error was coming up).
Hard fix: Rename /var/lib/jenkins/cb-envelope/ to /var/lib/jenkins/cb-envelope.OLD
Try Jenkins start now (ex: sudo /sbin/service jenkins stop/start and after start Jenkins should come up fine. cb-envelop folder will be recreated (if renamed/deleted) after Jenkins start.
Once this was done net.sf.json.JSONException: A JSONObject text must begin with '{' at character 0 of error went away and Jenkins installation went a little further BUT installation halted at a given plugin. I disabled that plugin (by going to $JENKINS_HOME/plugins/<plugin.jpi_or_hpi>.disabled and retried to start Jenkins, now it halted at another next plugin in the list of plugins during Initializing plugin .... plugin... steps/sub-steps.
To resolve that, we had to contact Cloudbees support team and disable support-core.jpi plugin. This plugin comes with Cloudbees Jenkins Operation Center / Master installation by default as one of the .jpi files.
After support-core.jpi file in plugins folder was renamed to support-core.jpi.disabled (you can move support-core folder to support-core.OLD if you want), we retried starting Jenkins and it came back up fine.
One last thing we noticed was, after Jenkins dashboard was finally UP, it was in QUITE and SHUTDOWN mode (by design). To remove state that, we had to click on:
As an admin level user account, Go to Jenkins > Manage Jenkins > Quiet Restart and UN-CHECK the check box which says Stay in "queiting down" state when restarted.
Remove the check from the following.
On Jenkins dashboard, click cancel on Shut down mode link (if any) and now Jenkins is up and running fine.
After building the app, the deployment process works. The SpringBoot (mysql) application is deployed and marked as the usual "Starting the application".
After a few seconds, there is a message saying it "kills" the application. Notice, I am just within the limits of my Openshift free quota.
What is the mesage? .../wildfly/bin/standalone.sh: line 307: 183 Killed "java" -D"[Standalone]"
See the line in bold.
On another time I see is that the application is started, I can even login, but then after a minute is is killed.
14:47:15,334 INFO [nl.xyz.Application] (ServerService Thread Pool --
66) Started Application in 26.49 seconds (JVM running for 57.227)
14:47:15,723 INFO [javax.enterprise.resource.webcontainer.jsf.config]
(ServerService Thread Pool -- 66) Initializing Mojarra 2.2.13.SP1
20160303-1204 for context '' /wildfly/bin/standalone.sh: line 307: 183
Killed "java" -D"[Standalone]" -server -XX:+UseParallelGC -Xms40m
-Xmx250m -XX:+AggressiveOpts -XX:MinHeapFreeRatio=20 -XX:MaxHeapFreeRatio=40 -XX:GCTimeRatio=4 -XX:AdaptiveSizePolicyWeight=90 -Dorg.apache.tomcat.util.LOW_MEMORY=true -DOPENSHIFT_APP_UUID= -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Dorg.jboss.resolver.warning=true -Djava.net.preferIPv4Stack=true -Dfile.encoding=UTF-8 -Djboss.node.name=solvedcaches-5-80fj7 -Djgroups.bind_addr=0.0.0.0 -Dorg.apache.coyote.http11.Http11Protocol.COMPRESSION=on "-Dorg.jboss.boot.log.file=/wildfly/standalone/log/server.log"
"-Dlogging.configuration=file:/wildfly/standalone/configuration/logging.properties"
-jar "/wildfly/jboss-modules.jar" -mp "/wildfly/provided_modules:/wildfly/modules" org.jboss.as.standalone
-Djboss.home.dir="/wildfly" -Djboss.server.base.dir="/wildfly/standalone" '-b' '0.0.0.0' '-bmanagement' '0.0.0.0'
The last 2 build failed due to a generic build failture. Strange, the code was hardly changed. It looks like rebuilding takes memory and is competing with the running application and vice versa. The last time I stopped the POD, then deployment was ok.
Thank you #Jiri Fialka for all your help, online / offline.
As you suggested, after configuring the memory over the 2 POD's the application is working fine. Now I use 410 Mb for Mysql and 600 Mb for SpringBoot was WAR on a Wildfly pod. Even SSH was added within minutes!.
I am enthusiastic about the new Openshift v3 facilities. Also the GUI and oc.
I installed sonarqube-6.3.1 in my machine and created a database in mysql db named 'sonarqubedb'. Now when I am making changes in sonar.properties file to use the database, sonarqube is not getting started and throwing error msg, but If I am using the default DB configuration (and NOT mysql), I am able to start.
Could somebody please provide me a solution what is going wrong when I am using mysql db.
My sonar.properties file be like:
sonar.jdbc.username=root
sonar.jdbc.password=
sonar.jdbc.url=jdbc:mysql://localhost:3306/sonarqubedb?useUnicode=true&characterEncoding=utf8&rewriteBatchedStatements=true&useConfigs=maxPerformance
and the sonar log file when I try to start service, be like:
--> Wrapper Started as Service
Launching a JVM...
WrapperManager class initialized by thread: main Using classloader:
sun.misc.Launcher$AppClassLoader#4e25154f
Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
Wrapper Manager: JVM #1
Running a 64-bit JVM.
Wrapper Manager: Registering shutdown hook
Wrapper Manager: Using wrapper
Load native library. One or more attempts may fail if platform specific
libraries do not exist.
Loading native library failed: wrapper-windows-x86-64.dll Cause:
java.lang.UnsatisfiedLinkError: no wrapper-windows-x86-64 in
java.library.path
Loaded native library: wrapper.dll
Calling native initialization method.
Initializing WrapperManager native library.
Java Executable: C:\ProgramData\Oracle\Java\javapath\java.exe
Windows version: 6.1.7600
Java Version : 1.8.0_45-b15 Java HotSpot(TM) 64-Bit Server VM
Java VM Vendor : Oracle Corporation
Control event monitor thread started.
Startup runner thread started.
WrapperManager.start(org.tanukisoftware.wrapper.WrapperSimpleApp#4f023edb, args[]) called by thread: main
Communications runner thread started.
Open socket to wrapper...Wrapper-Connection
Failed attempt to bind using local port 31000
Opened Socket from 31001 to 32000
Send a packet KEY : 4hhDEyNqmPXAiWpf
handleSocket(Socket[addr=/127.0.0.1,port=32000,localport=31001])
Received a packet LOW_LOG_LEVEL : 1
Wrapper Manager: LowLogLevel from Wrapper is 1
Received a packet PING_TIMEOUT : 0
PingTimeout from Wrapper is 0
Received a packet PROPERTIES : (Property Values)
Received a packet START : start
calling WrapperListener.start()
Waiting for WrapperListener.start runner thread to complete.
WrapperListener.start runner thread started.
WrapperSimpleApp: start(args) Will wait up to 2 seconds for the main
method to complete.
WrapperSimpleApp: invoking main method
2017.04.26 14:54:12 INFO app[][o.s.a.AppFileSystem] Cleaning or creating
temp directory C:\Program Files\Sonar\sonarqube-6.3.1\sonarqube-6.3.1\temp
2017.04.26 14:54:12 INFO app[][o.s.p.m.JavaProcessLauncher] Launch
process[es]: C:\Program Files\Java\jre1.8.0_45\bin\java -
Djava.awt.headless=true -Xmx1G -Xms256m -Xss256k -Djna.nosys=true -
XX:+UseParNewGC -XX:+UseConcMarkSweepGC -
XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -
XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=C:\Program
Files\Sonar\sonarqube-6.3.1\sonarqube-6.3.1\temp -javaagent:C:\Program
Files\Java\jre1.8.0_45\lib\management-agent.jar -cp
./lib/common/*;./lib/search/* org.sonar.search.SearchServer C:\Program
Files\Sonar\sonarqube-6.3.1\sonarqube-6.3.1\temp\sq-
process3041279828124660880properties
Send a packet START_PENDING : 5000
Send a packet START_PENDING : 5000
WrapperSimpleApp: start(args) end. Main Completed=false, exitCode=null
WrapperListener.start runner thread stopped.
returned from WrapperListener.start()
Send a packet STARTED :
Startup runner thread stopped.
Received a packet PING : ping
Send a packet PING : ok
Received a packet PING : ping
Send a packet PING : ok
2017.04.26 14:54:23 INFO app[][o.s.p.m.Monitor] Process[es] is up
2017.04.26 14:54:23 INFO app[][o.s.p.m.JavaProcessLauncher] Launch
process[web]: C:\Program Files\Java\jre1.8.0_45\bin\java -
Djava.awt.headless=true -Dfile.encoding=UTF-8 -Xmx512m -Xms128m -
XX:+HeapDumpOnOutOfMemoryError -Djava.io.tmpdir=C:\Program
Files\Sonar\sonarqube-6.3.1\sonarqube-6.3.1\temp -javaagent:C:\Program
Files\Java\jre1.8.0_45\lib\management-agent.jar -cp
./lib/common/*;./lib/server/*;C:\Program Files\Sonar\sonarqube-
6.3.1\sonarqube-6.3.1\lib\jdbc\mysql\mysql-connector-java-5.1.39.jar
org.sonar.server.app.WebServer C:\Program Files\Sonar\sonarqube-
6.3.1\sonarqube-6.3.1\temp\sq-process5745752416531116392properties
Received a packet PING : ping
Send a packet PING : ok
2017.04.26 14:54:28 INFO app[][o.s.p.m.Monitor] Process[es] is stopping
2017.04.26 14:54:28 ERROR app[][o.s.p.m.Monitor] Process[web] failed to
start
2017.04.26 14:54:28 INFO app[][o.s.p.m.Monitor] Process[es] is stopped
Wrapper Manager: ShutdownHook started
WrapperManager.stop(0) called by thread: Wrapper-Shutdown-Hook
Send a packet STOP : 0
Received a packet STOP :
Thread, Wrapper-Shutdown-Hook, handling the shutdown process.
calling listener.stop()
WrapperSimpleApp: stop(0)
returned from listener.stop() -> 0
shutdownJVM(0) Thread:Wrapper-Shutdown-Hook
Send a packet STOPPED : 0
Closing socket.
Server daemon shut down
Wrapper Manager: ShutdownHook complete
<-- Wrapper Stopped
Thanks in Advance :)
Go to your sonarqube/logs directory. You'll find several log files and one of them will contain the detailed error on why sonarqube won't start.(you'll have to scroll all the way down inside the files for the latest information iirc)
Go through this sonar documentation, you will get some help -
https://docs.sonarqube.org/display/SONAR/Installing+the+Server
I had the same issue.
The problem was that my MySQL version didn't meet the minimum requirements.
for the last four days i am facing a problem on my production server where Tomcat stops responding, and when i try to shut it down via shutdow.sh tomcat process stays alive. i will have to kill the process and start it again.
the below stack is logged directly before tomcat crashes and stops responding. i did lots of research but couldn't solve the problem just yet.
any help is appreciated
there are two SEVERE web application errors
Jun 9, 2012 4:50:08 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/beta] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Jun 9, 2012 4:50:08 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/beta] appears to have started a thread named [MultiThreadedHttpConnectionManager cleanup] but has failed to stop it. This is very likely to create a memory leak.
and one MySQL error
INFO: Illegal access: this web application instance has been stopped already. Could not load com.mysql.jdbc.SQLError. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1587)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1546)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3358)
at com.mysql.jdbc.MysqlIO.quit(MysqlIO.java:1695)
at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4388)
at com.mysql.jdbc.ConnectionImpl.cleanup(ConnectionImpl.java:1368)
at com.mysql.jdbc.ConnectionImpl.finalize(ConnectionImpl.java:2737)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
full stack below
INFO: Destroying Spring FrameworkServlet 'springMvcServlet'
Jun 9, 2012 4:50:08 PM org.apache.catalina.core.ApplicationContext log
INFO: Closing Spring root WebApplicationContext
Jun 9, 2012 4:50:08 PM org.apache.catalina.loader.WebappClassLoader clearReferencesJdbc
SEVERE: The web application [/beta] registered the JDBC driver [com.mysql.jdbc.Driver] but failed to unregister it when the web application was stopped. To prevent a memory leak, the JDBC Driver has been forcibly unregistered.
Jun 9, 2012 4:50:08 PM org.apache.catalina.loader.WebappClassLoader clearReferencesThreads
SEVERE: The web application [/beta] appears to have started a thread named [MultiThreadedHttpConnectionManager cleanup] but has failed to stop it. This is very likely to create a memory leak.
Jun 9, 2012 4:50:09 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/home/bratecp/public_html/beta/WEB-INF/lib/ImageEditor.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
Jun 9, 2012 4:50:09 PM org.apache.catalina.loader.WebappClassLoader validateJarFile
INFO: validateJarFile(/home/bratecp/public_html/beta/WEB-INF/lib/gwt-user.jar) - jar not loaded. See Servlet Spec 2.3, section 9.7.2. Offending class: javax/servlet/Servlet.class
Jun 9, 2012 4:50:09 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring root WebApplicationContext
Jun 9, 2012 4:50:11 PM org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already. Could not load com.mysql.jdbc.SQLError. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1587)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1546)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3358)
at com.mysql.jdbc.MysqlIO.quit(MysqlIO.java:1695)
at com.mysql.jdbc.ConnectionImpl.realClose(ConnectionImpl.java:4388)
at com.mysql.jdbc.ConnectionImpl.cleanup(ConnectionImpl.java:1368)
at com.mysql.jdbc.ConnectionImpl.finalize(ConnectionImpl.java:2737)
at java.lang.ref.Finalizer.invokeFinalizeMethod(Native Method)
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:83)
at java.lang.ref.Finalizer.access$100(Finalizer.java:14)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:160)
Jun 9, 2012 4:58:44 PM org.apache.catalina.core.AprLifecycleListener init
INFO: The APR based Apache Tomcat Native library which allows optimal performance in production environments was not found on the java.library.path: /usr/local/jdk1.6.0_29/jre/lib/amd64/server:/usr/local/jdk1.6.0_29/jre/lib/amd64:/usr/local/jdk1.6.0_29/jre/../lib/amd64:/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
Jun 9, 2012 4:58:44 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'compression' to 'on' did not find a matching property.
Jun 9, 2012 4:58:44 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'compressionMinSize' to '2048' did not find a matching property.
Jun 9, 2012 4:58:44 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'noCompressionUserAgents' to 'gozilla, traviata' did not find a matching property.
Jun 9, 2012 4:58:44 PM org.apache.catalina.startup.SetAllPropertiesRule begin
WARNING: [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'compressableMimeType' to 'text/xml,text/plain,application/json,application/javascript,text/css' did not find a matching property.
Jun 9, 2012 4:58:44 PM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '1' did not find a matching property.
Jun 9, 2012 4:58:44 PM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '0' did not find a matching property.
Jun 9, 2012 4:58:44 PM org.apache.tomcat.util.digester.SetPropertiesRule begin
WARNING: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'debug' to '1' did not find a matching property.
Jun 9, 2012 4:58:45 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-bio-8080"]
Jun 9, 2012 4:58:45 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["ajp-bio-8009"]
Jun 9, 2012 4:58:45 PM org.apache.catalina.startup.Catalina load
There are at least two problems occurring simultaneously:
You are registering a JDBC driver that is being loaded from within your webapp's WEB-INF/lib directory and failing to deregister the driver when your webapp shuts down. You can deregister a JDBC driver easily using java.sql.DriverManager.deregisterDriver() from a ServletContextListener.
You (likely) have a non-daemon thread outliving your webapp. The only way to determine what is going on here is to take a thread dump to find out which thread is still alive (after you run shutdown.sh and wait maybe 5 seconds for everything to settle down) and where it may have come from. Basically, anywhere you start a thread in your webapp needs to have a symmetrical stop-thread when your webapp shuts down. Remember that certain operations start threads without you realizing it (for instance, creating a TimerTask, performing certain AWT-related operations, etc.).
You probably want to fix both of these problems to improve the stability of your application server.
In your stack trace above, it's a Connection finalizer that is trying to "really" close the connection to the database. I wonder if you aren't properly shutting-down your Connection objects (or a connection pool) before your webapp shuts down, and so the Connection finalizers are running after they can actually accomplish their goals (after the ClassLoader is dead).
Am experienced with Django, Apache, WSGI, MySql, etc. and have this environment setup and working fine on another OS X computer running Lion. Django session backend is db, session middleware and app are both enabled properly in settings. During first request of site view, we set a request.session key/value, which this first time thru works fine. On subsequent view though when we check if the key/value exists, we get a server-level 500 error that even with Debug mode on doesn't make it to the python interpreter to generate a full stack trace exception. The apache log generates the following messages...
[Mon Apr 16 14:26:22 2012] [notice] Apache/2.2.21 (Unix) DAV/2 mod_wsgi/3.3 Python/2.7.1 mod_ssl/2.2.21 OpenSSL/0.9.8r configured -- resuming normal operations
[Mon Apr 16 14:26:27 2012] [info] mod_wsgi (pid=2362): Create interpreter 'snap.joe|'.
[Mon Apr 16 14:26:27 2012] [info] [client 127.0.0.1] mod_wsgi (pid=2362, process='snap', application='snap.joe|'): Loading WSGI script '/var/www/venvs/snap_env/snap/wsgi/wsgi.py'.
[Mon Apr 16 14:26:32 2012] [error] [client 127.0.0.1] Premature end of script headers: wsgi.py
[Mon Apr 16 14:26:33 2012] [notice] child pid 2362 exit signal Bus error (10)
Have checked that a MySQL db table django_session row is added properly with session_data and also the cookie set in the browser after the first request contains the proper/matching session_id. If I add a simple request.session.flush() just before the existing session code, we can bypass the error obviously because there is never anything in the session. One more thing, the value we are adding to the session key/value store is a Django QuerySet object. Again, this is working on a live CentOS server, and also another Mac OS X Lion machine (albeit running Python 2.6 instead of 2.7).
Any ideas folks? Many thanks!
See documented reasons at:
http://code.google.com/p/modwsgi/wiki/FrequentlyAskedQuestions#Apache_Process_Crashes
and follow further links for possible workarounds.
In short, can be shared library version conflict, using mod_python at same time, or extension module for Python that doesn't work with sub interpreter.
If this is only Python site on that Apache, set:
WSGIApplicationGroup %{GLOBAL}
for one potential quick solution.
A few thoughts:
1) are you running the same versions of modules? Django just recently updated (1.4), so if you did a pip install and didn't specify a version your production might have an older version while your development has the newer one.
2) Can you test this using just the manage.py runserver? My suspicions are that it's something to do with the apache config (so if it works fine in runserver, that means the issue is with apache).