JavaMelody JDBC Graph not working Tomcat 8.8 and mysql DB - java-melody

I have configured javamelody 1.86.0 on Tomcat 8.8 and all the other metrics are captured except from the JDBC. Below are the config I have on the Tomcat application:
Context.xml:
<ResourceLink type="javax.sql.DataSource"
name="jdbc/LocalAPIDB"
global="jdbc/APIDB"/>
Server.xml:
<Resource type="javax.sql.DataSource"
name="jdbc/APIDB"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/apiDB"
username="xxxx"
password="xxxxx"
initialSize="340"
maxActive="3770"
maxIdle="2330"
minIdle="890"
testOnBorrow="true"
validationQuery="SELECT 1"
validationInterval="30000"
timeBetweenEvictionRunsMillis="5000"
minEvictableIdleTimeMillis="60000"
removeAbandoned="true"
removeAbandonedTimeout="60"
abandonWhenPercentageFull="0"
logAbandoned="true"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;
org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer"/>
On mbeans I can see there are active connections but on the graph nothing is captured.
JDBC Graph
MBeans
Debugging Logs

I suggest that you move the Resource from server.xml to the xml context file of the webapp (conf/Catalina/localhost/MpesaIntegrations.xml) and that you remove the ResourceLink.
So the xml context file of the webapp is like this:
<?xml version="1.0" encoding="UTF-8" ?>
<Context>
<Resource type="javax.sql.DataSource"
name="jdbc/LocalAPIDB"
factory="org.apache.tomcat.jdbc.pool.DataSourceFactory"
driverClassName="com.mysql.jdbc.Driver"
url="jdbc:mysql://localhost:3306/apiDB"
username="xxxx"
password="xxxxx"
initialSize="340"
maxActive="3770"
maxIdle="2330"
minIdle="890"
testOnBorrow="true"
validationQuery="SELECT 1"
validationInterval="30000"
timeBetweenEvictionRunsMillis="5000"
minEvictableIdleTimeMillis="60000"
removeAbandoned="true"
removeAbandonedTimeout="60"
abandonWhenPercentageFull="0"
logAbandoned="true"
jdbcInterceptors="org.apache.tomcat.jdbc.pool.interceptor.ConnectionState;org.apache.tomcat.jdbc.pool.interceptor.StatementFinalizer;org.apache.tomcat.jdbc.pool.interceptor.ResetAbandonedTimer"
/>
</Context>
And/or, if that's not enough, you probably have a listener or filter in your webapp that is getting and keeping in cache the DataSource. In that case, put the javamelody listener and filter near the top of your WEB-INF/web.xml file.

For anybody scratching their head trying to find out why JavaMelody won't show database stats such as 'active JDBC connections', I finally solved it by doing 2 things. Note that in my case I'm using Java 11 running Apache Tomcat 9 hosting an Apache Derby 10.15.2 database:
Setting up the database in the 'apache-tomcat-9.0.52\conf\context.xml' file eather than in java code in my webapp. I think this made the database associated with the JNDI resource name so JavaMelody could see it. I added this XML to that context.xml file:
<Resource name="jdbc/DatabaseJNDIResourceName"
auth="Container"
type="javax.sql.DataSource"
driverClassName="org.apache.derby.jdbc.EmbeddedDriver"
url="jdbc:derby:../DatabaseName;"
username="someuser"
password="s3cret"
/>
Then you can get the DataSource in your webapp java code using:
InitialContext initialContextJNDI = new InitialContext();
DataSource dataSource = (DataSource)initialContextJNDI.lookup("java:comp/env/jdbc/DatabaseJNDIResourceName");
Note that a method to detect a potential JNDI resource name problem is to see if the resource name shows up in JavaMelody by navigating to the page 'JNDI tree' at the bottom, linked here http://localhost:8080/monitoring?part=jndi
Inserting the web.xml code referred to here. Interestingly, when that XML code was left out, javamelody would still run but it didn't report JDBC-related data.
<filter>
<filter-name>javamelody</filter-name>
<filter-class>net.bull.javamelody.MonitoringFilter</filter-class>
<async-supported>true</async-supported>
</filter>
<filter-mapping>
<filter-name>javamelody</filter-name>
<url-pattern>/*</url-pattern>
<dispatcher>REQUEST</dispatcher>
<dispatcher>ASYNC</dispatcher>
</filter-mapping>
<listener>
<listener-class>net.bull.javamelody.SessionListener</listener-class>
</listener>
Another thing, even after these fixes finally solved the problem and made JavaMelody aware of my database, I didn't immediately notice since the graphs barely change if your database queries are fast. I only confirmed that the database stats were being logged after executing a long SQL query and updating the 'opened jdbc connections' page, linked at the bottom of the javamelody stats page here, which showed one single connection while the database took a few seconds to process the large (1 million row table) query http://localhost:8080/monitoring?part=connections

Related

Google Omaha installer hangs

I'm trying to get Google Omaha to work, and even though I feel I'm really close for some reason it is not.
I've read the few guides and tutorials available online for it and finally managed to get the client built. I've also installed the docker package provided by Crystalnix and as far as I can tell everything is setup as it should.
For some reason however the installation process gets stuck after the API request for the package information is made.
The last response from the server looks as follows:
<response protocol="3.0" server="prod">
<daystart elapsed_seconds="62157" elapsed_days="4372"/>
<app status="ok" appid="{XXX-XXX-XXX}">
<updatecheck status="ok">
<urls>
<url codebase="http://updates.example.com/static/media/build/Product/stable/win/78065329307666/"/>
</urls>
<manifest version="71.0.57.18">
<packages>
<package required="true" hash="6uZwsU9+WCZ1oR3ovGnFyrTCqhM=" name="install.exe" size="59309528"/>
</packages>
<actions>
<action successsaction="default" run="install.exe" event="install"/>
</actions>
</manifest>
</updatecheck>
</app>
</response>
After this request is being retried for three times the installer hangs with this error: Unable to connect to the Internet. If you use a firewall, please whitelist ProductUpdate.exe
What could the issue for this be?
LE: this is the complete Omaha log file https://pastebin.com/QALnk7X7

Tomcat 8.0.9 manager app reporting "401 Unauthorized" despite my trying everything

I have an instance of Tomcat 8.0.9, running on GNU/Linux 2.6.32-642.6.2.e16.x86_64, that responds with "401 Unauthorized" when I try to access the manager UI, no matter what I try.
I know Tomcat is running, because I get that page.
I know it is that specific instance of Tomcat, because if I shut it down and revisit the URL (https: //host.name:port/manager/html), I get a connection refused.
The browser doesn't even prompt me for username and password; it just goes straight to the "401 Unauthorized" page.
The server's catalina.out log even reports that it has deployed the manager webapp (paraphrased):
*.a few seconds ago* INFO [localhost-startStop-1] o.a.c.s.H.deployDirectory Deploying web application directory /path/to/webapps/manager
*milliseconds later* INFO [localhost-startStop-1] o.a.c.s.H.deployDirectory Deployment of web application directory /path/to/webapps/manager has finished in 22 ms
I started with the Apache docs (https: //host.name:port/docs/manager-howto.html), and then with several versions of this question on SO for troubleshooting, including one that seems to have the most answers.
Since I've never used the manager before, my go-to theory is that 'obvious' was exactly what I was missing; however, I tried all of the obvious stuff below:
I restarted Tomcat, multiple times in this process, verifying that a reload of the URL was indeed firing a connection refused with the server stopped, and then back to the 401 error with the server started.
I added admin-gui to the user's roles:
<role rolename="admin-gui" />
<role rolename="manager-gui" />
<user username="tomcat" password="s3cret" roles="admin-gui,manager-gui" />
The XML above is not inside a comment block - if I edit it in Gvim with syntax highlighting turned on, this is very obvious.
tomcat-users.xml is owned by tomcat, and is readable (it's mode 775, in fact). I know it's being read, because if I add nonsense elements to it, catalina.out reports this on startup.
I also added the other role to tomcat-users.xml, just in case (manager-jmx, admin-script, etc.). (I'm still not sure what's supposed to happen if you left those out, misspelled them, etc. I imagine the manager webapp wouldn't recognize rolenames outside its specific set, but naturally I can't confirm this yet.)
The following element exists in server.xml, inside an Engine element:
<Realm className="org.apache.catalina.realm.LockOutRealm">
<Realm className="org.apache.catalina.realm.UserDatabaseRealm" resourceName="UserDatabase" />
</Realm>
In case it's relevant, there's also this, earlier in server.xml:
<GlobalNamingResources>
<Resource auth="Container" factory="org.apache.catalina.users.MemoryUserDatabaseFactory" name="UserDatabase" pathname="conf/tomcat-users.xml" type="org.apache.catalina.UserDatabase" />
</GlobalNamingResources>
In the Tomcat startup script, CATALINA_HOME points to the exact install path of Tomcat. There is no TOMCAT_HOME or JAVA_HOME, but JRE_HOME points to a symbolic link that points to an actual Java install (I followed that link and confirmed).
In the manager webapp, META-INF/context.xml, the Valve element is commented out.
Is there even a report of why I'm unauthorized, that gives me more information? I've tried scouring the manager webapp directory in the hopes there was yet another log there. (There isn't.)

How to deploy the broadleaf demo on my own Tomcat server?

I was able to start the Broadleaf demo application from the provided scripts adminstart.sh and startsite.sh. I was also able to change the database to MySQl. However, I do not understand how to create a war that can run on my Tomcat instance.
I see that broadleaf does things differently as compared to the common approach. Usually I put the context values into the file application-context. xml that is referred in web.xml. There is no need for modifying Tomcat context.xml. The example is
<context-param>
<param-name>contextConfigLocation</param-name>
<param-value>classpath:spring/application-context.xml</param-value>
</context-param
In the broadleaf configuration there are several context parameters
<context-param>
<param-name>webAppRootKey</param-name>
<param-value>blAdmin.root</param-value>
</context-param>
<context-param>
<param-name>patchConfigLocation</param-name>
<param-value>
classpath:/bl-open-admin-contentClient-applicationContext.xml
classpath:/bl-open-admin-contentCreator-applicationContext.xml
classpath:/bl-admin-applicationContext.xml
classpath:/bl-cms-contentClient-applicationContext.xml
classpath:/bl-cms-contentCreator-applicationContext.xml
classpath*:/blc-config/admin/bl-*-applicationContext.xml
classpath:/applicationContext.xml
classpath:/applicationContext-email.xml
/WEB-INF/applicationContext-datasource.xml
/WEB-INF/applicationContext-admin-security.xml
/WEB-INF/applicationContext-admin-filter.xml
/WEB-INF/applicationContext-admin.xml
</param-value>
</context-param>
<context-param>
<param-name>shutdownHookMethod</param-name>
<param-value>forceFlush</param-value>
</context-param>
Plus in addition in web.xml:
<!-- Initialize spring mvc -->
<servlet>
<servlet-name>admin</servlet-name>
<servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
<init-param>
<param-name>contextConfigLocation</param-name>
<param-value>
classpath:/applicationContext-servlet-open-admin.xml
classpath:/applicationContext-servlet-admin.xml
classpath:/applicationContext-servlet-cms-contentClient.xml
classpath:/applicationContext-servlet-cms-contentCreator.xml
classpath:/bl-menu-admin-applicationContext-servlet.xml
classpath*:/blc-config/admin/bl-*-admin-applicationContext-servlet.xml
/WEB-INF/applicationContext-servlet-admin.xml
</param-value>
</init-param>
<load-on-startup>1</load-on-startup>
</servlet>
The result is that the Tomcat instance contains context.xml in the conf directory that provides database pool configuration:
What is the best way of utilizing broadleaf in my application?
Configuration
Put your MySQL JDBC driver to tomcat7/lib folder
Open web.xml for both, site and admin modules in DemoSite
Add the following code in both web.xml
<resource-ref>
<description>WEB</description>
<res-ref-name>jdbc/web</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>
<resource-ref>
<description>STORAGE</description>
<res-ref-name>jdbc/storage</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>
<resource-ref>
<description>SECURE</description>
<res-ref-name>jdbc/secure</res-ref-name>
<res-type>javax.sql.DataSource</res-type>
<res-auth>Container</res-auth>
</resource-ref>
Open cmd prompt for windows or command shell for linux
Navigate to your JDK_directory/bin and create keystore for SSL switching
keytool -genkey -alias tomcat -keyalg RSA -storepass password
Open Tomcat7/conf/server.xml
Add the following code to enable SSL switching, just below your default connector port
<Connector
protocol="org.apache.coyote.http11.Http11NioProtocol"
port="8443" maxThreads="200"
scheme="https" secure="true"
SSLEnabled="true"
keystorePass="password"
clientAuth="false"
sslProtocol="TLS"
/>
Open Tomcat7/conf/tomcat-users.xml
Add following
<user username="tomcat" password="password" roles="manager-gui"/>
Populate database.properties with values for your database or
Configure JNDI for tomcat
In your admin or site folder there is file target/tomcat-server-config/context.xml.
Copy this file into tomcat7/conf/
Open this file and replace the following properties with your database values
username="${database.user}" : Replace with your database username (eg. broadleaf)
password="${database.password}" : Replace with your database password
driverClassName="${database.driver}" : Replace with your database classname (eg. org.postgresql.Driver)
url="${database.url}" : Replace with your database URL with database path, port and name. (eg. jdbc:postgresql://localhost:5432/broadleaf)
in your IDE Start Maven project install goal of lifecycle (of entire project or each module)
This will create wars for each module
Start tomcat (run startup.bat/startup.sh in ~/tomcat7/bin/ )
Go to app manager and enter tomcat username and key that you added into tomcat-users.xml
deploy admin and site wars
context path is url mapping (eg. /admin and /demo-site)
XML configuration url is full path to your targets context.xml (eg. ~/target/tomcat-server-conf/context.xml)
WAR directory is full path to your war in target directory
Don't forget to change blPU.hibernate.hbm2ddl.auto in development.properties from create to update (standalone tomcat have trouble to create tables - that's my expirience with postreSQL)

Can't create JDBC driver for class "" [duplicate]

I have an external MySQL server that's set up and working fine. I created a database connection in Eclipse and can view the database in the Data Source Explorer tab.
Now, I have a servlet that needs to access that database. How do I do it? Is there a way to reference that database connection created in the data source explorer, or do I have to define everything twice?
Also, what's the best way to open the connection? I've got the mysql-connector-java-5.1.11-bin.jar file included, and I've found two methods that work:
MysqlDataSource d = new MysqlDataSource();
d.setUser("user");
d.setPassword("pass");
d.setServerName("hostname.com");
d.setDatabaseName("db");
Connection c = d.getConnection();
and
Connection c = DriverManager.getConnection("jdbc:mysql://hostname.com/db","user","pass");
Neither is optimal, because first of all, they both use hard-coded strings for everything. This is a Java EE web app project, so is there a good place to put connection data? Or is there a way to forgo all that and just use the connection in the data source explorer?
A common practice is to configure this as a DataSource in the servlet container in question. It will provide you connection pooling facilities which will greatly improve performance. Also a common practice is to externalize the raw settings in some configuration file which is been placed in the classpath.
In case you're using Tomcat as servletcontainer, you need to configure the datasource as per its JNDI documentation. You'll see that there are several ways. Easiest way is to create a /META-INF/context.xml in the webcontent of your dynamic web project (to be clear, the /META-INF is at the same level as the /WEB-INF of the webapp) and fill it with something like:
<?xml version="1.0" encoding="UTF-8"?>
<Context>
<Resource
name="jdbc/db" type="javax.sql.DataSource"
maxActive="100" maxIdle="30" maxWait="10000"
url="jdbc:mysql://hostname.com/db"
driverClassName="com.mysql.jdbc.Driver"
username="user" password="pass"
/>
</Context>
This roughly means that Tomcat server should create a datasource with the JNDI name jdbc/db with a maximum of 100 active connections, a maximum of 30 idle connections and a maximum wait time of 10000 milliseconds before a connection should be returned from your application (actually: closed by your application, so your application has 10 seconds time between acquiring the connection and closing the connection). The remnant of the settings should be familiar and self-explaining enough to you; those are the JDBC settings.
Finally in your web project, edit the file /WEB-INF/web.xml to add the following entry:
<resource-env-ref>
<resource-env-ref-name>jdbc/db</resource-env-ref-name>
<resource-env-ref-type>javax.sql.DataSource</resource-env-ref-type>
</resource-env-ref>
This roughly means that the webapplication should use the server-provided datasource with the name jdbc/db.
Then change your connection manager to something like this:
private DataSource dataSource;
public Database(String jndiname) {
try {
dataSource = (DataSource) new InitialContext().lookup("java:comp/env/" + jndiname);
} catch (NamingException e) {
// Handle error that it's not configured in JNDI.
throw new IllegalStateException(jndiname + " is missing in JNDI!", e);
}
}
public Connection getConnection() {
return dataSource.getConnection();
}
..and replace all Class.forName(driver) calls by new Database("jdbc/db") and replace all DriverManager.getConnection() calls by database.getConnection(). You can if necessary obtain the value jdbc/db from some config file (Properties file?).
Alternatively, inject the DataSource via the #Resource annotation inside a container managed artifact, such as a #WebServlet servlet class:
#Resource(name="jdbc/db")
private DataSource dataSource;
That should be it. Just deploy your webapplication with the above changes and run it. Don't forget to place the database JDBC driver in the Tomcat/lib or to add its path to the shared.loader property of Tomcat/conf/catalina.properties, because the responsibility of loading the JDBC driver is now moved from the webapplication to the server. For more hints and other basic JDBC/JNDI examples you may find this article useful as well.
See also:
How to install JDBC driver in Eclipse web project without facing java.lang.ClassNotFoundexception
Where do I have to place the JDBC driver for Tomcat's connection pool?
Is it safe to use a static java.sql.Connection instance in a multithreaded system?
Show JDBC ResultSet in HTML in JSP page using MVC and DAO pattern
How to retrieve and display images from a database in a JSP page?
You could set up a data source in whatever app server you're deploying your WAR to and fetch a reference to it with JNDI. Or you could package your WAR in an EAR and define the data source in the EAR's data-sources.xml file (and fetch a reference to it with JNDI).

Cant connect to Mysql master/slave using Entityframework

I have mysql master slave configuration and i cant connect to it when i specify two ips. I search for connection string and found http://www.connectionstrings.com/mysql-connector-net-mysqlconnection/
Server=serverAddress1, serverAddress2, serverAddress3;Database=myDataBase;
Uid=myUsername;Pwd=myPassword;
So its looks simple but when i specify my two ips i get error "No such host is known"
I also found https://blogs.oracle.com/MySqlOnWindows/entry/how_to_using_replication_load and tried it but strange thing is that then it much slower. So now i am lost i cant use my replication.
PS. I am not sure if it make sense but i am running that on Azure
You should specify server in specific config section
<configSections>
<section name="MySQL" type="MySql.Data.MySqlClient.MySqlConfiguration, MySql.Data, Version=6.7.4.0, Culture=neutral, PublicKeyToken=c5687fc88969c44d"/>
</configSections>
<MySQL>
<Replication>
<ServerGroups>
<Group name="MasterSlave">
<Servers>
<Server name="master" IsMaster="true" connectionstring=" server=<server>;port=<port>;uid=<user>;password=<password>;"/>
<Server name="slave" IsMaster="false" connectionstring=" server=<server>;port=<port>;uid=<user>;password=<password>;"/>
</Servers>
</Group>
</ServerGroups>
</Replication>
</MySQL>
Then use following as a connection string:
"server=MasterSlave;database=sakila;"
Source: https://blogs.oracle.com/MySqlOnWindows/entry/how_to_using_replication_load