I successfully completed the DataNucleus Tutorial for JDO using RDBMS with a local MySQL instance on my Ubuntu 13.10 machine.
While attempting to get the same working on a Google CloudSQL instance, I ran into some problems.
To make the switch, I replaced the following:
<property name="javax.jdo.option.ConnectionURL" value="jdbc:mysql://127.0.0.1/nucleus?useServerPrepStmts=false"/>
<property name="javax.jdo.option.ConnectionDriverName" value="com.mysql.jdbc.Driver"/>
... with the following:
<property name="javax.jdo.option.ConnectionURL" value="jdbc:google:mysql://<my-project>:nucleus/nucleus?useServerPrepStmts=false"/>
<property name="javax.jdo.option.ConnectionDriverName" value="com.mysql.jdbc.GoogleDriver"/>
I setup the local MySQL and the CloudSQL instances to have the same credentials, so the javax.jdo.option.ConnectionUserName and javax.jdo.option.ConnectionPassword did not need to be changed.
Under the Google Cloud Console Cloud SQL my-project:nucleus MySQL instance, I created a static IP and also whitelisted my development machine's IP address. I verified that I was able to connect natively to it via 'mysql -u root -p -h '.
When I try to run 'mvn datanucleus:schema-create', I get this:
[DEBUG] Exit code: 1
[DEBUG] --------------------
[DEBUG] Standard output from the DataNucleus tool org.datanucleus.store.schema.SchemaTool :
[DEBUG] --------------------
[INFO] DataNucleus SchemaTool : Creation of the schema
An error was encountered creating a PersistenceManagerFactory : Error creating transactional connection factory - please consult the log for more information.
If I replace the connection settings with the following then it works the same as with the local MySQL instance:
<property name="javax.jdo.option.ConnectionURL" value="jdbc:mysql://<Cloud SQL IP>/nucleus?useServerPrepStmts=false"/>
<property name="javax.jdo.option.ConnectionDriverName" value="com.mysql.jdbc.Driver"/>
So, I've worked around the problem, but why didn't this work with the GoogleDriver? I don't know anything about the black magic that's going on inside that driver, but I'm curious if there was a way to make that work, and whether there's a reason I should use the GoogleDriver at all.
Now that you can use the stock MySQL driver with Cloud SQL, you should use that instead of the older GoogleDriver.
See http://googlecloudplatform.blogspot.com/2013/10/google-cloud-sql-now-accessible-from-any-application-anywhere.html
Hope this helps,
Rob
The special Google JDBC driver is for use from Google App Engine applications only. When using it, the CloudSQL instance does not require a static public IP address, which is a billable feature of CloudSQL.
This all becomes obvious after reading the Google CloudSQL docs here and here.
Related
SnappyData v0.5
My goal is to start a "spark-shell" from my SnappyData install's /bin directory and issue Scala commands against existing tables in my SnappyData store.
I am on the same host as my SnappyData store, locator, and lead (and yes, they are all running).
To do this, I am running this command as per the documentation here:
Connecting to a Cluster with spark-shell
~/snappydata/bin$ spark-shell --master local[*] --conf snappydata.store.locators=10.0.18.66:1527 --conf spark.ui.port=4041
I get this error trying to create a spark-shell to my store:
[TRACE 2016/08/12 15:21:55.183 UTC GFXD:error:FabricServiceAPI
tid=0x1] XJ040 error occurred while starting server :
java.sql.SQLException(XJ040): Failed to start datab
ase 'snappydata', see the cause for details.
java.sql.SQLException(XJ040): Failed to start database 'snappydata',
see the cause for details.
at com.pivotal.gemfirexd.internal.impl.jdbc.SQLExceptionFactory40.getSQLException(SQLExceptionFactory40.java:124)
at com.pivotal.gemfirexd.internal.impl.jdbc.Util.newEmbedSQLException(Util.java:110)
at com.pivotal.gemfirexd.internal.impl.jdbc.Util.newEmbedSQLException(Util.java:136)
at com.pivotal.gemfirexd.internal.impl.jdbc.Util.generateCsSQLException(Util.java:245)
at com.pivotal.gemfirexd.internal.impl.jdbc.EmbedConnection.bootDatabase(EmbedConnection.java:3380)
at com.pivotal.gemfirexd.internal.impl.jdbc.EmbedConnection.(EmbedConnection.java:450)
at com.pivotal.gemfirexd.internal.impl.jdbc.EmbedConnection30.(EmbedConnection30.java:94)
at com.pivotal.gemfirexd.internal.impl.jdbc.EmbedConnection40.(EmbedConnection40.java:75)
at com.pivotal.gemfirexd.internal.jdbc.Driver40.getNewEmbedConnection(Driver40.java:95)
at com.pivotal.gemfirexd.internal.jdbc.InternalDriver.connect(InternalDriver.java:351)
at com.pivotal.gemfirexd.internal.jdbc.InternalDriver.connect(InternalDriver.java:219)
at com.pivotal.gemfirexd.internal.jdbc.InternalDriver.connect(InternalDriver.java:195)
at com.pivotal.gemfirexd.internal.jdbc.AutoloadedDriver.connect(AutoloadedDriver.java:141)
at com.pivotal.gemfirexd.internal.engine.fabricservice.FabricServiceImpl.startImpl(FabricServiceImpl.java:290)
at com.pivotal.gemfirexd.internal.engine.fabricservice.FabricServerImpl.start(FabricServerImpl.java:60)
at io.snappydata.impl.ServerImpl.start(ServerImpl.scala:32)
Caused by: com.gemstone.gemfire.GemFireConfigException: Unable to
contact a Locator service (timeout=5000ms). Operation either timed out
or Locator does not exist. Configured list of
locators is "[dev-snappydata-1(null):1527]".
at com.gemstone.gemfire.distributed.internal.membership.jgroup.GFJGBasicAdapter.getGemFireConfigException(GFJGBasicAdapter.java:533)
at com.gemstone.org.jgroups.protocols.TCPGOSSIP.sendGetMembersRequest(TCPGOSSIP.java:212)
at com.gemstone.org.jgroups.protocols.PingSender.run(PingSender.java:82)
at java.lang.Thread.run(Thread.java:745)
hmm! I assume you are trying the Spark-shell from your desktop and connecting to the cluster in AWS?
Not sure this is going to work because the local JVM launched by spark-shell will attempt to connect to the p2p cluster in Snappydata which is not likely to work.
Snappy-shell on the other hand merely uses the JDBC client to connect (and, hence will work).
And, you cannot use the locator client port (1527), anyway. See here
Can you try with snappydata.store.locators=10.0.18.66:10334 NOT 1527 as the port ? Unlikely this will work but worth a try.
Maybe there is a way to open up all ports and access to these nodes on AWS. Not recommended for production, though.
I am curious for other responses from the engg team.
Until then, you may have to start the spark-shell from within the network (AWS node).
I'm working on a proof of concept to deploy using flyway's command-line tool from a centralized server to deploy to multiple database platforms. (MySQL, Postgres, and SQL Server)
I'm able to deploy successfully without SSL, however it is using unencrypted host information such as logins/passwords/ports to the destination Database Server. My concern is there's a chance the un-encrypted traffic could be seen.
Does anyone have experience with the flyway command line tool using SSL to deploy to:
MySQL
SQL Server
I did not see any information in the documentation unless I missed it.
Thanks for any help and suggestions!
In the flyway examples in flyway.conf it shows how to add additional values to the jdbc url for example
# MySQL : jdbc:mysql://<host>:<port>/<database>?<key1>=<value1>&<key2>=<value2>...
# PostgreSQL : jdbc:postgresql://<host>:<port>/<database>?<key1>=<value1>&<key2>=<value2>...
# Redshift : jdbc:postgresql://<host>:<port>/<database>?<key1>=<value1>&<key2>=<value2>...
So for Redshift/Postgres for example you can include the ssl=true flag
flyway.url=jdbc:postgresql://yourserver:5439/dbname?ssl=true
You need to add the public key that the DB server key was signed with to your hosts trust store (for Redshift see http://docs.aws.amazon.com/redshift/latest/mgmt/connecting-ssl-support.html for details on that), e.g
${JAVA_HOME}/bin/keytool -keystore ${JAVA_HOME}/lib/security/cacerts -import -alias <alias> -file <certificate_filename>
I then had to hack the flyway startup script /flyway to include the truststore and password in the JAVA_ARGS (it probably should have these as variables) e.g
JAVA_ARGS="-Djava.security.egd=file:/dev/../dev/urandom -Djavax.net.ssl.trustStore=/etc/pki/java/cacerts -Djavax.net.ssl.trustStorePassword=changeit"
For MySQL I used the following URL to connect using SSL.
jdbc:mysql://hostname:3306/wpastudy?useSSL=true
Note the useSSL=true parameter.
I created my scaled application on Openshift server with following command:
rhc app create MyApp jbossews-2.0 -s
Then add Mysql:
rhc cartridge add mysql-5.5 -a MyApp
My application using Struts2, Spring & Hibernate. I configured the datasource as follow:
<bean id="dataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiName" value="java:comp/env/jdbc/MysqlDS"/>
</bean>
The JNDI "MysqlDS" is defined in .openshift\config\context.xml with the connection url:
url="jdbc:mysql://5344d4de4382ec43c9000090-myapp.rhcloud.com:37941/mydb"
The problem is my scale app can not establish the connection to Mysql with an error:
org.apache.tomcat.dbcp.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (Could not create connection to database server. Attempted reconnect
3 times. Giving up.)
I'm sure the username & password to access the database is correct. It seem MySQL on Openshift server doesn't open its port. When I tried to use an external database on freemysqlhosting.net (with open host & port) the application run well. But I just want to use MySQL db on Openshift. Anyone who have experience on this please give me some suggestion. Thanks
Make sure that you restarted your application after adding the mysql cartridge, sometimes the environment variables don't show up correctly until you restart. Also try to ssh to your gear and see if you can use the "mysql" command to connect to your mysql database directly.
If you left the .openshift/config/context.xml unchanged, the JNDI name fore the MySQL datasource is actually jdbc/MySQLDS and not jdbc/MysqlDS.
This was changed some time ago.
The documentation here http://openshift.github.io/documentation/oo_cartridge_guide.html#tomcat-cartridge-integrations is unfortunately not correct.
I've developed an application using JSP, Spring Security, MySql. I deployed it on Cloudfoundry. But just war file and I binded mysql service. But I also have to configure MySQL database that consists of two tables, new user and it all in my own database. While i tested my application on localhost i prepared the database running manual scripts in MySQL command window.
Question: How can I make the same configurations in CloudFoundry? can I the same way run all commands and scripts manually or export the database somehow? if yes, how to do this. And what to write here now instead of localhost?
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://localhost:3306/security_filter" />
thank you
Caldecott (vmc tunnel) is the correct way to access your cloudoundry database (for me it works and i am using ruby 1.8): http://docs.cloudfoundry.com/tools/vmc/caldecott.html
If that does not work for you you will have to do something manually (much harder):
Create a sample jsp/servlet application that connects to a mysql database (retrieving connection string, username and password as input from user). Then it will just run the sql statement against the database ( sql statement will be also input from user)
Bundle this application in your war
Now you have to retrieve database connection string/username and password. You can retrieve them from the enviromental variable VCAP_SERVICES. Just log log it in a startup listener (eg ServletContextListener)
Deploy your war and get logs from server (vmc logs ApplicationName). Get connection string, username and password
Logon to your application and use your database application to access the database using the db info you collected in the previous step
Just note that this is very dangerous approach. Just be sure to secure this database application or after the initial import remove it from your war and redeploy the application
As a final note you can check if such a database console application already exists so that you dont have to create your own (for example grails has a nice application for this http://grails.org/plugin/dbconsole. Maybe something exists for jsp/servlets)
Hope it helps if you have no luck with the ruby problem
You would need to create a mysqldump of your database.
Once you have the mysqldump file, you then use caldecott and perform vmc tunnel to your MySQL service. After you have establish a connection with your MySQL service on Cloud Foundry, you then need to use the mysql command to manually import the mysqldump file into your service.
For how to use vmc tunnel, visit the documentation site at: http://docs.cloudfoundry.com/tools/vmc/caldecott.html
I'm in SQLSERVER (2008 R2 EXPRESS) administrators list. I'm in administrator role on my machine. And I can manually connect to SQLSERVER (via Management Tools) using ServerName (in my case it's S-PROG-T\SQLEXPRESS). During installation I decided to use only Windows Authentication mode. I can't connect to SS.
Error message is: Login failed for user S-PROG-T\admin. Cannot open database 'ProjectDB' requested by the login.
I'm using the following configuration for NHibernate:
<session-factory>
<property name="connection.provider">
NHibernate.Connection.DriverConnectionProvider
</property>
<property name="dialect">
NHibernate.Dialect.MsSql2008Dialect
</property>
<property name="connection.driver_class">
NHibernate.Driver.Sql2008ClientDriver
</property>
<property name="connection.connection_string">
Data Source=S-PROG-T\SQLEXPRESS;Initial Catalog=ProjectDB;Integrated Security=SSPI
</property>
<property name="show_sql">
true
</property>
<mapping resource="Project.Domain.Model.Entities.Mappings.Vehicle.hbm.xml" assembly="Project.Domain" />
</session-factory>
And I've tried to connect using .\SQLEXPRESS data source. Inbound rules don't restrict SSMS.
What's wrong?
Thanks!
EDIT: Unit tests
Everything (nhibernate configuration, and schema export trial) starts inside my unit test (I don't know if this matters).
[TestFixture]
public class VehicleFixture
{
[Test]
public void TestSchemaExport()
{
new SchemaExport(new Configuration().Configure()).Execute(false, true, false);
}
}
EDIT: Log information
I discovered the next information inside a log (path: %ProgramFiles%\Microsoft SQL Server\MSSQL10_50.SQLEXPRESS\MSSQL\Log\):
Error: 18456; State: 38.
Login failed for user ‘S-PROG-T\Admin’.
Reason: Failed to open the explicitly specified database.
According to the article: http://sql-articles.com/articles/troubleshooting/troubleshooting-login-failed-error-18456/ database doesn’t exist or login doesn’t have access to the database.
I'm totally confused. Should I have created the database (manually) before I tried to export schema (sorry if the question is silly)?
EDIT: Using Mixed Authentication mode
I've set Mixed mode Authentication and created new login with serveradmin & sysadmin roles. But still the error message is the same, except for user name. Again I can manually do the same thing (connect and create db) with the login and its password.
Ok. I misunderstood the whole process. Everything's ok with Authentication and rather with connection strings. I just thought that schema export stands for database automated-creation and then schema export. So I created a DB and successfully exported a schema.
Can you connect with NHibernate if you change database to 'master'?
Is ProjectDB database user S-PROG-T\admin mapped to server login that you use to connect using management tools?
Can you connect to database using user/password combination in property name="connection.connection_string" instead of integrated security = SSPI?
EDIT You write that you created new login. But still - login is something used for SERVER authentication. You must have database user in ProjectDB mapped to server login to access the database.
Look at this thread: TSQL to Map User to Database
or read here: http://www.nerdymusings.com/LPMArticle.asp?ID=41
The website will not be using the same identity, therefore will not be an administrator. You can either define a specific identity and set your ApplicationPool (or your own admin account) to use this. Alternatively, use a login/password rather than integrated security.