I'm trying to connect a data modeling tool (DbSchema) to a MySQL database running in Google Cloud SQL. The cloud instance requires SSL. I've downloaded the necessary keys to my Mac and can connect through certain tools, like Sequel Pro and MySQL Workbench. However, these tools give me a way to enter the key locations into their connection windows. But, DbSchema does not - all it does is allow me to modify the connection string it uses to connect to the DB via JDBC.
What I have so far is:
jdbc:mysql://<MY IP ADDRESS>:3306?useUnicode=true&characterEncoding=UTF8&zeroDateTimeBehavior=convertToNull&useOldAliasMetadataBehavior=true&useSSL=true&verifyServerCertificate=false
This ends up giving me a password error although the PW I've used is correct. I think the problem is that JDBC isn't using the SSL keys. Is there a way to specify the locations of the SSL keys in this connection string?
This MySQL JDBC (for SSL) link may help you. Please see Setting up Client Authentication:
Once you have the client private key and certificate files you want to
use, you need to import them into a Java keystore so that they can be
used by the Java SSL library and Connector/J. The following
instructions explain how to create the keystore file:
Convert the client key and certificate files to a PKCS #12 archive:
shell> openssl pkcs12 -export -in client-cert.pem -inkey client-key.pem \
-name "mysqlclient" -passout pass:mypassword -out client-keystore.p12
Import the client key and certificate into a Java keystore:
shell> keytool -importkeystore -srckeystore client-keystore.p12 -srcstoretype pkcs12 \
-srcstorepass mypassword -destkeystore keystore -deststoretype JKS -deststorepass mypassword
Set JDBC connection properties:
clientCertificateKeyStoreUrl=file:path_to_truststore_file
clientCertificateKeyStorePassword=mypassword
Related
I have a MySQL database that requires SSL to access (Google Cloud SQL but that should not affect anything). I can access it using a terminal command like this:
mysql -uroot -h $CLOUD_SQL_HOST \
--ssl-ca=$CLOUD_SQL_KEY_LOCATION/server-ca.pem \
--ssl-cert=$CLOUD_SQL_KEY_LOCATION/client-cert.pem \
--ssl-key=$CLOUD_SQL_KEY_LOCATION/client-key.pem
How do I configure the Dropwizard datasource with the corresponding information? Very grateful for any pointers.
I am trying to connect to a MySQL Database with SSL using a Client Certificate. I have created a truststore with the CA Certificate:
keytool -import -alias mysqlServerCACert -file ca.crt -keystore truststore
Then I created a keystore with my private key and my client certificate:
openssl pkcs12 -export -out bi.pfx -inkey bi.key -in bi.crt -certfile ca.crt
openssl x509 -outform DER -in bi.pem -out bi.der
keytool -importkeystore -file bi.der -keystore keystore -alias mysqlClientCertificate
I added useSSL=true and requireSSL=true to the jdbc URL and passed
-Djavax.net.ssl.keyStore=${db.keyStore}
-Djavax.net.ssl.keyStorePassword=${db.keyStore.pwd}
-Djavax.net.ssl.trustStore=${db.trustStore}
-Djavax.net.ssl.trustStorePassword=${db.keyStore.pwd}
to the kettle transformation from the surrounding job. I still get "Could not create connection to database server".
I can connect via SSL using the command line tool:
mysql --protocol=tcp -h myqlhost -P 3309 -u bi -p --ssl=on --ssl-ca=ca.crt --ssl-cert=bi.crt --ssl-key=bi.key db_name
Therefore my current guess is, that ther is an issue with the SSL Certificates.
Is there a way to make the MySQL JDBC Driver tell me more details, what went wrong?
Is my assumtion wrong, that kettle parameters can be used to set system properties? How do I do that instead then?
Establish Secure Connection (SSL) To AWS (RDS) Aurora / MySQL from Pentaho (PDI Kettle)
1. You need to create a new user id and Grant SSL rights to it. So this user id can connect to Aurora / MySQL only using Secured connection.
GRANT USAGE ON *.* TO 'admin'#'%' REQUIRE SSL
2. Download public RDS key (.pem fie) from AWS (http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Aurora.html#Aurora.Overview.Security.SSL)
3. Downloaded file contains certificates / keys for each region.
4. Split certificates from .PEM file into different .PEM files
5. Use JDK keytool command utility to import all these PEM files into a single truststore (xyz.jks) file
a. keytool -import -alias xyz.jks -file abc1.pem -keystore truststore
6. Configure JNDI entry for your Aurora / MySQL instance in Pentaho Properties File "data-integration\simple-jndi\jdbc.properties"
a. Sample JNDI configuration
-------------------------------------------------------------------------
RDSSecured/type=javax.sql.DataSource
RDSSecured/driver=com.mysql.jdbc.Driver
RDSSecured/user=admin
RDSSecured/password=password
RDSSecured/url=jdbc:mysql://REPLACE_WITH_RDS_ENDPOINT_HERE:3306/DATABASE_NAME?verifyServerCertificate=true&useSSL=true&requireSSL=true
-------------------------------------------------------------------------
7. Make sure you copied MySQL connector jar in "lib" directory of your pentaho installation. Use connector version 5.1.21 or higher.
8.
9. Create a copy of Spoon.bat / Spoon.sh based on your operating system E.g. Spoon_With_Secured_SSL_TO_RDS.bat or Spoon_With_Secured_SSL_TO_RDS.sh
10. Now we need to pass the truststore details to Pentaho at startup, so edit the copied script and append below mentioned arguments to OPT variable
a. -Djavax.net.ssl.trustStore="FULL_PATH\xyz.jks"
b. -Djavax.net.ssl.trustStorePassword="YOUR_TRUSTSTORE_PASSWORD"
11. Use new script to start Spoon here after to establish the secure connection
12. Open/create your Job / Transformation
13. Go To View Tab - Database Connections and create new connection
a. Connection Type: MySQL
b. Access: JNDI
c. JNDI Name: RDSSecured
i. Same as name used in JDBC.properties file
14. Test Connection and you are ready…. :)
OK, here is the solution, that I have found now:
The start scripts for the various kettle tools pass parameters to the JVM by reading an environment-variable "OPT". So I have set
export OPT="-Djavax.net.ssl.keyStore=/path/to/keystore -Djavax.net.ssl.keyStorePassword=private -Djavax.net.ssl.trustStore=/path/to/truststore -Djavax.net.ssl.trustStorePassword=private"
Now the MySQL JDBC Driver finds its certificates and private key and can establish the connection.
I have a mysql-server set up on one server. I have enabled SSL by adding the following lines into the [mysqld] section in my.cnf:
ssl-ca=/etc/mysql/ca-cert.pem
ssl-cert=/etc/mysql/server-cert.pem
ssl-key=/etc/mysql/server-key.pem
I have generated these certificates by following the steps in http://dev.mysql.com/doc/refman/5.1/en/creating-ssl-certs.html
Now on the same server when I run the mysql client with the following command, the connection is established with SSL enabled:
mysql -uuser -p --ssl-ca=/etc/mysql/ca-cert.pem
Now when I try to do the same from a remote server
mysql -uuser -hserver.asdf -p --ssl-ca=ca-cert.pem
, I get the following error:
ERROR 2026 (HY000): SSL connection error
where in I have copied the same ca-cert.pem file that i used previously. But when I run the following command, the connection gets established with SSL enabled:
mysql -uuser -hserver.asdf -p --ssl-cert=client-cert.pem --ssl-key=client-key.pem
From what I can understand, --ssl-cert and --ssl-key parameters are required in the mysql client command only if we want the server to authenticate the client and are not actually necessary. But why is it that I can connect when I use these parameters and not otherwise?
UPDATE
There were certain other important factors that I thought were unimportant during the time of asking this question. The mysql-server package version being used was Percona-Server-51-5.1.73 and the mysql-client package on the local server was Percona-client-51-5.1.73. But on the remote server the mysql-client package was Percona-client-51-5.1.61
It turns out that there were some major changes made with respect to Percona-mysql packages with respect to SSL in the more recent builds of Percona-mysql. All older builds of the percona-mysql package came with yaSSL support whereas more recent builds use OpenSSL.
This was done because packages with yaSSL builds apparently used to cause some programs to crash. More details regarding this bug can be found here: https://bugs.launchpad.net/percona-server/+bug/1104977
Since this issue was fixed from 5.1.68 onwards, obviously a package before that didn't function properly.
So after updating the mysql client package to the latest version, the problem was fixed.
I'm a little green at this, and I hope the issue I'm having is a simple one...edit: new information at bottom
I need to make a connection to a remote mysql (Amazon RDS) database.
After following a few tutorials, I have unixodbc and libmyodbc installed and configured on the client, but when I try to connect via isql, I get the error
[08S01][unixODBC][MySQL][ODBC 5.1 Driver]Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
[ISQL]ERROR: Could not SQLConnect
The most confusing part about this error is that I'm not trying to connect to a local database, but rather to a remote one. I do not have a mysql.sock file on the client...this isn't the issue though is it?
I'm sensing a configuration error but I'm just not sure what it could be.
If I run odbcinst -j then the output is:
DRIVERS............: /etc/unixODBC/odbcinst.ini
SYSTEM DATA SOURCES: /etc/unixODBC/odbc.ini
USER DATA SOURCES..: /root/.odbc.ini
The content of /etc/unixODBC/odbcinst.ini is:
[MySQL]
Description = ODBC for MySQL
Driver = /usr/lib64/libmyodbc5.so
Setup = /usr/lib64/unixODBC/libodbcmyS.so
UsageCount = 5
[MySQL ODBC 515 Driver]
Description = ODBC 5.515 for MySQL
DRIVER = /usr/lib64/libmyodbc5-5.1.5.so
SETUP = /usr/lib64/unixODBC/libodbcmyS.so
UsageCount = 3
Please note that I had to make up this configuration myself, I did a find for libmyodbc* and found these two .so files, thus set up a driver for each of them. A search for libodbcmyS* yields:
/usr/lib64/unixODBC/libodbcmyS.so.1
/usr/lib64/unixODBC/libodbcmyS.so
/usr/lib64/unixODBC/libodbcmyS.so.1.0.0
So, I don't know what else that configuration could be.
The content of /etc/unixODBC/odbc.ini is:
[target_db]
Driver = MySQL
Server = [servername.com]
Port = 3306
Database = [databasename]
Option = 2
User = [username]
Password = [password]
I've tried different options in "Driver", changing it from MySQL, to MySQL ODBC 515 Driver, to the path to the .so file (eg: /usr/lib64/libmyodbc5.so) and all yield the same result.
I'm running:
odbcinst -i -d -f /etc/unixODBC/odbcinst.ini
Followed by:
odbcinst -i -s -l -f /etc/unixODBC/odbc.ini
Followed by:
odbcinst -s -q
Which prints out the name of my connection, ie [target_db]
Then, I try the connect:
isql -v target_db user password
or just
isql -v target_db
and get the error shown above.
Anyone happen to know what I'm doing wrong here? Thanks a bunch-
EDIT:
Wanted to mention that I'm able to connect to the database from this server using the mysql command line tools.
I installed a local mysql database, and I'm able to connect to this using isql. It seems to be ignoring my odbc.ini file entirely, i have to enter a name with the command, ie isql -v test-database, but it still tries to connect to localhost despite my settings.
I feel as though I've tried everything but will keep at it and will post if i find a solution.
You could try to connect using the ip of your server instead of the dns entry on the "Server" line of odbc.ini.. Have you verified the driver is installed with phpinfo()?
Try to set the environment variable ODBCINI with the path of your odbc.ini file.
Keep in mind that the odbc.ini file you point to must be "write-accessible" by the user that is running the program (i.e. the user must have permissions to write in this file).
chmod g+w .odbc.ini did it for me since we run the DB with ORACLE-Start and the crs-User seems to be in charge
I am using AWS RDS and wish to backup my primary db hourly to a DR site in another region
What I need is a mysql command which will export the db and import into another mysql server running in a different location
I am looking into using something like the following
$ mysqldump -umy_user -pmy_pass_1 -R my_database my_table | mysql -umy_user -pmy_pass_2 my_database
However I cannot seem to specify a host for the export db, how can I acheieve this?
I also need to consider security so would like this over SSL, any idea?
[edit]: got this sort of working by the following process
On a linux node in my primary site ran a mysqldump and then ran a mysql command to import into the dr site e.g.
mysqldump -P 3306 -h primary.com -u user --password=pass mydb > test.sql;
mysql -P 3306 -h dr.com -u user --password=pass mydb < test.sql;
However I still need this to be secure
mysqldump accepts ssl arguments:
--ssl Enable SSL for connection (automatically enabled with
other flags).
--ssl-ca=name CA file in PEM format (check OpenSSL docs, implies
--ssl).
--ssl-capath=name CA directory (check OpenSSL docs, implies --ssl).
--ssl-cert=name X509 cert in PEM format (implies --ssl).
--ssl-cipher=name SSL cipher to use (implies --ssl).
--ssl-key=name X509 key in PEM format (implies --ssl).
--ssl-verify-server-cert
Verify server's "Common Name" in its cert against
hostname used when connecting. This option is disabled by
default.
Set up the proper certs and it should be straightforward. Good details are here.