I am trying to connect from Glue to a mysql instance hosted on an EC2.
I deployed mysql database on a public subnet (Yes this is just a POC to test it out).
I created a S3 VPC endpoint (gateway) and attached to my public subnet
For the security groups (inbound and outbound) Im using open to all (Yes this is just a POC to test out Glue connection).
my glue iam role has the glue full access policy and the administrator access policy (yes not best policy but im just trying to check why i cant connect)
I am hitting an error which states that I cant connect. But I can connect to it on a mysql client. So Im pretty sure its nothing to do with my connection string. Any help?
2021-06-11T09:34:19.680+08:00
Copy
Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
2021-06-11T09:34:19.680+08:00 Exiting with error code 30
Glue does not support Mysql 8 because it uses old JDBC driver. So you have to setup your own JDBC for glue:
This feature enables you to connect to data sources with custom drivers that were not natively supported in AWS Glue such as MySQL 8 and Oracle 18.
Alternatively, you can migrate to older MySQL.
Related
I've added the host URL ({hash}.rackspaceclouddb.com), port 3306, and I'm using the same user and password that has access to everything via PhpMyAdmin, but I'm getting a failure when I test the connection. I understand that they run their instances on a private network, which I'm sure is causing the difficulty. Is there some trick I'm not aware of, or is it not possible?
Error message:
Failed: Communications link failure. The driver has not received any packets from the server.
In earlier versions, we've had a similar issue when establishing a connection with MySql. Please update DataGrip to the latest version. Also, make sure to download the latest MySql driver version available in DataGrip.
After upgrading DataGrip to version 2021.3.2, my existing db connections were broken. I connect to various DBs (Oracle, MySql) via an SSH tunnel configured to connect through an AWS bastion host.
After the upgrade - DataGrip suggested that the MySql driver had to be updated to Amazon Aurora MySQL driver and it no longer worked to connect where it worked before the upgrade.
Switching between drivers I get two separate errors:
First Error using Amazon Aurora MySQL (suggested driver after update)
[08000][-1] Could not connect to address=(host=localhost)(port=53929)(type=master) : (conn=57522706) could not load system variables[08000][1220] (conn=57522706) Connection is closed.
and second error using original MySQL driver
[08S01]
Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
No appropriate protocol (protocol is disabled or cipher suites are inappropriate).
All of my connections worked immediately prior to the DataGrip upgrade - so seems that upgrade requires new drivers, which have a problem with the way I connect.
After some looking - it seems that Aurora has an existing race condition - found my answer here: https://jira.mariadb.org/browse/CONJ-824?focusedCommentId=165412&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-165412
My solution was to change my usePipelineAuth to false (not set prior) in the Advanced tab of the Data Sources configuration:
This fixed my connections and am back in business.
Finally, In my case, I fix the issue just replacing/using MariaDB Driver.
I am trying to connect to a MySQL database from Data Fusion, but I am getting the following error. Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. The database is accessed through public IP through port 3306, from my machine I can connect perfectly, but from Data Fusion I cannot.
As John Hanley pointed out in his comment, it's probably due to a connectivity issue with your SQL instance.
A possible reason would be that you have not enabled your instance to be connected via its public IP. If that's the case, go to your SQL instance and edit its configuration, adding a network (if you haven't done so previously) and providing an IP range to include your Data Fusion instance. Keep in mind that if you configure your instance to accept connections using its public IP address, also configure it to use SSL to keep your data secure. If that was the issue, now you should be able to connect properly.
Also, be sure to check that the Dataproc cluster that your data Fusion instance is using under the hood has the proper configuration (you shouldn't worry about this if you haven't changed anything about the Dataproc cluster).
This is the best advice I can give without further details. If this doesn't work for you, we're going to need more information.
My two cents:
I use one windows pc with several linux box at home.
My main station is the Windows one.
I'v been using Sqlyog just to prob tables on the remote db.
I use my class C addr. to address the server.
It works well. By design I want to access the db server
from anywhere in my home...
60% to 75%ishhh problems of that kind would be related
to the .cnt file configuration.
Regards
Steph
I'm having issues connecting Domo to a MySQL database hosted with AWS RDS. Whenever I try to authenticate I get this error:
"Failed to authenticate. Verify the credentials and try again. Domo is ready, but the credentials you entered are invalid. Verify your account credentials and try again. Error setting up SQL connection. Could not create connection to database server. Attempted reconnect 3 times. Giving up."
Its not security group settings. Someone suggested on this post:
https://dojo.domo.com/t5/Data-Sources-and-Connectors/MySQL-connector-issues/td-p/15462
that I should enable SSL in AWS database but I'm not sure how to do that.
I'll assume you're using the MySQL connector, not the MySQL SSH connector.
It sounds like you need to whitelist Domo's IP addresses within your AWS RDS's security groups.
Aside from that, make sure you're populating the credentials in Domo with the right pieces of information. Hostname should be the server's public IP address.
This connector follows the same general process as described in AWS's documentation here, with the exception that steps 5 and 6 are optional since SSH is not required for this connector.
Hi in this instance database security group is open for all inbound traffic (ALL ports - All sources).
I am also able to connect to the database fine in mysql workbench or Datagrip which for sure use jdbc connection string.
2019-05-21 14:12:03 INFO CatalogClient:651 - Got connection 'tem-sas-main' info from Catalog with url: jdbc:mysql://my-database-example:3306/sas_tem_central
2019-05-21 14:12:03 INFO CatalogClient:684 - JDBC configuration for connection tem-sas-main: JDBCConfiguration(url=jdbc:mysql://my-database-example:3306/sas_tem_central, hostname=my-database-example, port=3306, databaseVendor=mysql, databaseVersion=null, connectionName=tem-sas-main, path=sas_tem_central, subnetId=subnet-0717c4db096e84393, availabilityZone=eu-west-1a, securityGroups=[sg-074b074ebc51c2315], enforceSSL=false)
2019-05-21 14:12:03 INFO JdbcConnection:42 - Starting connecter. driver com.mysql.jdbc.Driver#7e5d9a50
2019-05-21 14:12:03 INFO JdbcConnection:60 - Attempting to connect with SSL host matching: jdbc:mysql://my-database-example:3306/sas_tem_central
2019-05-21 14:14:15 INFO JdbcConnection:69 - SSL connection to data store using host matching failed. Retrying without host matching.
2019-05-21 14:14:15 INFO JdbcConnection:83 - Attempting to connect with SSL: jdbc:mysql://my-database-example:3306/sas_tem_central
2019-05-21 14:16:26 INFO JdbcConnection:88 - SSL connection to data store failed. Retrying without SSL.
2019-05-21 14:16:26 INFO JdbcConnection:102 - Attempting to connect without SSL: jdbc:mysql://my-database-example:3306/sas_tem_central
Check that your connection definition references your JDBC database with correct URL syntax, username, and password. Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
Also note my JDBC connection string is
jdbc:mysql://my-database-example:3306/sas_tem_central
And Require SSL connection is set to false
After speaking to an AWS Architect, I can confirm that Glue does not work with MySql Version 8 at this time. This is NOT documented anywhere I could find.
Glue's supported Databases (not pointed out prominently in the Glue documentation / faqs)
The following table lists the JDBC driver versions that AWS Glue supports.
Microsoft SQL Server 6.x
MySQL 5.1
Oracle Database 11.2
PostgreSQL 42.x
Amazon Redshift 4.1
Now you can customize your own configuration to connect to MySQL 8 and other newer databases from AWS Glue Jobs.
Download the JDBC driver for MySQL 8
Upload to S3
Create a JDBC connection to your database via Glue - Tables - Connections
(Note this will not work using the "test connection" since the latest version supported is (currently) 5.7
Edit the Glue Job - Dependent jars path - e.g.
s3://yourS3bucket/path/mysql-connector-java-8.0.21.jar
Modify the Glue Job e.g. Python script
datasink4 = glueContext.write_dynamic_frame.from_jdbc_conf
(frame = dropnullfields3, catalog_connection = “GLUE-CONNECTION-NAME”,
connection_options = {
"customJdbcDriverS3Path": "s3://yourS3Bucket/path/mysql-connector-java-8.0.21.jar",
"customJdbcDriverClassName": "com.mysql.cj.jdbc.Driver",
"user": “#DBuserName#”, "password": “DbUsersPassword”,
"url":"jdbc:mysql://dbname.url.region.rds.amazonaws.com/schemaName,
"connectionType": "mysql", "dbtable": “tablenameInDb”,
"database": “schemaName”}, transformation_ctx = "datasink4")
(no new lines in that script)
Your connection string looks right.
I was struggling with this same issue and found that I had to troubleshoot a few issues:
Ensure that your security group that you are using for your RDS is self-referenced. This means that you should add the security group from your db instance on RDS to both the inbound and outbound rules for All TCP.
Ensure that you have created an endpoint for your VPC. To do this, navigate to your VPC, click on 'Endpoint', add an 'Endpoint' for the VPC that the RDS is in and create an endpoint for S3.
Finally, the version you create your MySQL RDS instance on should be version 5 (not version 8). I noticed that AWS Glue JDBC connector only worked with MySQL version 5 (5.7)
When you use the AWS RDS with the MySQL Engine, you don't need to create a JDBC connection. You can use the Amazon RDS when you are creating your connection.
New Connection Screen
I first tried with mysql8. It didn't work. It thrown same error.
Later i created mysql5.7 for testing connection. It worked now. I can able to connect Mysql5.7 from Glue. Makesure your VPC-s3 Endpoint are configured properly.
Thanks
Raaz