I configured a connection profile and I could ping it successfully, but the Data Source Explorer does not see any of the schemas (databases) in MySQL.
When I configure the connection for JPA, the database-specific validations fail and none of my TABLE names mapped to my entities are seen, and cause the failures.
I looked at previous questions and answers, and followed the suggestions, which allowed me to configure the connection successfully, but the explorer still does not see schemas.
Help would be greatly appreciated. I’ve burned a full day on this issue and I cannot find any answers from uncle Google.
The MySQL driver is configured correctly (otherwise the ping would fail).
URL: jdbc:mysql://localhost:3306/finances
Database Name: finances
Driver Class: com.mysql.jdbc.Driver
The driver Jar is attached and visible in the driver JAR list.
The JPA properties of the project point at the same connection as the Data Source Explore and the "connect" button connects successfully, the status bar at bottom of the window shows MyConnectionProfile (Connected).
The JPA properties are:
Platform: EclipseLink 2.5.x
User Library: EclipseLink 2.5.2
JPA Version: 2.1
The database-specific validation errors are
Table "XXXXX" cannot be resolved, for each of 15 tables.
I wanted to embed snapshots but I did not know how.
My very first question on SO! Be Gentle!
Related
Last night AWS RDS had an "Internet Connectivity Issue" that was resolved a short time later. However, my app (which runs in .NET Core and connects to an RDS MySQL instance via Pomelo.EntityFrameworkCore.MySql) could never re-establish a connection to the database even though the MySQL server was back online. I tested connecting from my own local machine and it worked just fine. I then re-deployed the .NET Core app it everything started working again.
Is there something that I need to re-create (the db context perhaps), or is there something that is being cached that I need to flush to try to connect again? I connect via hostname, and my connection string looks something like this:
server=something.somewhere.us-east-2.rds.amazonaws.com;userid=XXXX;password=YYYYY;database=ZZZZ
Here is the Exception being thrown:
MySqlException: Unable to connect to any of the specified MySQL hosts.
at MySqlConnector.Core.ServerSession+<ConnectAsync>d__56.MoveNext (C:\projects\mysqlconnector\src\MySqlConnector\Core\ServerSession.cs:239)
and here is how I create my db context in Startup.cs:
services.AddDbContext<BlayFapContext>(opt => opt.UseMySql(Settings.Instance.SQLConnectionString));
Any help would be greatly appreciated.
Giawa
Okay, we worked out what happened. Pomelo's MySQL wrapper had an issue as outlined in their git repo here: https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/issues/434
Basically, if a MySQL database is not available when the connection string is first used then it will be cached as invalid and will never work again. You can easily confirm this by launching a service with no MySQL connectivity, verify it doesn't work, then launch MySQL and confirm that the service still doesn't work. It can never establish a MySQL connection after the first connection string is found to be invalid.
They patched it shortly after the 2.0.1 release, but they haven't updated Nuget with a new version since then, despite the issue being found 6 months ago. So, the fix is to checkout their repository source code, and patch it ourselves. We found the fix here works just fine: https://github.com/PomeloFoundation/Pomelo.EntityFrameworkCore.MySql/pull/456
So, why was the connection string retried? We already had a successful connection! It turns out that the internet connectivity issue with the Ohio data center was not limited to RDS, but also affected EC2. Our EC2 instance was rebooted as part of the fix, and the MySQL connection wasn't valid when it reboot due to the continued connectivity issues. The state of that connection was cached, and even though the MySQL server came back online our service was toast.
Giawa
Before I begin, I'd like to start by saying I am completely new to Kafka and am fairly new to Linux, so if this ends up being a ridiculously simple answer, please be kind! :)
The high level idea of what I'm trying to do is use Confluent's Kafka Connect to read from a MySQL database that is having sensor data streamed to it on a minute or sub-minute basis and then use Kafka as an "ETL pipeline" to instantly route that data to a Data Warehouse and/or MongoDB for reporting or even tie in directly to Kafka from our web-app.
I am using Robin Moffatt's series as well as Confluent's JDBC Source Connector Quickstart as my initial guide. As far as where these are hosted, I am using an Amazon RDS MySQL database and a separate AWS EC2 t2.large instance with Ubuntu 16.04.2 to run Kafka Connect.
Using Robin's workflow, I am to the point where I have created the configuration file, but I am not using the json format he uses. I am using the format from the quickstart article.
name=jdbc_source_mysql_4427_Data
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
key.converter=io.confluent.connect.avro.AvroConverter
key.converter.schema.registry.url=http://localhost:8081
value.converter=io.confluent.connect.avro.AvroConverter
value.converter.schema.registry.url=http://localhost:8081
connection.url=jdbc:mysql://lndbtest.cdveaddpnevv.us-east-2.rds.amazonaws.com:3306/LNDBv1?user=adminRDS&password=*****
table.whitelist=4427_Data
mode=timestamp
timestamp.column.name=TmStamp
validate.non.null=false
topic.prefix=mysql-
And that is saved at:
/etc/kafka-connect-jdbc/kafka-connect-jdbc-source.properties
I then run:
/usr/bin/confluent load jdbc_source_mysql_4427_Data -d /etc/kafka-connect-jdbc/kafka-connect-jdbc-source.properties
and get this error:
{
"error_code": 400,
"message": "Connector configuration is invalid and contains the following 2 error(s):\nInvalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://lndbtest.cdveaddpnevv.us-east-2.rds.amazonaws.com:3306/LNDBv1?user=adminRDS&password=*** for configuration Couldn't open connection to jdbc:mysql://lndbtest.cdveaddpnevv.us-east-2.rds.amazonaws.com:3306/LNDBv1?user=adminRDS&password=***\nInvalid value java.sql.SQLException: No suitable driver found for jdbc:mysql://lndbtest.cdveaddpnevv.us-east-2.rds.amazonaws.com:3306/LNDBv1?user=adminRDS&password=*** for configuration Couldn't open connection to jdbc:mysql://lndbtest.cdveaddpnevv.us-east-2.rds.amazonaws.com:3306/LNDBv1?user=adminRDS&password=***\nYou can also find the above list of errors at the endpoint `/{connectorType}/config/validate`"
}
It seems to be a driver issue. My question at this point is, "Do I need to download the MySQL JDBC driver to my EC2 instance, or should that have been included in the Confluent Platform package?"
Also, does my overall idea sound like a good fit for Kafka Connect?
As I mentioned earlier, I am new to these technologies, but have found the best way to learn something is to jump right in and try to solve a problem. Any ideas and suggestions would be more than welcome. Thank you!
The overall concept makes sense to me. You do need to download the driver and add it to your worker classpath. It isn't packaged for licensing reasons I assume.
As #dawsaw says, you do need to make the MySQL JDBC driver available to the connector.
My observation here would be–given a free hand in all the application and architecture you describe– it would be best to stream from the sensor into Kafka, and then from there Kafka into MySQL, Mongo, webapp, etc.
Streaming into a DB to then stream out of the DB is not a perfect choice, if you have the option.
It's because there's no mysql driver in the distribution of confluent. I think you can solve the problem by downloading a mysql driver jar file, then putting it in confluent/share/java/kafka-connect-jdbc folder and re-run the program.
I'm trying to connect to a local MySQL database, with the anylogic object "database".
I'm using the type: "Other Database", and the connection URL: "Server=localhost;Database=anylogicdata;"
but I constantly get a RuntimeException saying: "not suitable driver found"
The help file says that you have to install the driver, but i don't know which or if it is my connection URL that is the problem.
Does anyone have some pointers to help me along the way?
You need a JDBC driver to be able to connect to a MySQL server from Java (that AnyLogic is based on) and you can find one here. After you have installed the driver you should find it in the list of available JDBC drivers in AnyLogic. The name should be com.mysql.jdbc.Driver if you chose the suggested one.
First you have to download and add the mysql-connector-java-*.jar file to the anylogic-project.
Then you have to type com.mysql.jdbc.Driver into the JDBC driver dropdown box. finally the connection string should look something like
jdbc:mysql:[host/db]?:[port]
another thing I found that might be helpfull to others, is that you can get the java connection from the anylogic db object: database.getConnection() this is very usefull if you want to create your own query. eg create a bulk insert instead of the single insert that Anylogic provides.
I downloaded and installed the driver setup file, DB2OLEDB.exe, from here:
http://download.microsoft.com/mwg-internal/de5fs23hu73ds/progress?id=HYLbKUfGNl
Using the connection string that worked on another PC, I tried to create a Connection Object in an SSIS package. When I tested the connection I got this error:
Test connection failed because of an error in initializing provider. A TCPIP socket error has occurred (10057): A request to send or receive data was disallowed because the socket is not connected and (when sending on a datagram socket using a sendto call) no address was supplied.
Any suggestions on what the cause of this error is and how I might resolve this issue?
By the way, when I use the DB2 Configuration set up utility and test a connection from within that, I am able to successfully connect.
What other info can I provide to help you answer this question?
Thank you
Could this be related to a blocked port?
If you follow all the steps illustrated here: http://www.bidn.com/blogs/PatrickLeBlanc/ssis/700/connecting-to-db2-using-ssis do you still get the same result?
Maybe a silly question, did you restart the computer after the installation?
Are you an admin user on one machine and not on the other?
You could try to verify the port connectivity with a quick telnet command:
telnet your-db-host your-db-listening-port
If it connects, that one is off the list.
Doing some research I've found two possible fixes.
The first link suggests calling BeginReceive after the EndAccept logic is complete. Are you using script code, or just using the GUI without any scripting?
TCP async sockets throwing 10057
The second link points to drivers / software on the PC. It could be that you are missing a windows update or have faulty hardware / drivers.
I think this is less likely the case since you could connect to a different machine with the same connection string(?). Can you verify this is a valid statement?
http://social.msdn.microsoft.com/Forums/en-US/1bc3df95-c86d-4d25-aa20-30f61ed00c63/odd-socket-errors
If you could show the connection strings used for both the working and non working, and give a little more detail about The "Other PC" in comparison to the non-working PC... that would be helpful =]
If neither of the posts I've linked are the solution, this specific Google search has proven to yield some seemingly helpful results
"socket" "10057" "no address was supplied."
The yii requirements page says PDO extension + the mysql driver works, phpinfo() says that PDO and the MySQL driver is installed, I have configured the 'db' component in the main config file for my project generated with yiic webapp, checked and double checked that the settings are correct (and yes, I am using mysql).
I have made a new migration script in /[mywebapp]/protected/migrations and now I'm trying to run the ./protected/yiic migrate command, but i just get an exception:
exception 'CDbException' with message 'CDbConnection failed to open the DB connection: could not find driver'
I have no idea what is wrong. I have been googling for 2 hours now and i find a lot of other users experiencing the same problems, but usually they are missing the drivers or something obvious. Is there anything i'm completely overlooking?
Despite the real answer being in the comments for this question, I am answering it here so that it appears as an answer. Yiic.php migrate uses the configuration stored in console.php. You need to set your database connection in there to use yiic.