I am trying to connect Talend (version 7.0.1) to MySQL Server (version 8.0.12), I can successfully connect and perform simple select queries to the database. However as soon as I try to perform a bulk Insert I get the following exception:
Starting job A_2_ProcessRestData_NEW at 17:59 20/08/2018.
[statistics] connecting to socket on port 3407
[statistics] connected
Exception in component tDBOutputBulkExec_1_tMBE (A_2_ProcessRestData_NEW)
java.sql.SQLException: Could not retrieve transation read-only status server
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1084)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:973)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:918)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:949)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:939)
at com.mysql.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:3976)
at com.mysql.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:3947)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:813)
at com.mysql.jdbc.StatementImpl.execute(StatementImpl.java:788)
at living_dashboard.a_2_processrestdata_new_0_1.A_2_ProcessRestData_NEW.tFileInputDelimited_1Process(A_2_ProcessRestData_NEW.java:4671)
at living_dashboard.a_2_processrestdata_new_0_1.A_2_ProcessRestData_NEW.runJobInTOS(A_2_ProcessRestData_NEW.java:6955)
at living_dashboard.a_2_processrestdata_new_0_1.A_2_ProcessRestData_NEW.main(A_2_ProcessRestData_NEW.java:6733)
Caused by: java.sql.SQLException: Unknown system variable 'tx_read_only'
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1084)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4232)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:4164)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2615)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2776)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2832)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2781)
at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1569)
at com.mysql.jdbc.ConnectionImpl.isReadOnly(ConnectionImpl.java:3970)
... 6 more
[statistics] disconnected
Job A_2_ProcessRestData_NEW ended at 17:59 20/08/2018. [exit code=1]
Do you have any clue on what might be?
Thanks in advance
I eventually solved the question. It turns out that the DB Version to be used cannot be MySQL 5 as I was specifying. I need to use MariaDB instead. Problem solved ;)
Related
MySQL 5.7, a transaction is running but thread is sleeping, client request(tomcat) is blocking, it will last for many many seconds, after killing connection in MySQL, tomcat receives below exception:
org.springframework.dao.RecoverableDataAccessException:
### Error querying database. Cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet successfully received from the server was 852,932 milliseconds ago. The last packet sent successfully to the server was 857,937 milliseconds ago.
at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:98)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:82)
at org.mybatis.spring.MyBatisExceptionTranslator.translateExceptionIfPossible(MyBatisExceptionTranslator.java:73)
at org.mybatis.spring.SqlSessionTemplate$SqlSessionInterceptor.invoke(SqlSessionTemplate.java:446)
at com.sun.proxy.$Proxy59.selectList(Unknown Source)
at org.mybatis.spring.SqlSessionTemplate.selectList(SqlSessionTemplate.java:230)
......
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:989)
at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2222)
at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1982)
at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:3407)
at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:470)
at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:3109)
at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:2334)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2733)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2549)
at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
at com.mysql.jdbc.PreparedStatement.execute(PreparedStatement.java:1192)
......
Caused by: java.io.EOFException: Can not read response from server. Expected to read 20,481 bytes, read 19,682 bytes before connection was unexpectedly lost.
at com.mysql.jdbc.MysqlIO.readFully(MysqlIO.java:3008)
at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:2205)
I use alibaba druid Connection Pool, testOnBorrow=true, mysql java driver version is 5.1.40
The above exception is thrown after connection is killed, tomcat is blocking until the connection is killed.
This case occurs several times in production evn, it is hard to repeat in develop env.
As the caused exception is Can not read response from server. Expected to read 20,481 bytes, read 19,682 bytes before connection was unexpectedly lost., client is waiting for more data from server, so I guess the connection borrowed from pool is valid at first, but why can't reading more data from server?
BTW: we recently use MySQL /*+ MAX_EXECUTION_TIME(xxx) */ optimizer hint. MySQL will throw Exception if query is timeout, I don't know whether it is related with my problem, but I guess it should not.
Presumably your query from your servlet ran for longer than your MAX_EXECUTION_TIME. At any rate, it looks like your query timed out when it had been running for 853 seconds.
And, your servlet crashed. The error trace you provided shows what happened.
If you want your servlet to recover from having its JDBC connections killed, you must program it to catch an exception when you KILL its database process, and then repeat the query.
You'll have to do some experimenting to get this right.
You can force the failure in development by using a much shorter MAX_EXECUTION_TIME. Or you can access your development MySQL server with a command client, look up the processid of the process serving your spring servlet, and KILL it manually.
The root cause: I believe your query takes far too long. You must optimize it somehow. Look at query-performance for ideas about that.
I am trying to generateChangeLog for a db on a percona server, i get the below error when i try to do so.
Starting Liquibase at Wed, 05 Dec 2018 22:34:37 EST (version 3.6.2 built at
2018-07-03 11:28:09)
Unexpected error running Liquibase: liquibase.exception.DatabaseException:
liquibase.exception.UnexpectedLiquibaseException: Error during testing for
MySQL/MariaDB JDBC driver bug: could not retrieve JDBC metadata information
for temporary table 'TMP_XDBOCVCKWHSQYXKP'
liquibase.exception.LiquibaseException:
liquibase.command.CommandExecutionException:
liquibase.exception.DatabaseException:
liquibase.exception.UnexpectedLiquibaseException: Error during testing for
MySQL/MariaDB JDBC driver bug: could not retrieve JDBC metadata information
for temporary table 'TMP_XDBOCVCKWHSQYXKP'
at liquibase.integration.commandline.Main.doMigration(Main.java:1043)
at liquibase.integration.commandline.Main.run(Main.java:191)
at liquibase.integration.commandline.Main.main(Main.java:129)
Caused by: liquibase.command.CommandExecutionException:
liquibase.exception.DatabaseException:
liquibase.exception.UnexpectedLiquibaseException: Error during testing
for MySQL/MariaDB JDBC driver bug: could not retrieve JDBC metadata
information for temporary table 'TMP_XDBOCVCKWHSQYXKP'
I am trying to use it via command line using below statement
liquibase --driver=com.mysql.cj.jdbc.Driver --classpath=C:/liquibase-3.6.2-
bin/jars/mysql-connector-java-8.0.13.jar --changeLogFile=db.changelog-
1.0.xml --url="jdbc:mysql://REMOTE_SERVER_IP:3306/DB_NAME?
autoReconnect=true" --username=USER_NAME --password=PASSWORD --logLevel=info
generateChangeLog
Has anyone encountered this issue before? I tried using the old and new JDBC drivers but did not help.
I hit the same issue. Setting the logLevel to debug you could see that Liquibase was creating and dropping a temporary table. That temporary table seemed to cause the issue.
I was able to work-around this issue by using MySQL Workbench to copy the schema to a regular MySQL instance then running generateChangeLog from there.
At the moment I am trying to follow this documentation in order to connect to my Second Generation Cloud SQL database. However, my application fails on startup since it cannot connect to it ..
As been told by the documentation, I have added both service accounts
my-project#appspot.gserviceaccount.com
1234-compute#developer.gserviceaccount.com
as Editor under IAM.
I am using the IPv4 of the Cloud SQL instance to connect. Here the connection string:
jdbc:mysql://w.x.y.z:3306/app_db?useSSL=false
but in the Stackdriver Logging view I see:
[main] ERROR com.mz.server.BootstrappingServerConfig - Error trying to migrate SQL scripts ..
org.flywaydb.core.api.FlywayException: Unable to obtain Jdbc connection from DataSource
at org.flywaydb.core.internal.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at org.flywaydb.core.Flyway.execute(Flyway.java:1385)
at org.flywaydb.core.Flyway.migrate(Flyway.java:1006)
...
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
...
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
...
Both instances are in the same region: us-central1-b for the compute instances and us-central1 for the SQL instances.
My app.yaml file:
runtime: java
vm: true
runtime_config:
jdk: openjdk8
server: jetty9
env_variables:
'ALPN_ENABLE': 'true' # OPTIONAL
beta_settings:
cloud_sql_instances: mz-test:us-central1:mz-test-cloudsql
health_check:
enable_health_check: false
I am not sure what's wrong here. Am I actually allowed to use the local IPv4 address of the SQL instance?
The documentation actually says
"In your application code, connect to the instance by opening a socket with the following name: /cloudsql/[INSTANCE_CONNECTION_NAME}"
But I am not sure what that would mean for the connection string.
I am also missing the information about what username/password is required or do I not need one here?
Why am I not able to connect?
For Java applications running on App Engine Flexible Environment applications use the following library: https://github.com/GoogleCloudPlatform/cloud-sql-mysql-socket-factory
I have a xampp server running on a AWS-EC2 Ubuntu instance, and I have a database there that is called "androiddb". I can access the database by writing in a navigator http://public_ip/phpmyadmin and entering with "root" user and no password.
In another side I have a Java application that needs to connect with that database, and I have the following line in my code to do it:
con = DriverManager.getConnection("jdbc:mysql://public_ip/androiddb",
"root", "");
When I run my application, I get the following error:
Error: Communications link failure due to underlying exception:
** BEGIN NESTED EXCEPTION **
java.net.SocketException
MESSAGE: java.net.ConnectException: Connection refused
STACKTRACE:
java.net.SocketException: java.net.ConnectException: Connection refused
at com.mysql.jdbc.StandardSocketFactory.connect(StandardSocketFactory.java:156)
at com.mysql.jdbc.MysqlIO.<init>(MysqlIO.java:284)
at com.mysql.jdbc.Connection.createNewIO(Connection.java:2569)
at com.mysql.jdbc.Connection.<init>(Connection.java:1485)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:266)
at java.sql.DriverManager.getConnection(DriverManager.java:571)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at Presentacion.Presentacion$2.mouseClicked(Presentacion.java:125)
at java.awt.Component.processMouseEvent(Component.java:6519)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3321)
at java.awt.Component.processEvent(Component.java:6281)
at java.awt.Container.processEvent(Container.java:2229)
at java.awt.Component.dispatchEventImpl(Component.java:4872)
at java.awt.Container.dispatchEventImpl(Container.java:2287)
at java.awt.Component.dispatchEvent(Component.java:4698)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4832)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4501)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4422)
at java.awt.Container.dispatchEventImpl(Container.java:2273)
at java.awt.Window.dispatchEventImpl(Window.java:2719)
at java.awt.Component.dispatchEvent(Component.java:4698)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:747)
at java.awt.EventQueue.access$300(EventQueue.java:103)
at java.awt.EventQueue$3.run(EventQueue.java:706)
at java.awt.EventQueue$3.run(EventQueue.java:704)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:87)
at java.awt.EventQueue$4.run(EventQueue.java:720)
at java.awt.EventQueue$4.run(EventQueue.java:718)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$1.doIntersectionPrivilege(ProtectionDomain.java:76)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:717)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:242)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:150)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:146)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:138)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:91)
** END NESTED EXCEPTION **
Last packet sent to the server was 19 ms ago.
Also I looked for the port where mysql was running and I used the code:
SHOW VARIABLES
WHERE Variable_name IN (
'hostname',
'port')
I write this piece of code in phpmyadmin in the SQL section, and I get the following output:
Variable_name Value
hostname ip-172-31-23-224
port 0
I don't know what does port 0 mean, it should be 3306 (default mysql port)...
Anyways, I don't know if I'm doing something wrong with the xampp configuration or if it's problem of AWS or anything else.
Could someone help me with this? Thank you very much.
You should ensure your security groups for AWS are open on the 3306 port (or whatever you configure) so they can access the database.
You could trouble shoot by trying to SSH into the machine, can you connect locally? If yes, then look at your security groups and open the port. If you can't connect locally than the database is down and you need to diagnose and configure the DB correctly.
But a connection refused is more than likely a port not being open in the security group.
Just to add what solved his issue:
He checked his security groups and opened the correct ports
Checked locally to see that mySQL wasn't listening on any port
netstat -tlnp to check to see SQL running locally
Checked the my.ini file to see that mysql was configured to listen on a port
Found it wasn't and reconfigured it to listen
It may help some working with phpMyAdmin and AWS and Amazon's relational database service (RDS) to know that phpMyAdmin will not work unless the RDS endpoint is edited into the "localhost" or "127.0.0.1" location in the config.inc.php file. (It is a necessary but not sufficient fix -- you may need to do other things...)
Generalizing a bit, if you are using foo and it has a config file, and you are looking to connect to a database instantiated under RDS, you'll need that endpoint as host entry.
I cannot figure out how to access a MySQL database while running in Jenkins CloudBees. I have tried following the Dev#Cloud MySQL, but there are a lot of warnings in the console output and I have to deal with localhost access vs. sockets, which is pretty tricky to debug without a command line.
Is there a way to use the CloudBees MySQL hosting? When I try and use the cloudbees JDBC driver, I get ClassNotFound error messages.
Update: The answer below clarified that I cannot use the CloudBees driver, so I am now using the regular MySQL connector through the c3p0 connection pool. Based on comments below, I have limited the number of connections in the pool to 10.
The exact same configuration and Maven command line works when I run it from my development computer, connecting to the Cloudbees Mysql instance.
Unfortunately, I get this error when trying to connect:
Caused by: com.googlecode.flyway.core.exception.FlywayException: Unable to obtain Jdbc connection from DataSource
at com.googlecode.flyway.core.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:56)
at com.googlecode.flyway.core.Flyway.execute(Flyway.java:864)
at com.googlecode.flyway.core.Flyway.migrate(Flyway.java:591)
at com.assuredlabor.db.DatabaseMigrator.migrate(DatabaseMigrator.java:13)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1414)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1375)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1335)
... 62 more
Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:106)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:529)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:128)
at com.googlecode.flyway.core.util.jdbc.JdbcUtils.openConnection(JdbcUtils.java:50)
... 72 more
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1319)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:557)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:477)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:525)
... 74 more
Any ideas would be appreciated.
You can't use CloudBees JDBC driver from outside the RUN#Cloud container, but you can use a standard MySQL JDBC driver with your RUN database using the host name and port that you can find on RUN#Cloud web console for your DB resource.