org.apache.openjpa.lib.jdbc.ReportingSQLException, MySQL, OpenJPA, Geronimo - mysql

Can anyone explain why I'm getting this error (full stack trace below):
Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TYPE = innodb' at line 1** {stmnt 763763032 CREATE TABLE ADMIN_USER (admin_id BIGINT NOT NULL AUTO_INCREMENT, first_name VARCHAR(255), last_name VARCHAR(255), user_name VARCHAR(255), PRIMARY KEY (admin_id)) TYPE = innodb} [code=1064, state=42000]*
I've got a web app with MySQL 5.5.11 and openJPA 1.2.2 and Geronimo 2.2.1 with Tomcat 6. I'm able to depoly the .ear fine. But when I submit my form to persist a Admin #Entity, I get the following error above.
Also, I'm using tranql-connector-mysql-local-1.5.rar (from the Geronimo repository) as my connector.
persistence.xml
<persistence xmlns="http://java.sun.com/xml/ns/persistence" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" version="1.0">
<persistence-unit name="AReyesPersistUnit" transaction-type="JTA">
<provider>org.apache.openjpa.persistence.PersistenceProviderImpl</provider>
<jta-data-source>AReyesMySql</jta-data-source>
<class>com.areyes.entity.Admin</class>
<properties>
<property name="openjpa.jdbc.SynchronizeMappings" value="buildSchema"/>
</properties>
</persistence-unit>
</persistence>
Admin #Entity
#Entity
#Table (name="ADMIN_USER")
public class Admin implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#GeneratedValue (strategy=GenerationType.IDENTITY)
#Column (name="admin_id")private long id;
#Column (name="user_name")private String userName;
#Column (name="first_name")private String firstName;
#Column (name="last_name")private String lastName;
...followed by getters/setters
Stack trace
2011-06-30 23:15:52,203 ERROR [[AddAdmin]] Servlet.service() for servlet AddAdmin threw exception
javax.ejb.EJBException: The bean encountered a non-application exception; nested exception is:
<openjpa-1.2.2-r422266:898935 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TYPE = innodb' at line 1 {stmnt 1243851750 CREATE TABLE ADMIN_USER (admin_id BIGINT NOT NULL AUTO_INCREMENT, first_name VARCHAR(255), last_name VARCHAR(255), user_name VARCHAR(255), PRIMARY KEY (admin_id)) TYPE = innodb} [code=1064, state=42000]
at org.apache.openejb.core.ivm.BaseEjbProxyHandler.convertException(BaseEjbProxyHandler.java:359)
at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:287)
at $Proxy65.create(Unknown Source)
at com.areyes.servlet.AddAdmin.doPost(AddAdmin.java:38)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:713)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:806)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.geronimo.tomcat.GeronimoStandardContext$SystemMethodValve.invoke(GeronimoStandardContext.java:420)
at org.apache.geronimo.tomcat.valve.GeronimoBeforeAfterValve.invoke(GeronimoBeforeAfterValve.java:47)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:857)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:396)
at org.apache.geronimo.pool.ThreadPool$1.run(ThreadPool.java:214)
at org.apache.geronimo.pool.ThreadPool$ContextClassLoaderRunnable.run(ThreadPool.java:344)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:680)
Caused by: <openjpa-1.2.2-r422266:898935 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TYPE = innodb' at line 1 {stmnt 1243851750 CREATE TABLE ADMIN_USER (admin_id BIGINT NOT NULL AUTO_INCREMENT, first_name VARCHAR(255), last_name VARCHAR(255), user_name VARCHAR(255), PRIMARY KEY (admin_id)) TYPE = innodb} [code=1064, state=42000]
at org.apache.openjpa.jdbc.meta.MappingTool.record(MappingTool.java:553)
at org.apache.openjpa.jdbc.meta.MappingTool.record(MappingTool.java:453)
at org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory.synchronizeMappings(JDBCBrokerFactory.java:159)
at org.apache.openjpa.jdbc.kernel.JDBCBrokerFactory.newBrokerImpl(JDBCBrokerFactory.java:119)
at org.apache.openjpa.kernel.AbstractBrokerFactory.newBroker(AbstractBrokerFactory.java:189)
at org.apache.openjpa.kernel.DelegatingBrokerFactory.newBroker(DelegatingBrokerFactory.java:142)
at org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:192)
at org.apache.openjpa.persistence.EntityManagerFactoryImpl.createEntityManager(EntityManagerFactoryImpl.java:56)
at org.apache.geronimo.persistence.CMPEntityManagerTxScoped.createEntityManager(CMPEntityManagerTxScoped.java:74)
at org.apache.geronimo.persistence.CMPEntityManagerTxScoped.getEntityManager(CMPEntityManagerTxScoped.java:55)
at org.apache.geronimo.persistence.CMPEntityManagerTxScoped.persist(CMPEntityManagerTxScoped.java:81)
at com.areyes.session.AdminManager.create(AdminManager.java:27)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:162)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:144)
at org.apache.openejb.monitoring.StatsInterceptor.record(StatsInterceptor.java:164)
at org.apache.openejb.monitoring.StatsInterceptor.invoke(StatsInterceptor.java:92)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext$Invocation.invoke(ReflectionInvocationContext.java:162)
at org.apache.openejb.core.interceptor.ReflectionInvocationContext.proceed(ReflectionInvocationContext.java:144)
at org.apache.openejb.core.interceptor.InterceptorStack.invoke(InterceptorStack.java:122)
at org.apache.openejb.core.stateless.StatelessContainer._invoke(StatelessContainer.java:221)
at org.apache.openejb.core.stateless.StatelessContainer.invoke(StatelessContainer.java:174)
at org.apache.openejb.core.ivm.EjbObjectProxyHandler.businessMethod(EjbObjectProxyHandler.java:217)
at org.apache.openejb.core.ivm.EjbObjectProxyHandler._invoke(EjbObjectProxyHandler.java:77)
at org.apache.openejb.core.ivm.BaseEjbProxyHandler.invoke(BaseEjbProxyHandler.java:282)
... 23 more
Caused by: org.apache.openjpa.lib.jdbc.ReportingSQLException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'TYPE = innodb' at line 1 {stmnt 1243851750 CREATE TABLE ADMIN_USER (admin_id BIGINT NOT NULL AUTO_INCREMENT, first_name VARCHAR(255), last_name VARCHAR(255), user_name VARCHAR(255), PRIMARY KEY (admin_id)) TYPE = innodb} [code=1064, state=42000]
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.wrap(LoggingConnectionDecorator.java:192)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator.access$700(LoggingConnectionDecorator.java:57)
at org.apache.openjpa.lib.jdbc.LoggingConnectionDecorator$LoggingConnection$LoggingStatement.executeUpdate(LoggingConnectionDecorator.java:762)
at org.apache.openjpa.jdbc.schema.SchemaTool.executeSQL(SchemaTool.java:1191)
at org.apache.openjpa.jdbc.schema.SchemaTool.createTable(SchemaTool.java:949)
at org.apache.openjpa.jdbc.schema.SchemaTool.add(SchemaTool.java:526)
at org.apache.openjpa.jdbc.schema.SchemaTool.add(SchemaTool.java:344)
at org.apache.openjpa.jdbc.schema.SchemaTool.run(SchemaTool.java:321)
at org.apache.openjpa.jdbc.meta.MappingTool.record(MappingTool.java:501)
... 54 more

After much digging, this is a bug that has already been fixed. The root issue is that MySQL changed the table_option 'type' to 'engine' in the 5.1ish time frame.
Either you can move to a newer level of OpenJPA(1.3.x, 2.0.x, etc), or don't use OpenJPA to create your tables.

Related

<Issue> Apache Hive: Can't insert record after adding column

I use Hive 3.1.0.3.1.0.0-78.
Create source table with statement:
CREATE TABLE IF NOT EXISTS core.source_jira
(
pk_jira BIGINT DEFAULT SURROGATE_KEY(),
test_1 STRING,
test_2 STRING,
valid BOOLEAN,
PRIMARY KEY (pk_jira) DISABLE NOVALIDATE
)
STORED AS ORC
LOCATION '/warehouse/tablespace/managed/hive/core.db/source_jira'
TBLPROPERTIES (
"orc.compress" = "SNAPPY",
"transactional" = "true",
"NO_AUTO_COMPACTION"="TRUE"
)
;
INSERT INTO TABLE core.source_jira(test_1, test_2, valid)
SELECT "text_1", "text_2", TRUE;
Check table data with select:
SELECT * from core.source_jira
pk_jira
test_1
test_2
valid
1099511627776
text_1
text_2
true
Create destination table with statement:
CREATE TABLE IF NOT EXISTS core.destination_jira
(
pk_jira BIGINT DEFAULT SURROGATE_KEY(),
test_1 STRING,
test_2 STRING,
valid BOOLEAN,
PRIMARY KEY (pk_jira) DISABLE NOVALIDATE
)
STORED AS ORC
LOCATION '/warehouse/tablespace/managed/hive/core.db/destination_jira'
TBLPROPERTIES (
"orc.compress" = "SNAPPY",
"transactional" = "true",
"NO_AUTO_COMPACTION"="TRUE"
)
;
Add column:
ALTER TABLE core.destination_jira ADD COLUMNS (jira_type STRING);
Insert data statement:
insert into table core.destination_jira (test_1, test_2, valid) select test_1, test_2, valid from core.source_jira
I got an error:
SQL Error [40000] [42000]: Error while compiling statement: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
org.jkiss.dbeaver.model.sql.DBSQLException: SQL Error [40000] [42000]: Error while compiling statement: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:577)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$1(SQLQueryJob.java:486)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:172)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:493)
at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:894)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:3645)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:123)
at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:172)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:121)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:4949)
at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:279)
at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:265)
at org.apache.hive.jdbc.HiveStatement.runAsyncOnServer(HiveStatement.java:303)
at org.apache.hive.jdbc.HiveStatement.execute(HiveStatement.java:244)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:329)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.lambda$0(JDBCStatementImpl.java:131)
at org.jkiss.dbeaver.utils.SecurityManagerUtils.wrapDriverActions(SecurityManagerUtils.java:94)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:131)
... 12 more
Caused by: org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: FAILED: RuntimeException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:335)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:199)
at org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:262)
at org.apache.hive.service.cli.operation.Operation.run(Operation.java:247)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:575)
at org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:561)
at sun.reflect.GeneratedMethodAccessor72.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
at org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
at org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
at com.sun.proxy.$Proxy47.executeStatementAsync(Unknown Source)
at org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:315)
at org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:566)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
at org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getColNameToDefaultValueMap(SemanticAnalyzer.java:750)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getColForInsertStmtSpec(SemanticAnalyzer.java:4620)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.handleInsertStatementSpec(SemanticAnalyzer.java:4735)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4580)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genSelectPlan(SemanticAnalyzer.java:4326)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPostGroupByBodyPlan(SemanticAnalyzer.java:10662)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genBodyPlan(SemanticAnalyzer.java:10601)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11530)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genPlan(SemanticAnalyzer.java:11400)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genOPTree(SemanticAnalyzer.java:12202)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.genOPTree(CalcitePlanner.java:620)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(SemanticAnalyzer.java:12292)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(CalcitePlanner.java:360)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:289)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1816)
at org.apache.hadoop.hive.ql.Driver.compileAndRespond(Driver.java:1811)
at org.apache.hadoop.hive.ql.reexec.ReExecDriver.compileAndRespond(ReExecDriver.java:126)
at org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:197)
... 26 more
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException:java.lang.NullPointerException
at org.apache.hadoop.hive.ql.metadata.Hive.getEnabledDefaultConstraints(Hive.java:5389)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.getColNameToDefaultValueMap(SemanticAnalyzer.java:744)
... 45 more
Caused by: java.lang.NullPointerException: null

Spark JDBC MySQL Syntax Error with column names with spaces

I'm querying a MySQL table using PySpark JDBC. The problem I'm facing is that said table has column names with spaces.
In MySQL Workbench I would do the query like this to get the data and that works fine.
select
`some id`
from
table
But when I do it via PySpark like this:
query = 'select `some id` from table'
df = sqlContext.read.format('jdbc') \
.options(
driver = 'com.mysql.jdbc.Driver'
, url = some_connection_url
, dbtable = query
, user = user
, password = password
).load()
I get the following error message:
Py4JJavaError: An error occurred while calling o497.load.
: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'select
`some id`
from
some_table WHERE 1=0' at line 1
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97)
at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeInternal(ClientPreparedStatement.java:953)
at com.mysql.cj.jdbc.ClientPreparedStatement.executeQuery(ClientPreparedStatement.java:1003)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRDD$.resolveTable(JDBCRDD.scala:61)
at org.apache.spark.sql.execution.datasources.jdbc.JDBCRelation$.getSchema(JDBCRelation.scala:210)
at org.apache.spark.sql.execution.datasources.jdbc.JdbcRelationProvider.createRelation(JdbcRelationProvider.scala:35)
at org.apache.spark.sql.execution.datasources.DataSource.resolveRelation(DataSource.scala:318)
at org.apache.spark.sql.DataFrameReader.loadV1Source(DataFrameReader.scala:223)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:211)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:167)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:244)
at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357)
at py4j.Gateway.invoke(Gateway.java:282)
at py4j.commands.AbstractCommand.invokeMethod(AbstractCommand.java:132)
at py4j.commands.CallCommand.execute(CallCommand.java:79)
at py4j.GatewayConnection.run(GatewayConnection.java:238)
at java.lang.Thread.run(Thread.java:748)
I've tried quoting the column names with [] and "" but I get the same error message.
I'm also using the latest MySQL jars: mysql-connector-java-8.0.21.jar and Spark 2.4.
Any insights are highly appreciated.
Enclose your query in brackets () then try to pass to read.format("jdbc").
Example:
query = '(select `some id` from table)e'
df = sqlContext.read.format('jdbc') \
.options(
driver = 'com.mysql.jdbc.Driver'
, url = some_connection_url
, dbtable = query
, user = user
, password = password
).load()

Innodb deadlock, not mentioned in ENGINE INNODB STATUS

I get a deadlock exception, but when I want to see some info about that from "SHOW ENGINE INNODB STATUS", it contains deadlock information from longer time ago (long before the exception), so it looks like it is just "not catched" by that command.
From the stacktrace I can see, that the last my application code executed was the $Proxy174.attemptToAcquireJobExecution , which is realization of the spring data interface:
#Transactional(TxType.REQUIRES_NEW)
#Modifying
#Query("UPDATE Job SET last_run = NOW(),
last_run_by=:runner
WHERE name = :name
AND (last_run + :interval ) < NOW()")
int attemptToAcquireJobExecution(
#Param("name") String var1,
#Param("interval") int var2,
#Param("runner") String var3);
It operates on such a simple table:
CREATE TABLE `jobs` (
`name` varchar(50) COLLATE utf8mb4_unicode_ci NOT NULL,
`last_run_by` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`last_run` datetime(3) DEFAULT NULL,
`context_data` varchar(100) COLLATE utf8mb4_unicode_ci DEFAULT NULL,
PRIMARY KEY (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Scheduled tasks, that are executing that command runs every few minutes (on 3 app instances), so there is really minimal risk of collisions, yet deadlock occurs very often... There is no #Transactional anywhere before that operation, so there should be no hanging transaction really. Anyway, pinging those commands yields empty results:
SELECT * FROM INFORMATION_SCHEMA.INNODB_LOCKS ;
SELECT * FROM INFORMATION_SCHEMA.INNODB_LOCK_WAITS;
Original stacktrace, my classes marked as "xxx", at the bottom:
com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(NativeConstructorAccessorImpl.java)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
at com.mysql.jdbc.Util.getInstance(Util.java:408)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:951)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3973)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3909)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2527)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2680)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2497)
at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1566)
... 2 frames excluded
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.tomcat.jdbc.pool.ProxyConnection.invoke(ProxyConnection.java:126)
at org.apache.tomcat.jdbc.pool.JdbcInterceptor.invoke(JdbcInterceptor.java:108)
at org.apache.tomcat.jdbc.pool.DisposableConnectionFacade.invoke(DisposableConnectionFacade.java:81)
at com.sun.proxy.$Proxy121.commit(Unknown Source)
at org.hibernate.resource.jdbc.internal.AbstractLogicalConnectionImplementor.commit(AbstractLogicalConnectionImplementor.java:80)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.commit(JdbcResourceLocalTransactionCoordinatorImpl.java:232)
at org.hibernate.engine.transaction.internal.TransactionImpl.commit(TransactionImpl.java:65)
at org.hibernate.jpa.internal.TransactionImpl.commit(TransactionImpl.java:61)
... 49 common frames omitted
Wrapped by: javax.persistence.PersistenceException: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1692)
at org.hibernate.jpa.spi.AbstractEntityManagerImpl.convert(AbstractEntityManagerImpl.java:1602)
at org.hibernate.jpa.internal.TransactionImpl.commit(TransactionImpl.java:75)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:517)
... 48 common frames omitted
Wrapped by: org.springframework.orm.jpa.JpaSystemException: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction; nested exception is javax.persistence.PersistenceException: com.mysql.jdbc.exceptions.jdbc4.MySQLTransactionRollbackException: Deadlock found when trying to get lock; try restarting transaction
at org.springframework.orm.jpa.EntityManagerFactoryUtils.convertJpaAccessExceptionIfPossible(EntityManagerFactoryUtils.java:418)
at org.springframework.orm.jpa.vendor.HibernateJpaDialect.translateExceptionIfPossible(HibernateJpaDialect.java:246)
at org.springframework.orm.jpa.JpaTransactionManager.doCommit(JpaTransactionManager.java:521)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:761)
at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:730)
at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:504)
at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:292)
at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:96)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.dao.support.PersistenceExceptionTranslationInterceptor.invoke(PersistenceExceptionTranslationInterceptor.java:136)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.jpa.repository.support.CrudMethodMetadataPostProcessor$CrudMethodMetadataPopulatingMethodInterceptor.invoke(CrudMethodMetadataPostProcessor.java:133)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.data.repository.core.support.SurroundingTransactionDetectorMethodInterceptor.invoke(SurroundingTransactionDetectorMethodInterceptor.java:57)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:213)
at com.sun.proxy.$Proxy174.attemptToAcquireJobExecution(Unknown Source)
at xxx.lib.scheduler.JobsService.checkIfJobShouldBeExecuted(JobsService.java:32)
at xxx.lib.scheduler.StandardBatchStarter.startScheduled(StandardBatchStarter.java:38)
at xxx.scheduled.ImportProductDescriptionJob.startScheduled(ImportProductDescriptionJob.java:46)
at xxx.scheduled.ImportProductDescriptionJob$$FastClassBySpringCGLIB$$494978af.invoke(<generated>)
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:721)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
at org.springframework.cloud.sleuth.instrument.scheduling.TraceSchedulingAspect.traceBackgroundThread(TraceSchedulingAspect.java:72)
... 2 frames excluded
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
... 21 frames truncated
AND (last_run + :interval ) < NOW()
Probably leads to a syntax error -- or a data error.
Please provide the generated SQL. I suspect it is generating
AND (last_run + 2) < NOW()
But it needs to be something like
AND (last_run + INTERVAL 2 DAY) < NOW()
Also, please provide SHOW CREATE Job.

Getting an error in update statement when accessing mysql from java

String sql = "update `library`.`memebers` set 'STATUS'='" + "1" +"' where mem_id = '"+str+"'";
int i = st.executeUpdate(sql);
Error in 2nd line above:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''status' = '1' where mem_id = '656597'' at line 1
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1052)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3609)
at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3541)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2002)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2163)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2618)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1749)
at com.mysql.jdbc.StatementImpl.executeUpdate(StatementImpl.java:1666)
at Frames.Addmembers.jButton1ActionPerformed(Addmembers.java:181)
at Frames.Addmembers.access$000(Addmembers.java:21)
at Frames.Addmembers$1.actionPerformed(Addmembers.java:83)
at javax.swing.AbstractButton.fireActionPerformed(AbstractButton.java:1995)
at javax.swing.AbstractButton$Handler.actionPerformed(AbstractButton.java:2318)
at javax.swing.DefaultButtonModel.fireActionPerformed(DefaultButtonModel.java:387)
at javax.swing.DefaultButtonModel.setPressed(DefaultButtonModel.java:242)
at javax.swing.plaf.basic.BasicButtonListener.mouseReleased(BasicButtonListener.java:236)
at java.awt.Component.processMouseEvent(Component.java:6263)
at javax.swing.JComponent.processMouseEvent(JComponent.java:3267)
at java.awt.Component.processEvent(Component.java:6028)
at java.awt.Container.processEvent(Container.java:2041)
at java.awt.Component.dispatchEventImpl(Component.java:4630)
at java.awt.Container.dispatchEventImpl(Container.java:2099)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.LightweightDispatcher.retargetMouseEvent(Container.java:4574)
at java.awt.LightweightDispatcher.processMouseEvent(Container.java:4238)
at java.awt.LightweightDispatcher.dispatchEvent(Container.java:4168)
at java.awt.Container.dispatchEventImpl(Container.java:2085)
at java.awt.Window.dispatchEventImpl(Window.java:2478)
at java.awt.Component.dispatchEvent(Component.java:4460)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:599)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:269)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:184)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:174)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:169)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:161)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:122)
Field name in single quotes. It should be either without quotes, or inside back single quotes: ```:
String sql = "update library.memebers set `STATUS`='1' where mem_id = '"+str+"'";
status is the name of a column, and as such, it should be either status or `status`.
Entities are either written plain, or they're quoted with ``. (Backticks, not apostrophes.)
Unless you're using reserved field names for column names (which you shouldn't do), you do not have to quote fields.
String sql = "update library.memebers set STATUS = 1 where mem_id = " + Integer.valueOf(str);
Note that I removed the quotes around the status and the mem_id. I'm assuming that both are integer fields, and if they are, you should not be quoting their values. Integer fields should have integer values, not strings that evaluate to integer values. (Note though that you would never want to just blindly assume a string is an integer and concatenate it into a query -- always make certain it's a proper integer first)

Not able to insert a record in mysql table with auto-increment key as PK

i am not able to insert a record. i am using hibernate and mysql.
files are as follows...
Route.hbm.xml
<id name="routeId" type="java.lang.Integer" unsaved-value="null">
<column name="ROUTE_ID" not-null="true" />
<generator class="increment" />
</id>
<property name="routeName" type="java.lang.String" column="ROUTE_NAME" />
<property name="from" type="java.lang.String" column="FROM" />
<property name="to" type="java.lang.String" column="TO" />
<property name="distance" type="java.lang.Integer" column="DISTANCE" />
<property name="time" type="java.lang.Integer" column="TIME" />
RouteDTO.java
private static final long serialVersionUID = 1L;
private Integer routeId;
private String routeName;
private String from;
private String to;
private Integer distance;
private Integer time;
public Integer getRouteId() {
return routeId;
}
public void setRouteId(Integer routeId) {
this.routeId = routeId;
}
.
.
.
.
Table Structure is
CREATE TABLE `route` (
`ROUTE_ID` int(5) NOT NULL AUTO_INCREMENT,
`ROUTE_NAME` varchar(20) NOT NULL,
`FROM` varchar(20) NOT NULL,
`TO` varchar(20) NOT NULL,
`DISTANCE` int(11) DEFAULT NULL,
`TIME` int(11) DEFAULT NULL,
PRIMARY KEY (`ROUTE_ID`)
)
when i am trying to save this record by RoutDAO.java
code is
getHibernateTemplate().save(routeDto);
it gives me error as follows....
Hibernate: select routedto0_.ROUTE_ID as ROUTE1_, routedto0_.ROUTE_NAME as ROUTE2_0_, routedto0_.FROM as FROM0_, routedto0_.TO as TO0_, routedto0_.DISTANCE as DISTANCE0_, routedto0_.TIME as TIME0_ from ROUTE routedto0_
Hibernate: insert into ROUTE (ROUTE_NAME, FROM, TO, DISTANCE, TIME, ROUTE_ID) values (?, ?, ?, ?, ?, ?)
Apr 15, 2011 9:04:03 PM org.hibernate.util.JDBCExceptionReporter logExceptions
WARNING: SQL Error: 1064, SQLState: 42000
Apr 15, 2011 9:04:03 PM org.hibernate.util.JDBCExceptionReporter logExceptions
SEVERE: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'FROM, TO, DISTANCE, TIME, ROUTE_ID) values ('A099', 'Hadapser', 'Pune Station', ' at line 1
Apr 15, 2011 9:04:03 PM org.hibernate.event.def.AbstractFlushingEventListener performExecutions
SEVERE: Could not synchronize database state with session
org.hibernate.exception.GenericJDBCException: Could not execute JDBC batch update
at org.hibernate.exception.ErrorCodeConverter.handledNonSpecificException(ErrorCodeConverter.java:92)
at org.hibernate.exception.ErrorCodeConverter.convert(ErrorCodeConverter.java:80)
at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:179)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:226)
at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:136)
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:274)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:27)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:726)
......
Caused by: java.sql.BatchUpdateException:
You have an error in your SQL syntax;
check the manual that corresponds to your MySQL server version
for the right syntax to use
near 'FROM, TO, DISTANCE, TIME, ROUTE_ID) values
('A099', 'Hadapser', 'Pune Station', ' at line 1
at
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2024)
at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1449)
at org.apache.commons.dbcp.DelegatingStatement.executeBatch(DelegatingStatement.java:294)
at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:57)
at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:172)
... 58 more
Please help me out...
This is because of the column calledFROM. This is a SQL keyword. The SQL being issued by Hibernate is therefore invalid. I'm mildly surprised MySQL let you create the table in the first place.
You should pick a different name for your column than FROM.
I think the problem is you are using MySQL reserved words for column names.