I'm using Hadoop 0.20.2 with Hive 0.11. I have succesfully inserted into hive/hdfs some csv-files in seperate tables. selects and joins work flawlessly. When trying to analyse some data, i needed to make use of the built in functions of hive like:
substr
to_date
rand
etc.
for example:
select sid, request_id, to_date(times), to_unix_timestamp(times) from contents where sid = '5000000032066010373';
sid and request id are strings here, times is a timestamp column
Unfortanetely i only get errors (always the same error stack) when using these functions:
java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:354)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
at org.apache.hadoop.mapred.Child.main(Child.java:170)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 5 more
Caused by: java.lang.RuntimeException: Error in configuring object
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:93)
at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:117)
at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
... 10 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:88)
... 13 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:121)
... 18 more
Caused by: java.lang.NoClassDefFoundError: org/codehaus/jackson/JsonFactory
at org.apache.hadoop.hive.ql.udf.generic.GenericUDTFJSONTuple.<clinit>(GenericUDTFJSONTuple.java:56)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:113)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:526)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.registerGenericUDTF(FunctionRegistry.java:520)
at org.apache.hadoop.hive.ql.exec.FunctionRegistry.<clinit>(FunctionRegistry.java:423)
at org.apache.hadoop.hive.ql.exec.DefaultUDFMethodResolver.getEvalMethod(DefaultUDFMethodResolver.java:59)
at org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge.initialize(GenericUDFBridge.java:154)
at org.apache.hadoop.hive.ql.udf.generic.GenericUDF.initializeAndFoldConstants(GenericUDF.java:111)
at org.apache.hadoop.hive.ql.exec.ExprNodeGenericFuncEvaluator.initialize(ExprNodeGenericFuncEvaluator.java:141)
at org.apache.hadoop.hive.ql.exec.Operator.initEvaluators(Operator.java:970)
at org.apache.hadoop.hive.ql.exec.Operator.initEvaluatorsAndReturnStruct(Operator.java:996)
at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:60)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
at org.apache.hadoop.hive.ql.exec.FilterOperator.initializeOp(FilterOperator.java:78)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:451)
at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:407)
at org.apache.hadoop.hive.ql.exec.TableScanOperator.initializeOp(TableScanOperator.java:186)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:543)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:100)
... 18 more
Caused by: java.lang.ClassNotFoundException: org.codehaus.jackson.JsonFactory
what am i doing wrong here?
SHOW FUNCTIONS;
shows me that these functions are in there ...
Try updating the jackson*.jar in your HADOOP_HOME/lib with the ones in HIVE_HOME/lib, restart Hadoop and try submitting the job.
I met the same problem while using hive + apache hadoop 0.20.2
the problem is that hadoop-0.20.2 doesn't have these jackson jars in the lib folder, so either you add it manually in the hive console every time:
add jars $HIVE_HOME/lib/jackson-core-asl-1.8.8.jar $HIVE_HOME/lib/jackson-jaxrs-1.8.8.jar $HIVE_HOME/lib/jackson-mapper-asl-1.8.8.jar $HIVE_HOME/lib/jackson-xc-1.8.8.jar;
or you copy all these jars to each machine in the cluster?
try updating the jackson*.jar in your HADOOP_HOME/lib with the ones in HIVE_HOME/lib, restart Hadoop and try submitting the job.
Thanks it worked
Related
I am trying to create an external table using hive with hadoop but somehow it failed. These are the error I get when I try to run my queries.
02:23:29.516 [HiveServer2-Background-Pool: Thread-39] ERROR hive.ql.exec.DDLTask - org.apache.hadoop.hive.ql.metadata.HiveException: Cannot validate serde: org.openx.data.jsonserde.JsonSerDe
at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3858)
at org.apache.hadoop.hive.ql.plan.CreateTableDesc.toTable(CreateTableDesc.java:700)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3960)
at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:333)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1077)
at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:235)
at org.apache.hive.service.cli.operation.SQLOperation.access$300(SQLOperation.java:90)
at org.apache.hive.service.cli.operation.SQLOperation$2$1.run(SQLOperation.java:299)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1926)
at org.apache.hive.service.cli.operation.SQLOperation$2.run(SQLOperation.java:312)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: Class org.openx.data.jsonserde.JsonSerDe not found
at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:2329)
at org.apache.hadoop.hive.ql.exec.DDLTask.validateSerDe(DDLTask.java:3852)
... 22 more
How can I solve it?
The exception says
java.lang.ClassNotFoundException: Class
org.openx.data.jsonserde.JsonSerDe not found
Install JSONSerDe (download JARS from http://www.congiu.net/hive-json-serde/ and put them into hive/lib), read instructions here: Hive-JSON-Serde
Also instead of putting jars into hive/lib you can try adding jars in hive session:
ADD JAR ADD JAR /usr/lib/hive/lib/json-serde-1.3.8-jar-with-dependencies.jar;
ADD JAR ADD JAR /usr/lib/hive/lib/json-udf-1.3.8-jar-with-dependencies.jar;
Alternatively you can try native Hive JSONSerDe: org.apache.hive.hcatalog.data.JsonSerDe - just change the class name in table DDL SerDe. It should be already installed. Read more details about differences here: https://docs.aws.amazon.com/athena/latest/ug/json-serde.html
I have the same issue when I use the hive command in cmd. But it works normally when I use the beeline with hive2 connection.
I tried to deploy a Java and Maven function app to Microsoft Azure. The deployment goes well but when I try to run the program it gets all these errors. I start getting all these communication errors. Everything works locally but when published it doesn't. I don't know what info I should give here. But if someone has got the same problem it would be appreciated if someone could help. I can provide more info if asked but don't know what I should put here.
> Result: Failure
Exception: KeyException: Key does not exist.
Stack: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.microsoft.azure.functions.worker.broker.JavaMethodInvokeInfo.invoke(JavaMethodInvokeInfo.java:22)
at com.microsoft.azure.functions.worker.broker.JavaMethodExecutor.execute(JavaMethodExecutor.java:54)
at com.microsoft.azure.functions.worker.broker.JavaFunctionBroker.invokeMethod(JavaFunctionBroker.java:51)
at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:33)
at com.microsoft.azure.functions.worker.handler.InvocationRequestHandler.execute(InvocationRequestHandler.java:10)
at com.microsoft.azure.functions.worker.handler.MessageHandler.handle(MessageHandler.java:45)
at com.microsoft.azure.functions.worker.JavaWorkerClient$StreamingMessagePeer.lambda$onNext$0(JavaWorkerClient.java:92)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.hibernate.exception.GenericJDBCException: Unable to acquire JDBC Connection
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:47)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:111)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:97)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:109)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getPhysicalConnection(LogicalConnectionManagedImpl.java:136)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.getConnectionForTransactionManagement(LogicalConnectionManagedImpl.java:254)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.begin(LogicalConnectionManagedImpl.java:262)
at org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorImpl$TransactionDriverControlImpl.begin(JdbcResourceLocalTransactionCoordinatorImpl.java:214)
at org.hibernate.engine.transaction.internal.TransactionImpl.begin(TransactionImpl.java:56)
at org.hibernate.internal.AbstractSharedSessionContract.beginTransaction(AbstractSharedSessionContract.java:409)
at io.madtrix.producerutil.dao.TokenRepository.getMadtrixTokenForManagerAccountAuthenticatedSourceWithClientId(TokenRepository.java:139)
at io.madtrix.MessageHandler.<init>(MessageHandler.java:38)
at io.madtrix.TwitterProducer.run(TwitterProducer.java:31)
... 16 more
Caused by: java.sql.SQLException: Connections could not be acquired from the underlying database!
at com.mchange.v2.sql.SqlUtils.toSQLException(SqlUtils.java:118)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:692)
at com.mchange.v2.c3p0.impl.AbstractPoolBackedDataSource.getConnection(AbstractPoolBackedDataSource.java:140)
at org.hibernate.c3p0.internal.C3P0ConnectionProvider.getConnection(C3P0ConnectionProvider.java:72)
at org.hibernate.internal.NonContextualJdbcConnectionAccess.obtainConnection(NonContextualJdbcConnectionAccess.java:35)
at org.hibernate.resource.jdbc.internal.LogicalConnectionManagedImpl.acquireConnectionIfNeeded(LogicalConnectionManagedImpl.java:106)
... 25 more
Caused by: com.mchange.v2.resourcepool.CannotAcquireResourceException: A ResourcePool could not acquire a resource from its primary factory or source.
at com.mchange.v2.resourcepool.BasicResourcePool.awaitAvailable(BasicResourcePool.java:1469)
at com.mchange.v2.resourcepool.BasicResourcePool.prelimCheckoutResource(BasicResourcePool.java:644)
at com.mchange.v2.resourcepool.BasicResourcePool.checkoutResource(BasicResourcePool.java:554)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutAndMarkConnectionInUse(C3P0PooledConnectionPool.java:758)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool.checkoutPooledConnection(C3P0PooledConnectionPool.java:685)
... 29 more
Caused by: java.sql.SQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:108)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:95)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:87)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:61)
at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:71)
at com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:932)
at com.mysql.cj.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:857)
at com.mysql.cj.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:444)
at com.mysql.cj.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:230)
at com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:226)
at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:175)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:220)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:206)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:203)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1138)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1125)
at com.mchange.v2.resourcepool.BasicResourcePool.access$700(BasicResourcePool.java:44)
at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1870)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:696)
Caused by: com.mysql.cj.exceptions.CJCommunicationsException: Communications link failure
The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server.
at sun.reflect.GeneratedConstructorAccessor36.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:59)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:103)
at com.mysql.cj.exceptions.ExceptionFactory.createException(ExceptionFactory.java:149)
at com.mysql.cj.exceptions.ExceptionFactory.createCommunicationsException(ExceptionFactory.java:165)
at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:355)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.negotiateSSLConnection(NativeAuthenticationProvider.java:789)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.proceedHandshakeWithPluggableAuthentication(NativeAuthenticationProvider.java:499)
at com.mysql.cj.protocol.a.NativeAuthenticationProvider.connect(NativeAuthenticationProvider.java:217)
at com.mysql.cj.protocol.a.NativeProtocol.connect(NativeProtocol.java:1411)
at com.mysql.cj.NativeSession.connect(NativeSession.java:165)
at com.mysql.cj.jdbc.ConnectionImpl.connectWithRetries(ConnectionImpl.java:876)
... 13 more
Caused by: javax.net.ssl.SSLException: java.security.ProviderException: java.security.KeyException: Key does not exist.
at sun.security.ssl.Alerts.getSSLException(Alerts.java:208)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1964)
at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1921)
at sun.security.ssl.SSLSocketImpl.handleException(SSLSocketImpl.java:1904)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1420)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1397)
at com.mysql.cj.protocol.ExportControlled.performTlsHandshake(ExportControlled.java:213)
at com.mysql.cj.protocol.StandardSocketFactory.performTlsHandshake(StandardSocketFactory.java:206)
at com.mysql.cj.protocol.a.NativeSocketConnection.performTlsHandshake(NativeSocketConnection.java:99)
at com.mysql.cj.protocol.a.NativeProtocol.negotiateSSLConnection(NativeProtocol.java:350)
... 19 more
Caused by: java.security.ProviderException: java.security.KeyException: Key does not exist.
at sun.security.mscapi.RSACipher.doFinal(RSACipher.java:310)
at sun.security.mscapi.RSACipher.engineDoFinal(RSACipher.java:334)
at javax.crypto.Cipher.doFinal(Cipher.java:2164)
at java.security.Signature$CipherAdapter.engineVerify(Signature.java:1319)
at java.security.SignatureSpi.engineVerify(SignatureSpi.java:278)
at java.security.Signature$Delegate.engineVerify(Signature.java:1228)
at java.security.Signature.verify(Signature.java:704)
at sun.security.ssl.RSASignature.engineVerify(RSASignature.java:182)
at sun.security.ssl.RSASignature.engineVerify(RSASignature.java:175)
at java.security.Signature$Delegate.engineVerify(Signature.java:1222)
at java.security.Signature.verify(Signature.java:655)
at sun.security.ssl.HandshakeMessage$ECDH_ServerKeyExchange.<init>(HandshakeMessage.java:1118)
at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:284)
at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1052)
at sun.security.ssl.Handshaker.process_record(Handshaker.java:987)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1072)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1385)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1413)
... 24 more
Caused by: java.security.KeyException: Key does not exist.
at sun.security.mscapi.RSACipher.encryptDecrypt(Native Method)
at sun.security.mscapi.RSACipher.doFinal(RSACipher.java:294)
... 41 more
The answer was in the connection to MySQL. It used SSL=true but when changed to SSL=false it started working normally.
We started having a broken pipe issue this past Saturday with a job that gets run via a quartz scheduler during the early morning of each day. Below are some details on this.
The job consists of a native java class that calls Talend based jobs. Each of these Talend based jobs are getting the broken pipe error. Curious enough, when I run the same java class that gets submitted via the scheduler, no errors get thrown and the jobs run successfully. Also, this job stream runs in Tomcat.
Is there something I should do within the Talend jobs to see if a connection is available first? As you will see in the following jdbc parameters used for the tMysqlConnection component, the autoreconnect option is set to on.
jdbc parameters are as follows are "noDatetimeStringSync=true&connectTimeout=0&socketTimeout=0&autoReconnect=true"
Below is the stacktrace from this morning's job failure.
Any input is greatly appreciated. Thank you.
Exception in component tMysqlConnection_1
com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 344,265,715 milliseconds ago. The last packet sent successfully to the server was 344,265,792 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
at sun.reflect.GeneratedConstructorAccessor840.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:532)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1117)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3871)
at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2484)
at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2664)
at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2788)
at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5282)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tMysqlConnection_1Process(CSXDemand_MasterService.java:2381)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tFileList_2Process(CSXDemand_MasterService.java:2284)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tFileExist_1Process(CSXDemand_MasterService.java:1146)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tJava_2Process(CSXDemand_MasterService.java:1054)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.tJava_1Process(CSXDemand_MasterService.java:972)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.runJobInTOS(CSXDemand_MasterService.java:4370)
at umaxbalancing_etl.csxdemand_masterservice_0_1.CSXDemand_MasterService.runJob(CSXDemand_MasterService.java:4079)
at com.rez1.dws.mid.etl.engine.extract.talend.RunTalendDailyJobs.runCSXDemandExtract(RunTalendDailyJobs.java:45)
at com.rez1.dws.mid.etl.engine.extract.talend.RunTalendDailyJobs.dailyJobs(RunTalendDailyJobs.java:28)
at com.rez1.dws.mid.etl.engine.master.service.DWMasterETLServiceImpl.processTalendDailyJobs(DWMasterETLServiceImpl.java:401)
at com.rez1.dws.mid.etl.engine.master.service.DWMasterETLServiceImpl$$FastClassByCGLIB$$13fc5e62.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:91)
at com.rez1.common.processmonitor.ProcessMonitor.handleNestedInvocation(ProcessMonitor.java:440)
at com.rez1.common.processmonitor.ProcessMonitor.handleJoinPoint(ProcessMonitor.java:404)
at com.rez1.op.core.performance.aspect.OperationalProgressMonitorAspect.operationalMonitor(OperationalProgressMonitorAspect.java:66)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:91)
at com.rez1.dws.mid.etl.engine.aspect.DWMasterETLExceptionMonitorAspect.monitorForExceptions(DWMasterETLExceptionMonitorAspect.java:44)
at sun.reflect.GeneratedMethodAccessor57.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
at com.rez1.dws.mid.etl.engine.master.service.DWMasterETLServiceImpl$$EnhancerByCGLIB$$ca0f48f4.processTalendDailyJobs(<generated>)
at com.rez1.apipublish.dws.facade.DataWarehouseFacadeImpl.talendDailyJob(DataWarehouseFacadeImpl.java:169)
at com.rez1.apipublish.dws.facade.DataWarehouseFacadeImpl$$FastClassByCGLIB$$fcc756cf.invoke(<generated>)
at net.sf.cglib.proxy.MethodProxy.invoke(MethodProxy.java:149)
at org.springframework.aop.framework.Cglib2AopProxy$CglibMethodInvocation.invokeJoinpoint(Cglib2AopProxy.java:688)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:91)
at com.rez1.common.processmonitor.ProcessMonitor.handleFirstInvocation(ProcessMonitor.java:369)
at com.rez1.common.processmonitor.ProcessMonitor.handleJoinPoint(ProcessMonitor.java:402)
at com.rez1.op.core.performance.aspect.OperationalProgressMonitorAspect.operationalMonitor(OperationalProgressMonitorAspect.java:66)
at sun.reflect.GeneratedMethodAccessor36.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:65)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.Cglib2AopProxy$DynamicAdvisedInterceptor.intercept(Cglib2AopProxy.java:621)
at com.rez1.apipublish.dws.facade.DataWarehouseFacadeImpl$$EnhancerByCGLIB$$521e5519.talendDailyJob(<generated>)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:309)
at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at org.springframework.remoting.support.RemoteInvocationTraceInterceptor.invoke(RemoteInvocationTraceInterceptor.java:77)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
at $Proxy75.talendDailyJob(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.springframework.remoting.support.RemoteInvocation.invoke(RemoteInvocation.java:205)
at org.springframework.remoting.support.DefaultRemoteInvocationExecutor.invoke(DefaultRemoteInvocationExecutor.java:38)
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invoke(RemoteInvocationBasedExporter.java:78)
at org.springframework.remoting.support.RemoteInvocationBasedExporter.invokeAndCreateResult(RemoteInvocationBasedExporter.java:114)
at org.springframework.remoting.httpinvoker.HttpInvokerServiceExporter.handleRequest(HttpInvokerServiceExporter.java:73)
at org.springframework.web.servlet.mvc.HttpRequestHandlerAdapter.handle(HttpRequestHandlerAdapter.java:49)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:790)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:719)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:644)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:560)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:298)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:852)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:588)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:636)
Caused by: java.net.SocketException: Broken pipe
at java.net.SocketOutputStream.socketWrite0(Native Method)
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:109)
at java.net.SocketOutputStream.write(SocketOutputStream.java:153)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3852)
Finally got to the bottom of the broken pipe error I encountered.
The broken pipe error was being thrown due to my not closing down certain database connections
from within the Talend jobs. These connections at issue were connections that were defined in native
java (via the tJava component). It was incorrectly assumed by me that by referencing a previously
defined connection (for connection attributes), the connection resources would be closed and
released.
With the above in mind, I was actually creating a new connection as listed below and had to
specifically commit the connection and then close it out. The item to take note of here is the use
of the commit on the connection. The broken pipe error was still being thrown with just the close
of the connection. It was not until the commit was added did the broken pipe errors cease and that
was due to the connection being closed properly.
Hindsight being what it is... when using the Talend components (and not native java code) for
the connection definition and closing of that connection, the autoCommit feature is available to the
tMysqlCOnnection as well as the commit option being available for the close connection option being
available on the tMysqlCommit component. These two options allow for the prevention of a broken
pipe error and is why I only encountered broken pipe errors with the connections used by the tJava
components. Makes sense now.
// code snippet from the tJava component using native java to read a table.
java.sql.Connection tableConnection = null;
tableConnection = (java.sql.Connection) globalMap.get("conn_tMysqlConnection_1");
Statement selectStatement = tableConnection.createStatement();
// Execute statement to get the row count from the table.
ResultSet resultSet = selectStatement.executeQuery("select count(*) from tableA");
// Position the result set to allow for access of its information.
resultSet.next();
// Get the row count from the result set.
context.numberOfRows = resultSet.getInt(1);
resultSet.close();
selectStatement.close();
tableConnection.commit();
tableConnection.close();
And... I would be remiss if I did not mention that I found out about needing to use the commit
by an article on the following link. http://mikeschubert.com/2006/08/03/javanetsocketex/
I have created a small game using Slick 2D and the LWJGL. When testing the program in Eclipse, everything works fine. As soon as I try to run my exported jar I get this message in the terminal:
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:58)
Caused by: java.lang.UnsatisfiedLinkError: no lwjgl in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1856)
at java.lang.Runtime.loadLibrary0(Runtime.java:845)
at java.lang.System.loadLibrary(System.java:1084)
at org.lwjgl.Sys$1.run(Sys.java:73)
at java.security.AccessController.doPrivileged(Native Method)
at org.lwjgl.Sys.doLoadLibrary(Sys.java:66)
at org.lwjgl.Sys.loadLibrary(Sys.java:95)
at org.lwjgl.Sys.<clinit>(Sys.java:112)
at org.lwjgl.opengl.Display.<clinit>(Display.java:132)
at org.newdawn.slick.AppGameContainer$1.run(AppGameContainer.java:39)
at java.security.AccessController.doPrivileged(Native Method)
at org.newdawn.slick.AppGameContainer.<clinit>(AppGameContainer.java:36)
I am really not sure what to do. Any help?
Use JarSplice :)
You just show it where to find the native files, the LWJGL and Slick JARs, and the JAR you create with eclipse.
I'm not that new to Java. I've done plenty of stuff with it before. But I'm migrating from PHP to Java Servlet/JSP (which I am new to) and I can't figure out this error to save my life. I installed Tomcat6 using Ubuntu 9.04's Synaptic Package Manager and it works like a charm. But when I put struts in my WEB-INF/lib directory and try to make a struts app, I get the below in the log file.
I used the struts-blank.war to start the project, so the web.xml is the same as provided in struts 2.1.8.1.
It appears that there is some issue with reading the struts-default.xml file.
Thanks in advance
SEVERE: Exception starting filter struts2
Unable to load configuration. - Class: java.security.AccessControlContext
File: AccessControlContext.java
Method: checkPermission
Line: 342 - java/security/AccessControlContext.java:342:-1
at org.apache.struts2.dispatcher.Dispatcher.init(Dispatcher.java:431)
at org.apache.struts2.dispatcher.ng.InitOperations.initDispatcher(InitOperations.java:69)
at org.apache.struts2.dispatcher.ng.filter.StrutsPrepareAndExecuteFilter.init(StrutsPrepareAndExecuteFilter.java:51)
at org.apache.catalina.core.ApplicationFilterConfig.getFilter(ApplicationFilterConfig.java:275)
at org.apache.catalina.core.ApplicationFilterConfig.setFilterDef(ApplicationFilterConfig.java:397)
at org.apache.catalina.core.ApplicationFilterConfig.(ApplicationFilterConfig.java:108)
at org.apache.catalina.core.StandardContext.filterStart(StandardContext.java:3709)
at org.apache.catalina.core.StandardContext.start(StandardContext.java:4359)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:791)
at org.apache.catalina.core.ContainerBase.access$000(ContainerBase.java:123)
at org.apache.catalina.core.ContainerBase$PrivilegedAddChild.run(ContainerBase.java:145)
at java.security.AccessController.doPrivileged(Native Method)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:769)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:525)
at org.apache.catalina.startup.HostConfig.deployDirectory(HostConfig.java:926)
at org.apache.catalina.startup.HostConfig.deployDirectories(HostConfig.java:889)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:492)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1149)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:311)
at org.apache.catalina.util.LifecycleSupport.fireLifecycleEvent(LifecycleSupport.java:117)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1053)
at org.apache.catalina.core.StandardHost.start(StandardHost.java:719)
at org.apache.catalina.core.ContainerBase.start(ContainerBase.java:1045)
at org.apache.catalina.core.StandardEngine.start(StandardEngine.java:443)
at org.apache.catalina.core.StandardService.start(StandardService.java:516)
at org.apache.catalina.core.StandardServer.start(StandardServer.java:710)
at org.apache.catalina.startup.Catalina.start(Catalina.java:578)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.commons.daemon.support.DaemonLoader.start(DaemonLoader.java:177)
Caused by: Unable to load configuration. - Class: java.security.AccessControlContext
File: AccessControlContext.java
Method: checkPermission
Line: 342 - java/security/AccessControlContext.java:342:-1
at com.opensymphony.xwork2.config.ConfigurationManager.getConfiguration(ConfigurationManager.java:58)
at org.apache.struts2.dispatcher.Dispatcher.init_PreloadConfiguration(Dispatcher.java:374)
at org.apache.struts2.dispatcher.Dispatcher.init(Dispatcher.java:418)
... 36 more
Caused by: Caught exception while loading file struts-default.xml - Class: java.security.AccessControlContext
File: AccessControlContext.java
Method: checkPermission
Line: 342 - java/security/AccessControlContext.java:342:-1
at com.opensymphony.xwork2.config.providers.XmlConfigurationProvider.loadConfigurationFiles(XmlConfigurationProvider.java:902)
at com.opensymphony.xwork2.config.providers.XmlConfigurationProvider.loadDocuments(XmlConfigurationProvider.java:143)
at com.opensymphony.xwork2.config.providers.XmlConfigurationProvider.init(XmlConfigurationProvider.java:110)
at com.opensymphony.xwork2.config.impl.DefaultConfiguration.reloadContainer(DefaultConfiguration.java:168)
at com.opensymphony.xwork2.config.ConfigurationManager.getConfiguration(ConfigurationManager.java:55)
... 38 more
Caused by: java.security.AccessControlException: access denied (java.util.PropertyPermission xwork.saxParserFactory read)
at java.security.AccessControlContext.checkPermission(AccessControlContext.java:342)
at java.security.AccessController.checkPermission(AccessController.java:553)
at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
at java.lang.SecurityManager.checkPropertyAccess(SecurityManager.java:1302)
at java.lang.System.getProperty(System.java:669)
at com.opensymphony.xwork2.util.DomHelper.parse(DomHelper.java:78)
at com.opensymphony.xwork2.config.providers.XmlConfigurationProvider.loadConfigurationFiles(XmlConfigurationProvider.java:893)
... 42 more
I had a brain storm and figured this out myself. Although I don't think this is the most secure solution, but it works for my purposes and I'm not that concerned because it's a local dev server that only listens on the loopback interface (127.0.0.1).
Anyway, the solution is to add this to the bottom of /etc/tomcat6/policy.d/50local.policy
grant codeBase "file:${catalina.base}/webapps/-" {
permission java.security.AllPermission;
};