When I tried to run a simple map reduce program talking to Cassandra, I get the following error. I am using Hadoop 2.2 and Cassandra 2.0.2. Can someone who solved this problem please respond back with the solution?
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected
at org.apache.cassandra.hadoop.AbstractColumnFamilyInputFormat.getSplits(AbstractColumnFamilyInputFormat.java:116)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:491)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:508)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:392)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1286)
Related
I have set the variable in ~/.bashrc
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/lcoal/Hbase/lib/hbase-client-1.2.4.jar
but when i compile the code
java -cp $HADOOP_CLASSPATH:/home/hadoopuser/Downloads/myjar.jar com.bigdata.uniquecoder.WordCountClass
It still gives me this error.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
at com.bigdata.uniquecoder.WordCountClass.main(WordCountClass.java:57)
Caused by:java.lang.ClassNotFoundException:org.apache.hadoop.hbase.HBaseConfiguration at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 1 more
NOTE: It works fine when i run it in eclipse but when running on top of hadoop gives this error.
Any help will be highly appreciated.
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/hadoop/hbase/HBaseConfiguration
The missing class will be present in hbase-common-x.y.z.jar
Update the $HADOOP_CLASSPATH with
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/Hbase/lib/hbase-common-1.2.4.jar
Or,
export HADOOP_CLASSPATH=$HADOOP_CLASSPATH:/usr/local/Hbase/lib/
this will load all the jars under $HBASE_HOME/lib
Make sure $HADOOP_CLASSPATH contains necessary HADOOP libraries. Else, use the below export command in ~/.bashrc
export HADOOP_CLASSPATH=$HADOOP_HOME/share/hadoop/common/:$HADOOP_HOME/share/hadoop/common/lib/:$HADOOP_HOME/share/hadoop/hdfs/:$HADOOP_HOME/share/hadoop/hdfs/lib/:$HADOOP_HOME/share/hadoop/yarn/:$HADOOP_HOME/share/hadoop/yarn/lib/:/usr/local/Hbase/lib/:$CLASSPATH
Trying to run a simple hdfs query failed with:
[ms#cosmosmaster-gi ~]$ hadoop fs -ls /user/ms/def_serv/def_servpath
Java HotSpot(TM) 64-Bit Server VM warning: Insufficient space for shared memory file:
/tmp/hsperfdata_ms/21066
Try using the -Djava.io.tmpdir= option to select an alternate temp location.
Exception in thread "main" java.lang.NoClassDefFoundError: ___/tmp/hsperfdata_ms/21078
Caused by: java.lang.ClassNotFoundException: ___.tmp.hsperfdata_ms.21078
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
Could not find the main class: ___/tmp/hsperfdata_ms/21078. Program will exit.
Any idea how to fix that or increase quota?
Thanks!
ms
Your quota has not been exceeded (see command below), but this was a problem with the cluster. It should be fixed now.
$ hadoop fs -dus /user/ms
hdfs://cosmosmaster-gi/user/ms 90731
I am using hadoop 1.0.3 and sqoop 1.4.6. I am trying to import a table from MySQL to hdfs. I am getting following error:
Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class org.apache.hadoop.mapreduce.JobContext, but interface was expected
at org.apache.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:65)
at com.cloudera.sqoop.config.ConfigurationHelper.getJobNumMaps(ConfigurationHelper.java:36)
at org.apache.sqoop.mapreduce.db.DataDrivenDBInputFormat.getSplits(DataDrivenDBInputFormat.java:125)
at org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:962)
at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:979)
at org.apache.hadoop.mapred.JobClient.access$600(JobClient.java:174)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:897)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:850)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:850)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:500)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:530)
at org.apache.sqoop.mapreduce.ImportJobBase.doSubmitJob(ImportJobBase.java:196)
at org.apache.sqoop.mapreduce.ImportJobBase.runJob(ImportJobBase.java:169)
at org.apache.sqoop.mapreduce.ImportJobBase.runImport(ImportJobBase.java:266)
at org.apache.sqoop.manager.SqlManager.importTable(SqlManager.java:673)
at org.apache.sqoop.manager.MySQLManager.importTable(MySQLManager.java:118)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:497)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
Any suggestion on this?
This is due to sqoop being compiled on hadoop 1.x,
you can download sqoop compiled on hadoop 2.x on the sqoop site,
goto link - releases / sqoop 1
should look something like this,
sqoop-1.4.5.bin__hadoop-1.0.0.tar.gz
sqoop-1.4.5.bin__hadoop-2.0.4-alpha.tar.gz
get the sqoop or hadoop2.x should solve the problem.
Issue could be due to versions. Also please give the command details you have used.
Issue can be due to sqoop/sqoop2. You can look here.
Or in general, due to different versions of sqoop and haddop. Look here
General compatibility for sqoop and haddop versions info here
I've been dealing with this error message for a while now and nobody has been able to help me...
I am trying to probe a java application using the standalone TPTP probekit agent with the following linux command:
java '-agentlib:JPIBootLoader=JPIAgent:server=standalone;ProbekitAgent:ext-pk-BCILibraryName=BCIEngProbe,ext-pk-probescript=ome/anis/qf-t/core/probe/testProbe.probescript' junit.textui.TestRunner quickfix.test.acceptance.AcceptanceTestSuite
My application is actually a junit testcase from the Quickfixj package (quickfixj.org)...
Running the above command, the probing starts and I am able to see the runtime traces. However, at some point during execution I am getting the runtime exception:
java.lang.NoClassDefFoundError: testProbe_probe$Probe_0
25-Jan-2012 10:25:11 AM quickfix.test.acceptance.ATServer run
SEVERE: error in AT server
java.lang.NoClassDefFoundError: testProbe_probe$Probe_0
at com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl.<init>(DocumentBuilderFactoryImpl.java)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at java.lang.Class.newInstance0(Class.java:355)
at java.lang.Class.newInstance(Class.java:308)
at javax.xml.parsers.FactoryFinder.newInstance(FactoryFinder.java:147)
at javax.xml.parsers.FactoryFinder.find(FactoryFinder.java:233)
at javax.xml.parsers.DocumentBuilderFactory.newInstance(DocumentBuilderFactory.java:123)
at quickfix.DataDictionary.load(DataDictionary.java:906)
at quickfix.DataDictionary.read(DataDictionary.java:893)
at quickfix.DataDictionary.<init>(DataDictionary.java:109)
at quickfix.DefaultSessionFactory.getDataDictionary(DefaultSessionFactory.java:325)
at quickfix.DefaultSessionFactory.createDataDictionary(DefaultSessionFactory.java:219)
at quickfix.DefaultSessionFactory.processFixtDataDictionaries(DefaultSessionFactory.java:258)
at quickfix.DefaultSessionFactory.create(DefaultSessionFactory.java:113)
at quickfix.mina.acceptor.AbstractSocketAcceptor.createSessions(AbstractSocketAcceptor.java:242)
at quickfix.mina.acceptor.AbstractSocketAcceptor.startAcceptingConnections(AbstractSocketAcceptor.java:99)
at quickfix.SocketAcceptor.initialize(SocketAcceptor.java:66)
at quickfix.SocketAcceptor.start(SocketAcceptor.java:59)
at quickfix.test.acceptance.ATServer.run(ATServer.java:193)
at java.lang.Thread.run(Thread.java:662)**
which states that my probe file is missing. But this file has been needed in the first place to print the traces!!???.....
Well, here are some facts:
I am able to probe other applications, including a simple junit test:
java '-agentlib:JPIBootLoader=JPIAgent:server=standalone;ProbekitAgent:ext-pk-BCILibraryName=BCIEngProbe,ext-pk-probescript=ome/anis/qf-t/core/probe/testProbe.probescript' junit.textui.TestRunner quickfix.test.acceptance.AnisJUnitTestExample
I am able to run the quickfix.test.acceptance.AcceptanceTestSuite using junit (no probing):
java junit.textui.TestRunner quickfix.test.acceptance.AcceptanceTestSuite
I have set the path to testProbe_probe$Probe_0 in CLASSPATH
I have called the System.getenv("CLASSPATH") from within my testcase and the CLASSPATH seems to be fine
My testcase deals with threads, sockets, etc
And my system configuration is:
-os: ubuntu 10.04-64bit (on VirtualBox!)
-java version "1.6.0_24"
-junit: junit-4.10.jar
-TPTP agent controller: agntctrl.linux_em64t-TPTP-4.7.2
Any ideas??
Thanks,
Anis
hibernate 3.6.8 final
c3p0 jar that came with hibernate 3.6.8 package -> c3p0-0.9.1.jar
1
15
40
0
5
2
The app seems to be working fine, however I get massive log calls with the following stacktrace:
org.apache.catalina.loader.WebappClassLoader loadClass
INFO: Illegal access: this web application instance has been stopped already. Could not load com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException. The eventual following stack trace is caused by an error thrown for debugging purposes as well as to attempt to terminate the thread which caused the illegal access, and has no functional impact.
java.lang.IllegalStateException
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1566)
at org.apache.catalina.loader.WebappClassLoader.loadClass(WebappClassLoader.java:1526)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1013)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:987)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:982)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:927)
at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2411)
at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2153)
at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:792)
at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
at sun.reflect.GeneratedConstructorAccessor38.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:381)
at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:305)
at com.mchange.v2.c3p0.DriverManagerDataSource.getConnection(DriverManagerDataSource.java:134)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:182)
at com.mchange.v2.c3p0.WrapperConnectionPoolDataSource.getPooledConnection(WrapperConnectionPoolDataSource.java:171)
at com.mchange.v2.c3p0.impl.C3P0PooledConnectionPool$1PooledConnectionResourcePoolManager.acquireResource(C3P0PooledConnectionPool.java:152)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquire(BasicResourcePool.java:1074)
at com.mchange.v2.resourcepool.BasicResourcePool.doAcquireAndDecrementPendingAcquiresWithinLockOnSuccess(BasicResourcePool.java:1061)
at com.mchange.v2.resourcepool.BasicResourcePool.access$800(BasicResourcePool.java:32)
at com.mchange.v2.resourcepool.BasicResourcePool$ScatteredAcquireTask.run(BasicResourcePool.java:1796)
at com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread.run(ThreadPoolAsynchronousRunner.java:620)
Any information on how to remove that INFO log would be very much helpful thanks!
UPDATE: Is this a critical error? Or can should I just ignore it?
After a search on the web regarding this issue, I found some similar issues reported by several people. All of them point to a common problem: Threads. Basically, if you start new threads in your application (either in your code or by using a third party tool like Quartz, you have to make sure that all of the threads are stopped appropriately when the application is undeployed from the server. Here are some quotes from the searches:
Mikolaj Rydzewski wrote:
It looks like after webapp's instance has been undeployed, background quartz thread wants to do something and then exception occurs.
Another (and better explanation) on jspwiki.org:
It is possible that this is caused by Tomcat unsuccessfully reloading the web application. The app is unloaded, but all threads don't get shut down properly. As a result, when the threads try to run, they get clobbered by the fact that Tomcat has shut down its classloader, and an error is logged.
So, in order to solve this issue you have to make sure all threads started by your application will be stopped at application undeployment (or redeployment, it's the same). You can do this by registering a ServletContextListener to your application server and stopping your threads inside contextDestroyed(ServletContextEvent) method.
If you are using log4j, change your logging settings to something like this (the word ERROR replace INFO) :
log4j.rootLogger=ERROR, file, stdout
log4j.logger.org.hibernate=ERROR
They are located in the the log4j.properties file in yr project.
OK I switched to boneCP, c3p0 does not really seem to work for java6!!!