when I enter /usr/cygnus/bin/cygnus-flume-ng agent --conf /usr/cygnus/conf/ -f /usr/cygnus/conf/agent_1.conf -n cygnusagent -Dflume.root.logger=DEBUG,console I get the output:
2015-04-22 14:45:14,308 (lifecycleSupervisor-1-4) [ERROR - es.tid.fiware.fiwareconnectors.cygnus.interceptors.DestinationExtractor.initialize(DestinationExtractor.java:84)] Runtime error (File not found. Details=/usr/cygnus/conf/matching_table.conf (No such file or directory))
2015-04-22 14:45:14,309 (lifecycleSupervisor-1-4) [ERROR - org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:253)] Unable to start EventDrivenSourceRunner: { source:org.apache.flume.source.http.HTTPSource{name:http-source,state:IDLE} } - Exception follows.
java.lang.IllegalStateException: Running HTTP Server found in source: http-source before I started one.Will not attempt to start.
at com.google.common.base.Preconditions.checkState(Preconditions.java:145)
at org.apache.flume.source.http.HTTPSource.start(HTTPSource.java:137)
at org.apache.flume.source.EventDrivenSourceRunner.start(EventDrivenSourceRunner.java:44)
at org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:701)
Any solution to the missing matching_table.conf?
Good catch! This is a bug that will be fixed in next release. Cygnus installer forgets to rename the /usr/cygnus/conf/matching_table.conf.template as /usr/cygnus/conf/mathing_table.conf. The workaround is as simple as renaming that file by hand.
Related
I was running integrations tests with Pax-Exam and Karaf, tests got executed successfully but while shutting Karaf, it stuck on below and never resume
Pax-Exam = 4.11
Karaf = 4.2
[main] DEBUG o.ops4j.store.intern.TemporaryStore - Exit store(): 66cf6a516d0d1a670e78bd6b0be97f3da2a380b3
[main] DEBUG o.o.p.e.c.remote.RBCRemoteTarget - Preparing and Installing bundle (from stream )..
[main] DEBUG o.o.p.e.r.c.RemoteBundleContextClient - Packing probe into memory for true RMI. Hopefully things will fill in..
[main] DEBUG o.o.p.e.c.remote.RBCRemoteTarget - Installed bundle (from stream) as ID: 86
[main] DEBUG o.o.p.e.c.remote.RBCRemoteTarget - call [[TestAddress:PaxExam-bc970a6c-c656-4aa6-9300-35ded2bcde50 root:PaxExam-f6737e31-8f28-43e
0-847e-1f3f49649233]]
[main] DEBUG o.o.p.e.k.c.i.KarafTestContainer - Shutting down the test container (Pax Runner)
Following is output of JConsole for blocking
Name: main
State: BLOCKED on java.lang.Object#d53a0bb owned by: KarafJavaRunner
Total blocked: 106 Total waited: 105
Stack trace:
org.ops4j.pax.exam.karaf.container.internal.runner.InternalRunner.shutdown(InternalRunner.java:71)
org.ops4j.pax.exam.karaf.container.internal.runner.KarafJavaRunner.shutdown(KarafJavaRunner.java:120)
- locked org.ops4j.pax.exam.karaf.container.internal.runner.KarafJavaRunner#279baf5b
org.ops4j.pax.exam.karaf.container.internal.KarafTestContainer.stop(KarafTestContainer.java:600)
- locked org.ops4j.pax.exam.karaf.container.internal.KarafTestContainer#25dcfa62
org.ops4j.pax.exam.spi.reactors.AllConfinedStagedReactor.invoke(AllConfinedStagedReactor.java:87)
org.ops4j.pax.exam.junit.impl.ProbeRunner$2.evaluate(ProbeRunner.java:267)
org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
org.junit.runners.ParentRunner.run(ParentRunner.java:309)
org.ops4j.pax.exam.junit.impl.ProbeRunner.run(ProbeRunner.java:98)
org.ops4j.pax.exam.junit.PaxExam.run(PaxExam.java:93)
org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
Update:
One more thing i observed if i forcefully shut it and then if i run "mvn clean install" i get following error and i have to wait to get it run again
[←[1;31mERROR←[m] Failed to execute goal ←[32morg.apache.maven.plugins:maven-clean-plugin:2.5:clean←[m ←[1m(default-clean)←[m on project ←[36mosgi-unit-tes
ts-sample←[m: ←[1;31mFailed to clean project: Failed to delete C:\Users\..\target\pax
exam\e266ddcb-5fed-4997-8178-3d4944251418\system\org\apache\felix\org.apache.felix.framework\5.6.10\org.apache.felix.framework-5.6.10.jar←[m -> ←[1m[Help 1
Update2:
After exiting prompt still its running
C:\Program Files\Java\jdk1.8.0_162\bin>jps -l
1552 sun.tools.jps.Jps
4144
1420 org.apache.karaf.main.Main
C:\Program Files\Java\jdk1.8.0_162\bin>jps -l 1420
RMI Registry not available at 1420:1099
Exception creating connection to: 1420; nested exception is:
java.net.SocketException: Network is unreachable: connect
Update3:
if i kill this process, Pax resume and display message successful execution of Tests. infact before shutting all tests are already successfull but it not able to shut.
TASKKILL /F /PID 10692
Now i have no clue to handle this locking issue.
Update4:
Name: main
State: WAITING on org.apache.felix.framework.util.ThreadGate#b3d26d8
Total blocked: 6 Total waited: 7
Stack trace:
java.lang.Object.wait(Native Method)
org.apache.felix.framework.util.ThreadGate.await(ThreadGate.java:79)
org.apache.felix.framework.Felix.waitForStop(Felix.java:1075)
org.apache.karaf.main.Main.awaitShutdown(Main.java:640)
org.apache.karaf.main.Main.main(Main.java:188)
Name: FelixDispatchQueue
State: WAITING on java.util.ArrayList#3276dd18
Total blocked: 353 Total waited: 342
Stack trace:
java.lang.Object.wait(Native Method)
java.lang.Object.wait(Object.java:502)
org.apache.felix.framework.EventDispatcher.run(EventDispatcher.java:1122)
org.apache.felix.framework.EventDispatcher.access$000(EventDispatcher.java:54)
org.apache.felix.framework.EventDispatcher$1.run(EventDispatcher.java:102)
java.lang.Thread.run(Thread.java:748)
Update5:
After spending lot of time i finally realize that by adding below bundles it got stuck, if i dont add them it works fine
wrappedBundle( maven("org.ops4j.pax.tinybundles", "tinybundles").versionAsInProject() ), //2.1.0
wrappedBundle( maven("biz.aQute.bnd", "bndlib").versionAsInProject() )//2.4.0
Regards,
I resolved issue by changing following jars version
maven("org.ops4j.pax.tinybundles", "tinybundles") from 2.1.0 to 3.0.0
maven("biz.aQute.bnd", "bndlib") from 2.4.0 to 3.5.0
I'm having a hard time managing how I could load a JSON file from a Linux file system in a Spark environment. I am using Spark 1.6 by the way.
The file is located at /home/wymeka/fields.json and I am trying this command line :
spark-submit --master yarn transform.jar --schema-file "file:///home/wymeka/fields.json" --cache
The line from Main class in charge of loading this file is as follows :
val df_schema = sqlContext.read.json(pathToSchemaFile)
Using all this, lead me to the following exception :
Caused by: java.io.FileNotFoundException: File file:/home/wymeka/fields.json does not exist
at org.apache.hadoop.fs.RawLocalFileSystem.deprecatedGetFileStatus(RawLocalFileSystem.java:542)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileLinkStatusInternal(RawLocalFileSystem.java:755)
at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:532)
at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:425)
at org.apache.hadoop.fs.ChecksumFileSystem$ChecksumFSInputChecker.<init>(ChecksumFileSystem.java:140)
at org.apache.hadoop.fs.ChecksumFileSystem.open(ChecksumFileSystem.java:341)
at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:778)
at org.apache.hadoop.mapred.LineRecordReader.<init>(LineRecordReader.java:109)
at org.apache.hadoop.mapred.TextInputFormat.getRecordReader(TextInputFormat.java:67)
at org.apache.spark.rdd.HadoopRDD$$anon$1.<init>(HadoopRDD.scala:237)
at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:208)
at org.apache.spark.rdd.HadoopRDD.compute(HadoopRDD.scala:101)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.rdd.MapPartitionsRDD.compute(MapPartitionsRDD.scala:38)
at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:306)
at org.apache.spark.rdd.RDD.iterator(RDD.scala:270)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
at org.apache.spark.scheduler.Task.run(Task.scala:89)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Or when I try this command line :
spark-submit --master yarn transform.jar --schema-file "file:\/\/\/home\/imachraoui\/fields.json" --cache
I get another exception :
java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: file:%5C/%5C/%5C/home%5C/wymeka%5C/fields.json
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.<init>(Path.java:172)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$11.apply(ResolvedDataSource.scala:170)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$$anonfun$11.apply(ResolvedDataSource.scala:169)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:108)
at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
at scala.collection.mutable.ArrayOps$ofRef.flatMap(ArrayOps.scala:108)
at org.apache.spark.sql.execution.datasources.ResolvedDataSource$.apply(ResolvedDataSource.scala:169)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:119)
at org.apache.spark.sql.DataFrameReader.load(DataFrameReader.scala:109)
at org.apache.spark.sql.DataFrameReader.json(DataFrameReader.scala:244)
at com.nexys.spark.transform.Main$.main(Main.scala:80)
at com.nexys.spark.transform.Main.main(Main.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:731)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:181)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:206)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:121)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
Caused by: java.net.URISyntaxException: Relative path in absolute URI: file:%5C/%5C/%5C/home%5C/wymeka%5C/fields.json
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.<init>(URI.java:752)
at org.apache.hadoop.fs.Path.initialize(Path.java:203)
... 24 more
Any help would be very welcome.
Edited
I tried afterwards this command line :
spark-submit --files /home/wymeka/fields.json --master yarn transform.jar --schema-file "fields.json" --cache
and thus changed my spark code as below :
val df_schema = sqlContext.read.json(SparkFiles.getRootDirectory()+"/"+pathToSchemaFile)
But still nothing !
That file should be available on all worker nodes in the same path, otherwise it should be HDFS path
You are submitting an application to the YARN cluster(--master yarn parameter). Spark expects that the file you specified is available locally by the path: /home/wymeka/fields.json on the cluster.
To run the program locally you should change spark-submit parameter.
--master local[*]
or specify proper hdfs location if you want to deploy to the YARN cluster.
Launching Applications with spark-submit
I'm trying to deploy a WAR on CloudBees Glassfish4 server. I've followed the instructions at the bottom of http://developer.cloudbees.com/bin/view/RUN/Glassfish4 to include the jar in the META-INF/lib directory.
When I deploy with:
bees app:deploy target/app.war -a myDomain/app -t glassfish4-full
I get the error:
ERROR: Server.InternalError - java.lang.IllegalArgumentException: Platform error -
plugin_setup_error: glassfish4-full 1 [main] INFO com.cloudbees.clickstack.glassfish.Setup - Setup clickstack com.cloudbees.clickstack:glassfish-clickstack:4-full-1.0.2 - 2013-12-12T13:06:29.572+0100, current dir /mnt/genapp/apps/1cabb3f9/.
[main] INFO com.cloudbees.clickstack.glassfish.Setup - Setup: Environment{,
appUser='app_1cabb3f9',
appId='1cabb3f9',
appPort=8336,
appDir=/var/genapp/apps/1cabb3f9,
logDir=/var/genapp/apps/1cabb3f9/.genapp/log,
genappDir=/var/genapp/apps/1cabb3f9/.genapp,
controlDir=/var/genapp/apps/1cabb3f9/.genapp/control,
clickstackDir=/mnt/genapp-tmp/genapp-remote-plugin-1389871636905879,
packageDir=/mnt/genapp-tmp/stax-genapp-1389871636.236927/app,
}, com.cloudbees.clickstack.domain.metadata.Metadata#385cbbb1
Exception in thread "main" java.lang.Exception: Exception deploying on 10.159.35.35
at com.cloudbees.clickstack.glassfish.Setup.main(Setup.java:147)
Caused by: java.lang.IllegalArgumentException
at com.sun.nio.zipfs.ZipPath.relativize(ZipPath.java:238)
at com.cloudbees.clickstack.util.Files2$3.visitFile(Files2.java:188)
at com.cloudbees.clickstack.util.Files2$3.visitFile(Files2.java:184)
at java.nio.file.FileTreeWalker.walk(Unknown Source)
at java.nio.file.FileTreeWalker.walk(Unknown Source)
at java.nio.file.FileTreeWalker.walk(Unknown Source)
at java.nio.file.Files.walkFileTree(Unknown Source)
at java.nio.file.Files.walkFileTree(Unknown Source)
at com.cloudbees.clickstack.util.Files2.unzipSubDirectoryIfExists(Files2.java:184)
at com.cloudbees.clickstack.util.ApplicationUtils.extractContainerExtraLibs(ApplicationUtils.java:49)
at com.cloudbees.clickstack.glassfish.Setup.installApplication(Setup.java:259)
at com.cloudbees.clickstack.glassfish.Setup.setup(Setup.java:154)
at com.cloudbees.clickstack.glassfish.Setup.main(Setup.java:139)
I got a reply from CloudBees support.
The documentation at http://developer.cloudbees.com/bin/view/RUN/Glassfish4 was wrong, you don't need to include the mysql connector in your project.
As I replied to you on our support platform. We fixed the bug on both "glassfish4-full" and "glassfish4" (web profile) ClickStacks.
Sorry for the inconvenience,
Cyrille
Clickstacks release notes:
https://github.com/CloudBees-community/glassfish4-clickstack/releases/tag/v4-web-1.0.1
https://github.com/CloudBees-community/glassfish4-clickstack/releases/tag/v4-full-1.0.3
I observe the following stack trace when I try to start my jboss server (from both eclipse IDE and the standalone.sh from my unix terminal). Could anybody please tell why?:
JAVA_OPTS: -server -XX:+UseCompressedOops -XX:+TieredCompilation -Xms64m -Xmx512m -XX:MaxPermSize=256m -Djava.net.preferIPv4Stack=true -Dorg.jboss.resolver.warning=true -Dsun.rmi.dgc.client.gcInterval=3600000 -Dsun.rmi.dgc.server.gcInterval=3600000 -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true -Djboss.server.default.config=standalone.xml
=========================================================================
16:41:48,878 INFO [org.jboss.modules] JBoss Modules version 1.1.1.GA
16:41:49,025 INFO [org.jboss.msc] JBoss MSC version 1.0.2.GA
16:41:49,066 INFO [org.jboss.as] JBAS015899: JBoss AS 7.1.1.Final "Brontes" starting
16:41:49,289 ERROR [org.jboss.as.server] JBAS015956: Caught exception during boot: org.jboss.as.controller.persistence.ConfigurationPersistenceException: JBAS014676: Failed to parse configuration
at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:141) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.server.ServerService.boot(ServerService.java:266) [jboss-as-server-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractControllerService$1.run(AbstractControllerService.java:155) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_05-icedtea]
Caused by: com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog
at [row,col {unknown-source}]: [1,0]
at com.ctc.wstx.sr.StreamScanner.throwUnexpectedEOF(StreamScanner.java:677)
at com.ctc.wstx.sr.BasicStreamReader.handleEOF(BasicStreamReader.java:2104)
at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2010)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1102)
at com.ctc.wstx.sr.BasicStreamReader.nextTag(BasicStreamReader.java:1125)
at org.jboss.staxmapper.XMLMapperImpl.parseDocument(XMLMapperImpl.java:67) [staxmapper-1.1.0.Final.jar:1.1.0.Final]
at org.jboss.as.controller.persistence.XmlConfigurationPersister.load(XmlConfigurationPersister.java:133) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
... 3 more
16:41:49,295 FATAL [org.jboss.as.server] JBAS015957: Server boot has failed in an unrecoverable manner; exiting. See previous messages for details.
16:41:49,301 INFO [org.jboss.as] JBAS015950: JBoss AS 7.1.1.Final "Brontes" stopped in 1ms
This means that there is some problem with your Standalone.xml. Take a fresh one, it should work.
Here are the relevant lines
JBAS014676: Failed to parse configuration
Caused by: com.ctc.wstx.exc.WstxEOFException: Unexpected EOF in prolog
Looks like you're missing a configuration file (or more likely it's empty).
I just came across the same problem and found out that standalone.xml was empty.
I don't know what could have caused this though... U can find back up of ur standalone.xml in ${AS7_installation}/standalone/configuration/standalone_xml_history
I'm having a problem that I can't figure it out. I'm testing a SSL code that connects to a remote website, and everything works great on my computer. BTW, I imported the certificate from the website with:
keytool -importcert -keystore /java/path/to/cacerts -trustcacerts -alias “Cert Alias” -file website_certificate.cer
The problem appears when I upload and test the code on the server. I get the following exceptions:
Caused by: sun.security.validator.ValidatorException: PKIX path validation failed: java.security.cert.CertPathValidatorException: timestamp check failed
Caused by: java.security.cert.CertPathValidatorException: timestamp check failed
Caused by: java.security.cert.CertificateExpiredException: NotAfter:
I checked the date on the server and it is up to date, and the certificate expires on 2012.
Any help will be appreciated.