Topology not working due to Found multiple defaults.yaml resources when deploying to remote cluster - build.gradle

I have a single node remote cluster set up (all nimbus , supervisor , zookeeper) running on the same machine . I deployed my Topology (simple Exclamation Topology) to this remote cluster . While topology and jar got submitted successfully , nothing is happening in the cluster .
When I checked the supervisor logs , I could see this :
2015-10-14T21:24:26.340+0000 b.s.d.supervisor [INFO] 42dd0337-1182-45b0-9385-14570c7e0b09 still hasn't started
Worker log files are empty .
On debugging a little in supervisor logs , I could see
Launching worker with command: (Some java command) ..Firing this java command I can see this error :
Caused by: java.lang.RuntimeException: Found multiple defaults.yaml resources. You're probably bundling the Storm jars with your topology jar.
I debugged more on internet and other stuff , and modified my build.gradle file too but still the same error whenever I deploy my topology .
This is my gradle file
dependencies {
compile group: 'org.springframework.boot', name: 'spring-boot-starter-actuator', version: springBootVersion
compile group: 'org.quartz-scheduler', name: 'quartz', version: quartzVersion
compile group: 'clj-stacktrace' , name: 'clj-stacktrace',version: cljStackTrace
compile group: 'org.apache.storm' , name: 'storm-core',version: stormVersion
ext {
fatJarExclude = true
}
}
task uberjar(type: Jar) {
from files(sourceSets.main.output.classesDir)
from {configurations.compile.collect {zipTree(it)}} {
exclude "META-INF/*.SF"
exclude "META-INF/*.DSA"
exclude "META-INF/*.RSA"
exclude "META-INF/LICENSE"
}
manifest {
attributes 'Main-Class': 'storm.topology.ExclamationTopology'
}
}

The jar file must not contain defaults.yaml file. Thus, you need to exclude it via
exclude "defaults.yaml"
Actually I would recommend the exclude all Storm dependencies from your jar. They are not needed and increase the size of the fat jar unnecessarily.

When packaging your topology jar, don't include the Storm jars as Storm will put those on the classpath for you.
Reference
http://storm.apache.org/releases/1.0.2/Troubleshooting.html

There are two options:
1- Delete using winrar "defaults.yaml" in the .jar
2- adding in the pom.xml storm-core as provided.
In my pom.xml:
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-core</artifactId>
<version>0.9.1-incubating</version>
<scope>provided</scope>
</dependency>

Related

spring boot with gradle not starting up

I am trying to build sample hello application using gradle build in Spring tool suite facing below design time error.
I have grade version 4.10.2 installed on my machine.
Error Msg:
Description Resource Path Location Type
Could not run build action using Gradle distribution 'https://services.gradle.org/distributions/gradle-4.10.2-bin.zip'.
Build file 'C:\TFS\Study\Springboot\GradleExamples\workspace\Gradle_Hello\build.gradle' line: 14
Plugin [id: 'org.springframework.boot', version: '2.1.2.RELEASE'] was not found in any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (could not resolve plugin artifact 'org.springframework.boot:org.springframework.boot.gradle.plugin:2.1.2.RELEASE')
Searched in the following repositories:
Gradle Central Plugin Repository
Plugin [id: 'org.springframework.boot', version: '2.1.2.RELEASE'] was not found in any of the following sources:
- Gradle Core Plugins (plugin is not in 'org.gradle' namespace)
- Plugin Repositories (could not resolve plugin artifact 'org.springframework.boot:org.springframework.boot.gradle.plugin:2.1.2.RELEASE')
Searched in the following repositories:
Gradle Central Plugin Repository build.gradle /Gradle_Hello line 14 Gradle Error Marker
Next time, please include your build.gradle file.
I believe your problem is caused by the fact you are trying to apply the plugin:'org.springframework.boot' without telling your gradle script where to find that plugin. Your buildscript (build.gradle file) actually depends on the org.springframework.boot plugin, however since it can't find it (and doesn't even know what it is), you are getting this issue.
you can fix that issue by adding the following code at the top of your file:
buildscript {
repositories {
mavenCentral()
}
dependencies {
// You are telling gradle that this script (Not the project) depends on the
// following plugin
classpath "org.springframework.boot:spring-boot-gradle-plugin:2.1.2.RELEASE"
}
}
apply plugin: "org.springframework.boot"
// The rest of your build.gradle file
I had the same problem, so I changed to Gradle 6.5 and also add settings.gradle file in the same dir as your build.gradle with the following info.
// settings.gradle
pluginManagement {
plugins {
id "org.springframework.cloud.contract" version "${verifierVersion}"
}
repositories {
// to pick from local .m2
mavenLocal()
// for snapshots
maven { url "https://repo.spring.io/snapshot" }
// for milestones
maven { url "https://repo.spring.io/milestone" }
// for GA versions
gradlePluginPortal()
}
}
In addition make sure that you can connect to your repositories. In my case if was a internal nexus server and gradle wasn't able to connect to the repositories because of SSL certificate missing from JDK truststore. I added the nexus SSL certificate in to the JDK truststore and it resolved

3rd party Framework Library not loaded: 'Image not found'

I am upgrading a framework to latest version. Earlier integration (>2 years old) had framework directly copied in the project; now getting cocoapod (0.39.0) to get framework integrated with project using xcode (7.2.1) and objective-c project.
Upon run, it generates following error:
dyld: Library not loaded: #rpath/name.framework/name
Referenced from:
/Users/xyz_xyz/Library/Developer/CoreSimulator/Devices/xxxxxxx/data/Containers/Bundle/Application/xxxxxxx/appname.app/appname
Reason: image not found
"Pod" xcode-project has correct reference and framework is present in corresponding folder
Found that nameFramework isn't linked (added) in any of the build phases. I am new to using cocoapods and not sure what changes would be necessary in Xcode build settings to make transition from directly-embedded framework to cocoapods based integration.
how to get past "dyld: Library not loaded" error?
What phase should I use to reference name.Framework during build as it's not getting generated?
How to copy bundle resources from Pod to project? Dragging-n-drop Pods/name/Resources/name.bundle prompts "copy item if needed" dialog. <- I don't think I need to do this when using cocoapods.
[update] Integration using cocoapods works fine when a sample or new project is used. It's something in the current project settings that's causing the issue.
Podfile:
platform :ios, '8.0'
# use_framework for swift based pod integration. requires cocoapod 0.39.0
#use_frameworks!
pod 'GTMOAuth2'
pod 'Typhoon'
pod 'Alamofire'
# Issue with name
pod 'name', podspec:'https://customers.pspdfkit.com/cocoapods/.../latest.podspec'
target :ABC do
pod '...', '~>1'
end
target :XYZ do
pod '...', :path => 'submodules/...'
end
[Update]
- Upgraded to CocoaPods 1.0.1 & modified the Podfile to uncomment use_frameworks!, and make other changes that are required for 0.39.0 to 1.0.1 migration. Here is the updated Podfile.
platform :ios, '8.0'
# use_framework is required for dynamic links (swift) based pod integration.
use_frameworks!
target 'XYZ' do
pod 'GTMOAuth2'
pod 'Alamofire'
pod 'name', podspec:'https://customers.name.com/cocoapods/.../latest.podspec'
target :XYZ-A do
pod 'XYZ-iOS-SDK', :path => 'submodules/xyz-ios-sdk'
end
end
Fixed errors such as following by adding $(inherited) flag (where applicable)
[!] The XYZ-v2 [Release] target overrides the OTHER_LDFLAGS build setting defined in ...
Progress after above changes, Pods/Target Supported Files/XYZ-v2/ has Pods-XYZ-v2-frameworks.sh and resources.sh; earlier frameworks.sh was missing. Following is partial content of the framworks.sh, and it does contain copy instructions.
if [[ "$CONFIGURATION" == "Debug" ]]; then
install_framework "$BUILT_PRODUCTS_DIR/GTMOAuth2/GTMOAuth2.framework"
install_framework "$BUILT_PRODUCTS_DIR/GTMSessionFetcher/GTMSessionFetcher.framework"
install_framework "$BUILT_PRODUCTS_DIR/GoogleAPIClient/GoogleAPIClient.framework"
install_framework "$BUILT_PRODUCTS_DIR/Mantle/Mantle.framework"
install_framework "${PODS_ROOT}/PSPDFKit/PSPDFKit.framework"
install_framework "$BUILT_PRODUCTS_DIR/SSKeychain/SSKeychain.framework"
fi
// and for "Release" & "Distribution" as well..
Now I am trying to resolve compile errors upon build, which are related to static vs dynamic library includes.
/path../Pods/SSKeychain/Sources/SSKeychain.h:65:1: Duplicate interface definition for class ‘SSKeychain'
[Updated] Posted a new question: CocoaPods 1.0.1 Redefinition of 'XYZ', Redefinition of enumerator 'ABC', Duplicate interface definition for 'MNO'
Related:
OS X Framework Library not loaded: 'Image not found'
Seems relevant: https://github.com/CocoaPods/CocoaPods/issues/4772
Try using use_frameworks! (it's currently commented out). PSPDFKit is a dynamic framework, so you need to enable this option.
Also try it with the newest Xcode and CocoaPods >= 1.0.0. Older versions might not work correctly.
You can find more information about PSPDFKit integration via CocoaPods here: https://pspdfkit.com/guides/ios/current/getting-started/using-cocoapods
If all of this doesn't help you can reach the PSPDFKit developers directly at https://pspdfkit.com/support/request
same issue on dyld: Library not loaded: #rpath/TwilioAccessManager.framework/TwilioAccessManager
Reason: image not found
I had the same problem, this fixed it for me.I Changed Framework status required to optional .

artifactoryPublish task is looking for default zip file of distribution plugin

I upgraded my enviornment from gradle 1.12 to gradle 2.10 & java7 to java8.After upgrading my environment all of sudden one task of my build.gradle is not working
artifactoryPublish.I am using distrobution plugin to create a custom zip files.Below is the error I am getting
What went wrong:
14:24:05 Execution failed for task ':artifactoryPublish'.
14:24:05 > File 'F:\jenkins03\workspace\DataGenerator\build\distributions\DataGenerator-28_42_00_00.zip' does not exists, and need to be published!
I am not sure why it is trying to look for default zip files
Include the distributions block in the gradle script where you have included the distribution plugin. Below is the sample.
distributions {
main {
baseName = archivesBaseName
contents {
...
...
}
}
}

Missing Java3D on tomcat cartridge

I have small gear with tomcat cartridge. When I try to execute war that that generate images with Java3D I get following exception:
Caused by: java.lang.ClassNotFoundException: javax.media.j3d.Node
As a first think I tried to add Java3D at classpath, I have added to my pom.xml:
<dependency>
<groupId>java3d</groupId>
<artifactId>j3d-core-utils</artifactId>
<version>1.3.1</version>
<scope>compile</scope>
</dependency>
This added to final war following artifacts:
[INFO] +- java3d:j3d-core-utils:jar:1.3.1:compile
[INFO] | +- java3d:vecmath:jar:1.3.1:compile
[INFO] | \- java3d:j3d-core:jar:1.3.1:compile
When I deployd adjusted war following exception raised:
Caused by: java.lang.UnsatisfiedLinkError: no J3D in java.library.path
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1886)
As far as I understand exception, it says that there are no native Java3D libraries at java.library.path. So I installed Java3D from suggested link, I also updated j3dcore.jar, j3dutils.jar and vecmath.jar. Also catalina.sh was updated:
export LD_LIBRARY_PATH=/var/lib/openshift/<my-application-id>/app-root/data/j3d-1_5_2-linux-amd64/lib/amd64
I suppose that there is no X11 server to work with, because of that Java3D have to run in headless mode. It could be set in catalina.sh like this:
JAVA_OPTS=${JAVA_OPTS}" -Djava.awt.headless=true"
Now it seems that all java3D classes and *.so libraries are found. Now there is another problem:
java.awt.HeadlessException
at sun.java2d.HeadlessGraphicsEnvironment.getDefaultScreenDevice(HeadlessGraphicsEnvironment.java:64)
Problem is that Java3D class Canvas3D can't work in headless mode. Only way could be to connect to some X11 server with screen. It could be done with export DISPLAY=:0.0
As far as I was able to test, it seems that there is no X11 server, providing screen to which could Java3D connect. Because of that it's not possible to run Java3D at OpenShift platform with tomcat cartridge.
Thanks for your help.
Have you tried adding it to your pom.xml to get installed via maven? Or add the .jar file to your project manually... http://mvnrepository.com/artifact/java3d/j3d-core-utils/1.3.1
You might require more than just the core package.
Since you are deploying a war file, and not using maven, i think you would need to download the jar files and embed them in your war file as libraries and use them.
You might also check out this article: https://www.openshift.com/kb/kb-e1087-how-to-include-libraries-jar-files-in-your-java-application-without-using-maven
It looks like there is also a .so file that you would need to include with something like -Djava.library.path
Here is the file with the jars & .so file on java.net http://download.java.net/media/java3d/builds/release/1.5.2/j3d-1_5_2-linux-amd64.zip
Speaking with the dev ops team, it does not seem that package is installed on the servers.

How to configure UglifyJS with Gradle?

Are there any ready-to-use Gradle plugins to use for UglifyJs? We are trying to configure Uglify something similar to what has been done here, but the owner of that project seems to have his own private artifactory to which he points to, thereby getting access to UglifyAntTask, which is a github-hosted project not following Gradle/Maven etc. (basically non-managed) JAR. We tried downloading this JAR to our project and tried configuring using the options suggested in gradle page as follows:
dependencies {
compile fileTree(dir: 'libs', include: '*.jar') (or)
compile files('uglifyjs-java-v1.0.jar')
}
Note: The (or) is not there in actual code, I mentioned only to indicate that we tried both options but it was not picking the JAR.
So at a later step, when we gave
ant.taskdef(name: "uglify", classname: "uglify.ant.UglifyTask", classpath: configurations.uglifyjs.asPath)
Gradle throws the following errror:
taskdef class uglify.ant.UglifyTask cannot be found
using the classloader AntClassLoader[]
I am hoping that at least some one must have had the need to include non-managed 3rd party JAR and have figured out how to do this, if so, please point the solution/mistake we have made.
Thanks,
Paddy
Here is how the offical Gradle documentation describe it:
configurations {
uglifyjs
}
dependencies {
uglifyjs files('uglifyjs-java-v1.0.jar')
}
task uglifyjs << {
ant.taskdef(name: 'uglifyjs', classname: 'uglify.ant.UglifyTask', classpath: configurations.uglifyjs.asPath)
ant.uglifyjs( ... UglifyJS Ant Task parameters ... )
}
See http://www.gradle.org/docs/current/userguide/ant.html#N11416
HTH