Jenkins: import external package from Jenkinsfile using declarative syntax - json

I had a groovy code wich contains "import groovy.json.JsonSlurper".
I have spent a day testing and i dont know how to load external libraries using declarative syntax.
This is my code:
pipeline {
agent any
import groovy.json.JsonSlurper
stages {
stage("test") {
steps {
}
}
}
}
I have read the jenkins documentation, and i have tried to use the next but without success:
#Grab('groovy.json.JsonSlurper')
import groovy.json.JsonSlurper
both import and #Grab is not recognized. Some idea?
Thanks!

What #Daniel Majano says is true about the import syntax, but the #Grab syntax I found holds differences of behavior between a Pipeline script maintained directly in Jenkins vs Pipeline script from SCM.
When I placed a Grab command in the Pipeline script for a tester pipeline job I found that it didn't make any difference whether the Grab command was there or if it was commented out.
However when used from a Pipeline script from SCM it would throw the following exception...
java.lang.RuntimeException: No suitable ClassLoader found for grab
I removed it from the SCM script and everything worked out in the end.
Additional Background
I'm not sure why the grab was choking in the SCM version, but there's definitely some working parts to the groovy editor because if you define a partial grab command it will give you some validation errors pointing to the broken line as you see in the red X box below, with the error The missing attribute "module" is required in #Grab annotations:
Therefore the script validator is aware of the Grab annotation as it calls it and that it has both a group and module attribute. I'm using the so called shorthand notation in this example.

Related

Error while creating a custom producer in scala

I have written a small code for custom producer in Kafka using scala and it is giving the below error. I have attached the code in code section. I have attached some code for reference.
Name: Compile Error
Message: <console>:61: error: not found: type KafkaProducer
val producer = new KafkaProducer[String, String](props)
^
I think I need to import a relevant package. I tried importing the packages but could not get the correct one.
val producer = new KafkaProducer[String, String](props)
for( i <- 1 to 10) {
//producer.send(new ProducerRecord[String, String]("jin", "test",
"test"));
val record = new ProducerRecord("jin", "key", "the end ")
producer.send(record)
I can't install a scala kernel for jupyter right now, but based on this github you should add Kafka as a dependency, then the library might be recognized
%%configure -f
{
"conf": {
"spark.jars.packages": "org.apache.spark:spark-streaming_2.11:2.1.0,org.apache.bahir:spark-streaming-twitter_2.11:2.1.0,org.apache.spark:spark-streaming-kafka-0-8_2.10:2.1.0,com.google.code.gson:gson:2.4",
"spark.jars.excludes": "org.scala-lang:scala-reflect,org.apache.spark:spark-tags_2.11"
}
}
If this doesn't work, try downloading the whole notebook from the git, and fire it yourself, to see if something else is needed
#Arthur , Magic command %%configure -f did not work in jupyter notebook. I tried downloading the Whole notebook from the git but that also does not work. Luckily I was
reading the apache toree documentation for adding the dependencies and found a command %%addDeps. After putting dependencies in the below format into jupyter notebook,
I managed to run the code.
%AddDeps org.apache.kafka kafka-clients 1.0.0
%AddDeps org.apache.spark spark-core_2.11 2.3.0
Just for the information of others, when we compile the code using SBT, we need to comment this code from jupyter notebook as we will add these in build.sbt file.
Thanks Arthur for showing the direction !

JavaScript import statements running code

I'm trying to diagnose a problem thats recently arisen for us, after upgrading our suppported browser (~40 -> ~60)
We have this effective code in an external (now unsupported) library, that sits in an iffe:
(function(window, undefined){
# external library
if(!window.terribleIdea){
window.terribleIdea = {}
}
<some code>
if(!window.terribleIdea.config.url){
window.terribleIdea.config.url = 'the wrong url'
}
localStorage.set('somethingImportant', getStuff(window.terribleIdea.config.url))
})( window );
Now we did have a bootstap type file that looked like this:
# appBootstrapper.js
import applyConfig from './app/configApplier';
import ALL_ANGULAR_MODULES from './app'; # contains angular.module set up for
# app and every dependency
fetchConfig()
.then(applyConfig)
.then () => angular.bootstrap(document, [ALL_ANGULAR_MODULES])
.catch( error => {
alert(`It borked: ${error.message}`)
});
Amongst other things applyConfig does:
window.terribleIdea = {
config: {
url: 'not terrible'
}
}
What now happens, is that the import statement of ALL_ANGULAR_MODULES ends up running the code in the external library, and setting local storage. What we think used to happen is that it was only called on angular.bootstrap running.
Now what I need to find out is, has the functionality of the import statement changed in a later version of chrome, or has it always been running this code and gone unnoticed?
All I can find is references to Dynamic Imports, and the order of scripts running, in <script></script> tags.
It is hard to debug without access to the project (see discussion in comments above). Here are some possibilities worth exploring while encountering such issues. It is of course possible that it was like this all along.
Change in the bundler configuration. Webpack accepts entries as arrays, and order matters in them.
Change in the way the bundler/dependency manager reacts to dynamic imports
Change in the way your application loads its dependency during the its bootstrap phase. It is not (imo) angular specific, as many app use some kind of "componentization" which translates in files executed in a different order they are imported (as they may only export constructors or whatnot).

Grails package change for domain class caused DuplicateMappingException

While working through a tutorial to start learning Grails, I made a mistake and ran:
grails create-domain-class com.FooBar
instead of:
grails create-domain-class com.acme.FooBar
It was immediately obvious I had made an error so I tried the following:
Searched for a function that reverses the create-domain-class command, it seems there isn't one.
Searched for advice on the web and the consensus is that you can delete a domain class file, any associated views and tests, then to be safe run a text search for your class name in your project directory for any references you may have missed. I have done all this.
Then I ran the correct command to create com.acme.FooBar, which worked.
After this the app fails to run and reports the following error:
org.hibernate.DuplicateMappingException: duplicate import: FooBar refers to both com.acme.FooBar and com.FooBar (try using auto-import="false")
After adding the following code to com.acme.FooBar:
...
static mapping = {
autoImport false
}
...
The app now runs as expected.
However as an experienced Java developer who occasionally does refactor a package I would like to understand how to do that without causing a DuplicateMappingException or resorting to the "autoImport false" solution.
Thanks.
You shouldn't be doing
static mapping = {
autoImport false
}
As, by doing this you said that don't check for domain just by name and look up for package as well. Hence, once you do that you will have to use Fully qualified name of the class in your queries / hqls which may itch sometimes.
You should be removing the Domain completely i.e.
remove the Domain
remove the view folder creating by default with very same name and so do the controller
Now, do grails clean-all(Make it a thumb rule to use grails clean-all first for any issue unexpectedly occuring).
To be more accurate do remove target directory from your project and then do run grails run-app.
I had done very same thing many times and got it resolved by above steps.
Hope it helps.

Using JUnit in Jython - NameError for assertTrue

Environment Details
Mac OS X 10.9
Oracle JDK 1.7.0_55 64-bit
jython-standalone-2.5.3.jar
junit-4.11
What I have done so far
I have added the junit jar to /Library/Java/Extensions.
I invoked Jython as follows java -jar jython-standalone-2.5.3.jar
In the Jython interpreter, I imported the following import org.junit.Assert, and this import was successful.
Problem
When I tried to use assertTrue, I got a NameError in the interpreter. Why is this so?
I understand that assertTrue is a static method. Not sure what implication this has when I try to use it in Jython.
Additional Context
I am using XMLUnit in Jython. Was able to successfully import the Diff class from org.custommonkey.xmlunit in Jython. Also able to use the methods in this class, and call them on a Diff object. The result of this method call is what I am trying to pass to assertTrue, when it throws the error.
from org.custommonkey.xmlunit import Diff
import org.junit.Assert
xml1 = ...some XML string...
xml2 = ...some XML string...
myDiff = Diff(xml1, xml2)
assertTrue(myDiff.similar())
Hope this additional information is useful in identifying a solution to this problem.
Latest Status
I narrowed it down to setting this property python.security.respectJavaAccessibility = false, since the Assert() constructor is protected.
Still trying to get it to work. Any help is greatly appreciated.
Figured it out.
In addition to junit.jar file, the hamcrest-core.jar file also needed to be copied to /Library/Java/Extensions.
Then I got rid of the jython.jar file, and instead installed it using the jython installer.
After the installation was completed, I updated the registry file in the installation folder, specifically setting this property python.security.respectJavaAccessibility = false.
Now I am able to see the assertTrue method, and no longer getting a NameError.

MEF: "Unable to load one or more of the requested types. Retrieve the LoaderExceptions for more information"

Scenario: I am using Managed Extensibility Framework to load plugins (exports) at runtime based on an interface contract defined in a separate dll. In my Visual Studio solution, I have 3 different projects: The host application, a class library (defining the interface - "IPlugin") and another class library implementing the interface (the export - "MyPlugin.dll").
The host looks for exports in its own root directory, so during testing, I build the whole solution and copy Plugin.dll from the Plugin class library bin/release folder to the host's debug directory so that the host's DirectoryCatalog will find it and be able to add it to the CompositionContainer. Plugin.dll is not automatically copied after each rebuild, so I do that manually each time I've made changes to the contract/implementation.
However, a couple of times I've run the host application without having copied (an updated) Plugin.dll first, and it has thrown an exception during composition:
Unable to load one or more of the requested types. Retrieve the LoaderExceptions for more information
This is of course due to the fact that the Plugin.dll it's trying to import from implements a different version of IPlugin, where the property/method signatures don't match. Although it's easy to avoid this in a controlled and monitored environment, by simply avoiding (duh) obsolete IPlugin implementations in the plugin folder, I cannot rely on such assumptions in the production environment, where legacy plugins could be encountered.
The problem is that this exception effectively botches the whole Compose action and no exports are imported. I would have preferred that the mismatching IPlugin implementations are simply ignored, so that other exports in the catalog(s), implementing the correct version of IPlugin, are still imported.
Is there a way to accomplish this? I'm thinking either of several potential options:
There is a flag to set on the CompositionContainer ("ignore failing imports") prior to or when calling Compose
There is a similar flag to specify on the <ImportMany()> attribute
There is a way to "hook" on to the iteration process underlying Compose(), and be able to deal with each (failed) import individually
Using strong name signing to somehow only look for imports implementing the current version of IPlugin
Ideas?
I have also run into a similar problem.
If you are sure that you want to ignore such "bad" assemblies, then the solution is to call AssemblyCatalog.Parts.ToArray() right after creating each assembly catalog. This will trigger the ReflectionTypeLoadException which you mention. You then have a chance to catch the exception and ignore the bad assembly.
When you have created AssemblyCatalog objects for all the "good" assemblies, you can aggregate them in an AggregateCatalog and pass that to the CompositionContainer constructor.
This issue can be caused by several factors (any exceptions on the loaded assemblies), like the exception says, look at the ExceptionLoader to (hopefully) get some idea
Another problem/solution that I found, is when using DirectoryCatalog, if you don't specify the second parameter "searchPattern", MEF will load ALL the dlls in that folder (including third party), and start looking for export types, that can also cause this issue, a solution is to have a convention name on all the assemblies that export types, and specify that in the DirectoryCatalog constructor, I use *_Plugin.dll, that way MEF will only load assemblies that contain exported types
In my case MEF was loading a NHibernate dll and throwing some assembly version error on the LoaderException (this error can happen with any of the dlls in the directory), this approach solved the problem
Here is an example of above mentioned methods:
var di = new DirectoryInfo(Server.MapPath("../../bin/"));
if (!di.Exists) throw new Exception("Folder not exists: " + di.FullName);
var dlls = di.GetFileSystemInfos("*.dll");
AggregateCatalog agc = new AggregateCatalog();
foreach (var fi in dlls)
{
try
{
var ac = new AssemblyCatalog(Assembly.LoadFile(fi.FullName));
var parts = ac.Parts.ToArray(); // throws ReflectionTypeLoadException
agc.Catalogs.Add(ac);
}
catch (ReflectionTypeLoadException ex)
{
Elmah.ErrorSignal.FromCurrentContext().Raise(ex);
}
}
CompositionContainer cc = new CompositionContainer(agc);
_providers = cc.GetExports<IDataExchangeProvider>();