Spinnaker Jenkins Integration unable to fetch jobs from Jenkins - exception

We have completed all the steps as described in the hello-spinnaker example below.We have used the AWS spinnaker image to directly configure spinnaker in AWS.
www.spinnaker.io/docs/hello-spinnaker.
I am trying to create a sample pipeline as noted in the above example.But while I create trigger in the first step and select jenkins ,the jobs are not getting populated and am getting below error in browser.
GET http://localhost:8084/v2/builds/Jenkins/jobs 429 (Too Many Requests)
The actual issue looks like while retrofit is trying to map the response from jenkins getjobs into the JobList class its finding an attribute _class in jenkins response xml and which is not present in JobList groovy class.Below is how we tried finding the issue
1)Login to AWS Spinnaker instance
2)Gate service is exposed at port 8084.
curl http://localhost:8084/v2/builds/Jenkins/jobs.
{"failureCause":"retrofit.RetrofitError: 429 Too Many Requests","error":"Too Many Requests","message":"429 Too Many Requests","status":429,"url":"http://localhost:8088/jobs/Jenkins","timestamp":1462793944530}
3)Igor service is exposed at port 8088.
curl http://localhost:8088/jobs/Jenkins
{"fallbackException":"java.lang.UnsupportedOperationException: No fallback available.","failureType":"COMMAND_EXCEPTION","failureCause":"retrofit.converter.ConversionException: org.simpleframework.xml.core.AttributeException: Attribute '_class' does not have a match in class com.netflix.spinnaker.igor.jenkins.client.model.JobList at line 1","error":"Hystrix Failure","message":"jenkins-Jenkins-getJobs failed and no fallback available.","status":429,"timestamp":1462793896853}
When I check in the igor logs,there are few exceptions which are occuring during the getprojects by jenkins poll
Caused by: retrofit.converter.ConversionException: org.simpleframework.xml.core.AttributeException: Attribute '_class' does not have a match in class com.netflix.spinnaker.igor.jenkins.client.model.ProjectsList at line 2
at retrofit.converter.SimpleXMLConverter.fromBody(SimpleXMLConverter.java:38)
at retrofit.RestAdapter$RestHandler.invokeRequest(RestAdapter.java:367)
... 39 common frames omitted
Caused by: org.simpleframework.xml.core.AttributeException: Attribute '_class' does not have a match in class com.netflix.spinnaker.igor.jenkins.client.model.ProjectsList at line 2
4)Connect to jenkins and get the jobs as its being done in spinnaker code https://github.com/spinnaker/igor/blob/master/igor-web/src/main/groovy/com/netflix/spinnaker/igor/jenkins/client/JenkinsClient.groovy
resp = requests.get('http://jenkinserverip:8080/api/xml?tree=jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name,jobs[name]]]]]]]]]]',auth=('admin','password'))
print resp.text
<hudson _class='hudson.model.Hudson'><job _class='hudson.model.FreeStyleProject'><name>Hello Build</name></job><job _class='hudson.model.FreeStyleProject'><name>Hello Poll</name></job></hudson>
So as the jenkins response is having the _class attribute ,retrofit is throwing an error at this line http://grepcode.com/file/repo1.maven.org/maven2/com.squareup.retrofit/retrofit/1.9.0/retrofit/RestAdapter.java#383
I wanted to see how can we quickly fix this as it looks like some version in compatibility of jenkins.

I'm seeing a similar issue in spinnaker 1.8.5. I had to reformat the jenkins url from myjenkins.server.com:8080 to http://myjenkins.server.com/ and it corrected the issue.

this is a bug around the jenkins api in later version. I believe 2.2 is the last compatible version, we run 1.6 internally.

Related

Wazuh active response with VirusTotal is not working

Wanted to integrate with VirusTotal and Yara but it seems like active response doesn't work as expected by following the steps in the link below:
https://documentation.wazuh.com/current/user-manual/capabilities/active-response/ar-use-cases/removing-malware.html
After adding/downloading eicar.com in /root directory, and read ossec.log, I get the following output:
About VirusTotal
I just followed the documentation and it worked well for me in Wazuh Manager 4.3.4 and a Wazuh Agent of the same version.
I got those same messages in /var/ossec/logs/ossec.log of the Wazuh Agent, those appear when the files do not exist or the proper permissions are not assigned, those files were replaced already in 4.2 but still show up in the log, since you are trying to use the script from the documentation then do not worry about those messages.
If you check under /var/ossec/logs/active-responses.log do you get any error?
What version of Wazuh Manager and Wazuh Agent are you using?
About Yara
It shouldn't be related to VirusTotal and probably deserves a different post, there is an issue open here but seems it is working, probably this comment helps you troubleshooting that one
The Active Response module is managed from the Wazuh Manager in /var/ossec/etc/ossec.conf, from here you can enable the response you need to execute using an <active response> configuration block that will use a "command" as a response. For example, if you are going to enable "remove-threat" as an Active Response on any agent that triggers the VirusTotal rule, you should have a <command> block and also an <active-response> block for that particular case, the same goes for any other AR case you may want to use.
<command>
<name>remove-threat</name>
<executable>remove-threat.sh</executable>
<timeout_allowed>no</timeout_allowed>
</command>
<active-response>
<disabled>no</disabled>
<command>remove-threat</command>
<location>local</location>
<rules_id>87105</rules_id>
</active-response>
The Response (script) needs to be present on each agent under /var/ossec/active-response/bin/. If you are only using the "remove-threat" Active Response, you should only have a single <active-response> block on the Manager's configuration file. Each <active-response> block within the Manager's "ossec.conf" must have a matching <command> block that is basically the response (script) the module is going to use. Perhaps you can share with us this configuration file so we can take a look.
Also, the following output from the Manager will be useful to see if the integration with Virustotal is being activated:
cat /var/ossec/logs/ossec.log | grep wazuh-integratord
I hope this helps,
Let us know

Why my Soffid JSON REST Web Services Connector does not update an object in the target system?

I am trying to connect my Soffid 3 server with our custom web application named Schrift. I am using а JSON REST Web Services Connector for this purpose. I added REST Web service plugin and then configured an agent with JSON/XML/SOAP Rest webservice type.
Loading of objects is working fine. My REST connector connects to the web service successfully and gets data of the accounts.
The problem is when I am trying to update some data (for example, I am trying to lock an account), nothing happens. And unfortunately I don't know what should be happening. When should REST connector send updated data to the managed system and in which way? I didn't find any log entries saying that REST connector was trying to update an object on managed system. Maybe I did smth wrong or missed something.
I would appreciate for any help. I can post any conf or log details if you need.
Update#1
(I did some investigation after the first answer)
I checked the agent settings: Read only and Manual account creation are set to no
The account was set to unmanaged type, but I succeeded in changing its type to shared and then to single without getting an error. Now it is set to single
The task queue is empty.
Also I've checked that update method is present and update properties are set correctly. updateParams is not set (it means that all attributes should be sent to the managed system).
But when I change status of the account (from Enable to Disable), nothing happens.
In the console log I can see only these lines
14-Sep-2021 13:26:29.708 INFO [BPM-Scheduler:192.168.7.121:1] com.soffid.iam.bpm.job.JobExecutorThread.run No job to execute
When I manually run the task Analize impact for changes on Schrift, Execution log shows
Changes detected for accounts
=============================
NO CHANGE DETECTED
Changes detected for roles
=============================
NO CHANGE DETECTED
Update#2
After many attempts I made some progress. Now when I make some changes in the account, the task named UpdateAccount baklykov#irf.com.ua#Schrift appears, but runs with an error.
At first it was 415 Unsupported Media Type error as I wrote in comments, but now it looks a little different
Throws exception updating object : Extensible object [type = account]
EmployeeEmail: baklykov#irf.com.ua
IsLockedOut: true (log truncated) ...
caused by Unexpected response, Content-Type: null
Update#3
I found out that soffid's request for updating the object was in improper format (all the parameters were passed in the html request instead of putting them in json body)
After researching I found a method's property called Encoding and set it to application/json value.
Now the parameters are passed in json body (that's what I need), but now the problem is that soffid puts all the parameters in json body, including the key parameter by which the object for updating should be determined. My guess this is the reason why the object in the target system is still not updated.
In other words my application expects a request like this:
https://myapp.mysite.com/api/v1/Soffid/Employees?EmployeeEmail=baklykov%40irf.com.ua :
{"EmployeeLastName":"Baklykov","EmployeeFirstName":"Ivan"}
but Soffid sends this:
https://myapp.mysite.com/api/v1/Soffid/Employees:
{"EmployeeLastName":"Baklykov","EmployeeFirstName":"Ivan","EmployeeEmail":"baklykov#irf.com.ua"}
The system should have created a UpdateAccount task in the task queue. Please, verify:
The task engine is in automatic mode. In read-only or manual mode, no task will be created.
If you are updating an account, check the account is not set as unmanaged. In that case, no tasks is created.
Finally, verify the task queue has not held the task up.
Have you checked the engine mode? Look at Main Menu > Administration > Configure Soffid > Integration engine > Smart engine settings
It should be set to automatic.

Couchbase Java SDK times out with BUCKET_NOT_AVAILABLE

I am doing a lookup operation Couchbase Java SDK 3.0.9 which looks like this:
// Set up
bucket = cluster.bucket("my_bucket")
collection = bucket.defaultCollection()
// Look up operation
val specs = listOf(LookupInSpecStandard.get("hash"))
collection.lookupIn(id, specs)
The error I get is BUCKET_NOT_AVAILABLE. Here are is the full message:
com.couchbase.client.core.error.UnambiguousTimeoutException: SubdocGetRequest, Reason: TIMEOUT {"cancelled":true,"completed":true,"coreId":"0xdb7f8e4800000003","idempotent":true,"reason":"TIMEOUT","requestId":608806,"requestType":"SubdocGetRequest","retried":39,"retryReasons":["BUCKET_NOT_AVAILABLE"],"service":{"bucket":"export","collection":"_default","documentId":"export:main","opaque":"0xcfefb","scope":"_default","type":"kv"},"timeoutMs":15000,"timings":{"totalMicros":15008977}}
The strange part is that this code hasn't been touched for months and the lookup broke out of a sudden. The CB cluster is working fine. Its version is
Enterprise Edition 6.5.1 build 6299.
Do you have any ideas what might have gone wrong?
Note that in Couchbase Java SDK 3.x, the Cluster::bucket method returns instantly, and continues opening a bucket in the background. So the first operation you perform - a lookupIn here - needs to wait for that resource opening to complete before it can proceed. It looks like it took a little longer to access the Couchbase bucket than usual and you got a timeout.
I recommend using the Bucket::waitUntilReady method after opening a bucket, to block until the resource opening is complete:
bucket = cluster.bucket("my_bucket")
bucket.waitUntilReady(Duration.ofMinutes(1));
This problem can occur because of firewall. You need to allow these ports.
Client-to-node
Unencrypted: 8091-8097, 9140 [3], 11210
Encrypted: 11207, 18091-18095, 18096, 18097
You can check more from below
https://docs.couchbase.com/server/current/install/install-ports.html#_footnotedef_2

unable to deploy .war file to a local glassfish container using hudson

ERROR: Publisher hudson.plugins.deploy.DeployPublisher aborted due to exception
org.codehaus.cargo.util.CargoException: The call to deploy returned 1
at org.codehaus.cargo.container.glassfish.GlassFish2xInstalledLocalDeployer.doDeploy(GlassFish2xInstalledLocalDeployer.java:79)
at org.codehaus.cargo.container.glassfish.internal.AbstractGlassFishInstalledLocalDeployer.redeploy(AbstractGlassFishInstalledLocalDeployer.java:97)
at hudson.plugins.deploy.CargoContainerAdapter.deploy(CargoContainerAdapter.java:64)
at hudson.plugins.deploy.CargoContainerAdapter$1.invoke(CargoContainerAdapter.java:93)
at hudson.plugins.deploy.CargoContainerAdapter$1.invoke(CargoContainerAdapter.java:77)
at hudson.FilePath.act(FilePath.java:793)
at hudson.FilePath.act(FilePath.java:775)
at hudson.plugins.deploy.CargoContainerAdapter.redeploy(CargoContainerAdapter.java:77)
at hudson.plugins.deploy.DeployPublisher.perform(DeployPublisher.java:47)
at hudson.tasks.BuildStepMonitor$3.perform(BuildStepMonitor.java:51)
at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:735)
at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:713)
at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:689)
at hudson.model.Build$RunnerImpl.post2(Build.java:163)
at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:655)
at hudson.model.Run.run(Run.java:1458)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:82)
at hudson.model.Executor.run(Executor.java:137)
Finished: FAILURE
The call to deploy returned 1
Why did the call to deploy return 1?
Btw, glassfish log as such a moment just say:
[#|2015-02-27T12:16:32.676+0100|INFO|sun-appserver2.1|javax.enterprise.system.tools.admin|_ThreadID=21;_ThreadName=httpWorkerThread-4848-1;/var/tmp/s1astempCIserver-1206725054/*.ear;|ADM1006:Uploading the file to:[/var/tmp/s1astempCIserver-1206725054/*.ear]|#]
[#|2015-02-27T12:16:35.165+0100|INFO|sun-appserver2.1|javax.enterprise.system.tools.deployment|_ThreadID=23;_ThreadName=Thread-2440;|deployed with moduleid = *|#]
So eveything seems fine (note, the star/asterix * replaced the actual name here) but when looking into glassfish admin gui right after that => nothing changed so something did not work as mentioned in the log above but it seems no where logged what actually brokes up.

How do I get the Junit test results using Email-ext in Jenkins

What do I need to add to the default html_gmail.jelly script to have it show the classes that were tested including how many tests were ran within each class?
When a Jenkins job is complete you can drill down to the Junit Test Results in an address that looks like:
http://somecompany.jenkins.com/view/App_Automation/job/Application_Under_Test/129/testReport/com.AUT.testing.mobile/
The test results are generated by the build.xml so is it just a matter of pointing to that xml file?
The email-ext page shows a clean example but not the tokens that are used to achieve that: http://wiki.hudson-ci.org/download/attachments/3604514/html.jpg
Currently using the ${FAILED_TESTS} token generates a nice Tested; Failed; Skipped number, but nothing that points to which tests passed/failed/skipped. I would like to show the total number of tests including which tests were actually ran.
Thanks ahead of time
OK I figured out how to display the pass and failed methods by adding var=pass or var=fail to the token of those assignments.
First go to the Jelly script in the this path:
~/.hudson/plugins/email-ext/WEB-INF/classes/hudson/plugins/emailext/templates/automation.jelly
$DEFAULT_SUBJECT (${build.testResultAction?.failCount} ${build.testResultAction?.failureDiffString})
SETTING UP THE CONFIG IN JENKINS
DEFAULT SUBJECT:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS!
DEFAULT CONTENT:
$PROJECT_NAME - Build # $BUILD_NUMBER - $BUILD_STATUS:
Check console output at $BUILD_URL to view the results.
Changes:
${CHANGES}
Changes Since Last Success
${CHANGES_SINCE_LAST_SUCCESS}
Failed Tests:
${FAILED_TESTS}
Build Log:
${BUILD_LOG}
Total Amount of Tests:
${TEST_COUNTS, var}
Total = $TEST_COUNTS
Failed = ${TEST_COUNTS,var="fail"}
Total = $TEST_COUNTS
Passed = ${TEST_COUNTS,var="pass"}
Job Description:
${JOB_DESCRIPTION}
Place this in the email job
${JELLY_SCRIPT,template="html-with-health-and-console"}
Note the templates available are noted in the path ~/.hudson/plugins/email-ext/WEB-INF/classes/hudson/plugins/emailext/templates/automation.jelly or create your own.