i got stuck on splitting the values in json. I want to split the values for the key: ServerIPList. Here's the JSON format:
{
ServerIPList : [
<PrivateIP> <HOSTNAME> <REGION> <AWS ACCOUNT>
{ 172.00.00.00 ,ip-172-00-00-00.ec2.internal ,us-east-1,123456789123 } ,
] ,
Operation : start,
SNowTicket : RITM00001
}
Here's my code:
import groovy.json.*
def message
pipeline {
agent any
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
}
stages {
stage('Recieve SQS Message') { steps {
script {
def aws = new AwsUtil()
def success = aws.receiveSqsMessages(env.AWS_SQS_QUEUE, 1, { json ->
message = new Util().jsonSlurper(json.Body)
println message
println message.ServerIPList
println message.Operation
result = sh (
script: "python ec2InstanceState.py ${message.ServerIPList} ${message.Operation}",
returnStdout: true
).trim()
output of my code:
Body:{
"ServerIPList" : "172.24.8.73","ip-172-00-00-00.ec2.internal","us-east-1","123456789123"
"Operation" : "start",
"SNowTicket" : "RITM1062357"
},
"172.24.8.73","ip-172-00-00-00.ec2.internal","us-east-1","123456789123"
"start"
"python ec2InstanceState.py "172.24.8.73","ip-172-00-00-00.ec2.internal","us-east-1","123456789123" start,
Now, here i would like to grep only PrivateIp from the key: "ServerIPList". How to get it:
python ec2InstanceState.py 172.24.8.73 start
Related
i am new to jenkins, i have a pipline job with parameters.
I want to create a JSON file and write my parameters there.
(and then let my jar file read that JSON file and run according to it)
how can i do this in groovy?
this is my jenkins file:
pipeline {
agent {
label "create_pass_criteria"
}
parameters {
string(name: 'IP', description: 'Please enter your ip')
password(name: 'PASSWORD',description: 'Please enter your mx password')
string(name: 'NAME', description: 'Please enter the name ')
}
tools {
maven 'maven-3.3.9'
}
options
{
buildDiscarder(logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '20'))
gitLabConnection('gitlab')
}
stages {
stage('Git Clone') {
steps {
updateGitlabCommitStatus name: 'Build', state: 'running'
checkout([
$class : 'GitSCM',
branches : [[name: '*/master']],
doGenerateSubmoduleConfigurations: false,
extensions : [],
submoduleCfg : [],
userRemoteConfigs : [[credentialsId: GIT_CRED, url: GIT_PATH]]
])
}
}
stage('Build') {
steps {
sh 'mvn install'
}
}
stage('run') {
steps {
sh 'java -jar /var/lib/jenkins/workspace/create_pass_criteria/target/create_pass_criteria-8.0.125-SNAPSHOT.jar'
}
}
}
post {
success {
updateGitlabCommitStatus name: 'Build', state: 'success'
emailext(
to: EMAIL_ADDR,
subject: "Success Pipeline: ${currentBuild.fullDisplayName}",
body: "Pipeline URL: ${env.BUILD_URL}",
mimeType: 'text/html'
)
}
failure {
updateGitlabCommitStatus name: 'Build', state: 'failed'
emailext(
to: EMAIL_ADDR,
subject: "Failed Pipeline: ${currentBuild.fullDisplayName}",
body: "Pipeline URL: ${env.BUILD_URL}",
mimeType: 'text/html'
)
}
}
} // pipeline
i don't know if it is correct but this is what i need to add to my Jenkins file?:
node{
//to create json declare a sequence of maps/arrays in groovy
//here is the data according to your sample
def data = [
attachments:[
[
mxIp : params.MX_IP,
mxPassword : params.MX_PASSWORD,
policyName : params.POLICY_NAME,
]
]
]
writeJSON(file: 'parameters.json', json: data)
}
if yes, at which part does it is has to be?
You could put this code in a script block like this:
stage('run') {
steps {
script {
def data = [
attachments:[
[
mxIp : params.MX_IP,
mxPassword : params.MX_PASSWORD,
policyName : params.POLICY_NAME,
]
]
]
writeJSON(file: 'parameters.json', json: data)
}
sh 'java -jar /var/lib/jenkins/workspace/create_pass_criteria/target/create_pass_criteria-8.0.125-SNAPSHOT.jar'
}
}
In complex pipelines I try to create clean code by adhering to the single level of abstraction principle. In this case I would extract the script and sh steps into a separate function, which could then be called from the pipeline section as a single step:
stage('run') {
steps {
createPassCriteria()
}
}
Define the function after the closing } of the pipeline section:
void createPassCriteria() {
def data = [
attachments:[
[
mxIp : params.MX_IP,
mxPassword : params.MX_PASSWORD,
policyName : params.POLICY_NAME,
]
]
]
writeJSON(file: 'parameters.json', json: data)
sh 'java -jar /var/lib/jenkins/workspace/create_pass_criteria/target/create_pass_criteria-8.0.125-SNAPSHOT.jar'
}
How do you capture a JSON object as a prettified string using a Jenkins declarative-syntax pipeline?
pipeline {
agent any
stages {
stage( "Set up" ) {
steps {
script {
hostname = "bld-machine"
reply_email = "jenkins#${hostname}.company.com"
actor_email = "user#company.com"
status_json = initStatusJson()
}
}
}
/** Try figure out the difference between "global" and "env." variables. */
stage( "Capture variables" ) {
steps {
script {
status_json.env["var"] = "${env.var}" as String
status_json.env["var2"] = "${var}" as String
}
}
}
}
post {
always {
script {
def pretty_json = writeJSON( returnText: true, json: status_json )
}
emailext( subject: "CI/CD | ${currentBuild.currentResult}",
from: "${reply_email}",
to: "${actor_email}",
mimeType: "text/plain",
body: "${pretty_json}" )
}
}
}
def initStatusJson() {
def json_obj = readJSON text: '{}'
json_obj.job = readJSON text: '{}'
json_obj.env = [:]
json_obj.job.name = "${JOB_BASE_NAME}" as String
json_obj.job.number = "${BUILD_ID}" as String
json_obj.job.server = "${JENKINS_URL}" as String
json_obj.job.visualization = "${JENKINS_URL}/blue/organizations/jenkins/${JOB_BASE_NAME}/detail/${JOB_BASE_NAME}/${BUILD_ID}/pipeline" as String
return json_obj
}
The def pretty_json =... statement in the above Jenkinsfile triggers the following error:
WARNING: Unknown parameter(s) found for class type WARNING: Unknown parameter(s) found for class type 'org.jenkinsci.plugins.pipeline.utility.steps.json.WriteJSONStep': returnText
[Pipeline] }
[Pipeline] // script
Error when executing always post condition:
java.lang.IllegalArgumentException: You have to provided a file for writeJSON.
at org.jenkinsci.plugins.pipeline.utility.steps.json.WriteJSONStepExecution.run(WriteJSONStepExecution.java:61)
at org.jenkinsci.plugins.pipeline.utility.steps.json.WriteJSONStepExecution.run(WriteJSONStepExecution.java:43)
at org.jenkinsci.plugins.workflow.steps.SynchronousNonBlockingStepExecution.lambda$start$0(SynchronousNonBlockingStepExecution.java:47)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
What I have tried:
The def pretty_json = writeJSON( returnText: true, json: status_json ) statement is inspired by these resources:
Jenkinsfile pipeline construct JSON object and write to file
https://www.jenkins.io/doc/pipeline/steps/pipeline-utility-steps/#writejson-write-json-to-a-file-in-the-workspace
I also tried def pretty_json = writeJSON returnText: true, json: status_json which resulted in an identical error.
status_json.toString() returns a valid, but non-prettified JSON string.
I tried def pretty_json = JsonOutput.toJson(status_json) based on Create JSON strings from Groovy variables in Jenkins Pipeline, and it generates this error:
Error when executing always post condition:
groovy.lang.MissingPropertyException: No such property: JsonOutput for class: groovy.lang.Binding
Tried def pretty_json = groovy.json.JsonOutput.prettyPrint(status_json) based on https://gist.github.com/osima/1161966, and it generated this error:
Error when executing always post condition:
groovy.lang.MissingMethodException: No signature of method: java.lang.Class.prettyPrint() is applicable for argument types: (net.sf.json.JSONObject)
Update: Attempted #daggett's solution as follows:
post {
always {
script {
def pretty_json = status_json.toString(2)
}
emailext( subject: "CI/CD | ${currentBuild.currentResult}",
from: "${reply_email}",
to: "${actor_email}",
mimeType: "text/plain",
body: "${pretty_json}" )
}
}
...and also tried some variations like pretty_json = ... (instead of def pretty_json = ...) and also moving the pretty_json assignment outside the script{...} scope...but none worked.
Inside the script{...} context, the .toString(2) generated this error:
Scripts not permitted to use method net.sf.json.JSON toString int.
Outside the script{...} context, it generated what I interpret to be a "syntax error":
WorkflowScript: 79: Expected a step # line 79, column 7.
pretty_json = status_json.toString(2)
According to last error message
groovy.lang.MissingMethodException:
No signature of method: java.lang.Class.prettyPrint()
is applicable for argument types: (net.sf.json.JSONObject)
You have net.sf.json.JSONObject in status_json variable.
that's really strange - seems you are getting status_json not in a standard way for jenkins
however according to documentation of this class
http://json-lib.sourceforge.net/apidocs/jdk15/net/sf/json/JSONObject.html#toString(int)
just do following to make pretty json:
def pretty_json = status_json.toString(2)
If you have Scripts not permitted to use method XYZ exception:
for security reasons a lot of non-standard methods are disabled in jenkins.
refer this answer to resolve this kind of issue: https://stackoverflow.com/a/39412951/1276664
and finally - almost every case from your question should work:
writeJSON( returnText: true, json: status_json ) :
update pipeline-utility-steps jenkins plugin to the latest version to support returnText parameter
the same as above
...
JsonOutput.toJson(status_json) : JsonOutput class located in groovy.json package. you could import this package at t
he beginning of the script import groovy.json or call it like this: groovy.json.JsonOutput.toJson(status_json). note that this method returns non-formatted json.
groovy.json.JsonOutput.prettyPrint(status_json) : check the documentation for JsonOutput.prettyPrint - it could be called for string and not for object. so this could work: groovy.json.JsonOutput.prettyPrint(status_json.toString()) but only in case when status_json.toString() returns a valid json and JsonOutput.prettyPrint allowed to be called in jenkins admin.
I just did a test and it gave results :
def pretty_json = writeJSON( returnText: true, json: status_json , pretty: 4)
Note : Ensure you have the plugin Pipeline Utility Steps installed. Or reinstall it again.
Below is the script example:
#!groovy
import hudson.model.Result
import groovy.json.*
pipeline
{
agent any
stages
{
stage ('Set up')
{
steps
{
script
{
hostname = "bld-machine"
reply_email = "jenkins#${hostname}.company.com"
actor_email = "user#company.com"
status_json = initStatusJson()
println (status_json)
}
}
}
stage ('Capture variables')
{
steps
{
script
{
// Added just for test
status_json.env["var"] = "Alt" as String
status_json.env["var2"] = "su" as String
println (status_json)
}
}
}
}
post {
always {
script {
def pretty_json = writeJSON( returnText: true, json: status_json , pretty: 4)
println (pretty_json)
emailext( subject: "CI/CD | ${currentBuild.currentResult}",
from: "${reply_email}",
to: "${actor_email}",
mimeType: "text/plain",
body: "${pretty_json}" )
}
}
}
}
def initStatusJson() {
def json_obj = readJSON text: '{}'
json_obj.job = readJSON text: '{}'
json_obj.env = [:]
json_obj.job.name = "${JOB_BASE_NAME}" as String
json_obj.job.number = "${BUILD_ID}" as String
json_obj.job.server = "${JENKINS_URL}" as String
json_obj.job.visualization = "${JENKINS_URL}/blue/organizations/jenkins/${JOB_BASE_NAME}/detail/${JOB_BASE_NAME}/${BUILD_ID}/pipeline" as String
return json_obj
}
Output log :
is there a shorthand way figuring out parameter type based on the json evaluated by readJSON groovy package. I am using resulting event_processor_parameters in a job such as
build job: "dvmt-event-processor-dev", wait: false, parameters: event_processor_parameters
I have this working but I would like to have more cleaner way.
props = readJSON text: env.hb_job_params
for ( param in props.get(application_server)) {
if (param.value.getClass() == Boolean){
event_processor_parameters.add([$class: 'BooleanParameterValue', name: param.key, value: param.value])
}
else if (param.value.getClass() == String){
event_processor_parameters.add([$class: 'StringParameterValue', name: param.key, value: param.value])
}
}
env.hb_job_params ==>
{
"server1": {
"ENV": "DEV",
"dev_xbar_host": "xbarserver1",
"platform_type" : "o2",
"dev_app_host" : "server1",
"VERSION" : "1.0.0.23",
force_build: false
}
}
as a variant:
for ( param in props.get(application_server)) {
def clazz = "${param.value.getClass().getSimpleName()}ParameterValue"
event_processor_parameters.add([$class: clazz, name: param.key, value: param.value])
}
Below is my groovy code where I am framing a command using json data, but json data have different types data like list in array, or only single variable so can any one please tell me how to find data type of a variable.
Below code I have high-lighted the one element "jsondata[key]" that is my values of keys, and I want to check the data type of this values like in my JSON data i have params(key) with 4 list of arrays(values) so before using params values i have check the data type.
my expected result:
key : [1 2 3]
if (typeof(key) == list) {
for (value in key) {
#here i want to frame a command using previous keys..
}
}
like typeof() in python
import groovy.json.JsonSlurper
def label = "test testname params"
File jsonFile = newFile("/home/developer/Desktop/kramdeni/vars/PARAMS.json")
def jsondata = new JsonSlurper().parse(jsonFile)
println jsondata.keySet()
println "jsondata: " + jsondata
def command = ""
keys = label.split(" ")
println "keys: " + keys
for (key in keys) {
command += "-" + key + " " + **jsondata[key]** + " "
}
println "command: " + command
my json data:
{
"test": "iTEST",
"testname": "BOV-VDSL-link-Rateprofile-CLI-Test-1",
"params": [
{
"n2x_variables/config_file": "C:/Program Files (x86)/Agilent/N2X/RouterTester900/UserData/config/7.30 EA SP1 Release/OSP Regression/BOV/Bov-data-1-single-rate-profile.xml"
},
{
"n2x_variables/port_list": "303/4 303/1"
},
{
"n2x_variables/port_list": "302/3 303/4"
},
{
"n2x_variables/port_list": "301/3 303/5"
}
]
}
jsondata.each{ entry->
println entry.value.getClass()
}
The following code illustrates instanceof for String and List types:
import groovy.json.JsonSlurper
def label = "test testname params"
def jsonFile = new File("PARAMS.json")
def jsondata = new JsonSlurper().parse(jsonFile)
def command = ""
def keys = label.split(" ")
for (key in keys) {
def value = jsondata[key]
if (value instanceof String) {
println "${key} ${value}"
} else if (value instanceof List) {
value.each { item ->
println "${key} contains ${item}"
}
} else {
println "WARN: unknown data type"
}
}
example output for the JSON specified (I'm not sure how you want to build the command, so this is simple output. It should be easy to build up command as desired):
$ groovy Example.groovy
test iTEST
testname BOV-VDSL-link-Rateprofile-CLI-Test-1
params contains [n2x_variables/config_file:C:/Program Files (x86)/Agilent/N2X/RouterTester900/UserData/config/7.30 EA SP1 Release/OSP Regression/BOV/Bov-data-1-single-rate-profile.xml]
params contains [n2x_variables/port_list:303/4 303/1]
params contains [n2x_variables/port_list:302/3 303/4]
params contains [n2x_variables/port_list:301/3 303/5]
I am facing a problem in building the json message, i need a json message to be in the below format:
{
success:true,
count:3,
data: [
{id:1, data: SUCCESS},
{id:2, data: FAILURE},
{id:3, data: Not Declared}
]
}
Im anot sure how to do this plz help
You can do it the following way:
class Data { String id; String data }
def yourdata = [
new Data("1","SUCCESS")
new Data("2","FAILURE")
new ...
]
def builder = new groovy.json.JsonBuilder()
def root = builder {
success true
count 3
data yourdata.collect { d ->
["id":d.id,
"data":d.data]
}
}
return builder.toPrettyString()