Remove a block from JSON payload using Groovy script - json

Please find an example payload, where I am trying to remove one block based on the updated date. I am looking to get the message block where the updated date is less than a week from the current date.
[
{
"firstName": "person1",
"lastName": "lname",
"createdDate": "2021-11-18T03:08:50.000Z",
"externalId": null,
"id": "123",
"updatedDate": "2022-03-18T19:33:28.792Z",
"title": "XXXX",
"email": "123#gmail.com",
"enabled": true
},
{
"firstName": "P2",
"lastName": "lname2",
"createdDate": "2021-10-26T19:12:16.000Z",
"externalId": null,
"id": "125",
"updatedDate": "2022-03-01T18:01:19.762Z",
"title": ".",
"email": "125#gmail.com",
"enabled": true
}
]
Using the following code to get the desired result. The remove statement seems to be not removing the collection from body1, or I am dong something wrong here. Tried adding the body1 to the message body, is not working. Tried using the builder, this is giving a completely different error. Any help would be highly appreciated.
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap; import groovy.json.*;
import groovy.json.JsonSlurper;
import groovy.json.JsonBuilder
def Message processData(Message message) {
//Body
def body = message.getBody(java.lang.String) as String;
def body1 = new JsonSlurper().parseText(body)
def jsonOp
def builder = new JsonBuilder()
def today = new Date()
def targetDate = today - 7
pastweek = (targetDate.format("yyyy-MM-dd\'T\'HH:mm:ss'Z'"))
body1.collect.eachWithIndex { it,index->
if ( it.updatedDate < pastweek )
{
it.remove(index)
}
else {
builder {
'firstName' it.firstName
'lastName' it.lastName
'email' it.email
'updatedDate' it.updatedDate
'enabled' it.enabled
}
}
}
def finalJson = builder.toPrettyString()
message.setBody(finalJson.toString())
return message;
}

import groovy.json.*
def Message processData(Message message) {
def body = message.getBody(java.lang.String) as String
def data = new JsonSlurper().parseText(body)
def pastweek = (new Date()-7).format("yyyy-MM-dd'T'HH:mm:ss.SSS'Z'")
data.removeAll{it.updatedDate < pastweek}
message.setBody(new JsonBuilder(data).toPrettyString())
return message
}

Related

Preparing JSON array of Objects from multiple array list

I am very new to the Groovy scripts and would like to build a JSON output from the below JSON input. Kindly help!
My JSON input looks like this:
{
"id":"1222",
"storageNode": {
"uuid": "22255566336",
"properties": {
"BuinessUnit": [
"Light",
"Fan",
"Watch"
],
"Contact": [
"abc#gmail.com",
"fhh#gmail.com"
],
"Location": [
"Banglore",
"Surat",
"Pune"
]
}
}
}
Expected Output:
[
{
"BuinessUnit": "Light",
"Contact": "abc#gmail.com",
"Location": "Banglore"
},
{
"BuinessUnit": "Fan",
"Contact": "fhh#gmail.com",
"Location": "Surat"
},
{
"BuinessUnit": "Watch",
"Contact": "",
"Location": "Pune"
}
]
Please note that in case any array is not matching the value count that will always be the last one and in that case, a blank value ("") has to be populated. The "BusinessUnit" object can be referred for array size validation.
My code looks like this:
import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;
import groovy.json.*;
def Message processData(Message message) {
//Body
def body = message.getBody(String.class);
def jsonSlurper = new JsonSlurper()
def list = jsonSlurper.parseText(body)
String temp
def BU = list.storageNode.properties.get("BusinessUnit")
def builder = new JsonBuilder(
BU.collect {
[
BusinessUnit: it
]
}
)
message.setBody(builder.toPrettyString())
return message
}
It is only returning this:
[
{
"BusinessUnit": "Light"
},
{
"BusinessUnit": "Fan"
},
{
"BusinessUnit": "Watch"
}
]
Now how will I add other parts to it? Please help!
I have come up with the following solution that converts source JSON string to the target JSON string:
import groovy.json.JsonBuilder
import groovy.json.JsonSlurper
def json = '''
{
"id":"1222",
"storageNode": {
"uuid": "22255566336",
"properties": {
"BusinessUnit": [
"Light",
"Fan",
"Watch"
],
"Contact": [
"abc#gmail.com",
"fhh#gmail.com"
],
"Location": [
"Banglore",
"Surat",
"Pune"
]
}
}
}
'''
println convert(json)
String convert(String json) {
def list = new JsonSlurper().parseText(json)
List<String> units = list.storageNode.properties.BusinessUnit
List<String> contacts = list.storageNode.properties.Contact
List<String> locations = list.storageNode.properties.Location
def result = []
units.eachWithIndex { unit, int index ->
result << [
BusinessUnit: unit,
Contact : contacts.size() > index ? contacts[index] : '',
Location : locations.size() > index ? locations[index] : '',
]
}
return new JsonBuilder(result).toPrettyString()
}
I've omitted the logic of getting string from the message and packaging transformed JSON into message.
I hope it will help you to move forward. Please let me know if you need further assistance here.
You can use the built-in Groovy facilities, like transpose():
import groovy.json.*
def json = new JsonSlurper().parseText '''{ "id":"1222", "storageNode": { "uuid": "22255566336", "properties": {
"BuinessUnit": [ "Light", "Fan", "Watch" ],
"Contact": [ "abc#gmail.com", "fhh#gmail.com" ],
"Location": [ "Banglore", "Surat", "Pune" ] } } }'''
def names = json.storageNode.properties*.key
def values = json.storageNode.properties*.value
int maxSize = values*.size().max()
// pad lists with trainiling spaces
values.each{ v -> ( maxSize - v.size() ).times{ v << '' } }
def result = values.transpose().collect{ tuple -> [ names, tuple ].transpose().collectEntries{ it } }
assert result.toString() == '[[BuinessUnit:Light, Contact:abc#gmail.com, Location:Banglore], [BuinessUnit:Fan, Contact:fhh#gmail.com, Location:Surat], [BuinessUnit:Watch, Contact:, Location:Pune]]'
This piece of code can process everything under storageNode.properties.

Save result to a new csv file instead of println Groovy

How can I save result of groovy script to a new file? C:/temp/all1.csv. I want to parse json file to csv, script is working fine but I don't know how can I save result in a new file. Please help.
import groovy.json.*
import java.io.File
def json ='''
{
"expand": "schema,names",
"startAt": 0,
"maxResults": 50,
"total": 21,
"issues": [
{
"expand": "operations,versionedRepresentations",
"id": "217580",
"self": "issue/217580",
"key": "ART-4070",
"fields": {"summary": "#[ART] Pre.3 Verification \\"S\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217579",
"self": "issue/217579",
"key": "ART-4069",
"fields": {"summary": "Verification \\"C\\""}
},
{
"expand": "operations,versionedRepresentations",
"id": "217577",
"self": "issue/217577",
"key": "ART-4068",
"fields": {"summary": "#[ART] Enum type"}
}
]
}
'''
File csvFile = new File( 'C:/temp/all1.csv')
def jsonSlurper = new JsonSlurper()
def config = [ // header -> extractor
"key": { it.key },
"summary": { it.fields.summary }
]
def encode(e) { // help with nulls; quote the separator
(e ?: "").replaceAll(";", "\\;")
}
def csvLine(items) { // write items as "CSV"
println(items.collect{ encode it }.join(";"))
}
def obj = new JsonSlurper().parseText(json)
csvLine(config.keySet())
obj.issues.each{ issue ->
csvLine(config.values().collect{ f -> f issue })
}
result:
key;summary
ART-4070;#[ART] Pre.3 Verification "S"
ART-4069;Verification "C"
ART-4068;#[ART] Enum type
To go with the current code, you could use csvFile.append(...) instead of println inside your
csvLine function and depending on your amount of real data, this might
be a good compromise between performance and resource.
Or you can write the whole CSV at once. E.g.
// prepare whole table
def data = [config.keySet()]
data.addAll(
obj.issues.collect{ issue ->
config.values().collect{ f -> f issue }
}
)
// write table as csv
def csvFile = "/tmp/out.csv" as File
csvFile.text = data.collect{
it.collect{ encode it }.join(";")9
}.join("\n")

Can I combine Json.Builder with JsonOutput in groovy? CSV to JSON

I have to parse csv file (with some modifications) to json in groovy.When I'm trying to execute this code i have a problem with spliting some values.
Content of csv file:
TestKey;Finished;user;status
RWS.PT.001;2020-07-20T23:01:21+02:00;admin;PASS;
RWS.PT.002;2020-07-20T23:02:21+02:00;admin;PASS;
my code in groovy:
import groovy.json.*
def builder = new groovy.json.JsonBuilder()
def root = builder {
testExecutionKey 'DEMO-303'
info (
user: 'admin')
tests 'rows':'ghgh','uuuuu'
}
print JsonOutput.prettyPrint(JsonOutput.toJson(root))
def csvfile = new File('C:/temp/raportTest.csv').readLines()
def keys = csvfile[0].split(';')
def rows = csvfile[1..-1].collect { line ->
def i = 0, vals = line.split(';')
keys.inject([:]) { map, key -> map << ["$key": vals[i++]] }
}
print JsonOutput.prettyPrint(JsonOutput.toJson(rows))
my target file should looks like this:
{
"testExecutionKey": "DEMO-303",
"info" : {
"user" : "admin"
},
"tests" : [
{
"testKey" : "RWS.PT.001",
"finished" : "2020-07-20T23:01:21+02:00",
"status" : "PASS"
},
{
"testKey" : "RWS.PT.002",
"finished" : "2020-07-20T23:01:21+02:00",
"status" : "PASS"
}
]
}
Now I have:
{
"testExecutionKey": "DEMO-303",
"info": {
"user": "admin"
},
"tests": [
{
"rows": "ghgh"
},
"uuuuu"
]
}[
{
"TestKey": "RWS.PT.001",
"Finished": "2020-07-20T23:01:21+02:00",
"user": "admin",
"status": "PASS"
},
{
"TestKey": "RWS.PT.002",
"Finished": "2020-07-20T23:02:21+02:00",
"user": "admin",
"status": "PASS"
}
]
How can I input code from def rows (from JsonOutput) into JsonBuilder (instead of "rows": "ghgh").
Please help!
You over-engineered it a bit. You actually don't have to combine anything, just use the JsonBuilder and some Groovy magic:
import groovy.json.*
def kindaFile = '''
TestKey;Finished;user;status
RWS.PT.001;2020-07-20T23:01:21+02:00;admin;PASS;
RWS.PT.002;2020-07-20T23:02:21+02:00;admin;PASS;
'''.trim()
def keys
def testList = []
//parse CSV
kindaFile.splitEachLine( /;/ ){ parts ->
if( !keys )
keys = parts
else{
def test = [:]
parts.eachWithIndex{ val, ix -> test[ keys[ ix ] ] = val }
testList << test
}
}
def builder = new JsonBuilder()
def root = builder {
testExecutionKey 'DEMO-303'
info user: 'admin'
tests testList
}
println JsonOutput.prettyPrint(JsonOutput.toJson(root))
prints
{
"testExecutionKey": "DEMO-303",
"info": {
"user": "admin"
},
"tests": [
{
"TestKey": "RWS.PT.001",
"Finished": "2020-07-20T23:01:21+02:00",
"user": "admin",
"status": "PASS"
},
{
"TestKey": "RWS.PT.002",
"Finished": "2020-07-20T23:02:21+02:00",
"user": "admin",
"status": "PASS"
}
]
}

how to parse CSV to JSON from 2 CSV Files in Groovy

Please help with parse CSV to JSON from 2 CSV Files in groovy
For example :
CSV1:
testKey,status
Name001,PASS
Name002,PASS
Name003,FAIL
CSV2:
Kt,Pd
PT-01,Name001
PT-02,Name002
PT-03,Name003
PT-04,Name004
I want to input in "testlist" data from CSV2.val[1..-1],CSV1.val[1..-1]
Result should be like :
{
"testExecutionKey": "DEMO-303",
"info": {
"user": "admin"
},
"tests": [
{
"TestKey": "PT-01",
"status": "PASS"
},
{
"TestKey": "PT-02",
"status": "PASS"
},
{
"TestKey": "PT-03",
"status": "FAIL"
}
]
code without this modification (from only 1 csv):
import groovy.json.*
def kindaFile = '''
TestKey;Finished;user;status
Name001;PASS;
Name002;PASS;
'''.trim()
def keys
def testList = []
//parse CSV
kindaFile.splitEachLine( /;/ ){ parts ->
if( !keys )
keys = parts
else{
def test = [:]
parts.eachWithIndex{ val, ix -> test[ keys[ ix ] ] = val }
testList << test
}
}
def builder = new JsonBuilder()
def root = builder {
testExecutionKey 'DEMO-303'
info user: 'admin'
tests testList
}
println JsonOutput.prettyPrint(JsonOutput.toJson(root))
Your sample JSON doesn't match the CSV definition. It looks lile you're using fields [1..-1] from CSV 1, as you stated, but fields [0..-2] from CSV 2. As you only have 2 fields in each CSV that's the equivalent of csv1[1] and csv2[0]. The example below uses [0..-2]. Note that if you always have exactly two fields in your input files then the following code could be simplified a little. I've given a more generic solution that can cope with more fields.
Load both CSV files into lists
File csv1 = new File( 'one.csv')
File csv2 = new File( 'two.csv')
def lines1 = csv1.readLines()
def lines2 = csv2.readLines()
assert lines1.size() <= lines2.size()
Note the assert. That's there as I noticed you have 4 tests in CSV2 but only 3 in CSV1. To allow the code to work with your sample data, it iterates through through CSV1 and adds the matching data from CSV2.
Get the field names
fieldSep = /,[ ]*/
def fieldNames1 = lines1[0].split( fieldSep )
def fieldNames2 = lines1[0].split( fieldSep )
Build the testList collection
def testList = []
lines1[1..-1].eachWithIndex { csv1Line, lineNo ->
def mappedLine = [:]
def fieldsCsv1 = csv1Line.split( fieldSep )
fieldsCsv1[1..-1].eachWithIndex { value, fldNo ->
String name = fieldNames1[ fldNo + 1 ]
mappedLine[ name ] = value
}
def fieldsCsv2 = lines2[lineNo + 1].split( fieldSep )
fieldsCsv2[0..-2].eachWithIndex { value, fldNo ->
String name = fieldNames2[ fldNo ]
mappedLine[ name ] = value
}
testList << mappedLine
}
Parsing
You can now parse the list of maps with your existing code. I've made a change to the way the JSON string is displayed though.
def builder = new JsonBuilder()
def root = builder {
testExecutionKey 'DEMO-303'
info user: 'admin'
tests testList
}
println builder.toPrettyString()
JSON Output
Running the above code, using your CSV1 and CSV 2 data, gives the JSON that you desire.
for CSV1:
testKey,status
Name001,PASS
Name002,PASS
Name003,FAIL
and CSV2:
Kt,Pd
PT-01,Name007
PT-02,Name001
PT-03,Name003
PT-05,Name002
PT-06,Name004
PT-07,Name006
result is:
{
"testExecutionKey": "DEMO-303",
"info": {
"user": "admin"
},
"tests": [
{
"status": "PASS",
"testKey": "PT-01"
},
{
"status": "PASS",
"testKey": "PT-02"
},
{
"status": "FAIL",
"testKey": "PT-03"
}
]
}
but I need exactly the same values for testKey (testKey from CSV1=Kt from CSV2)
{
"testExecutionKey": "DEMO-303",
"info": {
"user": "admin"
},
"tests": [
{
"testKey": "PT-02",
"status": "PASS"
},
{
"testKey": "PT-05",
"status": "PASS"
},
{
"testKey": "PT-03",
"status": "FAIL"
}
]
}

Unexpected character (g) at position 0 when trying to parse json - HttpResponseDecorator

Whenever I try to execute below script I keep getting error, saying unexpected character (g).
Basically I want to be able to parse the json response and be able to get the upstream job name from it.
Script:
#Grapes([
#Grab(group='org.codehaus.groovy.modules.http-builder', module='http-builder', version='0.7.1'),
#Grab(group='commons-collections', module='commons-collections', version='3.2.1'),
#Grab(group='org.jsoup', module='jsoup', version='1.10.2'),
#Grab(group='org.json', module='json', version='20190722'),
#Grab(group='com.googlecode.json-simple', module='json-simple', version='1.1.1')
])
import static groovyx.net.http.ContentType.*
import groovyx.net.http.HttpResponseException
import groovyx.net.http.RESTClient
import groovy.json.JsonSlurper
import groovy.json.JsonOutput
import java.net.*
import java.util.*
import org.json.simple.*
import org.json.simple.parser.JSONParser;
def getRestClient(){
String jenkinsUrl="http://somedomainname:8080"
def restClient = new RESTClient(jenkinsUrl)
return restClient
}
def getJobsInfo(String jobname,RESTClient restClient){
def requrl= '/job/'+jobname+'/lastBuild/api/json/?pretty=true'
def response = restClient.get( path : requrl)
return response
}
def writeToPropertyFile(){
def jsonResponseObject = getJobsInfo("sampleJobName",getRestClient())
println "\n\n\n\n\n Json String :---- "+ jsonResponseObject.toString()
JSONParser jsonParser = new JSONParser();
JSONObject jsonObject = (JSONObject) jsonParser.parse(jsonResponseObject.toString());
JSONArray upstreamJobInfoArray = jsonObject.getJSONArray("causes");
for (int i = 0; i < upstreamJobInfoArray.length(); i++) {
JSONObject jobCauses = upstreamJobInfoArray.getJSONObject(i);
String upstreamProjectName = jobCauses.getString("upstreamProject");
println upstreamProjectName
}
}
writeToPropertyFile()
Error :
Json String :---- groovyx.net.http.HttpResponseDecorator#6b063470
Caught: Unexpected character (g) at position 0.
Unexpected character (g) at position 0.
at org.json.simple.parser.Yylex.yylex(Yylex.java:610)
at org.json.simple.parser.JSONParser.nextToken(JSONParser.java:269)
at org.json.simple.parser.JSONParser.parse(JSONParser.java:118)
at org.json.simple.parser.JSONParser.parse(JSONParser.java:81)
at org.json.simple.parser.JSONParser.parse(JSONParser.java:75)
at org.json.simple.parser.JSONParser$parse.call(Unknown Source)
at getUpstreamJob.writeToPropertyFile(getUpstreamJob.groovy:39)
at getUpstreamJob.run(getUpstreamJob.groovy:50)
EDIT 1 : START
JSON response that I am trying to parse :
{
"_class": "hudson.model.FreeStyleBuild",
"actions": [
{
"_class": "hudson.model.CauseAction",
"causes": [
{
"_class": "hudson.model.Cause$UpstreamCause",
"shortDescription": "Started by upstream project \"sampleJobName\" build number 712",
"upstreamBuild": 712,
"upstreamProject": "sampleJobName",
"upstreamUrl": "job/sampleJobName/"
},
{
"_class": "hudson.model.Cause$UserIdCause",
"shortDescription": "Started by user Malick, Asif",
"userId": "asifma00",
"userName": "Malick, Asif"
},
{
"_class": "com.sonyericsson.rebuild.RebuildCause",
"shortDescription": "Rebuilds build #300",
"upstreamBuild": 300,
"upstreamProject": "sampleJobName",
"upstreamUrl": "view/ABCProjectView/job/sampleJobName/"
}
]
},
{
"_class": "hudson.model.ParametersAction",
"parameters": [
{
"_class": "hudson.model.StringParameterValue",
"name": "SNAPSHOTNAME",
"value": "ABCDE_12121.2000-2121212121212"
},
{
"_class": "hudson.model.StringParameterValue",
"name": "BUILD_LABEL",
"value": "ABCDE_12121.2000"
}
]
},
{},
{},
{},
{},
{
"_class": "hudson.plugins.parameterizedtrigger.BuildInfoExporterAction"
},
{},
{},
{},
{}
],
"artifacts": [],
"building": false,
"description": null,
"displayName": "#301",
"duration": 1199238,
"estimatedDuration": 1194905,
"executor": null,
"fullDisplayName": "sampleJobName #301",
"id": "301",
"keepLog": false,
"number": 301,
"queueId": 189076,
"result": "SUCCESS",
"timestamp": 1583500786857,
"url": "http://somedomainname:8080/job/sampleJobName/301/",
"builtOn": "Server12345",
"changeSet": {
"_class": "hudson.scm.EmptyChangeLogSet",
"items": [],
"kind": null
},
"culprits": []
}
EDIT 1 : END
I have tried looking at several Stack Overflow issues, but still haven't been able to resolve it.
Please guide.
It's because you're not getting the actual JSON string, you're getting the toString() output on a groovyx.net.http.HttpResponseDecorator class.
You can see it printing it out here (it's easy to miss though... I missed it the first look 🙂):
Json String :---- groovyx.net.http.HttpResponseDecorator#6b063470
I think you need to call jsonResponseObject.data to get the parsed data, and it might even return you a Map parsed form the JSON (so you can get rid of all your JSONObject code). Test it by changing to:
def data = jsonResponseObject.data
println "\n\n\n\n\n Json ${data.getClass().name} :---- $data" jsonResponseObject.toString()
If it is a java.util.Map, you can simplify your second part from:
JSONParser jsonParser = new JSONParser();
JSONObject jsonObject = (JSONObject) jsonParser.parse(jsonResponseObject.toString());
JSONArray upstreamJobInfoArray = jsonObject.getJSONArray("causes");
for (int i = 0; i < upstreamJobInfoArray.length(); i++) {
JSONObject jobCauses = upstreamJobInfoArray.getJSONObject(i);
String upstreamProjectName = jobCauses.getString("upstreamProject");
println upstreamProjectName
}
To (using data from above):
data.actions.causes.upstreamProject.flatten().each { println it }