The following Groovy code can create a new CSV file (in this example testfile.csv) and write JSON data in the in CSV. I do not want to create a new CSV file but just want to add (Append) few more lines to the existing testfile.csv file without overwriting the file. May someone please help what to change in the following code to force it to append the file instead of writing a new one? I heard about StandardOpenOption.APPEND but no idea where to put that. Thanks
import groovy.json.JsonSlurper;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import com.oracle.e1.common.OrchestrationAttributes;
import java.text.SimpleDateFormat;
HashMap < String, Object > main(OrchestrationAttributes orchAttr, HashMap inputMap) {
HashMap < String, Object > returnMap = new HashMap < String, Object > ();
returnMap.put("CSVComplete", "false");
// Write the view number after jsonIn.fs_DATABROWSE_
def jsonIn = new JsonSlurper().parseText(inputMap.get("Vendor Data"));
def jsonData = jsonIn.fs_DATABROWSE_GettingJsonDataFromSomewhere.data.gridData.rowset;
if (jsonData.size() == 0) {
returnMap.put("CSVComplete", "empty");
return returnMap;
}
def fileName = orchAttr.getTempFileName("testfile.csv");
returnMap.put("CSVOutFileName", fileName);
//class writer to write file def
def sw = new StringWriter();
//build the CSV writer with a header
//def csv = new CSVPrinter(sw, CSVFormat.DEFAULT.withHeader("Business Unit", "Document Number", "LT", "SUB","Amount","HardcodedTHREAD","ApprovedBudget","fromview003"));
def csv = new CSVPrinter(sw, CSVFormat.DEFAULT); //No header
// create output file
fileCsvOut = new File(fileName);
def count=0;
// build the CSV
def an8Map = new ArrayList();
for (int i = 0; i < jsonData.size(); i++) {
def businessunit = jsonData[i].table1_column1;
if (an8Map.contains(businessunit)) {
continue;
}
an8Map.add(businessunit);
count++;
csv.printRecord(businessunit, jsonData[i].table_column,
jsonData[i].table1_column1, jsonData[i]. table1_column2, jsonData[i]. table1_column3, "Fixed text1 "Fixed text2", "Fixedtext3");
}
csv.close();
//writing csv to file
fileCsvOut.withWriter('UTF-8') {
writer ->
writer.write(sw.toString())
}
orchAttr.writeDebug(sw.toString());
returnMap.put("csv", sw.toString());
returnMap.put("CSVComplete", "true");
returnMap.put("CSVcount", Integer.toString(count));
return returnMap;
}
use withWriterAppend instead of withWriter
https://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/File.html#withWriterAppend(java.lang.String,%20groovy.lang.Closure)
Related
I am quite new to Groovy and I am parsing a JSON file with following code:
void getItemData()
{
def jsonSlurper = new JsonSlurper()
def reader = new BufferedReader(new InputStreamReader(new FileInputStream("data.json"),"UTF-8"));
data = jsonSlurper.parse(reader);
data.TESTS.each{println it.MEMBER_ID}
}
And I am getting printed properly the value of MEMBER_ID.
I want to Parameterize the above function like:
void getItemData(String item)
{
...
}
so that I can call this function with the desired item I want. For example: I want the MEMBER_ADDRESS. I want to call the function like:
getItemData("MEMBER_ADDRESS")
Now How I should change the statement:
data.TESTS.each{println it.MEMBER_ID}
I tried with
it.${item}
and some other ways not working.
Please teach me how to do it.
My JSON file is like below:
{
"TESTS":
[
{
"MEMBER_ID": "my name",
"MEMBER_ADDRESS": "foobarbaz",
}
]
}
Those here from google, looking for an answer to parse JSON File.
void getItemData(String item) {
def jsonSlurper = new JsonSlurper()
def data = jsonSlurper.parseText(new File("data.json").text)
println data.TESTS.each{ println it["$item"] }
}
getItemData("MEMBER_ADDRESS")
You need to add quotes around ${item} like:
import groovy.json.*
void getItemData(String item) {
def jsonSlurper = new JsonSlurper()
def reader = new BufferedReader(new InputStreamReader(new FileInputStream("/tmp/json"),"UTF-8"))
data = jsonSlurper.parse(reader)
data.TESTS.each { println it."$item" }
}
getItemData("MEMBER_ADDRESS")
I am trying to build a REST service which accepts XML and convert it into JSON and call external Service which accepts JSON and put my JSON into it. I am able to put the json without pretty but I want to PUT the json in pretty format. Please suggest how to do, below is my code ...
package com.mypackge
import grails.converters.JSON
import grails.rest.RestfulController
import grails.plugins.rest.client.RestBuilder
class RestCustomerController extends RestfulController {
/*
static responseFormats = ['json', 'xml']
RestCustomerController() {
super(Customer)
}
*/
def index() {
convertXmlToJson()
}
def myJson = ''
def convertXmlToJson() {
def xml = ''' <Customer>
<customerid>9999999999999</customerid>
<ssn>8888</ssn>
<taxid>8888</taxid>
<address>
<addressline1>Yamber Ln</addressline1>
<addressline1>8664 SE</addressline1>
<city>CCCCC</city>
<state>CC</state>
<zipcode>97679</zipcode>
</address>
<firstname>Scott</firstname>
<middlename></middlename>
<lastname>David</lastname>
<account>
<accountno>576-294738943</accountno>
<accounttype>Lease</accounttype>
<accountsubtype></accountsubtype>
<accountstatus>complete</accountstatus>
<firstname>Scott</firstname>
<middlename></middlename>
<lastname>David</lastname>
<businessname></businessname>
<billingsystem>yoiuhn</billingsystem>
<brand></brand>
<plantype></plantype>
<billingaddress>
<addressline1>Yamber Ln</addressline1>
<addressline1>8664 SE </addressline1>
<city>CCCCC</city>
<state>CC</state>
<zipcode>97679</zipcode>
</billingaddress>
<job>
<jobid>8276437463728</jobid>
<jobstatus>SUCCESS</jobstatus>
</job>
</account>
</Customer>
'''.stripMargin()
// Parse it
def parsed = new XmlParser().parseText( xml )
def myId = parsed.customerid.text()
// Deal with each node:
def handle
handle = { node ->
if( node instanceof String ) {
node
}
else {
[ (node.name()): node.collect( handle ) ]
}
}
// Convert it to a Map containing a List of Maps
def jsonObject = [ (parsed.name()): parsed.collect { node ->
[ (node.name()): node.collect( handle ) ]
} ]
def json = new groovy.json.JsonBuilder(jsonObject) //.toPrettyString()
// Check it's what we expected
def mmyresp
try{
mmyresp = putRequest(myId,json)
}catch(Exception e) {
mmyresp = 'Please Validate JSON ....'
}
}
def putRequest(String id, JSON myJson) {
String url = "http://foo.com/customer/external/"+id
def rest = new RestBuilder()
def resp = rest.put(url){
contentType "application/json"
json{
myJson
}
}
return resp
}
}
The record is added in below format ...
{"Customer":[{"customerid":["9999999999999"]},{"ssn":["8888"]},
{"taxid":["8888"]},{"address":[{"addressline1":["Yamber Ln"]},
{"addressline1":["8664 SE"]},{"city":["CCCCC"]},{"state":["CC"]},{"zipcode":["97679"]}]},
{"firstname":["Scott"]},{"middlename":[]},{"lastname":["David"]},{"businessname":[]},
{"account":[{"accountno":["576-294738943"]},{"accounttype":["Lease"]},{"accountsubtype":[]},
{"accountstatus":["complete"]},{"firstname":["Scott"]},{"middlename":[]},{"lastname":["David"]},
{"businessname":[]},{"billingsystem":["yoiuhn"]},{"brand":[]},{"plantype":[]},
{"billingaddress":[{"addressline1":["Yamber Ln"]},{"addressline1":["8664 SE"]},
{"city":["CCCCC"]},{"state":["CC"]},{"zipcode":["97679"]}]},{"job":[{"jobid":["8276437463728"]},
,{"jobstatus":["SUCCESS"]}]}]}]}
But I want this to be inserted in pretty format. I tried .toPrettyString() but got casting exception when try to put as json. I am trying the REST services for the first time, not sure where I am doing wrong. Please suggest me on this.
You should set following field in you Config.groovy.
grails.converters.default.pretty.print = true
This will pretty print for both the xml and json.
you could optionally set it up for xml or json only like below:
For json:
grails.converters.json.pretty.print = true
For xml
grails.converters.xml.pretty.print = true
A sample of Config.groovy entry is:
environments {
development {
grails.converters.json.pretty.print = true
}
}
Hope it helps!!!
For Grails 4, try this:
def json = x as JSON
json.prettyPrint = true;
log.info(json.toString())
I'm trying to figure out if there's a way of accessing column headings in a CSV using OpenCSV in groovy? This is what I have:
#GrabConfig( systemClassLoader=true )
#Grab( 'mysql:mysql-connector-java:5.1.27' )
#Grab(group = 'net.sf.opencsv', module = 'opencsv', version = '2.3')
import groovy.sql.Sql
import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.CSVParser
def sql = Sql.newInstance("jdbc:mysql://localhost:3306/nid", "developer","whatever", "com.mysql.jdbc.Driver")
def notes = sql.dataSet("vdc_notifications")
def TEST_FILE_NAME = 'C:\\Users\\me\\Desktop\\test.csv'
List<String[]> rows = new CSVReader(new FileReader(new File(TEST_FILE_NAME)), CSVParser.DEFAULT_SEPARATOR, CSVParser.DEFAULT_ESCAPE_CHARACTER, CSVParser.DEFAULT_QUOTE_CHARACTER).readAll()
What i'm trying to get to is being able to do:
rows.each() { row -> println row.some_column_name }
usually the first row of a CSV file contains the names of the columns. so you can either use a simple parsing approach and take the first row for the header:
#Grab('net.sf.opencsv:opencsv:2.3')
import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.bean.CsvToBean;
import au.com.bytecode.opencsv.bean.HeaderColumnNameMappingStrategy;
def csv = """\
name,age
Charlie,23
Billy,64"""
// read the files and keep the first one as the header
def csvr = new CSVReader(new StringReader(csv))
def header
while ((line=csvr.readNext())) {
if (!header) {
header = line
} else {
// create a map from the header and the line
println([header,line].transpose().collectEntries())
}
}
Or you can use the CsvToBean and the HeaderColumnNameMappingStrategy to create beans:
class Person {
String name
Integer age
}
// use the mapper
def ctb = new CsvToBean<Person>()
def hcnms = new HeaderColumnNameMappingStrategy<Person>()
hcnms.type = Person
println ctb.parse(hcnms, new StringReader(csv))
Hi below is my code to extract particular metadata tags and write those tags to a json file. And i imported json.lib.jar and tika-app.jar into my build path.
File dir = new File("C:/pdffiles");
File listDir[] = dir.listFiles();
for (int i = 0; i < listDir.length; i++)
{
System.out.println("files"+listDir.length);
String file=listDir[i].toString();
File file1 = new File(file);
InputStream input = new FileInputStream(file1);
Metadata metadata = new Metadata();
BodyContentHandler handler = new BodyContentHandler(10*1024*1024);
AutoDetectParser parser = new AutoDetectParser();
parser.parse(input, handler, metadata);
Map<String, String> map = new HashMap<String, String>();
map.put("File name: ", listDir[i].getName());
map.put("Title: " , metadata.get("title"));
map.put("Author: " , metadata.get("Author"));
map.put("Content type: " , metadata.get("Content-Type"));
JSONObject json = new JSONObject();
json.accumulateAll(map);
FileWriter file2;
file2 = new FileWriter("C:\\test.json");
file2.write(json.toString());
file2.flush();
}
But it is writing only single file metadata to the json file. Is there any problem with my code, please suggest me.
may be you should use-
file2.write(json.toJSONString());
instead of this line -
file2.write(json.toString());
I have a csv file (details.csv) like
ID,NAME,ADDRESS
1,"{foo,bar}","{123,mainst,ny}"
2,"{abc,def}","{124,mainst,Va}"
3,"{pqr,xyz}","{125,mainst,IL}"
when I use (Note: I have other closure above this which reads all csv files from directory)
if(file.getName().equalsIgnoreCase("deatails.csv")) {
input = new FileInputStream(file)
reader = new BufferedReader(new InputStreamReader(input))
reader.eachLine{line-> def cols = line.split(",")
println cols.size() }
Instead of getting size 3 I am getting 6 with values
1
"{foo
bar}"
"{123
mainst
ny}"
spilt(",") is splitting data by comma(,) but I want my results as
1
"{foo,bar}"
"{123,mainst,ny}"
How can I fix this closure. Please help! Thanks
Writing a csv parser is a tricky business.
I would let someone else do the hard work, and use something like GroovyCsv
Here is how to parse it with GroovyCsv
// I'm using Grab instead of just adding the jar and its
// dependencies to the classpath
#Grab( 'com.xlson.groovycsv:groovycsv:1.0' )
import com.xlson.groovycsv.CsvParser
def csv = '''ID,NAME,ADDRESS
1,"{foo,bar}","{123,mainst,ny}"
2,"{abc,def}","{124,mainst,Va}"
3,"{pqr,xyz}","{125,mainst,IL}"'''
def csva = CsvParser.parseCsv( csv )
csva.each {
println it
}
Which prints:
ID: 1, NAME: {foo,bar}, ADDRESS: {123,mainst,ny}
ID: 2, NAME: {abc,def}, ADDRESS: {124,mainst,Va}
ID: 3, NAME: {pqr,xyz}, ADDRESS: {125,mainst,IL}
So, to get the NAME field of the second row, you could do:
def csvb = CsvParser.parseCsv( csv )
println csvb[ 1 ].NAME
Which prints
{abc,def}
Of course, if the CSV is a File, you can do:
def csvc = new File( 'path/to/csv' ).withReader {
CsvParser.parseCsv( it )
}
Then use it as above
There are two ways of doing.
One is using collect
def processCsvData(Map csvDataMap, File file)
{
InputStream inputFile = new FileInputStream(file);
String[] lines = inputFile.text.split('\n')
List<String[]> rows = lines.collect {it.split(',')}
// Add processing logic
}
Here problem is it is removing commas in between braces ({}) i.e "{foo,bar}" becomes "{foo bar}"
Another way of using java, and this works just fine
public class CSVParser {
/*
* This Pattern will match on either quoted text or text between commas, including
* whitespace, and accounting for beginning and end of line.
*/
private final Pattern csvPattern = Pattern.compile("\"([^\"]*)\"|(?<=,|^)([^,]*)(?:,|$)");
private ArrayList<String> allMatches = null;
private Matcher matcher = null;
private int size;
public CSVParser() {
allMatches = new ArrayList<String>();
matcher = null;
}
public String[] parse(String csvLine) {
matcher = csvPattern.matcher(csvLine);
allMatches.clear();
String match;
while (matcher.find()) {
match = matcher.group(1);
if (match!=null) {
allMatches.add(match);
}
else {
allMatches.add(matcher.group(2));
}
}
size = allMatches.size();
if (size > 0) {
return allMatches.toArray(new String[size]);
}
else {
return new String[0];
}
}
}
Hope this helps!