I'm trying to figure out if there's a way of accessing column headings in a CSV using OpenCSV in groovy? This is what I have:
#GrabConfig( systemClassLoader=true )
#Grab( 'mysql:mysql-connector-java:5.1.27' )
#Grab(group = 'net.sf.opencsv', module = 'opencsv', version = '2.3')
import groovy.sql.Sql
import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.CSVParser
def sql = Sql.newInstance("jdbc:mysql://localhost:3306/nid", "developer","whatever", "com.mysql.jdbc.Driver")
def notes = sql.dataSet("vdc_notifications")
def TEST_FILE_NAME = 'C:\\Users\\me\\Desktop\\test.csv'
List<String[]> rows = new CSVReader(new FileReader(new File(TEST_FILE_NAME)), CSVParser.DEFAULT_SEPARATOR, CSVParser.DEFAULT_ESCAPE_CHARACTER, CSVParser.DEFAULT_QUOTE_CHARACTER).readAll()
What i'm trying to get to is being able to do:
rows.each() { row -> println row.some_column_name }
usually the first row of a CSV file contains the names of the columns. so you can either use a simple parsing approach and take the first row for the header:
#Grab('net.sf.opencsv:opencsv:2.3')
import au.com.bytecode.opencsv.CSVReader
import au.com.bytecode.opencsv.bean.CsvToBean;
import au.com.bytecode.opencsv.bean.HeaderColumnNameMappingStrategy;
def csv = """\
name,age
Charlie,23
Billy,64"""
// read the files and keep the first one as the header
def csvr = new CSVReader(new StringReader(csv))
def header
while ((line=csvr.readNext())) {
if (!header) {
header = line
} else {
// create a map from the header and the line
println([header,line].transpose().collectEntries())
}
}
Or you can use the CsvToBean and the HeaderColumnNameMappingStrategy to create beans:
class Person {
String name
Integer age
}
// use the mapper
def ctb = new CsvToBean<Person>()
def hcnms = new HeaderColumnNameMappingStrategy<Person>()
hcnms.type = Person
println ctb.parse(hcnms, new StringReader(csv))
Related
The following Groovy code can create a new CSV file (in this example testfile.csv) and write JSON data in the in CSV. I do not want to create a new CSV file but just want to add (Append) few more lines to the existing testfile.csv file without overwriting the file. May someone please help what to change in the following code to force it to append the file instead of writing a new one? I heard about StandardOpenOption.APPEND but no idea where to put that. Thanks
import groovy.json.JsonSlurper;
import org.apache.commons.csv.CSVFormat;
import org.apache.commons.csv.CSVPrinter;
import com.oracle.e1.common.OrchestrationAttributes;
import java.text.SimpleDateFormat;
HashMap < String, Object > main(OrchestrationAttributes orchAttr, HashMap inputMap) {
HashMap < String, Object > returnMap = new HashMap < String, Object > ();
returnMap.put("CSVComplete", "false");
// Write the view number after jsonIn.fs_DATABROWSE_
def jsonIn = new JsonSlurper().parseText(inputMap.get("Vendor Data"));
def jsonData = jsonIn.fs_DATABROWSE_GettingJsonDataFromSomewhere.data.gridData.rowset;
if (jsonData.size() == 0) {
returnMap.put("CSVComplete", "empty");
return returnMap;
}
def fileName = orchAttr.getTempFileName("testfile.csv");
returnMap.put("CSVOutFileName", fileName);
//class writer to write file def
def sw = new StringWriter();
//build the CSV writer with a header
//def csv = new CSVPrinter(sw, CSVFormat.DEFAULT.withHeader("Business Unit", "Document Number", "LT", "SUB","Amount","HardcodedTHREAD","ApprovedBudget","fromview003"));
def csv = new CSVPrinter(sw, CSVFormat.DEFAULT); //No header
// create output file
fileCsvOut = new File(fileName);
def count=0;
// build the CSV
def an8Map = new ArrayList();
for (int i = 0; i < jsonData.size(); i++) {
def businessunit = jsonData[i].table1_column1;
if (an8Map.contains(businessunit)) {
continue;
}
an8Map.add(businessunit);
count++;
csv.printRecord(businessunit, jsonData[i].table_column,
jsonData[i].table1_column1, jsonData[i]. table1_column2, jsonData[i]. table1_column3, "Fixed text1 "Fixed text2", "Fixedtext3");
}
csv.close();
//writing csv to file
fileCsvOut.withWriter('UTF-8') {
writer ->
writer.write(sw.toString())
}
orchAttr.writeDebug(sw.toString());
returnMap.put("csv", sw.toString());
returnMap.put("CSVComplete", "true");
returnMap.put("CSVcount", Integer.toString(count));
return returnMap;
}
use withWriterAppend instead of withWriter
https://docs.groovy-lang.org/latest/html/groovy-jdk/java/io/File.html#withWriterAppend(java.lang.String,%20groovy.lang.Closure)
Hi here i have whole bunch of products with different json objects. I have to get the product name based on the price. If price is not zero or not null get that product.
Here is the script:
Below script will traverse thru the json and extracts the product name where there is price and also put the data into a map called productPriceMap which can be used later.
def productPriceMap = [:]
def jsonParsed = new groovy.json.JsonSlurper().parseText(json)
jsonParsed.products.collect{ product ->
product.productRatePlans.collect { ratePlan ->
ratePlan.productRatePlanCharges.collect { charge ->
charge.productRatePlanChargeTiers.collect{ tier ->
if (tier.price) {
log.info "${tier.price} - ${product.name}"
productPriceMap[product.name] = tier.price
}
}
}
}
}
log.info "Product and price map\n$productPriceMap"
You can quickly try this online Demo
I am sure this can be shortened / more groovified. But this is what I could get it.
What you are doing is filtering a certain list. So it gives you some result, but has to be stored somewhere to use it further.
Like this
def toList(priceNullCheck){
def filteredList = [priceNullCheck].flatten().findAll { it != null}
log.info filteredList
}
Else you can return [priceNullCheck].flatten().findAll { it != null} directly
UPDATED
import javax.xml.transform.Transformer
import javax.xml.transform.TransformerFactory
import javax.xml.transform.dom.DOMSource
import javax.xml.transform.stream.StreamResult
def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def response = groovyUtils.getXmlHolder("TestRequest#Response")
def collectResult = []
use (groovy.xml.dom.DOMCategory) {
for( node in response.getDomNodes("//*:products//*:e")) {
def eachNodeAsXml = groovyUtils.getXmlHolder(nodeToString(node))
String priceTagValue = eachNodeAsXml.getNodeValue("//*:price")
float price = priceTagValue?Float.parseFloat(priceTagValue):0
if(price>0)
collectResult.add(eachNodeAsXml.getNodeValue("/*:name"))
}
}
println collectResult
def nodeToString(def node)
{
StringWriter writer = new StringWriter();
Transformer transformer = TransformerFactory.newInstance().newTransformer();
transformer.transform(new DOMSource(node), new StreamResult(writer));
return writer.toString();
}
I am trying to build a REST service which accepts XML and convert it into JSON and call external Service which accepts JSON and put my JSON into it. I am able to put the json without pretty but I want to PUT the json in pretty format. Please suggest how to do, below is my code ...
package com.mypackge
import grails.converters.JSON
import grails.rest.RestfulController
import grails.plugins.rest.client.RestBuilder
class RestCustomerController extends RestfulController {
/*
static responseFormats = ['json', 'xml']
RestCustomerController() {
super(Customer)
}
*/
def index() {
convertXmlToJson()
}
def myJson = ''
def convertXmlToJson() {
def xml = ''' <Customer>
<customerid>9999999999999</customerid>
<ssn>8888</ssn>
<taxid>8888</taxid>
<address>
<addressline1>Yamber Ln</addressline1>
<addressline1>8664 SE</addressline1>
<city>CCCCC</city>
<state>CC</state>
<zipcode>97679</zipcode>
</address>
<firstname>Scott</firstname>
<middlename></middlename>
<lastname>David</lastname>
<account>
<accountno>576-294738943</accountno>
<accounttype>Lease</accounttype>
<accountsubtype></accountsubtype>
<accountstatus>complete</accountstatus>
<firstname>Scott</firstname>
<middlename></middlename>
<lastname>David</lastname>
<businessname></businessname>
<billingsystem>yoiuhn</billingsystem>
<brand></brand>
<plantype></plantype>
<billingaddress>
<addressline1>Yamber Ln</addressline1>
<addressline1>8664 SE </addressline1>
<city>CCCCC</city>
<state>CC</state>
<zipcode>97679</zipcode>
</billingaddress>
<job>
<jobid>8276437463728</jobid>
<jobstatus>SUCCESS</jobstatus>
</job>
</account>
</Customer>
'''.stripMargin()
// Parse it
def parsed = new XmlParser().parseText( xml )
def myId = parsed.customerid.text()
// Deal with each node:
def handle
handle = { node ->
if( node instanceof String ) {
node
}
else {
[ (node.name()): node.collect( handle ) ]
}
}
// Convert it to a Map containing a List of Maps
def jsonObject = [ (parsed.name()): parsed.collect { node ->
[ (node.name()): node.collect( handle ) ]
} ]
def json = new groovy.json.JsonBuilder(jsonObject) //.toPrettyString()
// Check it's what we expected
def mmyresp
try{
mmyresp = putRequest(myId,json)
}catch(Exception e) {
mmyresp = 'Please Validate JSON ....'
}
}
def putRequest(String id, JSON myJson) {
String url = "http://foo.com/customer/external/"+id
def rest = new RestBuilder()
def resp = rest.put(url){
contentType "application/json"
json{
myJson
}
}
return resp
}
}
The record is added in below format ...
{"Customer":[{"customerid":["9999999999999"]},{"ssn":["8888"]},
{"taxid":["8888"]},{"address":[{"addressline1":["Yamber Ln"]},
{"addressline1":["8664 SE"]},{"city":["CCCCC"]},{"state":["CC"]},{"zipcode":["97679"]}]},
{"firstname":["Scott"]},{"middlename":[]},{"lastname":["David"]},{"businessname":[]},
{"account":[{"accountno":["576-294738943"]},{"accounttype":["Lease"]},{"accountsubtype":[]},
{"accountstatus":["complete"]},{"firstname":["Scott"]},{"middlename":[]},{"lastname":["David"]},
{"businessname":[]},{"billingsystem":["yoiuhn"]},{"brand":[]},{"plantype":[]},
{"billingaddress":[{"addressline1":["Yamber Ln"]},{"addressline1":["8664 SE"]},
{"city":["CCCCC"]},{"state":["CC"]},{"zipcode":["97679"]}]},{"job":[{"jobid":["8276437463728"]},
,{"jobstatus":["SUCCESS"]}]}]}]}
But I want this to be inserted in pretty format. I tried .toPrettyString() but got casting exception when try to put as json. I am trying the REST services for the first time, not sure where I am doing wrong. Please suggest me on this.
You should set following field in you Config.groovy.
grails.converters.default.pretty.print = true
This will pretty print for both the xml and json.
you could optionally set it up for xml or json only like below:
For json:
grails.converters.json.pretty.print = true
For xml
grails.converters.xml.pretty.print = true
A sample of Config.groovy entry is:
environments {
development {
grails.converters.json.pretty.print = true
}
}
Hope it helps!!!
For Grails 4, try this:
def json = x as JSON
json.prettyPrint = true;
log.info(json.toString())
i am trying to import file CSV like this image..
this image means..
when i import this file..line 1 with read and save in to Table SeniorHighSchool
then it will get :
name : Alexander
age : 15
muchcourse : 3
after that i want make a condition, When "muchcourse is filled" it will read the next row...
example :
at this case, the "muchcourse" is "3", then "look at the image"..row 2,3,4 (3 lines)will inserted to other table .. because the "muchcourse" is "3"
this is my coding what i have tried .
def upload = {
withForm{
def f = request.getFile('filecsv')
def orifilename = f.getOriginalFilename()
def homeDir = new File(System.getProperty("user.home"))
def homeurl = "Documents/Uploads/"
File fileDest = new File(homeDir,homeurl+orifilename)
f.transferTo(fileDest)
request.getFile(new File(fileDest)).InputStream.splitEachLine(',') {fields ->
def student= new SeniorHighSchool(
name: fields[0].trim(),
age: fields[1].trim(),
muchcourse: fields[2].trim()
)
if (student.hasErrors() || student.save(flush: true) == null)
{
log.error("Could not import domainObject ${student.errors}")
}
}
redirect(action:"list")
}
}
i confused to make a condition..
def upload = {
withForm{
def f = request.getFile('filecsv')
def orifilename = f.getOriginalFilename()
def homeDir = new File(System.getProperty("user.home"))
def homeurl = "Documents/Uploads/"
File fileDest = new File(homeDir,homeurl+orifilename)
f.transferTo(fileDest)
request.getFile(new File(fileDest)).InputStream.splitEachLine(',') {fields ->
def student= new SeniorHighSchool(
name: fields[0].trim(),
age: fields[1].trim(),
muchcourse: fields[2].trim()
)
if (student.hasErrors() || student.save(flush: true) == null)
{
log.error("Could not import domainObject ${student.errors}")
}
if(fields[2]) {
def score = new Score(
course: //the problem at this line..how?
//it will insert 3 times then back to the row 5 to insert into "Student" again
)
}
}
redirect(action:"list")
}
}
if(fields.size()>2){
store 3 values in one table(student)
}
else{
store 2 values in another table(score)
}
If muchcourse field is occured then fields size is 3 then save three data in one table. otherwise size is 2 then save that two data in another table.I think it will solve your problem.
#th3morg like thiss?
request.getFile(new File(fileDest)).InputStream.splitEachLine(',') {fields ->
if(fields.size()>2){
def student= new SeniorHighSchool(
name: fields[0].trim(),
age: fields[1].trim(),
muchcourse: fields[2].trim()
)
}
else{
def score = new Score(
course:fields[0].trim(),
score:fields[1].trim()
)
}
}
If the class SeniorHighStudent "static hasMany = [scores: Score]" then the following should do the trick:
def currentStudent
request.getFile(new File(fileDest)).InputStream.splitEachLine(',') {fields ->
if(fields.size()>2){
if(currentStudent){
/*we've found a new student, so save the previous one*/
currentStudent.save()
}
currentStudent = new SeniorHighSchool(
name: fields[0].trim(),
age: fields[1].trim(),
muchcourse: fields[2].trim()
)
}
else{
/*add the score to the currentStudent's scores*/
currentStudent.addToScores(new Score(
course:fields[0].trim(),
score:fields[1].trim()
))
}
}
/*when the loop is done, save the last student because it hasn't been saved yet*/
currentStudent.save()
I have a csv file (details.csv) like
ID,NAME,ADDRESS
1,"{foo,bar}","{123,mainst,ny}"
2,"{abc,def}","{124,mainst,Va}"
3,"{pqr,xyz}","{125,mainst,IL}"
when I use (Note: I have other closure above this which reads all csv files from directory)
if(file.getName().equalsIgnoreCase("deatails.csv")) {
input = new FileInputStream(file)
reader = new BufferedReader(new InputStreamReader(input))
reader.eachLine{line-> def cols = line.split(",")
println cols.size() }
Instead of getting size 3 I am getting 6 with values
1
"{foo
bar}"
"{123
mainst
ny}"
spilt(",") is splitting data by comma(,) but I want my results as
1
"{foo,bar}"
"{123,mainst,ny}"
How can I fix this closure. Please help! Thanks
Writing a csv parser is a tricky business.
I would let someone else do the hard work, and use something like GroovyCsv
Here is how to parse it with GroovyCsv
// I'm using Grab instead of just adding the jar and its
// dependencies to the classpath
#Grab( 'com.xlson.groovycsv:groovycsv:1.0' )
import com.xlson.groovycsv.CsvParser
def csv = '''ID,NAME,ADDRESS
1,"{foo,bar}","{123,mainst,ny}"
2,"{abc,def}","{124,mainst,Va}"
3,"{pqr,xyz}","{125,mainst,IL}"'''
def csva = CsvParser.parseCsv( csv )
csva.each {
println it
}
Which prints:
ID: 1, NAME: {foo,bar}, ADDRESS: {123,mainst,ny}
ID: 2, NAME: {abc,def}, ADDRESS: {124,mainst,Va}
ID: 3, NAME: {pqr,xyz}, ADDRESS: {125,mainst,IL}
So, to get the NAME field of the second row, you could do:
def csvb = CsvParser.parseCsv( csv )
println csvb[ 1 ].NAME
Which prints
{abc,def}
Of course, if the CSV is a File, you can do:
def csvc = new File( 'path/to/csv' ).withReader {
CsvParser.parseCsv( it )
}
Then use it as above
There are two ways of doing.
One is using collect
def processCsvData(Map csvDataMap, File file)
{
InputStream inputFile = new FileInputStream(file);
String[] lines = inputFile.text.split('\n')
List<String[]> rows = lines.collect {it.split(',')}
// Add processing logic
}
Here problem is it is removing commas in between braces ({}) i.e "{foo,bar}" becomes "{foo bar}"
Another way of using java, and this works just fine
public class CSVParser {
/*
* This Pattern will match on either quoted text or text between commas, including
* whitespace, and accounting for beginning and end of line.
*/
private final Pattern csvPattern = Pattern.compile("\"([^\"]*)\"|(?<=,|^)([^,]*)(?:,|$)");
private ArrayList<String> allMatches = null;
private Matcher matcher = null;
private int size;
public CSVParser() {
allMatches = new ArrayList<String>();
matcher = null;
}
public String[] parse(String csvLine) {
matcher = csvPattern.matcher(csvLine);
allMatches.clear();
String match;
while (matcher.find()) {
match = matcher.group(1);
if (match!=null) {
allMatches.add(match);
}
else {
allMatches.add(matcher.group(2));
}
}
size = allMatches.size();
if (size > 0) {
return allMatches.toArray(new String[size]);
}
else {
return new String[0];
}
}
}
Hope this helps!