Coldfusion CSV to Spreadsheet - csv

I have a few processes that utilize the CFSpreadsheet tag to import and then manipulate Excel data. This works great for .XLS & .XLSX files, however, it doesn't work if the data is sent as a .CSV file since CFSpreadsheet apparently was never updated to import .CSV files. At the end of the day I just want a simple pre-processor that takes a .CSV file and re-writes it as an .XLSX file so that my other process can take it from there.
My environment is the developer edition of Coldfusion 2018 and I've tried importing the data manually (which can work if I know all of the column definitions---but I won't always know that). My latest attempt has been with Ben Nadel's CSVToArray function ( https://www.bennadel.com/blog/2041-update-parsing-csv-data-files-in-coldfusion-with-csvtoarray.htm ) which works---I can easily get the .CSV file into an array---but I can't figure out how to go from that array to something like a query that I can write a spreadsheet with using CFSpreadsheet.
Here's an EXAMPLE:
<!--- This include is the function from Ben Nadel referenced above --->
<cfinclude template="Function_CSVtoArray.cfm">
<cfset result = csvToArray(file="TEST_File.csv") />
<cfdump var="#result#" label="TESTING">
<!--- *** The above WORKS up to this point ***--->
<!--- Create a new query. --->
<cfset qPartsTwo = QueryNew( "" ) />
<!--- Loop over keys in the struct. --->
<cfloop index="strKey" list="#StructKeyList(result)#" delimiters=",">
<!--- Add column to new query with default values. --->
<cfset QueryAddColumn(qPartsTwo,strKey,"VARCHAR",objParts[strKey]) />
</cfloop>
<!--- This code FAILS with a "You have attempted to dereference a scalar variable of type class coldfusion.runtime.Array as a structure with members" error message --->
I'd like to end up at something like this (although right now "result" is an array of some kind and not a query):
<cfspreadsheet action="write" filename="<my path>\TEST.xlsx" query="result">
Any ideas would be appreciated!

It looks like your UDF returns a multi-dimensional array, not an array of structures. Instead of trying to coerce the array into a query object, try using spreadsheet functions to write the array data to an xlsx file.
DEMO / Sample data
result = [ ["First Name", "Last Name", "Address"]
, ["John", "Doe", "123 Anywhere Ave"]
, ["Mary", "Smith", "456 Somewhere Street"]
, ["Charles", "Doe", "789 Anywhere Court"]
];
Code:
// create spreadsheet
xlsx = SpreadSheetNew("The Results", true);
// populate with array data
SpreadSheetAddRows( xlsx, result );
// save to file
SpreadSheetWrite( xlsx, "c:/path/to/test.xlsx", true );
.. or as James A Mohler suggested, you could also use member functions:
xlsx = SpreadSheetNew("The Results", true);
xlsx.addRows( result );
xlsx.write( "c:/path/to/test.xlsx", true );

I bet you could do something like this
<cfinclude template="Function_CSVtoArray.cfm">
<cfset result = csvToArray(file="TEST_File.csv") />
<cfdump var="#result#" label="TESTING">
<!--- setup the columns that you need --->
<cfset qPartsTwo = queryNew("id,name,amount","Integer,Varchar,Integer", result) />
<cfspreadsheet action="write" filename="<my path>\TEST.xlsx" query="result">
CSVToArray() looks like it makes an array of structs.

If you already have an array of structures from CSVToArray. Can you then use the ArrayOfStructuresToQuery function: https://cflib.org/udf/ArrayOfStructuresToQuery

Related

ColdFusion 8: Convert OData Response to Array

I'm attempting to update an existing CF8 application to consume and load an array with the results of a newly updated RESTful API response using OData.
Here's the code in question... After pulling the data from the API that responds with the OData jSON string, the code blows up on the last line that inits the loop
<!--- Returned data is in json format so must change to an array. --->
<cfset local.result = deserializeJSON(myResult)>
<!--- Reference the array collection of categories --->
<cfset local.collection = local.result>
<!--- Initialize the output object --->
<cfset local.output = arrayNew(1)>
<!--- Loop over the collection --->
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
...
This was working fine using the previous JSON response:
[
{
"id": 1,
"name": "Blah, blah",
}
]
The only change introduced is the updated JSON response:
[
{
"#odata.context": "string",
"value": [
{
"id": 1,
"name": "Blah, blah"
}
]
}
]
I'm sure I'm missing something basic, but I've never worked on CF before so it's new territory here.
Thoughts?
Thanks!
UPDATE:
Apologies on not providing more detail. Here's how the app currently uses the response:
<!--- Loop over the collection --->
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<!--- Create a reference to the array element --->
<cfset local.objectInstance = local.collection[local.arrayIndex]>
<!--- Create a new object reference --->
<cfset local.thisObject = structNew()>
<!--- Seed the object properties --->
<cfset local.thisObject.categoryId = local.objectInstance.id>
<cfset local.thisObject.categoryName = local.objectInstance.name>
<!--- Place the new object in the collection array --->
<cfset arrayAppend(local.output, duplicate(local.thisObject))>
</cfloop>
And here's the error I'm receiving:
Error Occurred While Processing Request
Object of type class coldfusion.runtime.Struct cannot be used as an array
The error occurred in <path to file> line 97
"Line 97" is the begin loop available in the update above:
I did try using the "newJSON" approach offered by Miguel (thank you very much for that!), but unfortunately, I'm running into the same error.
Thanks again!
-Rich
Update after user posted more information
If you are still getting an error then you did something wrong. You must change the way you are referencing the new JSON data object. I created a new Gist using the updated code that you supplied so you can see how it works - TryCF Gist 2
Basically the code within your <cfloop> needs to look like this. Again, notice that there are actually two <cfloop> blocks. This is because the new JSON format generates an array that contains another array.
<!--- Loop over the collection --->
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<cfloop from="1" to="#arrayLen(local.collection[local.arrayIndex].value)#" index="local.arrayIndex2">
<!--- Create a reference to the array element --->
<cfset local.objectInstance = local.collection[local.arrayIndex].value>
<!--- Create a new object reference --->
<cfset local.thisObject = structNew()>
<!--- Seed the object properties --->
<cfset local.thisObject.categoryId = local.objectInstance[local.arrayIndex2].id>
<cfset local.thisObject.categoryName = local.objectInstance[local.arrayIndex2].name>
<!--- Place the new object in the collection array --->
<cfset arrayAppend(local.output, duplicate(local.thisObject))>
</cfloop>
</cfloop>
See the Gist for more details but this assigns the local.output array as it was before. In your original code the local.objectInstance within the loop was a structure. With the new JSON format the local.objectInstance within the loop now contains an array of structures. So you need to reference it as such.
Original answer before question was updated
With the updated JSON you will need to update how your code references the data (which you did not include in your original post). Making some assumptions I can show you how to reference the data using the examples you gave.
First for your original example. Here is some code that would reference and output the data for you. Notice that I have included a <cfdump> tag. You will want to use that in situations like this where you need to see the data. The deserializeJSON() function parses the JSON for you and creates a ColdFusion array of structures.
<cfset oldJSON = '[ { "id": 1, "name": "Blah, blah" } ]'>
<!--- Returned data is in json format so must change to an array. --->
<cfset local.result = deserializeJSON(oldJSON)>
<!--- Reference the array collection of categories --->
<cfset local.collection = local.result>
<!--- Initialize the output object --->
<cfset local.output = arrayNew(1)>
<cfdump var="#local.result#" label="Old JSON">
<!--- Loop over the collection --->
<cfoutput>
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<p>#local.arrayIndex# - #local.collection[local.arrayIndex].id# - #local.collection[local.arrayIndex].name#</p>
</cfloop>
</cfoutput>
That code gives this output:
Here is an example of the updated code needed to retrieve the same values from the new JSON format. Notice that I added another cfloop to reference the data because there are now two arrays.
<cfset newJSON = '[ { "#odata.context": "string", "value": [ { "id": 1, "name": "Blah, blah" } ] } ]'>
<!--- Returned data is in json format so must change to an array. --->
<cfset local.result = deserializeJSON(newJSON)>
<!--- Reference the array collection of categories --->
<cfset local.collection = local.result>
<!--- Initialize the output object --->
<cfset local.output = arrayNew(1)>
<cfdump var="#local.result#" label="New JSON">
<!--- Loop over the collection --->
<cfoutput>
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<cfloop from="1" to="#arrayLen(local.collection[local.arrayIndex].value)#" index="local.arrayIndex2">
<p>#local.arrayIndex# - #local.arrayIndex2# - #local.collection[local.arrayIndex].value[local.arrayIndex2].id# - #local.collection[local.arrayIndex].value[local.arrayIndex2].name#</p>
</cfloop>
</cfloop>
</cfoutput>
That code gives this output:
I created a gist with all of this code that you can play around with - TryCF Gist 1

Presence of single or double quotes in SERIESLABEL or ITEMLABEL giving error in CFCHART

I'm using CF Pie chart for one of my application. But it is working weird. The following code is the chart code and it is giving an error. It doesn't even display the chart. I know that, it is due to the presence of double quotes in the value of query column, col1.
<cfoutput>
<script type="text/javascript">
function Navigate(test){
alert(test);
}
</script>
<cfset testquery = queryNew("col1,Col2", "varchar,varchar") >
<cfset queryAddRow(testquery, 1)>
<cfset querySetCell(testquery, "col1", 'This is the "first" row') >
<cfset querySetCell(testquery, "Col2", 5000) >
<cfset queryAddRow(testquery, 1)>
<cfset querySetCell(testquery, "col1", 'This is the second row') >
<cfset querySetCell(testquery, "Col2", 2500) >
<cfset queryAddRow(testquery, 1)>
<cfset querySetCell(testquery, "col1", 'This is the third row') >
<cfset querySetCell(testquery, "Col2", 8500) >
<CFCHART Format="Html" CHARTWIDTH="600" CHARTHEIGHT="650" TITLE="Pie Chart in CF11" URL="javascript:Navigate('$SERIESLABEL$')">
<CFCHARTSERIES TYPE="pie" COLORLIST="##CA5940,##6FCF42,##4286CF" >
<CFLOOP FROM="1" TO="#testquery.RecordCount#" INDEx="i">
<CFCHARTDATA ITEM="#testquery.col1[i]#" VALUE="#testquery.Col2[i]#">
</CFLOOP>
</CFCHARTSERIES>
</CFCHART>
</cfoutput>
I've checked the chart's JSON in the viewsource, it is fine. But code giving the above error. So not sure why it is giving error. Without double quotes the code is working as expected, but I need the double quotes, it'll impact the application, if I remove the same.
I also tried by replacing the double quotes to single quotes, in that case, the chart is displaying, but if we click on the first row area, it is giving the same error in console.
So using quotes is the major problem here. But I need the above code to display the chart and while clicking on the area, it should show the corresponding label as it is.
I'm not sure I've missed something, or else anything wrong in the code.
Have you tried escaping the quotes within your JSON string?
Like this for example:
<cfset querySetCell(testquery, "col1", 'This is the \"first\" row') >
I created a gist of your sample code on trycf.com and it seems to work
http://trycf.com/gist/e3321edb3481411078b75ad187cae52b/acf11?theme=monokai
I also tried that exact same code on one of our ColdFusion 11 servers and it works fine too. So are you saying your sample code is not working or are you saying your actual code is still not working? If it is your actual code then there must be something else at play that is not shown in your example. If your data is coming from a database then you need to be sure that those characters are being escaped correctly at the point that ColdFusion is parsing it.
Now that you see how the character needs to be escaped you can use ScottJibben suggestion from the comments and just call JSStringFormat() to escape these characters for you.
<CFCHARTDATA ITEM="#JSStringFormat(testquery.col1[i])#" VALUE="#testquery.Col2[i]#">

Using CFFile to write csv file to server for download. How to prevent data from being saved as HTML Encoded?

Using CF9, I want to write output to a csv file for user download.
<cfinvoke component="RAMP.cfcData.reportMonthlyTotals" method="report" xport="1" returnvariable="rpt" />
<cfset columnList = "Project Number,Project Name,Organization,Division,Group" />
<cffile action="write" file="#xportPath#" charset="utf-8" output="#columnList#" />
<cfset csvRow = "" />
<cfloop array="#rpt.data#" index="row">
<cfset csvRow = arrayToList(row,",") />
<cffile action="append" file="#xportPath#" charset="utf-8" output="#csvRow#" />
</cfloop>
The output of the method report is an array of arrays. The returnformat of the function is "JSON". This same function returns data used in a JQuery DataTables grid.
rtn.data = [
["itemOne", "itemTwo", "itemThree"],
["itemOne", "itemTwo", "itemThree"],
]
The problem I am having is that some of the data has special characters. This is output from the same function to a textarea on a test page.
CC2014118463,"Employee Townhall with CSO Q3 - São Paulo","CMO Communications & Public Relations","CALA Comms"
The csv file has the same line as:
CC2014118463,"Employee Townhall with CSO Q3 - São Paulo","CMO Communications & Public Relations","CALA Comms"
What is causing the special character in San to be coverted to &#227 ?
I have tried different charset settings when I write and append the file.
I have also tried changing return format of the cfc, in the event it was an json encoding issue. But so far, no luck.
Thanks in advance for the assistance.
Gary

Send plain text in json format in an api webservice in coldfusion

I have an api I'm working on, I need to be able to send the coverletter of an applicant, which is stored like plain text in a table. I have a function which grabs the text as below. Do I have to use SerializeJSON(coverlettertext) before I return the text? In order for it to be consumed and get the proper data on the other end. Please advice.
<cffunction name="query1" access="remote" returnformat="JSON" output="true">
<cfquery name="local.cover_letter" datasource="#variables.dsn#">
SELECT cover_letter
FROM table1
WHERE userid = <cfqueryparam cfsqltype="cf_sql_integer" value="12345" />
AND jobid = <cfqueryparam cfsqltype="cf_sql_integer" value="456" />
</cfquery>
<cfif local.get_cover_letter.recordCount>
<cfreturn local.cover_letter/>
</cfif>
</cffunction>
ColdFusion functions with returnFormat, serializes it for you, but its ColdFusions format of Serialize.. it would take a returned query or struct, and serialize it as it sees fit. This is not always the expected format, as it has one row of cols, then several rows of data... compared to colnames and values in each and every row (CF11 allows you to select the serializing format or right your own but only if you call it manually I believe).
So you could say the return type is query, and then the format is json, and it will do it for you... it will be JSON, will it work with the service you're using, that DEPENDS on what the other end is EXPECTING.
If you require it in a specific format, ie, not ColdFusions, then you would want to change the ReturnFormat to plain, and then return a string, encoded how you want to.
From the context of your question, I'm assuming that you are only getting 1 letter. If I'm mistaken, then that would alter my response. That said, +1 to Gavin for identifying that the format will be different from what you may expect if you serialize the query object. You can instead, make a CF struct out of the output that you want and then serialize that. Take a look at the two variants in the code below.
As everyone else has noted, it all depends on what the consumer of the service is expecting.
<cffunction name="query1" access="remote" returnformat="JSON" output="true">
<cfscript>
local.cover_letter = queryNew(
"cover_letter",
"varChar",
[
{
cover_letter:"This is the text of my letter. It could be much longer"
}
]
);
</cfscript>
<cfif local.cover_letter.recordCount>
<cfreturn serializeJson(local.cover_letter)/>
</cfif>
</cffunction>
<cffunction name="query2" access="remote" returnformat="JSON" output="true">
<cfscript>
local.cover_letter = queryNew(
"cover_letter",
"varChar",
[
{
cover_letter:"This is the text of my letter. It could be much longer"
}
]
);
</cfscript>
<cfif local.cover_letter.recordCount>
<cfset local.str = {cover_letter= local.cover_letter.cover_letter} >
<cfreturn serializeJson(local.str) />
</cfif>
</cffunction>
<cfoutput>
<h1>query1 output</h1>
#query1()#
<h1>query2 output</h1>
#query2()#
</cfoutput>
You output should be:
query1 output
{"COLUMNS":["COVER_LETTER"],"DATA":[["This is the text of my letter. It could be much longer"]]}
query2 output
{"COVER_LETTER":"This is the text of my letter. It could be much longer"}

Using JSON Data with Coldfusion

I've worked with JSON data in the past - mainly 'fudging' my way to a solution, and not really understanding why or how things work. I've come across an issue where the data being returned looks somewhat different to what I've seen before, and I can't find any examples that match it.
Here's an example of the data returned (via an API);
{"domain.co.uk":{"status":"available","classkey":"thirdleveldotuk"},"domain.net":{"status":"available","classkey":"dotnet"},"domain.com":{"status":"available","classkey":"domcno"}}
On my front-end, I need to return something like this -
domain.co.uk - available
domain.net - available
domain.com - available
Because the 'domain.com' etc value will always change, I can't map the names as I usually would (although it will always be 3 'rows' returned)
I've checked every CF Book I own, and read the online CF Docs, but I'm totally at a loss as to where to even start with this one!
Pointers much appreciated!
If you run this with deserializeJSON(data), you'll see that you just end up with structures with nested structures. So, you can loop through your struct, grab the keys and then grab that key's status. In JSON terms, your JSON object has nested objects.
<cfset data = deserializeJSON(apiData) />
<cfset formattedData = [] />
<cfset tmp = {} />
<cfloop collection=#data# item="domain">
<cfset tmp.domain = domain />
<cfset tmp.status = data[domain]["status"] />
<cfset arrayAppend(formattedData,duplicate(tmp)) />
</cfloop>
<cfdump var=#formattedData# />
(This is really more of a comment, but is a bit too long ... )
I've worked with JSON data in the past - mainly 'fudging' my way to a
solution, and not really understanding why or how things work
JSON strings are essentially just a representations of two objects:
arrays which are denoted by [] and
structures (or objects) which are denoted by {}
Looking at the API string, the braces {} indicate you are dealing with a structure:
{ "theKey": "theValue" }
In your case, the domain name is the structure key:
{ "domain.co.uk": "theValue" }
.. and the value is a nested structure containing two static keys: "status" and "classkey"
{ "theKey": {"status":"available","classkey":"thirdleveldotuk"} }
As with any structure, you can iterate through the keys dynamically using a for .. in loop, a collection loop if you prefer cfml.
for (theKey in theStruct) {
WriteDump( theKey ); // ie "domain.co.uk"
}
Then inside the loop use associative array notation to grab the value, ie:
theStatus = theStruct[ theKey ]["status"]; // "available"
// ... OR
theValue = theStruct[ theKey ];
theStatus = theValue.status;
That is all there is to it. You can use similar logic to access any type of nested structures.