ColdFusion 8: Convert OData Response to Array - json

I'm attempting to update an existing CF8 application to consume and load an array with the results of a newly updated RESTful API response using OData.
Here's the code in question... After pulling the data from the API that responds with the OData jSON string, the code blows up on the last line that inits the loop
<!--- Returned data is in json format so must change to an array. --->
<cfset local.result = deserializeJSON(myResult)>
<!--- Reference the array collection of categories --->
<cfset local.collection = local.result>
<!--- Initialize the output object --->
<cfset local.output = arrayNew(1)>
<!--- Loop over the collection --->
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
...
This was working fine using the previous JSON response:
[
{
"id": 1,
"name": "Blah, blah",
}
]
The only change introduced is the updated JSON response:
[
{
"#odata.context": "string",
"value": [
{
"id": 1,
"name": "Blah, blah"
}
]
}
]
I'm sure I'm missing something basic, but I've never worked on CF before so it's new territory here.
Thoughts?
Thanks!
UPDATE:
Apologies on not providing more detail. Here's how the app currently uses the response:
<!--- Loop over the collection --->
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<!--- Create a reference to the array element --->
<cfset local.objectInstance = local.collection[local.arrayIndex]>
<!--- Create a new object reference --->
<cfset local.thisObject = structNew()>
<!--- Seed the object properties --->
<cfset local.thisObject.categoryId = local.objectInstance.id>
<cfset local.thisObject.categoryName = local.objectInstance.name>
<!--- Place the new object in the collection array --->
<cfset arrayAppend(local.output, duplicate(local.thisObject))>
</cfloop>
And here's the error I'm receiving:
Error Occurred While Processing Request
Object of type class coldfusion.runtime.Struct cannot be used as an array
The error occurred in <path to file> line 97
"Line 97" is the begin loop available in the update above:
I did try using the "newJSON" approach offered by Miguel (thank you very much for that!), but unfortunately, I'm running into the same error.
Thanks again!
-Rich

Update after user posted more information
If you are still getting an error then you did something wrong. You must change the way you are referencing the new JSON data object. I created a new Gist using the updated code that you supplied so you can see how it works - TryCF Gist 2
Basically the code within your <cfloop> needs to look like this. Again, notice that there are actually two <cfloop> blocks. This is because the new JSON format generates an array that contains another array.
<!--- Loop over the collection --->
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<cfloop from="1" to="#arrayLen(local.collection[local.arrayIndex].value)#" index="local.arrayIndex2">
<!--- Create a reference to the array element --->
<cfset local.objectInstance = local.collection[local.arrayIndex].value>
<!--- Create a new object reference --->
<cfset local.thisObject = structNew()>
<!--- Seed the object properties --->
<cfset local.thisObject.categoryId = local.objectInstance[local.arrayIndex2].id>
<cfset local.thisObject.categoryName = local.objectInstance[local.arrayIndex2].name>
<!--- Place the new object in the collection array --->
<cfset arrayAppend(local.output, duplicate(local.thisObject))>
</cfloop>
</cfloop>
See the Gist for more details but this assigns the local.output array as it was before. In your original code the local.objectInstance within the loop was a structure. With the new JSON format the local.objectInstance within the loop now contains an array of structures. So you need to reference it as such.
Original answer before question was updated
With the updated JSON you will need to update how your code references the data (which you did not include in your original post). Making some assumptions I can show you how to reference the data using the examples you gave.
First for your original example. Here is some code that would reference and output the data for you. Notice that I have included a <cfdump> tag. You will want to use that in situations like this where you need to see the data. The deserializeJSON() function parses the JSON for you and creates a ColdFusion array of structures.
<cfset oldJSON = '[ { "id": 1, "name": "Blah, blah" } ]'>
<!--- Returned data is in json format so must change to an array. --->
<cfset local.result = deserializeJSON(oldJSON)>
<!--- Reference the array collection of categories --->
<cfset local.collection = local.result>
<!--- Initialize the output object --->
<cfset local.output = arrayNew(1)>
<cfdump var="#local.result#" label="Old JSON">
<!--- Loop over the collection --->
<cfoutput>
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<p>#local.arrayIndex# - #local.collection[local.arrayIndex].id# - #local.collection[local.arrayIndex].name#</p>
</cfloop>
</cfoutput>
That code gives this output:
Here is an example of the updated code needed to retrieve the same values from the new JSON format. Notice that I added another cfloop to reference the data because there are now two arrays.
<cfset newJSON = '[ { "#odata.context": "string", "value": [ { "id": 1, "name": "Blah, blah" } ] } ]'>
<!--- Returned data is in json format so must change to an array. --->
<cfset local.result = deserializeJSON(newJSON)>
<!--- Reference the array collection of categories --->
<cfset local.collection = local.result>
<!--- Initialize the output object --->
<cfset local.output = arrayNew(1)>
<cfdump var="#local.result#" label="New JSON">
<!--- Loop over the collection --->
<cfoutput>
<cfloop from="1" to="#arrayLen(local.collection)#" index="local.arrayIndex">
<cfloop from="1" to="#arrayLen(local.collection[local.arrayIndex].value)#" index="local.arrayIndex2">
<p>#local.arrayIndex# - #local.arrayIndex2# - #local.collection[local.arrayIndex].value[local.arrayIndex2].id# - #local.collection[local.arrayIndex].value[local.arrayIndex2].name#</p>
</cfloop>
</cfloop>
</cfoutput>
That code gives this output:
I created a gist with all of this code that you can play around with - TryCF Gist 1

Related

Coldfusion JSON

Currently I'm getting this output (QueryBean):
But I want the "normal" JSON output, like this one:
[
{
"EventType": "active",
"InstanceId": "6728E65C-XXXX-XXXXX",
"CustomerId": "1000324234",
"Name": "bilderbuchverein Hermsen"
"Amount": 999999,
"StartDate": "August, 01 2019 00:00:00",
"ExpirationDate": null
},
{
"EventType": "active",
"InstanceId": "956FA492-XXXX-XXXXX",
"Name": "Phantasialand"
"CustomerId": "12345678999",
"Amount": 123456789,
"StartDate": "August, 14 2019 00:00:00",
"ExpirationDate": null
}
]
How can I manage to change the output format? My function has the parameter produces="application/json"
<cffunction name="listEvents" access="remote" returnType="any" produces="application/JSON" httpmethod="GET" restpath="/events">
<cfquery datasource="hostmanager" name="plannedInstances">
SELECT * FROM licenses
</cfquery>
<cfreturn plannedInstances>
</cffunction>
To get JSON output data as an array of structure, you can use Application-level defining. In Application.cfc by adding this.serialization.serializeQueryAs = "struct" you can get serialize JSON output data as an array of structure like below.
[
{
"CATEGORYID":20,
"CATEGORYNAME":"php",
"CATEGORYDESC":"php"
},
{
"CATEGORYID":21,
"CATEGORYNAME":"cf",
"CATEGORYDESC":"cf"
},
{
"CATEGORYID":22,
"CATEGORYNAME":".Net",
"CATEGORYDESC":".net"
}
]
I have used the same code with my test table. (please see the example code result screenshot)
Also, you can refer SerializeJSON 'Additional format for query serialization' I think you can't able to handle this issue with produces="application/json"
I hope It's useful for you!
There are two steps to achieve your goal.
Create and return an array of structures instead of a query ( Which fully based on CFML side )
Use the ParseJson method in jQuery to parse the response from CF.
Here is an example of returning an array of structures in JSON format. It creates a single array, and populates it with individual structures inside the query loop. Insert the query column names as the structure key, and finally append the structure into the root array. Finally, return that array from the function.
<cffunction name="sampleCall" access="remote" returntype="any" returnformat="JSON">
<cfquery name="read" datasource="myDataSource">
SELECT * FROM myTable limit 10
</cfquery>
<cfset myArray = [] >
<cfloop query="read">
<cfset myStr = {} >
<cfset structInsert(myStr, "logid", read.logid)>
<cfset structInsert(myStr, "log_datetime", read.log_datetime)>
<cfset structInsert(myStr, "log_Details", read.log_Details)>
<cfset arrayAppend(myArray,myStr)>
</cfloop>
<cfreturn myArray / >
</cffunction>
Note : The example uses sample columns from my query. You can use your own column names.
If you dump the result like the image below

Coldfusion CSV to Spreadsheet

I have a few processes that utilize the CFSpreadsheet tag to import and then manipulate Excel data. This works great for .XLS & .XLSX files, however, it doesn't work if the data is sent as a .CSV file since CFSpreadsheet apparently was never updated to import .CSV files. At the end of the day I just want a simple pre-processor that takes a .CSV file and re-writes it as an .XLSX file so that my other process can take it from there.
My environment is the developer edition of Coldfusion 2018 and I've tried importing the data manually (which can work if I know all of the column definitions---but I won't always know that). My latest attempt has been with Ben Nadel's CSVToArray function ( https://www.bennadel.com/blog/2041-update-parsing-csv-data-files-in-coldfusion-with-csvtoarray.htm ) which works---I can easily get the .CSV file into an array---but I can't figure out how to go from that array to something like a query that I can write a spreadsheet with using CFSpreadsheet.
Here's an EXAMPLE:
<!--- This include is the function from Ben Nadel referenced above --->
<cfinclude template="Function_CSVtoArray.cfm">
<cfset result = csvToArray(file="TEST_File.csv") />
<cfdump var="#result#" label="TESTING">
<!--- *** The above WORKS up to this point ***--->
<!--- Create a new query. --->
<cfset qPartsTwo = QueryNew( "" ) />
<!--- Loop over keys in the struct. --->
<cfloop index="strKey" list="#StructKeyList(result)#" delimiters=",">
<!--- Add column to new query with default values. --->
<cfset QueryAddColumn(qPartsTwo,strKey,"VARCHAR",objParts[strKey]) />
</cfloop>
<!--- This code FAILS with a "You have attempted to dereference a scalar variable of type class coldfusion.runtime.Array as a structure with members" error message --->
I'd like to end up at something like this (although right now "result" is an array of some kind and not a query):
<cfspreadsheet action="write" filename="<my path>\TEST.xlsx" query="result">
Any ideas would be appreciated!
It looks like your UDF returns a multi-dimensional array, not an array of structures. Instead of trying to coerce the array into a query object, try using spreadsheet functions to write the array data to an xlsx file.
DEMO / Sample data
result = [ ["First Name", "Last Name", "Address"]
, ["John", "Doe", "123 Anywhere Ave"]
, ["Mary", "Smith", "456 Somewhere Street"]
, ["Charles", "Doe", "789 Anywhere Court"]
];
Code:
// create spreadsheet
xlsx = SpreadSheetNew("The Results", true);
// populate with array data
SpreadSheetAddRows( xlsx, result );
// save to file
SpreadSheetWrite( xlsx, "c:/path/to/test.xlsx", true );
.. or as James A Mohler suggested, you could also use member functions:
xlsx = SpreadSheetNew("The Results", true);
xlsx.addRows( result );
xlsx.write( "c:/path/to/test.xlsx", true );
I bet you could do something like this
<cfinclude template="Function_CSVtoArray.cfm">
<cfset result = csvToArray(file="TEST_File.csv") />
<cfdump var="#result#" label="TESTING">
<!--- setup the columns that you need --->
<cfset qPartsTwo = queryNew("id,name,amount","Integer,Varchar,Integer", result) />
<cfspreadsheet action="write" filename="<my path>\TEST.xlsx" query="result">
CSVToArray() looks like it makes an array of structs.
If you already have an array of structures from CSVToArray. Can you then use the ArrayOfStructuresToQuery function: https://cflib.org/udf/ArrayOfStructuresToQuery

How to build structure from cfquery?

I have cffunction that should return JSON structure. There is more than 50 columns that I have to return. Instead of building my structure manually I would like to build that dynamically. So first loop through query then loop through each table column. Here is example:
<cffunction name="getRecords" access="remote" output="true" returnformat="JSON">
<cfargument name="userID" type="string" required="true">
<cfset fnResults = StructNew()>
<cfquery name="myQuery" datasource="test">
SELECT
ur_first,
ur_last,
ur_dob,
ur_gender,
ur_email,
ur_address,
... and the rest of the columns
FROM Users
WHERE ur_id = <cfqueryparam value="#trim(arguments.userID)#" cfsqltype="cf_sql_char" maxlength="15">
ORDER BY ur_createDt
</cfquery>
<cfset fnResults.recordcount = myQuery.recordcount>
<cfloop query="myQuery">
<cfset qryRecs = StructNew()>
<cfloop array="#myQuery.getColumnList()#" index="columnName">
<cfset qryRecs.'#columnName#' = URLEncodedFormat('#columnName#')>
</cfloop>
</cfloop>
<cfset fnResults.data = qryRecs>
<cfreturn fnResults>
</cffunction>
This error I'm getting back after Ajax call:
CFML variable name cannot end with a "." character.
The variable qryRecs. ends with a "." character. You must either provide an additional structure key or delete the "." character.
Referencing to this line:
443 : <cfset qryRecs.'#columnName#' = URLEncodedFormat('#columnName#')>
I want to set column name to structure qryRecs like this:
<cfset qryRecs.ur_first = URLEncodedFormat(myQuery.ur_first)>
This way I don't have to set 50 plus columns manually. They all should be created dynamically. If anyone can help please let me know.
I created an ArrayCollection object that can convert a ColdFusion query to a few different JSON formats. Take a look and see if this fits your needs.
For example, this query:
<cfquery name="rs.q" datasource="cfbookclub">
SELECT DISTINCT
bookid,
title,
genre
FROM
books
WHERE
title LIKE <cfqueryparam value="%#arguments.term#%" cfsqltype="cf_sql_varchar" />
ORDER BY
genre, title
</cfquery>
will be converted to this JSON:
{
"data": [
{
"bookid": 8,
"genre": "Fiction",
"title": "Apparition Man"
},
{
"bookid": 2,
"genre": "Non-fiction",
"title": "Shopping Mart Mania"
}
]
}
I'm also working on an update that adds meta data to the return message:
{
"success": true,
"message": "Array Collection created.",
"meta": {
"offset": 0,
"pageSize": 0,
"totalRecords": 0
},
"data": []
};

how do I change the label for each object in SerializeJSON

I'm trying to use ColdFusion's SerializeJSON() to return JSON data.
So far I have done this:
<cfset rows = [] />
<cfloop query="rsProducts">
<!--- Create a row struct. --->
<cfset row = {} />
<!--- Add each column to our struct. --->
<cfloop
index="column"
list="#rsProducts.columnList#"
delimiters=",">
<cfset row[ column ] = rsProducts[ column ][ rsProducts.currentRow ] />
</cfloop>
<!--- Append the row struct to the row array. --->
<cfset arrayAppend( rows, row ) />
</cfloop>
<cfreturn SerializeJSON(rows, true)>
This works fine and produces JSON like this:
[Object {PRICE: 89.99, PRODUCTSTATUS: 1, COUNTRY: US}, Object {PRICE: 110.50, PRODUCTSTATUS: 4, COUNTRY: UK}, Object {PRICE: 41.20, PRODUCTSTATUS: 1, COUNTRY: IN}]
However instead of a label of "Object" for each item, I'd like it to be "ProductItem" instead. It just makes it easier for me to deal with in jQuery later on.
How could I have it so it labels each object in JSON as "ProductItem"?
You can loop over the data in this manner easily.
.success(function(data) {
var ProductItem = JSON.parse(data);
$.each(ProductItem,function(key,value){
console.log(value.PRICE +" "+ value.PRODUCTSTATUS + " " + value.COUNTRY);
});

How do I use this coldfusion code to read a large XML file and insert the data into a database?

I am using ColdFusion (openBlueDragon) to insert the data from a large (200MB) xml file into a database without having to load the entire file into memory which is how I traditionally would do it. I did find a VERY SIMILAR QUESTION here: Looping over a large XML file that seems to be the answer I am looking for.
However, I'm not skilled enough in java to understand and adapt the code to my needs. I found no way to respond to the expert (#orangepips) who posted the code or else I would not have posted such a similar question.
My xml file looks like this:
<allItems>
<item>
<subject>The subject text</subject>
<date>2007-05-21 04:03:00</date>
<content>text content often contains many paragraphs of text</content>
<author>JPass78</author>
</item>
</allItems>
This is the code, courtesy orangepips, that I'm trying to adapt for my purpose. I've modified it a bit to include my own field names:
<cfset fis = createObject("java", "java.io.FileInputStream").init(
"#getDirectoryFromPath(getCurrentTemplatePath())#/file.xml")>
<cfset bis = createObject("java", "java.io.BufferedInputStream").init(fis)>
<cfset XMLInputFactory = createObject("java", "javax.xml.stream.XMLInputFactory").newInstance()>
<cfset reader = XMLInputFactory.createXMLStreamReader(bis)>
<cfloop condition="#reader.hasNext()#">
<cfset event = reader.next()>
<cfif event EQ reader.START_ELEMENT>
<cfswitch expression="#reader.getLocalName()#">
<cfcase value="allItems">
<!--- root node, do nothing --->
</cfcase>
<cfcase value="item">
<!--- set values used later on for inserts, selects, updates --->
</cfcase>
<cfcase value="subject">
<!--- some selects and insert --->
</cfcase>
<cfcase value="contentdate">
<!--- insert or update --->
</cfcase>
<cfcase value="content">
</cfcase>
<cfcase value="author">
</cfcase>
</cfswitch>
</cfif>
</cfloop>
<cfset reader.close()>
I have a single table and I am trying to figure out how do I access the values from each XML element so I may insert it one row at a time? like this: INSERT INTO content (subject,contentdate, content, author)
VALUES ("The subject text", 2007-5-21 04:03:00, "text content here","JPass78");
Instead of using COLDFUSION to import large XML files into a MYSQL database, use the MYSQL command "LOAD XML INFILE".
Here's the simple, light and fast code that worked for me:
LOAD XML INFILE 'pathtofile/file.xml' INTO TABLE table_name ROWS IDENTIFIED BY '<item>';
My xml file uses the same exact field names as my database table. ROWS IDENTIFIED BY tells the command that the field names in my xml file will correspond to the database fields in my table and they will be found in between the <item></item> tags.
FYI, <item> is my own naming format. Your file will likely have another tag name that relates to the data you're working with. For example, if your xml file is for employee data, you might instead use <employee>
Available in MYSQL5.5 - Reference for LOAD XML INFILE can be found at:
http://dev.mysql.com/doc/refman/5.5/en/load-xml.html
One possibility is to initialize a data structure each time you encounter the <item> element. As the child elements go by (<subject>, <date>, ...), extract their text and add it to your structure. Then when you reach the </item> element do your validation/insert. There may be better approaches. But that should give you something to work with ..
Update: I had a hunch a database bulk loading tool would be a better option. Turns out it was ;) See JPass' answer for details.
<cfset fis = createObject("java", "java.io.FileInputStream").init(pathToYourFile)>
<cfset bis = createObject("java", "java.io.BufferedInputStream").init(fis)>
<cfset XMLInputFactory = createObject("java", "javax.xml.stream.XMLInputFactory").newInstance()>
<cfset reader = XMLInputFactory.createXMLStreamReader(bis)>
<cfloop condition="#reader.hasNext()#">
<cfset event = reader.next()>
<cfif event EQ reader.START_ELEMENT>
<cfswitch expression="#reader.getLocalName()#">
<cfcase value="item">
<!--- start a new data row --->
<cfset row = {}>
</cfcase>
<cfcase value="subject">
<!--- extract the subject text --->
<cfset row.subject = reader.getElementText()>
</cfcase>
<cfcase value="date">
<!--- extract the date text --->
<cfset row.date = reader.getElementText()>
</cfcase>
<cfcase value="content">
<!--- extract the content text --->
<cfset row.content = reader.getElementText()>
</cfcase>
<cfcase value="author">
<!--- extract the author text --->
<cfset row.author = reader.getElementText()>
</cfcase>
</cfswitch>
<cfelseif event EQ reader.END_ELEMENT>
<!--- we have reached the end of the row. time to insert the data --->
<cfif reader.getLocalName() eq "item">
<cfdump var="#row#" label="Debug Row Data">
<!--- ... validate / insert "row" data into database --->
</cfif>
</cfif>
</cfloop>
<cfset fis.close()>
<cfset reader.close()>