Output ADODB.RecordSet as JSON - json

I'm trying to change my application so that it outputs JSON instead of HTML when it makes an AJAX request for some data. I have an ADODB RecordSet. I need to loop through it row-by-row and add/change/remove different values. Then I need to take all the modified rows and response.write them as JSON. I'm using JSON2.asp so my application already supports JSON.parse & JSON.stringify but I can't get it to spit out the multi-dimensional array as JSON.
set rs = conn.execute(strQuery)
if Not rs.EOF Then
rsArray = rs.GetRows() 'This pulls in all the results of the RecordSet as a 2-dimensional array
columnCount = ubound(rsArray,1)
rowCount = ubound(rsArray,2)
For rowIndex = 0 to rowCount 'Loop through rows as the outer loop
rsArray(3,0) = "somethingelse"
Next 'Move on to next row if there is one
response.write JSON.stringify(rsArray) & " _______ "
End If
I just need to be able to go through the results of my database query, modify the results, and then output the modified results in JSON format. What's the right way to do this?

The JSON2.asp implementation doesn't have a "Load From Database" function which means you will have to implement something to convert the ADODB.Recordset to a JSON structure yourself.
If you are willing to use a different script there is an implementation by RCDMK on GitHub that does have a LoadRecordset() method, it's called JSON object class 3.5.3.
This makes loading data from an ADODB.Recordset really straightforward.
<!-- #include virtual="/jsonObject.class.asp" -->
<%
Response.LCID = 2057
'...
Dim rs: Set rs = conn.execute(strQuery)
Dim JSON: Set JSON = New JSONobject
Call JSON.LoadRecordset(rs)
Call Response.Clear()
Response.ContentType = "application/json"
Call JSON.Write()
%>
Code has been tested using a disconnected recordset, the ... here denote assumed code to setup your recordset, connection etc
It's worth noting you could write this yourself, it's not a huge leap to loop through an ADODB.Recordset and build a JSON string. However, I would argue against for a few reasons;
It is a time-consuming exercise.
Very easy to miss something (like checking for numeric data types, when generating output).
Depending on how it is coded can make it awkward to maintain (For example, if not injecting property names directly from the recordset and choosing to "hardcode" them instead).
Why reinvent the wheel ? There are a lot of public implementations in the wild that deal with the issues raised here. Admittedly, some are better than others, but it takes five minutes to include them and give it a try.
Just for completeness here is my local test code using a disconnected recordset
<!-- #include virtual="/jsonObject.class.asp" -->
<%
Call init()
Sub init()
Dim fields: fields = Array(Array("title", adVarChar, 50), Array("firstname", adVarChar, 50), Array("lastname", adVarChar, 50), Array("age", adInteger, 4))
Dim rs: Set rs = Server.CreateObject("ADODB.Recordset")
Call InsertRow(rs, fields, Array("Mr", "Joe", "Bloggs", 31))
Call InsertRow(rs, fields, Array("Mr", "John", "Smith", 42))
Response.LCID = 2057
Dim JSON: Set JSON = New JSONobject
Call JSON.LoadRecordset(rs)
Call Response.Clear()
Response.ContentType = "application/json"
Call JSON.Write()
End Sub
Sub InsertRow(ByVal rs, fields, values)
With rs
If rs.State <> adStateOpen Then
For Each fld In fields
Call .Fields.Append(fld(0), fld(1), fld(2))
Next
.CursorLocation = adUseClient
.CursorType = adOpenDynamic
Call .Open()
End If
Call .AddNew()
For i = 0 To UBound(fields, 1)
.Fields(fields(i)(0)).Value = values(i)
Next
Call .Update()
Call .MoveFirst()
End With
End Sub
%>
Output:
{"data":[{"title":"Mr","firstname":"Joe","lastname":"Bloggs","age":31},{"title":"Mr","firstname":"John","lastname":"Smith","age":42}]}

Here ya go. This works for me.
set rs = conn.execute(strQuery)
c=0
Response.write "["
Do Until rs.eof
'Assign variables here with whatever you need to change
title = rs(0)
fName = rs(1)
lName = rs(2)
empID = rs(3)
With Response
if c > 0 then .write ", "
.write "{" & chr(34) & "Title" & chr(34) & " : " & chr(34) & title & chr(34) & ", " & chr(34) & "FirstName" & chr(34) & " : " & chr(34) & fName & chr(34) & ", "
.write chr(34) & "LastName" & chr(34) & " : " & chr(34) & lName & chr(34) & ", " & chr(34) & "EmpID" & chr(34) & " : " & chr(34) & empID & chr(34) & "}"
End With
c = c + 1
rs.MoveNext
Loop
Response.write "]"
This will write your JSON object directly to the page.

try setting content-type to "application/json" on top of your asp page.
<%#LANGUAGE="VBSCRIPT" CODEPAGE="65001"%>
<%
Option Explicit
Response.Buffer=True
Response.ContentType="application/json"
Response.Charset="utf-8"
'' rest of your code.. your db operations
'' response write your json
%>

Related

Is there any way to transpose a HTML table using VBA?

I have a macro allows me send emails of monthly performance to each manager. Codes are as follow:
Sub OutlookEmailsSend()
Dim objOutlook As Outlook.Application
Dim objMail As Outlook.MailItem
Dim lCounter As Long
Dim endColumnNo As Long
Dim a As Long
Dim sFile As String
endColumnNo = ThisWorkbook.Sheets("Sheet1").UsedRange.Columns.Count
Set objOutlook = Outlook.Application
For lCounter = 2 To 3
'
Set objMail = objOutlook.CreateItem(olMailItem)
objMail.To = Sheet1.Range("B" & lCounter).Value
objMail.Subject = "Sales Summary"
sFile = "Dear,<br><br>Please refer to below table for your performance<br><br><table border=1>"
For a = 1 To endColumnNo
sFile = sFile & "<tr><td>" & Cells(1, a) & "</td><td>" & Cells(lCounter, a) & "</td></tr>"
Next
objMail.HTMLBody = sFile
objMail.Display
Set objMail = Nothing
Next
End Sub
The macro produce table like this
Dear,
Please refer to below table for your performance
Name Tom
Email sgcjack#163.com
Item Phone
Sales 123
Bonus 3213
However, I would like the table presents as follow
Name Email Item Sales Bonus
Jack jacksun#citics.com.hk Computer 342 23123
Is there any way can do this?
For the sake of better readibility it might be helpful to organize the html creation in a function and to assign the function result to objMail.HTMLBody omitting the loops.
Btw you forgot the closing table tag </table> which wouldn't result in a valid html structure. - Of course, the most direct approach following the original code would be to follow the recommendation in comment to add the <tr>..</tr> tags outside the loop not forgetting the closing </table> tag.
With Sheet1
objMail.HTMLBody = getBody(.Range("A1",.Cells(1,EndColumnNo)),"Dear xx")`
End With
The help function getBody() joins (a) headers and (b) table data based on a (c) clearly defined table structure.
Note: You can play around and change that definition to a more sophisticated html code with separate header tags, too..
Function getBody(rng As Range, _
Optional greetings As String = "", _
Optional HeaderList As String = "Name,Email,Item,Sales,Bonus")
Const Blanks As String = " "
'a) get headers
Dim headers As String
headers = " <td>" & Replace(HeaderList, ",", "</td><td>") & "</td>"
'b) join table data "<td>..</td>"
Dim data As String
data = Blanks & _
Join(rng.Parent.Evaluate("""<td>""&" & rng.Address(0, 0) & " & ""</td>""") _
, vbNewLine & Blanks)
'c) define table structure
Dim tags()
tags = Array(greetings, _
"<table border='1'>", _
" <tr>", headers, " </tr>", _
" <tr>", data, " </tr>", _
"</table>")
'd) return joined function result
getBody = Join(tags, vbNewLine)
End Function

Why is VBA code slowing down when processing larger tables

I got problem with one of my subroutines, which job is to convert any passed ListObject (ussually generated by powerquery) into multiple MySQL queries, then send them to database. Queries and progress are shown on userform, that refresh with every query. My problem is that for some reason with some large tables, code starts out very quickly, but at some point it instantly slows down to fraction of speed it started and excel ram usage is increasing by +-1MB/s while running, and after code finish, it stays there.
With smaller tables (low column count, or small values in cells) it can process tens of thousands rows very fast without slowing, but problem comes with some large tables (either higher column count, or big values in cells, for ex. long strings etc...) after like 3k rows.
This sub is responsible for looping thru table, and building insert queries, then every few rows (depending on query length) calls function, that can send any query into selected DB. The problem is in "For i" loop, but i including whole code here.
Public Sub UploadniPayload(DBtabulka As String, Zdroj As ListObject, Optional Databaze As String = "tesu")
If ErrorMode = False Then On Error Resume Next
Dim Prikaz As String, Radek As String, Payload As String, i As Long, x As Long, PocetRadku As Long, PocetSloupcu As Long, DBsloupce As Long
Call VyplnNetInfo(DBIP)
AutoUploader.loading_sql.Value = 0
PocetRadku = Zdroj.DataBodyRange.Rows.Count
PocetSloupcu = Zdroj.DataBodyRange.Columns.Count
DBsloupce = DBPocetSloupcu(DBtabulka, Databaze)
If JeTabulkaPrazdna(Zdroj) = False Then
If (Zdroj.DataBodyRange.Columns.Count + 1) = DBsloupce Then
'PROBLEM APPEARING IN THIS LOOP
For i = 1 To PocetRadku
For x = 1 To PocetSloupcu
If x <= 0 Then Exit For
If x = 1 Then
Payload = "'','" & Zdroj.DataBodyRange(i, x).Text & "'"
Else
Payload = Payload & ",'" & Zdroj.DataBodyRange(i, x).Text & "'"
End If
Next x
Radek = "(" & Payload & ")"
If Prikaz <> vbNullString Then Prikaz = Prikaz & ", " & Radek Else Prikaz = Radek
If i = PocetRadku Or Len(Prikaz) > 2500 Then
AutoUploader.loading_sql.Value = i / PocetRadku
AutoUploader.txtStatus.Caption = "Zpracovávám " & i & "/" & PocetRadku & " řádků"
Prikaz = "INSERT INTO `" & Databaze & "`.`" & DBtabulka & "` VALUES " & Prikaz
Call PrikazSQL(Prikaz, Databaze)
Prikaz = vbNullString
Payload = vbNullString
End If
Next i
Else
Call Zaloguj("System", "Error - počet sloupců v " & Zdroj.Name & " (" & PocetSloupcu & "+1 ID) nesouhlasí s počtem sloupců v " & DBtabulka & "(" & DBsloupce & ")", False)
End If
Else
Call Zaloguj("System", "Error - pokus o upload prázdné tabulky (" & Zdroj.Name & ") do DB (" & DBtabulka & ")", False)
End If
If AutoUploader.chb_Uklizecka.Value = True Then Call VycistiTabulku(Zdroj)
End Sub
And this is my function responsible for sending queries into database.
Sometimes i use it for pulling single value from database, so it acts as string, but when i need only insert, i just using Call. DBIP, DBUser and DBPass are global variables.
Public Function PrikazSQL(ByRef Prikaz As String, Optional Databaze As String = "tesu") As String
On Error GoTo ErrHandler
AutoUploader.IconDirectSQL.BackColor = vbGreen
AutoUploader.txtKUK.Value = Prikaz
'If ErrorMode = True Then Call Zasifruj
DoEvents
Dim Pripojeni As ADODB.Connection, RS As ADODB.Recordset
Set Pripojeni = CreateObject("ADODB.Connection")
Set RS = CreateObject("ADODB.RecordSet")
Pripojeni.Open "" & _
"DRIVER={MySQL ODBC 8.0 UNICODE Driver}" & _
";SERVER=" & DBIP & _
";DATABASE=" & Databaze & _
";USER=" & DBUser & _
";PASSWORD=" & DBPass & _
";Option=3"
With RS
.CursorLocation = adUseClient
.LockType = adLockBatchOptimistic
.Open Prikaz, Pripojeni
.ActiveConnection = Nothing
End With
Pripojeni.Close
Set Pripojeni = Nothing
If RS.Fields.Count > 0 Then PrikazSQL = RS(0)
Set RS = Nothing
AutoUploader.IconDirectSQL.BackColor = vbWhite
DoEvents
Exit Function
ErrHandler:
RS.ActiveConnection = Nothing
If Not Pripojeni Is Nothing Then
Pripojeni.Close
Set Pripojeni = Nothing
End If
If RS.Fields.Count > 0 Then PrikazSQL = RS(0)
Set RS = Nothing
AutoUploader.IconDirectSQL.BackColor = vbWhite
DoEvents
Call Debuger("ERROR:" & vbCrLf & Err.Description & vbCrLf & vbCrLf & "QUERY:" & vbCrLf & Prikaz, "PrikazSQL")
End Function
Code above is only part of the autonomous bot, on start it apply these settings:
Application.DisplayAlerts = False
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
Application.EnableEvents = False
DoEvents is used only for refreshing userform, instead of repaint.
I try to unload any object or variable, that i dont need, but i think i am missing something important. Any other part of code runs fine. Any help would be very appreciated.

VBA returning a value and error information together

I am writing some VBA in MS Access, although the principle of my question would apply just as well to Excel or Word VBA. I have written a function GetStringParameterFromTable which returns a string value. It is possible that the function may result in a VBA-generated error, despite my best efforts to write it so that it does not. If an error happens, I don't want the code to crash, so I must use error handling. However, I don't want the code to display an error message and stop within the function if there is an error. I want the function to finish executing and return control to the calling procedure, and then I want the calling procedure to display the error message and tidy up, e.g. close open files. My question is: how does the calling procedure know that there has been an error in the function it called, and how does it get the error message?
I have thought of three ways of implementing this:
(1) Make GetStringParameterFromTable into a Sub, and pass it ParameterValue, ErrorFlag and ErrorMessage by reference.
(2) Keep GetStringParameterFromTable as a Function, define ErrorFlag and ErrorMessage as global variables and have the function alter ErrorFlag and ErrorMessage.
(3) Keep GetStringParameterFromTable as a Function and define a type with three components – ParameterValue, ErrorFlag and ErrorMessage – and make GetStringParameterFromTable return a value of the type I have defined.
I think that my requirement must be quite common, but I can’t find any examples of how it’s implemented. Does anyone have any views on which of my suggestions is the best way, or whether there is a better way that I haven’t thought of?
I have been contemplating the same thing since C#.net has implemented Tuples. I have implemented Tuples using VBA's type to create my tuples. What I have done is the following:
Public Type myTuple
Value as String 'Or whatever type your value needs to be
ErrCode as Long
ErrDesc as String
End Type
Public Function DoWork (ByRef mObject as MyClass) as myTuple
Dim retVal as myTuple
'Do whatever work
If Err.Number <> 0 then
retVal.Value = Nothing
retVal.ErrNumber = Err.Number
retVal.ErrDesc = Err.Description
Else
Set retVal.Value = Whatever Makes Sense
retVal.ErrNumber = 0
retVal.ErrDesc = VbNullString
End If
DoWork = retVal
End Function
I would like to be more specific, but you didn't provide a code example.
I am doing it like this and log the errors in a table:
' Lookups Replacements
'---------------------
Function DLook(Expression As String, Domain As String, Optional Criteria) As Variant
On Error GoTo Err_Handler
Dim strSQL As String
strSQL = "SELECT " & Expression & " FROM " & Domain 'DLookup
'DCount: strSQL = "SELECT COUNT(" & Expression & ") FROM " & Domain
'DMax: strSQL = "SELECT MAX(" & Expression & ") FROM " & Domain
'DMin: strSQL = "SELECT SUM(" & Expression & ") FROM " & Domain
'DFirst: strSQL = "SELECT FIRST(" & Expression & ") FROM " & Domain
'DLast: strSQL = "SELECT LAST(" & Expression & ") FROM " & Domain
'DSum: strSQL = "SELECT SUM(" & Expression & ") FROM " & Domain
'DAvg: strSQL = "SELECT AVG(" & Expression & ") FROM " & Domain
If Not IsMissing(Criteria) Then strSQL = strSQL & " WHERE " & Criteria
DLook = DBEngine(0)(0).OpenRecordset(strSQL, dbOpenForwardOnly)(0)
Exit Function
Err_Handler:
'Can be made as Error Sub as well
Dim ErrNumber as Integer
Dim ErrDescription as String
ErrNumber = Err.Number
ErrDescription = Err.Description
Err.Clear
On Error Resume Next
Dim strSQL as String
strSQL = "INSERT INTO tblErrorLog (ErrorNumber, ErrorDescription) VALUES (" & ErrNumber & ", '" & ErrDescription & "')"
Currentdb.Excecute strSQL, dbFailOnError
End Function
Called with:
If DLook("Column2", "Table1", "Column1 = " & ID) = 0 Then
'Do stuff
End If
If DLook("Column2", "Table1") = 0 Then
'Do other stuff
End If

Get value of node in MS Access using JSOn

I am using VB-JSON parser to get data from an API and saving the data in MS access table.
I am not able to figure out how to access the -KLj9kXnKd-9txfyIqM8 and -KLjJoT7gXCMq_jHx2_z.
I have Table structure as below, and want to save the data as shown below.
|ServerID |Name |Mobile
|-KLj9kXnKd-9txfyIqM8 |Adarsh |9987
|-KLjJoT7gXCMq_jHx2_z |Manas |022
JSON
{
"-KLj9kXnKd-9txfyIqM8": {
"personmobile": "9987",
"personname": "Adarsh"
},
"-KLjJoT7gXCMq_jHx2_z": {
"personmobile": "022",
"personname": "Manas"
}
}
VBA
Public Sub GetPerson()
'I have code here which gets the json as above from api.
Dim egTran As String
If reader.Status = 200 Then
Set db = CurrentDb
Set rs = db.OpenRecordset("tblPerson", dbOpenDynaset, dbSeeChanges)
egTran = "[" & reader.responseText & "]"
Set coll = Json.parse(egTran)
For Each contact In coll
rs.AddNew
rs!Name = contact.Item("personname")
rs!Mobile = contact.Item("personmobile")
rs!ServerID = contact.Item("??????")
what do I write in ??????
rs.Update
Next
End If
End Sub
I am also open to using any other parser. The API is based on Firebase Database
I'm not familiar with VB-JSON, but obviously "??????" is not an item of contact.
Thus, if I run this test function:
Public Sub TestJsonText()
Dim DataCollection As Collection
Dim ResponseText As String
ResponseText = _
"{" & _
" ""-KLj9kXnKd-9txfyIqM8"": {" & _
" ""personmobile"": ""9987""," & _
" ""personname"": ""Adarsh""" & _
" }," & _
" ""-KLjJoT7gXCMq_jHx2_z"": {" & _
" ""personmobile"": ""022""," & _
" ""personname"": ""Manas""" & _
" }" & _
"}"
If ResponseText <> "" Then
Set DataCollection = CollectJson(ResponseText)
MsgBox "Retrieved" & Str(DataCollection.Count) & " root member(s)", vbInformation + vbOKOnly, "Web Service Success"
End If
Call ListFieldNames(DataCollection)
Set DataCollection = Nothing
End Sub
using the Json modules from VBA.CVRAPI it will print:
root
-KLj9kXnKd-9txfy
personmobile 9987
personname Adarsh
-KLjJoT7gXCMq_jH
personmobile 022
personname Manas
From the function ListFieldNames you can pick MemberName for the field name and DataCollection(Index)(CollectionItem.Data) for the field value to add records.

VBA read CSV with delimiter in string

I'm trying to read a .csv to work with it in an .accdb
The file has ; as delimiter and "" as string qualifier.
Young and naive as I was I just split the file at the delimiter:
Set oFSO = New FileSystemObject
Set oStream = oFSO.OpenTextFile(sFilePath, ForReading)
Do Until oStream.AtEndOfStream
sLine = oStream.ReadLine
sArray = Split(sLine, ";")
....
Now I got a line that reads:
"String";"Str;ing";0;0;0;"String"
So I have delimiter inside one of the strings which makes the code above not work. Any ideas how to solve this?
EDIT:
I've found someone with a similar problem, only with a comma as delimiter. And they solved it using regular expressions.
The problem: I'm absolutely not good with regular expressions. In the example the used this expression and code:
Function regLine(sLine As String) As String
Dim oRegEx As RegExp
Set oRegEx = New RegExp
oRegEx.IgnoreCase = True
oRegEx.Global = True
' Pattern: ",(?=([^"]*"[^"]*")*(?![^"]*"))"
oRegEx.Pattern = ",(?=([^" & Chr(34) & "]*" & Chr(34) & "[^" & Chr(34) & "]*" & Chr(34) & ")*(?![^" & Chr(34) & "]*" & Chr(34) & "))"
regLine = oRegEx.Replace(sLine, ";")
End Function
So I don't really understand the expression. My first idea was to replace the comma with a semicolon but that didn't work.
Option Explicit
Dim line
line ="""String"";""Str;ing"";0;0;0;""String"""
WScript.Echo line
Dim aFields
With New RegExp
.Pattern = "(""[^""]*"")?;"
.Global = True
aFields = Split(.Replace(line, "$1"&Chr(0)),Chr(0))
End With
Dim field
For Each field In aFields
WScript.Echo field
Next
Code is .vbs, but shows how to use the regular expression to replace semicolons not enclosed in quotes with a null character and use the null character to split the line into its fields.
I solved the problem now by writing a loop, that deletes the delimiter if it is in a string.
Function fixLine(sLine As String)
Dim i As Long
Dim bInString As Boolean
bInString = False
fixLine = ""
For i = 1 To Len(sLine)
If Mid(sLine, i, 1) = Chr(34) Then
If bInString Then
bInString = False
Else
bInString = True
End If
End If
If bInString And Mid(sLine, i, 1) = ";" Then
Else
fixLine = fixLine & Mid(sLine, i, 1)
End If
Next
End Function
It kind of feels quick and dirty and I'm not sure about the performance but it works.
EDIT:
I also worked with theabove example I found. It replaces the delimiter in a line outside of strings. So I replaced the delimiter with Chr(0) which I know won't apear in a line and then split at the new delimiter.
Function regLine(sLine As String) As String()
Dim oRegEx As RegExp
Dim sLine2() As String
Set oRegEx = New RegExp
oRegEx.Global = True
'Pattern: ";(?=([^"]*"[^"]*")*(?![^"]*"))"
oRegEx.Pattern = ";(?=([^" & Chr(34) & "]*" & Chr(34) & "[^" & Chr(34) & "]*" & Chr(34) & ")*(?![^" & Chr(34) & "]*" & Chr(34) & "))"
sLine2 = oRegEx.Replace(sLine, Chr(0))
regLine = Split(sLine2, Chr(0))
End Function
My first question is: Is there any case where a ";" in the string values is a valid string? If so, I don't see any way other than manually verifying the data.
If not, how large is the input file? If it's not too big (for various definitions of "too" :-) ) then just manually scan it for errors.
If it is very large, I'd simple write a preprocesser program that reads the string values then deletes any ";" in those where it occurs. Such a program is only about a dozen lines long. Then run the clean file into Access.