Anybody have some VBA code that will store a byte array into a MySQL blob column?
Here is some code. Requires a reference to Microsoft Active Data Objects 2.x Library. It uses the OLE DB provider for MySQL (Might need to install that on the client machine).
Sub StoreBLOB(data() As Byte, key As Double)
'stores the BLOB byte array into the row identified by the key
'requires reference to Microsoft Active Data Objects 2.x Library
On Error GoTo handler:
Dim con As New ADODB.Connection
Dim rs As New ADODB.Recordset
Dim conStr As String
Dim strSQL As String
'have it return only the record you want to store your blob
strSQL = strSQL & "SELECT * FROM YOURTABLE WHERE KEY = " & key
'setup connection
conStr = conStr & "Provider=MySQLProv;"
conStr = conStr & "Data Source=mydb;"
conStr = conStr & "User Id=myUsername;"
conStr = conStr & "Password=myPassword;"
con.ConnectionString = conStr
con.Open
rs.Open strSQL, con, adOpenDynamic, adLockOptimistic
If rs.RecordCount > 1 Then
Err.Raise 1001, "StoreBLOB", "Too many records returned from dataset. Check to make sure you have the right key value"
Else
Err.Raise 1002, "StoreBLOB", "No Records found that match the key"
End If
rs.Fields("BLOBFIELDNAME").Value = data
rs.Update 'store the contents to the database
rs.Close
con.Close
Set rs = Nothing
Set con = Nothing
Exit Sub
handler:
Err.Raise 1003, "StoreBLOB", "Unexpected Error in StoreBLOB. Check that server is running"
End Sub
Assuming you are using ADO to access mysql, there's a KB article on the subject.
I have some code, I replicated the mysql_real_escape_string_quote C function in VBA so that one can escape the necessary characters and build your SQL as you would for regular text:
Function mysql_real_escape_string_quote(toStr() As Byte, fromStr() As Byte, length As Long, quote As String) As Long
mysql_real_escape_string_quote = 0
Dim CharMap() As Byte: CharMap = StrConv(String(256, 0), vbFromUnicode)
CharMap(0) = Asc("0"): CharMap(39) = Asc("'"): CharMap(34) = Asc(""""): CharMap(8) = Asc("b"): CharMap(10) = Asc("n"): CharMap(13) = Asc("r"):
CharMap(9) = Asc("t"): CharMap(26) = Asc("z"): CharMap(92) = Asc("\"): CharMap(37) = Asc("%"): CharMap(95) = Asc("_"):
Dim i As Long: Dim n As Long: n = 0
If length > UBound(fromStr) + 1 Then Exit Function
For i = 0 To length - 1 '---count escapable chars before redim---
n = n + 1
If CharMap(fromStr(i)) <> 0 Then n = n + 1
Next i
ReDim toStr(n - 1) As Byte
n = 0
For i = 0 To length - 1 '---test chars---
If CharMap(fromStr(i)) = 0 Then
toStr(n) = fromStr(i)
Else '---escape char---
toStr(n) = Asc(quote): n = n + 1
toStr(n) = CharMap(fromStr(i))
End If
n = n + 1
Next i
mysql_real_escape_string_quote = n
End Function
Function mysql_real_escape_string(InputString As String) As String
mysql_real_escape_string = ""
Dim toStr() As Byte: Dim fromStr() As Byte
fromStr = StrToChar(InputString)
If mysql_real_escape_string_quote(toStr, fromStr, UBound(fromStr) + 1, "\") = 0 Then Exit Function
mysql_real_escape_string = StrConv(toStr(), vbUnicode)
End Function
Function StrToChar(str As String) As Byte()
Dim ans() As Byte
ans = StrConv(str, vbFromUnicode)
ReDim Preserve ans(Len(str)) As Byte
ans(Len(str)) = 0
StrToChar = ans
End Function
Sub testit()
Dim toStr() As Byte: Dim fromStr() As Byte
fromStr = StrToChar("hello world's")
MsgBox (mysql_real_escape_string_quote(toStr, fromStr, UBound(fromStr) + 1, "\"))
MsgBox (mysql_real_escape_string("hello world's"))
For i = 0 To UBound(toStr)
Debug.Print i & " " & toStr(i)
Next i
End Sub
It's been optimized for large amounts of data without a ridiculous amount of conditionals (ifs).
Related
I have adapted some code I found to extract a mySQL table and write it to a worksheet. However, it is slow for some of the larger tables(30,000+). I am trying to find a better way to import the values and avoid looping. I was hoping to be able to assign it directly to a range, but have been unsuccessful. From my research, it seems Excel is limited when it comes to mySQL. Any suggestions?
Dim password As String
Dim sqlstr As String
Dim dbTable As String
'OMIT Dim Cn statement
Dim server_Name As String
Dim user_ID As String
Dim database_Name As String
Dim lRow As Integer, lCol As Integer
'Start timer
Dim Count As Long
Dim BenchMark As Double
BenchMark = Timer
Application.ScreenUpdating = False
Application.EnableEvents = False
'OMIT Dim rs statement
Set rs = CreateObject("ADODB.Recordset") 'EBGen-Daily
server_Name = Sheet10.Range("b1").Value
database_Name = Sheet10.Range("b2").Value ' Name of database
user_ID = Sheet10.Range("b3").Value 'id user or username
password = Sheet10.Range("b4").Value 'Password
dbTable = Sheet10.Range("tbl_name").Value
sqlstr = "SELECT * FROM " & dbTable
Set cn = New ADODB.Connection
'On Error Goto ErrorHandler
cn.Open "DRIVER={MySQL ODBC 8.0 Unicode Driver};" & _
"SERVER=" & server_Name & ";" & _
"DATABASE=" & database_Name & ";" & _
"USER=" & user_ID & ";" & _
"PASSWORD=" & password & ";" & _
"Option=16427"
rs.Open sqlstr, cn, adOpenStatic
'MsgBox cn.Execute("SELECT COUNT(*) As row_count FROM elite_advocacy;")!row_count + 1
Dim myArray()
myArray = rs.GetRows()
kolumner = UBound(myArray, 1)
rader = UBound(myArray, 2)
'Delete existing table
On Error Resume Next
Sheet2.ListObjects("tbl_data").Delete
On Error GoTo 0
'Write array to sheet <<< Slow for large datasets
For k = 0 To kolumner ' Using For loop data are displayed
Sheet2.Range("rng_s_data").Offset(0, k).Value = rs.Fields(k).Name
For r = 0 To rader
Sheet2.Range("rng_s_data").Offset(r + 1, k).Value = myArray(k, r)
Next
Next
'Write array to range <<< Failed
'Attempt 2
'Dim r1 As Range, rBase As Range
'Dim L As Long, U As Long
'Set rBase = Sheet2.Range("rng_s_data")
'L = LBound(myArray)
'U = UBound(myArray)
'r1 = rBase.Resize(1, rader - kolumner + 1)
'r1 = myArray
'Find lRow and lCol
lRow = Cells(Rows.Count, Range("rng_s_data").Column).End(xlUp).Row
lCol = Cells(Range("rng_s_data").Row, Columns.Count).End(xlToLeft).Column
'Create a table from Data
'Sheet2.ListObjects.Add(xlSrcRange, Sheet2.Range("A$5:$Z$100"), , xlYes).Name = "tbl_data"
Sheet2.ListObjects.Add(xlSrcRange, Sheet2.Range(Sheet2.Cells(Sheet2.Range("rng_s_data").Row, Sheet2.Range("rng_s_data").Column), _
Sheet2.Cells(lRow, lCol)), , xlYes).Name = "tbl_data"
Sheet2.ListObjects("tbl_data").TableStyle = "TableStyleLight1"
'Autofit Sheet
Cells.EntireColumn.AutoFit
Application.ScreenUpdating = True
Application.EnableEvents = True
'End Timer
MsgBox Timer - BenchMark
Errorhandler:
rs.Close
Set rs = Nothing
cn.Close
Set cn = Nothing
End Sub
I don't have mySQl for testing, but something like this would be a generic approach to querying any database from Excel using ADO.
Performance is optimum if you avoid any looping which involves cell-by-cell access, and do as much as you can with arrays, before transferring the final array to the worksheet in a single operation.
It's worth putting in extra effort to create re-usable pieces of code as standalone Subs or Functions - that allows your main logic to stay focused on the task at hand.
Sub Tester()
Dim cn As ADODB.Connection, rs As ADODB.Recordset
Dim sql As String, dbTable As String, data, rngTbl As Range
Dim BenchMark As Double
BenchMark = Timer
Set cn = GetConnection()
Set rs = New ADODB.Recordset
dbTable = Sheet10.Range("tbl_name").Value
sql = "SELECT * FROM " & dbTable
rs.Open sql, cn, adOpenStatic
data = RecordSetToArray(rs) 'Includes field names
'data = RecordSetToArray(rs,False) 'False = no field names
'Delete existing table
On Error Resume Next
sheet2.ListObjects("tbl_data").Delete
On Error GoTo 0
'put the data on the worksheet
Set rngTbl = ArrayToSheetRange(data, sheet2.Range("rng_s_data"))
With sheet2.ListObjects.Add(xlSrcRange, rngTbl, , xlYes)
.Name = "tbl_data"
.TableStyle = "TableStyleLight1"
.Range.EntireColumn.AutoFit
End With
Debug.Print "Done in " & Timer - BenchMark
End Sub
'return an opened connection object
Function GetConnection() As ADODB.Connection
Dim serverNm As String, userId As String, dbNm As String, pw As String
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
serverNm = Sheet10.Range("b1").Value
dbNm = Sheet10.Range("b2").Value ' Name of database
userId = Sheet10.Range("b3").Value 'id user or username
pw = Sheet10.Range("b4").Value 'Password
cn.Open "DRIVER={MySQL ODBC 8.0 Unicode Driver};" & _
"SERVER=" & serverNm & ";" & _
"DATABASE=" & dbNm & ";" & _
"USER=" & userId & ";" & _
"PASSWORD=" & pw & ";" & _
"Option=16427"
Set GetConnection = cn
End Function
'Create a 2-D array from a recordset
Function RecordSetToArray(rs As ADODB.Recordset, _
Optional IncludeFieldNames As Boolean = True)
Dim tmp, nC As Long, nR As Long, data, r As Long, c As Long, rowNum As Long
tmp = rs.GetRows() 'cols x rows
nC = UBound(tmp, 1) + 1 'zero-based --> 1-based
nR = UBound(tmp, 2) + 1
ReDim data(1 To nR + IIf(IncludeFieldNames, 1, 0), 1 To nC) 'allow for headers?
If IncludeFieldNames Then
For c = 1 To nC
data(1, c) = rs.Fields(c - 1).Name
Next c
rowNum = 1
End If
For r = 1 To nR
rowNum = rowNum + 1
For c = 1 To nC
data(rowNum, c) = tmp(c - 1, r - 1)
Next c
Next r
RecordSetToArray = data
End Function
'Fill an array to a worksheet starting at `rng`, and return the filled range
Function ArrayToSheetRange(data, rng As Range) As Range
Dim rv As Range
Set rv = rng.Cells(1).Resize(UBound(data, 1), UBound(data, 2))
rv.Value = data
Set ArrayToSheetRange = rv
End Function
i am trying to get the frequency of terms within a collection of variable length strings.The context is descriptions in an Access database. Would prefer to keep the solution in VBA. Delimiter is " " (space) character
Dim db As DAO.Database
Set db = CurrentDb()
Call wordfreq
End Sub
Function wordfreq()
Dim myCol As Collection
Dim myArray() As String
Dim strArray As Variant
Dim strDescr, strTerm, strMsg As String
Dim i, j As Integer
Set myCol = New Collection
strDescr = "here it should accept the table and display the result in seperate table"
' db.Execute "select columns from table"
myArray = Split(strDescr, " ")
For Each strArray In myArray
On Error Resume Next
myCol.Add strArray, CStr(strArray)
Next strArray
For i = 1 To myCol.Count
strTerm = myCol(i)
j = 0
For Each strArray In myArray
If strArray = strTerm Then j = j + 1
Next strArray
'placeholder
strMsg = strMsg & strTerm & " --->" & j & Chr(10) & Chr(13)
Next i
'placeholder
'save results into a table
MsgBox strMsg
End Function
See an example below using a Scripting.Dictionary object.
Function wordfreq()
Dim objDict As Object
Dim myArray() As String
Dim strInput As String
Dim idx As Long
Set objDict = CreateObject("Scripting.Dictionary")
strInput = "here it should accept the table and display the result in seperate table"
myArray = Split(strInput, " ")
For idx = LBound(myArray) To UBound(myArray)
If Not objDict.Exists(myArray(idx)) Then
'Add to dictionary with a count of 1
objDict(myArray(idx)) = 1
Else
'Increment counter
objDict(myArray(idx)) = objDict(myArray(idx)) + 1
End If
Next
'Test it
Dim n As Variant
For Each n In objDict.Keys
Debug.Print "Word: " & n, " Count: " & objDict(n)
Next
End Function
Output:
'Word: here Count: 1
'Word: it Count: 1
'Word: should Count: 1
'Word: accept Count: 1
'Word: the Count: 2
'Word: table Count: 2
'Word: and Count: 1
'Word: display Count: 1
'Word: result Count: 1
'Word: in Count: 1
'Word: seperate Count: 1
Edit
The process:
Loop through the Input recordset.
Split the Description into words.
Check if the word exist in Dictionary and add or
increment.
Add the Keys (words) and Values (count) of the aforementioned
Dictionary to the Output table.
To achieve this two helper functions have been set up:
One loops through the description recordset and returns a
Dictionary object filled with unique words as Keys and their
count as Values.
The other takes the above Dictionaryobject and adds it to the Output table.
You need to change [TABLE] to the name of your Input and Output tables.
Option Explicit
Sub WordsFrequency()
On Error GoTo ErrTrap
Dim rs As DAO.Recordset
Set rs = CurrentDb().OpenRecordset("SELECT Description FROM [TABLE] WHERE Description Is Not Null;", dbOpenSnapshot)
If rs.EOF Then GoTo Leave
With rs
.MoveLast
.MoveFirst
End With
If AddDictionaryToTable(ToDictionary(rs)) Then
MsgBox "Completed successfully.", vbInformation
End If
Leave:
If Not rs Is Nothing Then rs.Close
Set rs = Nothing
On Error GoTo 0
Exit Sub
ErrTrap:
MsgBox Err.Description, vbCritical
Resume Leave
End Sub
' Returns Scripting.Dictionary object
Private Function ToDictionary(rs As DAO.Recordset) As Object
Dim d As Object 'Dictionary
Dim v As Variant 'Words
Dim w As String 'Word
Dim i As Long, ii As Long 'Loops
Set d = CreateObject("Scripting.Dictionary")
For i = 1 To rs.RecordCount
v = Split(rs![Description], " ")
For ii = LBound(v) To UBound(v)
w = Trim(v(ii))
If Not d.Exists(w) Then d(w) = 1 Else d(w) = d(w) + 1
Next
rs.MoveNext
Next
Set ToDictionary = d
End Function
' Adds Dictionary object to table
Private Function AddDictionaryToTable(objDict As Object) As Boolean
On Error GoTo ErrTrap
Dim rs As DAO.Recordset
Dim n As Variant
Set rs = CurrentDb().OpenRecordset("[TABLE]")
With rs
For Each n In objDict.Keys
.AddNew
.Fields("Words").Value = n
.Fields("Counts").Value = objDict(n)
.Update
Next
End With
'all good
AddDictionaryToTable = True
Leave:
If Not rs Is Nothing Then rs.Close
Set rs = Nothing
On Error GoTo 0
Exit Function
ErrTrap:
MsgBox Err.Description, vbCritical
Resume Leave
End Function
So I have this SQL Query
SELECT *
FROM [Employee To Manager]
WHERE [Employee To Manager].[Manager UID] In(getMyTeamUserNames());
Which has a VBA function getMyTeamUserNames()
Public Function getMyTeamUserNames() As String
Dim rs As DAO.Recordset
Dim dbs As DAO.Database
Set dbs = CurrentDb
getMyTeamUserNames = commaDelimitArray(getTeamUserNames(getUserName, dbs))
End Function
Public Function commaDelimitArray(arrayStr) As String
Dim sepStr As String
sepStr = "','"
commaDelimitArray = "'" & Join(arrayStr, sepStr)
End Function
Public Function getTeamUserNames(username, dbs) As String()
Dim sqlstatement As String
sqlstatement = "SELECT * FROM [Employee to Manager] WHERE [Employee to
Manager].[Manager UID] = '" & username & "'"
Set rs = dbs.OpenRecordset(sqlstatement, dbOpenSnapshot)
Dim ComputerUsernames() As String
Dim FindRecordCount As Integer
If rs.EOF Then
FindRecordCount = 0
Exit Function
Else
rs.MoveLast
FindRecordCount = rs.RecordCount
End If
ReDim ComputerUsernames(FindRecordCount) As String
Dim i As Integer
i = 0
rs.MoveFirst
Do Until rs.EOF = True
ComputerUsernames(i) = rs("Computer Username")
If (ComputerUsernames(i) <> "") Then
i = i + 1
End If
If (ComputerUsernames(i - 1) <> username) Then
Dim recurResult() As String
recurResult = getTeamUserNames(ComputerUsernames(i - 1), dbs)
Dim resultSize As Integer
If Len(Join(recurResult)) > 0 Then
resultSize = UBound(recurResult) - LBound(recurResult) + 1
ReDim Preserve ComputerUsernames(UBound(ComputerUsernames) + resultSize)
For Each resultStr In recurResult
ComputerUsernames(i) = resultStr
If (ComputerUsernames(i) <> "") Then
i = i + 1
End If
Next resultStr
End If
End If
rs.MoveNext
Loop
ReDim Preserve ComputerUsernames(i - 1)
getTeamUserNames = ComputerUsernames
End Function
Query runs and I get no data.
However if I take the result from getMyTeamUserNames() and put it in the query by hand it works. getMyTeamUserNames() result varies from possibly 2 results to 40 (recursively gets subordinates all the way down the tree).
So a C Perkins specifically pointed out this would never work so I have rebuilt the query with some other queries.
Given below is the working code. Previously I was using the .Name property that didn't work.
Previous code:
For Each s In rs.Fields
word = Replace(strArray(count), """", "")
count = count + 1
'the below line shows error
s.Name = word
Next
New Complete working code. It opens a dialog for user to select the .csv file and then imports all the data into the table from that csv file.
strMsg = "Select the file from which you want to import data"
mypath = GetPath(strMsg, True)
mypath = mypath
Dim strFilename As String: strFilename = mypath
Dim strTextLine As String
Dim strArray() As String
Dim count As Integer
Dim regex As New RegExp
regex.IgnoreCase = True
regex.Global = True
'This pattern matches only commas outside quotes
'Pattern = ",(?=([^"]*"[^"]*")*(?![^"]*"))"
regex.Pattern = ",(?=([^""]*""[^""]*"")*(?![^""]*""))"
Dim iFile As Integer: iFile = FreeFile
Open strFilename For Input As #iFile
Dim db As DAO.Database
Dim rs As DAO.Recordset
Set db = CurrentDb
count = 0
Do Until EOF(1)
Line Input #1, strTextLine
count = 0
'regex.replaces will replace the commas outside quotes with <???> and then the
'Split function will split the result based on our replacement
On Error GoTo ErrHandler
strTextLine = regex.Replace(strTextLine, "<???>")
strArray = Split(regex.Replace(strTextLine, "<???>"), "<???>")
Set rs = db("AIRLINES").OpenRecordset
Dim word As Variant
With rs
.AddNew
For Each s In rs.Fields
word = Replace(strArray(count), """", "")
count = count + 1
'the below line shows error
s.Value = word
Next
.Update
.Close
End With
lpp:
Loop
db.Close
Close #iFile
MsgBox ("Imported Successfully")
Exit Sub
ErrHandler:
Resume lpp
Don't use the Name property. Use Value.
How are you populating the array? If it has base index of 0, then increment Count after setting the field value.
Ok so i have a complex reason field from one of our logging servers, and i need to break it down to make some sense, problem is the format changes depending on the status.
I managed to find some strings that i can compare the the reason to to get some sense out of it, but I want to distill it down to one reason code.
I scratched my head a bit and got it down to 7 reasons with different criterion, put the criteria in a table and came up with some vb code to do the comparison.
Problem is its dead slow, and half the reporting relies on the Reason code. The basic VBA function is below, This basically loads the criteria into an array and then compares the value against the array to return the ID.
Function Reason_code(LongReason As String) As Integer
Dim NoReason As Integer
Dim I As Integer
Dim J As Integer
Dim x As Boolean
NoReason = recordCount("RejReason") - 1
Dim conExpr() As String
ReDim conExpr(NoReason)
For I = 0 To (NoReason - 1)
conExpr(I) = GetVal("Criterior", "RejReason", "id", CStr(I + 1))
Next I
For J = 0 To (NoReason - 1)
x = LongReason Like conExpr(J)
If x = True Then
GoTo OutOfLoop
End If
Next J
OutOfLoop:
Reason_code = J + 1
End Function
I have used similar in VB before and it tends to be quite fast, so i am reconing that my GetVal function is the problem, but my VBA is rusty and my SQL is pretty non existent, so any help would be appreciated. I tried LSQL and SQL2 as one line but VBA doesnt like it.
Function GetVal(FieldNm As String, TableNm As String, IndexField As String, IndexNo As String) As String
Dim db As Database
Dim Lrs As DAO.Recordset
Dim LSQL As String
Dim LGST As String
Dim SQL2 As String
'Open connection to current Access database
Set db = CurrentDb()
'Create SQL statement to retrieve value from GST table
LSQL = CStr("SELECT " + FieldNm + " FROM " + TableNm)
SQL2 = CStr(LSQL + " WHERE " + IndexField + " = " + IndexNo)
Set Lrs = db.OpenRecordset(SQL2, dbOpenDynaset, dbReadOnly)
'Retrieve value if data is found
If Lrs.EOF = False Then
LGST = Lrs(0)
Else
LGST = "Item Not found"
End If
Lrs.Close
Set Lrs = Nothing
GetVal = LGST
End Function
Thanks in advance,
I Scratched my head for a bit and worked out i could speed it up by doing the read and compare at the same time, its not lightning, but its better
Function ReasonCode(LongReason As String) As String
Dim cdb As Database
Dim rs As DAO.Recordset
Dim RejRea()
Dim NoReason As Integer
Dim result As Boolean
Dim i As Integer
Set cdb = CurrentDb()
Set rs = cdb.OpenRecordset("RejReason", dbOpenDynaset, dbReadOnly)
rs.MoveLast
rs.MoveFirst
NoReason = rs.recordCount - 1
RejRea() = rs.GetRows(rs.recordCount)
For i = 0 To NoReason
result = LongReason Like CStr(RejRea(2, i))
If result = True Then
ReasonCode = CStr(RejRea(1, i))
GoTo outloop
End If
Next i
If ReasonCode = "" Then ReasonCode = "Not Found"
outloop:
Set rs = Nothing
Set cdb = Nothing
End Function
Still not sure its the best way to do it, but in the abscence of any other suggestions it will do for now.