I need to remove/replace a few troublesome characters in an imported table. The following is my current approach; fine for <5K record sets, but painfully slow on larger sets >20K records.
I'm guessing there are better ways to accomplish than stepping thru each record and performing a Replace:
Set rs = db.OpenRecordset("TblMatch")
Do While Not rs.EOF
strDesc = Replace(Nz(rs!DESC), Chr(39), Chr(39) & Chr(39))
strDesc = Replace(Nz(rs!DESC), Chr(34), "")
strDesc = Replace(Nz(rs!Validated_DESC), Chr(39), Chr(39) & Chr(39))
strDesc = Replace(Nz(rs!Validated_DESC), Chr(34), "")
rs.MoveNext
Loop
Would the following be more efficient?
db.Execute "UPDATE TblMatch SET TblMatch.Desc = Replace([Desc],Chr(39),Chr(39) & Chr(39));"
Any suggestions or pointers?
Thank you!
Executing a SQL statement will almost always be quicker than iterating over and modifying the content of individual records in a Recordset using VBA.
As such, you could perform the operations of your code using the following SQL update statement:
update
tblmatch t
set
t.desc = replace(replace(t.desc, "'", "''"), """", ""),
t.validated_desc = replace(replace(t.validated_desc, "'", "''"), """", "")
where
t.desc like "*['""]*" or t.validated_desc like "*['""]*"
Though, since you are replacing a character with a duplication of itself (' with ''), you will need to be careful not to run the above more than once on your dataset.
You could easily test to find out which approach is better, though I'd guess using queries would be faster.
That said do it in Excel - it will chew though 20k rows faster than you can blink. You could import / export the sheet with a bit of VBA.
Use the REPLACE and SEARCH functions e.g. =IF(SEARCH(CHAR(23),A1)>0, REPLACE(A1,SEARCH(CHAR(23),A1),1," "), A1)
Related
I am going to VBA for deleting data table.
It shows the errors shown in the screenshot.
Please help me to resolve it.
Private Sub cmdxoa_Click()
If Not (Me.frmformsub1.Form.Recordset.EOF And Me.frmformsub1.Form.Recordset.BOF) Then
If MsgBox("Do you wwant to delete ?", vbYesNo) = vbYes Then
CurrentDb.Execute "DELETE from db " & _
" where NOLC = " & Me.frmformsub1.Form.Recordset.Fields("nolc")
Me.frmformsub1.Form.Requery
End If
End If
End Sub
So if NOLC in the table is of type text, your criteria expression has to be:
"where NOLC = '" & Me.frmformsub1.Form.Recordset.Fields("nolc").Value & "'"
As you see you have to surround the value with '.
Remark: .Value isn't necessary, but it enhances the readability and assures that you are interested in the value and not in the object itself (the control in that case).
BUT: You should use parametrized queries instead string concatenation to avoid SQL injection:
How do I use parameters in VBA in the different contexts in Microsoft Access?
I hope you can help.
I need to update a very old website that is using classic ASP code and inline sql queries. There’s a lot of bad practice going on but I need to quickly protect the site as best I can while we have the resources to update the site and move it over to a more secure environment.
Basically, what I need is a regular expression or function that will blacklist all of the usual suspects (ie words and characters) that are used as SQL injection. I fully appreciate that there is no concrete way to totally protect the site against SQL injection by using a blacklist (or whitelist). However, I just need to buy myself a little time while I figure everything out, and have the time, to update the entire scripting.
Unfortunately, I’m not that great on classic asp coding but what I have found so far are these three functions:
------------FUNCTION 1--------------
function SQLInject(strWords)
dim badChars, newChars, i
badChars = array("select", "drop", ";", "--", "insert", "delete", "xp_")
newChars = strWords
for i = 0 to uBound(badChars)
newChars = replace(newChars, badChars(i), "")
next
newChars = newChars
newChars= replace(newChars, "'", "''")
newChars= replace(newChars, " ", "")
newChars= replace(newChars, "'", "|")
newChars= replace(newChars, "|", "''")
newChars= replace(newChars, "\""", "|")
newChars= replace(newChars, "|", "''")
SQLInject=newChars
end function
------------FUNCTION 1--------------
------------FUNCTION 2--------------
function SQLInject2(strWords)
dim badChars, newChars, tmpChars, regEx, i
badChars = array( _
"select(.*)(from|with|by){1}", "insert(.*)(into|values){1}", "update(.*)set", "delete(.*)(from|with){1}", _
"drop(.*)(from|aggre|role|assem|key|cert|cont|credential|data|endpoint|event|f ulltext|function|index|login|type|schema|procedure|que|remote|role|route|sign| stat|syno|table|trigger|user|view|xml){1}", _
"alter(.*)(application|assem|key|author|cert|credential|data|endpoint|fulltext |function|index|login|type|schema|procedure|que|remote|role|route|serv|table|u ser|view|xml){1}", _
"xp_", "sp_", "restore\s", "grant\s", "revoke\s", _
"dbcc", "dump", "use\s", "set\s", "truncate\s", "backup\s", _
"load\s", "save\s", "shutdown", "cast(.*)\(", "convert(.*)\(", "execute\s", _
"updatetext", "writetext", "reconfigure", _
"/\*", "\*/", ";", "\-\-", "\[", "\]", "char(.*)\(", "nchar(.*)\(")
newChars = strWords
for i = 0 to uBound(badChars)
Set regEx = New RegExp
regEx.Pattern = badChars(i)
regEx.IgnoreCase = True
regEx.Global = True
newChars = regEx.Replace(newChars, "")
Set regEx = nothing
next
newChars = replace(newChars, "'", "''")
SqlInject2 = newChars
end function
------------FUNCTION 2--------------
------------FUNCTION 3--------------
Function isURL(strURL)
Dim Slug, re, re2
'Everything to lower case
Slug = lcase(strURL)
' Replace - with empty space
Slug = Replace(Slug, "-", " ")
' Replace unwanted characters with space
Set re = New RegExp
re.Pattern = "[^a-z0-9\s-]"
re.Global = True
Slug = re.Replace(Slug, " ")
' Replace multple white spaces with single space
Set re2 = New RegExp
re2.Pattern = "\s+"
re2.Global = True
Slug = re2.Replace(Slug, " ")
Slug = Trim(Slug)
' Replace white space with -
Slug = Replace(Slug," ", "-")
isURL = Slug
End Function
------------FUNCTION 3--------------
Can anyone let me know if the above is any good and if so which one is the best one? If not, can anyone suggest a sample script I can use just to get by for the moment? Any help would be fully appreciated.
Best regards
Rod from the UK
Read about parameterised queries https://vikaskanani.wordpress.com/2012/05/07/classic-asp-sql-injection-prevention-by-using-query-parameter/
Above Functions
On the surface, any/all of those should work. (Not having tested them myself.)
I would suggest that you take the time to test them.
Whatever you do, there'll be something that you will have to adjust later on. The bad guys are very creative.
Also, while it may seem like a good idea to run all the functions through a parent function that calls the three above, you may find that you take a performance hit - depending on how many times they're all run.
Alternate
Have you considered replicating something like PHP's AddSlashes() function? That should break some of the SQL insertion attempts. You could customise it to target extra things that don't technically need \'s, but would neuter the SQL injection.
If you don't want to mess about with ASP classic & Regex, you could achieve this with a few simple replace(input,"'","\'") style lines.
You're basically trying to invent your own Web Application Firewall. This is a huge, complex task, and hard to get right unless you are a security expert.
https://learn.microsoft.com/en-us/azure/application-gateway/application-gateway-web-application-firewall-overview
I would recommend you skip this step. I don't think you will "buy yourself time"—you will just delay fixing your code properly by using SQL query parameters.
Has anyone got a better way to do this? I want the users to be able to filter to a set of records, and then use a sql statement to do something to just the filtered records.
My code at the moment looks something like the following, and it works. But I do wonder if there is any better (or other?) way to achieve this that could avoid the clunky Replace statement? I wondered if I could do it using a recordset instead, but unless I'm missing something, there's no way to do set operations over a recordset, I'd have to loop through the records individually to do that.
Any other suggestions?
Dim db As Database
Dim frm As Form
Dim sqlCmd As String
Dim filter As String
Set db = CurrentDb
Set frm = Forms("frmReplen").Controls("FrmReplenSheet").Form
filter = Replace(frm.filter, "[FrmReplenSheet].", "")
sqlCmd = "UPDATE tblRCmDataSheetExtract " & _
"SET Completed = 1 " & _
"WHERE " & filter
db.Execute sqlCmd
The only thing you could do is to shorten your procedure. You could do something like this:
currentdb.Execute "UPDATE tblRCmDataSheetExtract SET Completed = 1 WHERE " & Replace(Forms!frmReplen!FrmReplenSheet.Form.Filter, "[FrmReplenSheet].", "")
Or when you're running it from within frmReplen you could use
currentdb.Execute "UPDATE tblRCmDataSheetExtract SET Completed = 1 WHERE " & Replace(Me.FrmReplenSheet.Form.Filter, "[FrmReplenSheet].", "")
or from within the subform "FrmReplenSheet":
currentdb.Execute "UPDATE tblRCmDataSheetExtract SET Completed = 1 WHERE " & Replace(Me.Form.Filter, "[FrmReplenSheet].", "")
that puts your whole procedure into one single command. It's up to you if you like better readability or shorter procedures.
I'm using Access 2007 by itself with no connections to SQLserver or anything for this process.
I want to take the result of a few DateDiff functions and use an Update SQL statement to put them into fields on a table. My table's fields are number fields, and I am under the impression that DateDiff returns a number.
I try this, but I get a data type mismatch error on the first DateDiff (Pause1). I tried taking the quotes off of the fields but then I get a different error (can't find the field '|' referred to in your expression).
Here is my code. It really starts at the comment TIME REPORTING CODE HERE:
Private Sub StopNextButton_Click()
'
GetID = Forms!frm_MainMenu!AssocIDBox
CurRecord = Forms!frm_EC_L1_L2![L#].Value
'
DoCmd.RunSQL "UPDATE tbl_Data SET tbl_Data.[tsEndAll] = Now WHERE tbl_Data.[L#] = " & CurRecord & " AND (tbl_Data.[ECName] Like 'L1*' OR tbl_Data.[ECName] Like 'L2*') "
'
'TIME REPORTING CODE HERE'
'
Pause1 = DateDiff("s", "[tsPause1]", "[tsResume1]")
Pause2 = DateDiff("s", "[tsPause2]", "[tsResume2]")
ECTime = (DateDiff("s", "[tsECStart]", "[tsUpdated]") - (Pause1 + Pause2))
LTime = DateDiff("s", "[tsStartAll]", "[tsEndAll]")
'
DoCmd.RunSQL "UPDATE tbl_Data SET [ECTime] = " & ECTime & ", [LoanTime] = " & LTime & " WHERE tbl_Data.[L#] = " & CurRecord & " AND (tbl_Data.[ECName] Like 'L1*' OR tbl_Data.[ECName] Like 'L2*') "
'
'END OF TIME REPORTING CODE'
'
DoCmd.GoToRecord , , acNext
'
ETC.
Based off of your comment I assume that those fields are on the record your form is currently 'viewing'. If so you can just refer to them as Me.tsPause1 without [] or quotes. Pretty sure you can also do just tsPause1 but I find Me.tsPause1 makes it more obvious what you are doing.
However I think you are updating the field you are currently viewing and then immediately trying to access those updated fields. I am fairly certain you will need to a Me.Refresh before those fields' new values are accessible. Hopefully someone with more specific experience will correct me if I am wrong. I think something like this should work for you:
Me.Refresh
Pause1 = DateDiff("s", Me.tsPause1, Me.tsResume1)
Pause2 = DateDiff("s", Me.tsPause2, Me.tsResume2)
ECTime = (DateDiff("s", Me.tsECStart, Me.tsUpdated) - (Pause1 + Pause2))
LTime = DateDiff("s", Me.tsStartAll, Me.tsEndAll)
I was just testing my database and I realized that I run into problems wherever a text entry in my database contains a ' character (single quote). My solution for now is that before any .execute operations on a string, I call escape(string, "'", " "'" ").
Summarized example below:
qr = "INSERT INTO tblExample VALUES ( " & "'" & me.testparam & "'" & ");"
qr = Replace(qr, "'", " "'" ")
db.execute qr
'also tried qr = "INSERT INTO tblExample VALUES ( " & "'" & replace(me.testparam,"'"," ") & "'" & ");"
This was what I assumed to be the correct workaround to prevent errors from values such as Tourette's.
There's two problems with this. First of all, it's not working. Second, I have over 50 locations in code throughout my app where I call the statement db.execute qr where qr is a string that could potentially contain a single quote. I need the field in the table to contain the single quote, so I can't just replace it with a space or something similar.
Two part question:
Is there a better solution than going through all of my code calling Replace on every string that is to be executed as a query?
Why is my current implementation failing? - I still get syntax error in query expression even when escaping the single quote to a space.
First examine these 2 lines.
"VALUES ( " & "'" & me.testparam & "'" & ");"
"VALUES ( '" & me.testparam & "');"
Both will produce the exact same string. The difference for me is that my brain comprehends the second version faster.
Now, here is what the comments are telling you to do ... replace each single quote in your source string with two single quotes. I added Debug.Print so you can view the finished string in the Immediate window (go there with Ctrl+g) ... you can then see the actual string rather than trying to imagine what it looks like.
qr = "INSERT INTO tblExample VALUES ( '" & _
Replace(Me.testparam, "'", "''" & "');"
Debug.Print qr
db.Execute qr, dbFailOnError
Since I assumed db is a DAO.Database object variable, I included the dbFailOnError option. You should include an error handler in your code to deal with any problems dbFailOnError exposes.
When you run into trouble with a VBA function in a query, drop to the Immediate window and test your function expression there. This one triggers a compile error, "Expected: list separator or )":
? Replace("Tourette's", "'", " "'" ")
But this one works:
? Replace("Tourette's", "'", "''")
Tourette''s
I mentioned that because it's useful in general, and also because your title starts with "Escaping unwanted characters, mainly single quotes". So if you want to remove/replace other characters, not just single quotes, experiment in the Immediate window until you find a Replace() expression which works. Then use that expression in your query.
For example, if unwanted characters include line breaks ...
MyString = "foo" & vbCrlf & "bar" : ? MyString
foo
bar
? Replace(MyString, Chr(13) & Chr(10), " ")
foo bar
Note: I used Chr(13) & Chr(10) rather than vbCrlf as the find target because the db engine can use the Chr() function but doesn't know about the named constant (vbCrlf).
Your query is failing because you have not said where to insert :
Dim qd As QueryDef
qr = "INSERT INTO tblExample (AText) VALUES ( [avalue] );"
Set qd = CurrentDB.CreateQueryDef("",qr)
qd.Parameters("avalue").Value = me.testparam
qd.Execute dbFailOnError
Another method is to define a quote as constant (Const Quote = """") and use that to build SQL Statements. It is not possible to define a quote as Const Quote = Chr(34) as a constant definition can't be based on a function so one has to use four double quotes in a row. The third quote is what you are saving, the second quote is to excape the third quote and the first and last quote are because the value you are assigning is a string.
You will then be able to build SQL statements such as:
SQL = SELECT * FROM tblSyndromes
WHERE Syndrome = " & Quote & "Tourette's" & Quote & ";"
It will no longer matter that there are single quotes in your data.
I don't use parameters as if I upscale my database to sql server and convert my queries to pass-through queries, I can't use parameters. I rarely upscale but I write all my code with that assumption. Also if your query is not working as expected, how do find out what went wrong. If I have a variable called SQL, then I can always print the SQL statement and run it in a new query to see what it does.