Access VBA code to prevent duplicate import from Excel - ms-access

I have built this simple Access DB (2010) that imports 11 fields from Excel spreadsheet into a single table. What I have not done yet is to check if record in table exists and to not import it again creating duplicates.
Business Case:
User receives hundreds of spreadsheets per day (a form is used by employees in a field). User saves all these spreadsheets in a designated folder from where, VBA code picks up only some data and inserts in a single table in Access DB. When import is complete I then ask user to move all those spreadsheets into archive folder manually. I understand this process is very prone to where same spreadsheet may be imported again. I would like add a check to my VBA code to check if record exists and ignore it. It would need to check 3 fields: employee name, date, and location as there should only be one report per employee per day per location.
I am a novice and am just learning VBA so some solutions I found online are not sufficient enough for me at this point. Would like a specific example of code that I could reuse. My current code (which I also found online and modified to work for me) is something like this.
Function DoImport()
Dim strPathFile As String
Dim strFile As String
Dim strPath As String
Dim blnHasFieldNames As Boolean
Dim intWorksheets As Integer
Dim strWorksheets(1 To 1) As String
Dim strTables(1 To 1) As String
strWorksheets(1) = "data"
strTables(1) = "my_table"
blnHasFieldNames = True
strPath = "folder path were user originally saves all reports"
For intWorksheets = 1 To 1
strFile = Dir(strPath & "*.xlsm")
Do While Len(strFile) > 0
strPathFile = strPath & strFile
'MsgBox strPathFile
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel12Xml, strTables(intWorksheets), strPathFile, blnHasFieldNames, strWorksheets(intWorksheets) & "$"
strFile = Dir()
Loop
Next intWorksheets
End Function
Thank you vary much.

In your table, create a Unique Index using the three fields. This example assumes the names of the fields are EmployeeName, RecordDate, and Location:
This will not allow duplicates of the three fields to be inserted.

Related

Importing thousands of spreadsheets into Access Database table using VBA

I have created this process by lifting code from this (and some other) sites, and it worked like a charm when I tested it, but when I deployed it, it crashed hard... I am novice when it come to VBA and having trouble finding appropriate solution so thought I will ask for help.
Use Case
Accountant receives 100+ spreadsheets per day(!) from employees in the field as a form of required reporting. Before I got involved, 3 accountants would open each spreadsheet received via e-mail and copy/paste certain cell contents into a "master" spreadsheet that will be used for reconciliation at the end of the month. Needless to say, this has become, let's say, inefficient.
What I did
I created an Access DB and used TransferSpreadsheet method to import data. There are only 11 cells that we need imported from each spreadsheet, so I modified spreadsheets that field employees use to pull all this data into a hidden tab, where all data is in one row and all row goes into one table in Access. As I mentioned, when I and accountants tested the solution, it worked beautifully.
What Broke
First issue was that people in a field did not have same versions MS Office, and some used OpenOffice instead and we would get errors when trying to import some spreadsheets. However, since my simple solution was built for "prefect path" only, it was impossible to identify which spreadsheets failed, especially, when there are 2000+ of them sitting in a folder.
What I would like to be able to do
Short-term, until I have time to master VBA, I would love to add some sort of error handler. Or even if after import, "good" spreadsheets would be sent to one folder, and "bad" ones to another. From my experience,about 80% of reports import fine. Once it loops through the whole folder, accountants can check "failed imports" folder and enter these manually. My questions to you, Access VBA experts, is this doable and would be a reasonable solution? If so, can you please direct me to it?
Below is my current code that I adapted from the internets. Thank you for all your help!
Function DoImport()
Dim strPathFile As String
Dim strFile As String
Dim strPath As String
Dim blnHasFieldNames As Boolean
Dim intWorksheets As Integer
Dim strWorksheets(1 To 1) As String
' the number of worksheets to be imported
' from each EXCEL file (this code assumes that each worksheet
' with the same name is being imported into a separate table
' for that specific worksheet name)
Dim strTables(1 To 1) As String
' worksheet names ;
' add / delete code lines so that there is one code line for
' each worksheet that is to be imported from each workbook file
strWorksheets(1) = "Data"
strTables(1) = "my_table"
' Change this next line to True if the first row in EXCEL worksheet
' has field names
blnHasFieldNames = True
'update the path to where the Excel files to be imported are
strPath = "\mypathhere\"
' the number of worksheets to be imported
' from each EXCEL file
For intWorksheets = 1 To 1
strFile = Dir(strPath & "*.xlsm")
Do While Len(strFile) > 0
strPathFile = strPath & strFile
'MsgBox strPathFile
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel12Xml, strTables(intWorksheets), strPathFile, blnHasFieldNames, strWorksheets(intWorksheets) & "$"
strFile = Dir()
Loop
Next intWorksheets
End Function
This is off the top of my head (so there may be sytax errors) but the idea is to set the error handling to jump to code that saves the filename and then jump back into the process:
ON ERROR GOTO IMPORT_ERROR
For intWorksheets = 1 To 1
strFile = Dir(strPath & "*.xlsm")
Do While Len(strFile) > 0
strPathFile = strPath & strFile
'MsgBox strPathFile
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel12Xml, strTables(intWorksheets), strPathFile, blnHasFieldNames, strWorksheets(intWorksheets) & "$"
strFile = Dir()
GetNextFile:
Loop
Next intWorksheets
EXIT FUNCTION
IMPORT_ERROR:
'ADD CODE HERE TO HANDLE ERROR
ON ERROR GOTO IMPORT_ERROR
GOTO GetNextFile
End Function

Import different excel files into MS Access 2010 tables automatically

I'd like to import all Excel files (with different data and columns) from some directory into MS Access 2010 database, creating new table for each file. I've found the code to import files into one table:
Option Compare Database
Option Explicit
Function DoImport()
Dim strPathFile As String, strFile As String, strPath As String
Dim strTable As String
Dim blnHasFieldNames As Boolean
' Change this next line to True if the first row in EXCEL worksheet
' has field names
blnHasFieldNames = True
' Replace C:\Documents\ with the real path to the folder that
' contains the EXCEL files
strPath = "C:\Documents and Settings\myName\My Documents\Access Test\"
' Replace tablename with the real name of the table into which
' the data are to be imported
strTable = "tablename"
strFile = Dir(strPath & "*.xls")
Do While Len(strFile) > 0
strPathFile = strPath & strFile
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel9, _
strTable, strPathFile, blnHasFieldNames
' Uncomment out the next code step if you want to delete the
' EXCEL file after it's been imported
' Kill strPathFile
strFile = Dir()
Loop
End Function
But I need to create new table each time. Is it possible in VBA?
I think all you need to do is change the destination table name (the value of strTable) each time before you do DoCmd.TransferSpreadsheet.
In a comment you said you want the table name to be derived from the workbook file name. And, each time through your loop, another variable (strFile) contains the file name. So I think you could strip the file extension from that file name and use it as the Access table name.
Here is an Immediate window example which demonstrate how that can be done ...
strFile = "foo.xls"
strTable = Left(strFile, Len(strFile) - 4)
? strTable
foo
If that approach is suitable, revise the loop in your VBA code like this (untested) code snippet ...
strFile = Dir(strPath & "*.xls")
Do While Len(strFile) > 0
strPathFile = strPath & strFile
strTable = Left(strFile, Len(strFile) - 4)
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel9, _
strTable, strPathFile, blnHasFieldNames
strFile = Dir()
Loop
I used to be a MOS Access 2003. Now everyone is using 2010 but many things have not changed.
When you do a manual import or export, you can save the layout as a specification.
This process can be automated by a macro.
Check out the link below for more details and steps.
http://office.microsoft.com/en-us/access-help/run-a-saved-import-or-export-operation-HA001226020.aspx?CTT=5&origin=HA001226307
As for the other stuff, buttons, modules, etc, please read the on line help / documentation first.
We are here to help but not do the work for you.
J
Okay, I do not know if it is an issue with my computer not being on the current office CU.
http://msdn.microsoft.com/en-us/library/office/ff192475(v=office.14).aspx
Here is a link to how to use the ImportExport Macro. Use to be in the macro section.
I did read that you had to trust the location. So I tried both my location c:\msdn plus the default for the wizards.
Still was not able have it the option come up.
I tried creating a specification to see if one was needed for the option to show, no dice.
However, there is a DoCmd.TransferText and DoCmd.TransferSpreadSheet.
Both will allow you to import.
Create a function. Call the function from a macro (RunCode). Another way is to create a main menu form. Have a button. On the click command, run the code.
Please tell me if you ever get the ImportExportData Macro to show. I think it is a bug. I will need to bring down the latest Cumulative Updates and try again.

I need to dynamically (and quickly) import part of the data in an Access table (a large one) into Excel

I'm trying to create a dynamic report in Excel. I have lots of sales data and I want the user to be able to slice and dice it according to his needs.
Normally I would use two sheets: one hidden, containing the raw data, and one visible, containing all the buttons and form controls so that the user can dinamically select and visualize only a small subset of the original data at the time.
The problem is that this time I need to handle 6.000.000+ rows of data (and counting). Storing it all in an excel sheet is not an option. Besides, the data is already in the form of an Access table.
I tried accessing it dinamically via a query that "filters out" the un-needed information based on what the user selects in the form control on the Excel sheet. For some reason, this is very slow. It takes 4-5 minutes to pull out as little as 10 rows of data.
There has to be a quicker way to do this! I need this whole process to feel "instantaneous".
Any thoughts?
Edit: Ok, so the problem seems to be related to the fact that my access table is actually a linked table pointing to a *.txt file. This slows the import down a lot.
I tried both of the suggested solutions.
iDevlop's idea works quite fast (200k rows imported in 10-15 secs), but it has the downside of me having to update the table every time. I'll post another question, like he suggested, to see how and if the process can be automated.
Remou's script works perfectly too now (I had a hard time getting it right but he was really open and helpful so know I got it) and, although slower, it has the advantage of not requiring any database mantainance.
There's a few more things I need to get straight before choosing which approach to use. For now, all I want to say is thank you guys for you help! I could have never made it without you!!!
Don't bother going through Access if you have a text file. This may hold you until you can get a better system in place.
Dim cn As Object
Dim rs As Object
Dim strFile As String
Dim strCon As String
Dim strSQL As String
Dim s As String
Dim i As Integer, j As Integer
strFile = "z:\docs\"
''Note that if HDR=No, F1,F2 etc are used for column names,
''if HDR=Yes, the names in the first row of the range
''can be used.
''
''Connection strings : http://www.connectionstrings.com/excel
strCon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & strFile _
& ";Extended Properties=""Text;HDR=Yes;IMEX=1"";"
''Late binding, so no reference is needed
Set cn = CreateObject("ADODB.Connection")
Set rs = CreateObject("ADODB.Recordset")
cn.Open strCon
strSQL = "SELECT * " _
& "FROM [test.txt] a " _
& "WHERE a.FirstName ='Bernard'"
rs.Open strSQL, cn, 3, 3
''Pick a suitable empty worksheet for the results
Worksheets("Sheet3").Cells(2, 1).CopyFromRecordset rs
''Tidy up
rs.Close
Set rs = Nothing
cn.Close
Set cn = Nothing
If your file is tab delimited, you can use a schema.ini (http://msdn.microsoft.com/en-us/library/ms709353(VS.85).aspx). It must be in the same directory as you text file and need only contain two lines:
[Ordini BO new.txt]
Format=TabDelimited
Your connection string should read:
strCon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & strFile _
& ";Extended Properties=""Text;HDR=No;IMEX=1;FMT=Delimited"";"
As says Remou, check xour indexes, but also make sure your criteria are entered in a way that allows indexes to be used !
e.g : WHERE Format(myDate,"yyyy-mm") = "2011-09" does not allow the date index optimisation,
while WHERE myDate BETWEEN #09/01/2011# AND #09/30/2011# does allow index optimisation.
Edit:
If you have some kind of unique identifier in your text file and you translate that into a PK in your table design, you can then import the whole thing on a regular basis, and the duplicates will be discarded by the PK.
The import could be automated, even with a .vbs, you don't need Access to do it. Make that another question if you're stuck.
You could also ask the IT guys to delete the older records every month or so.

MS Access Auto Link Excel Spreadsheets

I have a directory structure where I am managing the requirements of a system with each component of that system having its own directory. the requirements of each component are stored in a excel workbook that has multiple worksheets(# of worksheets are static). I am currently using access as a central location to view the information in these sheets and perform queries on them. I hate having to manually link new excel files every time a new component documentation is added to the directory. Is there a way that when everytime I start access it will search the directory tree of stored excel files and automatically link them to access if they're not linked and update my save queries to include the new files. I was thinking that I could save the sub directory names in a table and all the filenames in those sub directory in another table that references the other table, so as it searches the filesystem it compares names to the table. Is this possible if so could someone point me in the right direction.
You can use Dir or the FileSystemObject recursively to get files from a directory tree. Access stores the link information of files in the connect property of the TableDef or you can get it from:
SELECT msysobjects.Database
FROM msysobjects
WHERE (((msysobjects.Database) Is Not Null));
You can get worksheets like so:
''Requires reference to the Microsoft Excel x.x Object Library
Dim strFileName As String
Dim objXL As New Excel.Application
Dim wkb As Excel.Workbook
Dim wks As Object
''objXL.Visible = True
strFileName = "C:\Docs\LTD.xls"
Set wkb = objXL.Workbooks.Open(strFileName)
For Each wks In wkb.Worksheets
DoCmd.TransferSpreadsheet acLink, acSpreadsheetTypeExcel9, _
wks.Name, strFileName, True, wks.Name & "$"
Next
''Tidy up
wkb.Close
Set wkb = Nothing
objXL.Quit
Set objXL = Nothing
Or using an ADOX.Catalogue: http://forum.lessthandot.com/viewtopic.php?f=95&t=3712

A data diff tool for finding the difference between two Access MDB Files

I have two different MDB files [with the same structure]. Is there an existing tool that can report the difference between the two files [row by row]?
I found a program called "MDBDiff" on SF, however the program is no longer available for download.
I've made an AccdbMerge utility that is able to compare data and programming objects as well. In scope of "row by row" comparison - it will show what records were added/modified/removed, for modified records it will highlight fields with updated values.
See the following page and go down a bit for a list of utilities to compare Access databases
http://www.granite.ab.ca/access/thirdparty.htm One of those might be what you're looking for.
I wanted to do the same (basically use DIFF to see differences row by row) so
1) I exported all the tables:
Option Explicit
Option Compare Database
Private Sub ExportAllTables()
Dim myDatabase As Database
Dim myTableDef As TableDef
Dim strTableName As String
Set myDatabase = CurrentDb
For Each myTableDef In myDatabase.TableDefs
DoEvents
strTableName = myTableDef.Name
DoCmd.TransferText _
acExportDelim, _
, _
strTableName, _
Environ("USERPROFILE") & "\DeskTop\dump\" & strTableName & ".CSV", _
True
Next myTableDef
MsgBox "Done"
End Sub
2) concatenated them into one file
type *.csv > all.txt
CAT will do as well if you have it
3) diff'ed them
diff all.txt all2.txt
Try using SQL Data Compare from Redgate, http://www.red-gate.com/products/SQL_Data_Compare/index.htm
and then use this trick,
http://www.red-gate.com/messageboard/viewtopic.php?p=15296#15296
I haven't tried it yet but this tool looks like it would do the job http://www.datanamic.com/download/download-datadiff-for-msaccess.html