Import csv, export queries, delete queries then repeat - ms-access

I have researched the code for this online for weeks but still cannot find what I need so any help will be greatly appreciated:
I need to:
1) Import the first CSV in "C:\Documents" and create a table called "Data" with import specs I have already created.
2) Create and export all fields in a (SELECT?) query (Sport = Football, Event = Match Odds) to an excel binary file called "Match Odds 001"
I then wish to delete the data contained in this query from the Table "Data".
3) Repeat step 2) two hundred times with different Events exporting each query to an excel binary file with the name "(Event) 001" then deleting that query data from the table.
4) After all queries have run and been deleted, any remaining data in the table will be exported to an excel binary file named "Misc 001" and then this data deleted from the Table "Data" (or maybe even delete the Table "Data" completely.)
5) Repeat from 1) importing the 2nd CSV file from "C:\Documents" and exporting queries to Excel Binary files named "(Event) 002" so as not to replace the previous files.
This would continue until all CSVs have been imported and split.
As there are 200 queries I'd prefer to create them in VBA code.
I have just started using VBA in Access and so far have found code which will import all csv files in a folder so I am hoping to insert code to create, export then delete the queries.
Any help on the vba coding required is hugely appreciated.
.
.
UPDATE: With massive thanks to Mike I now have the following code but a few small issues have surfaced.
1) How can I compact the Database to reduce the filesize after the "DoCmd.RunSQL "DROP TABLE Data"" command?
I wish to repeatedly import 500MB CSV's then delete them so after I import and drop the first then import the second the filesize becomes 1GB and is increasing.
2) How difficult is it to include a second column in the Events Table so that my SELECT query becomes WHERE Event=Event & Selection=Selection and to combine both these fields to create the filename?
3) The Events Table used to create the Queries and file names sometimes contains characters that cannot be used in file names, for Example "/" & "?". Can these be easily dropped to create the filenames or might it be better to add a further column to the Events Table which would contain the filename to be used (ie a combination of Event and Selection but with the disallowed characters removed)
If I can solve these issues I will have the perfect code for my needs, again with all credit to Mike.
Sub ImportAndSplit()
Dim fileCounter As String
Dim rs As New ADODB.Recordset
Dim sql As String
Dim qdf As QueryDef
Dim file As String
Const strPath As String = "C:\Users\Robbie\Documents\Data\" 'Directory Path
Dim strFile As String 'Filename
Dim strFileList() As String 'File Array
Dim intFile As Integer 'File Number
strFile = Dir(strPath & "*.csv") 'Loop through the folder & build file list
While strFile <> ""
intFile = intFile + 1 'add files to the list
ReDim Preserve strFileList(1 To intFile)
strFileList(intFile) = strFile
strFile = Dir()
Wend
If intFile = 0 Then 'see if any files were found
MsgBox "No files found"
Exit Sub
End If
DoCmd.SetWarnings False
Set qdf = CurrentDb.QueryDefs("Export")
sql = "SELECT Event FROM Events"
rs.Open sql, CurrentProject.Connection, adOpenStatic, adLockReadOnly
For intFile = 1 To UBound(strFileList) 'cycle through the list of files & import to Access creating a new table called Data
DoCmd.TransferText acImportDelim, "Data", "Data", strPath & strFileList(intFile)
fileCounter = Format(intFile, "000") 'format i so that when you use it in file names, the files sort intuitively
Do While Not rs.EOF
sql = "SELECT * FROM Data WHERE Event='" & rs!Event & "'" 'select the records to export and export them
file = "C:\Users\Robbie\Documents\Data Split\" & rs!Event & " " & fileCounter & ".xlsb" 'use the counter to distinguish between which csv your exporting from
qdf.sql = sql
DoCmd.TransferSpreadsheet acExport, acSpreadsheetTypeExcel12, qdf.Name, file
sql = "DELETE FROM Data WHERE Event='" & rs!Event & "'" 'delete the records from the source table
DoCmd.RunSQL sql
rs.MoveNext
Loop
rs.MoveFirst
'export remaining data
file = "C:\Users\Robbie\Documents\Data Split\Misc " & fileCounter & ".xlsb" 'use the counter to distinguish between which csv your exporting from
sql = "SELECT * FROM Data"
qdf.sql = sql
DoCmd.TransferSpreadsheet acExport, acSpreadsheetTypeExcel12, qdf.Name, file
'delete remaining data
'DoCmd.RunSQL "DELETE FROM Data"
'or delete the table
DoCmd.RunSQL "DROP TABLE Data"
Next
MsgBox UBound(strFileList) & " Files were Imported and Split"
rs.Close
DoCmd.SetWarnings True
End Sub

Create a table with your Events in it (in my example it's called "Events" and has one field "Event").
Create a query called "Export". Doesn't matter what it does, we're going to overwrite it 200 times anyway.
Add the ActiveX Data Objects Library reference
Code:
Dim i As Integer
Dim fileCounter As String
Dim rs As New ADODB.Recordset
Dim sql As String
Dim qdf As QueryDef
Dim file As String
DoCmd.SetWarnings False
Set qdf = CurrentDb.QueryDefs("Export")
sql = "SELECT Event FROM Events"
rs.Open sql, CurrentProject.Connection, adOpenStatic, adLockReadOnly
For i = 1 To x 'where x is however many csvs you're importing
fileCounter = Format(i, "000") 'format i so that when you use it in file names, the files sort intuitively
'run your import code here
Do While Not rs.EOF
'select the records to export and export them
sql = "SELECT * FROM Data WHERE Event='" & rs!Event & "'"
file = "C:\Documents\" & rs!Event & fileCounter & ".xlsb" 'use the counter to distinguish between which csv your exporting from
qdf.sql = sql
DoCmd.TransferSpreadsheet acExport, acSpreadsheetTypeExcel12, qdf.Name, file
'delete the records from the source table
sql = "DELETE FROM Data WHERE Event='" & rs!Event & "'"
DoCmd.RunSQL sql
rs.MoveNext
Loop
rs.MoveFirst
'export remaining data
file = "C:\Documents\MISC" & fileCounter & ".xlsb" 'use the counter to distinguish between which csv your exporting from
sql = "SELECT * FROM Data"
qdf.sql = sql
DoCmd.TransferSpreadsheet acExport, acSpreadsheetTypeExcel12, qdf.Name, file
'delete remaining data
DoCmd.RunSQL "DELETE FROM Data"
'or delete the table
'DoCmd.RunSQL "DROP TABLE Data"
Next i
rs.Close
DoCmd.SetWarnings True
Make sure none of those files exist before running this, it will not overwrite files. Either delete them manually or you could use the Windows Script Host in the loop to delete the files if they exist before creating them...
Dim fso As New FileSystemObject
If fso.FileExists(file) Then
fso.DeleteFile (file)
End If
In response to your updates:
Consider linking them instead of importing (DoCmd.TransferText acLink, "Data"...). I don't believe it's possible to compact while you're working in a db, it would have to close.
You basically answered your own question. If you want to do that, add the column to the table and update the SQL exactly how you said.
You could use the Replace function but you'd have to run it for each illegal character and you might end up missing one. It's probably best to do what you suggested and just create them yourself in a new column.

Related

Import Separate Excel 2016 Worksheets into Separate Access 2016 Tables

First off, this is a compilation of code from various sources and as much as I would like to give credit, I cannot find the originator. My apologies.
Dilemma, I have 5 spreadsheets, all different names, there is one tab on each spreadsheet with the same name as the file and in Access there is a corresponding table name that is the same(keeping it simple.) In the code below, it imports the first worksheets tab data into the corresponding table in Access and prompts me with the message box saying it is done. However, none of the other tables are imported, which makes sense. I am missing a component to call each file versus just one worksheet. FYI, I cannot combine these files into one worksheet, because some of the worksheets have a ton of data. They are all .xlsb files. Am I missing a step?
Function Fnc_ImportData()
Dim strPathFile As String, strFile As String, strPath As String
Dim blnHasFieldNames As Boolean
Dim intWorksheets As Integer
' delete the current data in all 5 tables in Access
DoCmd.OpenQuery "qry_F2F_Alloc_tblDataDelete"
DoCmd.OpenQuery "qry_GDM_USD_tblDataDelete
DoCmd.OpenQuery "qry_GDM_USD_BDGT_tblDataDelete
DoCmd.OpenQuery "qry_IntraHR_Data_tblDataDelete
DoCmd.OpenQuery "qry_IT_ProjectCosts_tblDataDelete
' Replace (1 to #) with the number of worksheets to be imported
' from each EXCEL file
Dim strWorksheets(1 To 5) As String
' Replace (1 to #) with the number of worksheets to be imported
' from each EXCEL file (this code assumes that each worksheet
' with the same name is being imported into a separate table
' for that specific worksheet name)
Dim strTables(1 To 5) As String
' Replace generic worksheet names with the real worksheet names;
' add / delete code lines so that there is one code line for
' each worksheet that is to be imported from each workbook file
' strWorksheets(1) = "GenericWorksheetName1"
strWorksheets(1) = "tbl_F2F_Alloc"
strWorksheets(2) = "tbl_GDM_USD"
strWorksheets(3) = "tbl_GDM_USD_BDGT"
strWorksheets(4) = "tbl_IT_ProjectCosts"
strWorksheets(5) = "tbl_IntraHR_Data"
' Replace generic table names with the real table names
' strTables(1) = "GenericTableName1"
strTables(1) = "tbl_F2F_Alloc"
strTables(2) = "tbl_GDM_USD"
strTables(3) = "tbl_GDM_USD_BDGT"
strTables(4) = "tbl_IT_ProjectCosts"
strTables(5) = "tbl_IntraHR_Data"
' Change this next line to True if the first row in EXCEL worksheet
' has field names
blnHasFieldNames = True
' Replace C:\Documents\ with the real path to the folder that
' contains the EXCEL files
strPath = "\\admpls173m\findata\Functions_Finance\HR Monthly PnL\2-Database\Files\TEST MAIN FILES\"
' Replace # with the number of worksheets to be imported
' from each EXCEL file
' For intWorksheets = 1 To #
For intWorksheets = 1 To 5
strFile = Dir(strPath & "*.xlsb")
Do While Len(strFile) > 0
strPathFile = strPath & strFile
DoCmd.TransferSpreadsheet acImport, _
acSpreadsheetTypeExcel12, strTables(intWorksheets), _
strPathFile, blnHasFieldNames, _
strWorksheets(intWorksheets) & "$"
strFile = Dir()
Loop
Next intWorksheets
DoCmd.SetWarnings True
'Message box telling you the data has been imported into the Tbl_Summary
MsgBox "Access Has Finished Importing the Files." & Chr(13) & "Data Is Ready To Be Reviewed.", vbInformation
Exit_File1_Click:
Exit Function
Err_File1_Click:
MsgBox Err.Description
Resume Exit_File1_Click
End Function

Searching for a value in Access Table VBA

I have written code that imports excel files into a access table. As each file is imported the file name is recorded and saved to a separate table named 'FilesDownloaded'.
I would like to add vba code that prior to importing the file it will check to see if the name of the file (myfile) is already saved on the 'FilesDownloaded' table. This will prevent the same file from being imported twice.
Code:
Function Impo_allExcel()
Dim myfile
Dim mypath
Dim que As Byte
Dim rs As DAO.Recordset
que = MsgBox("This proces will import all excel items with the .xls in the folder C:\MasterCard. Please make sure that only the files you want imported are located in this folder. Do you wish to proceed?", vbYesNo + vbQuestion)
If que = 7 Then
Exit Function
Else
'do nothing and proceed with code
End If
DoCmd.SetWarnings (False)
DoCmd.RunSQL "DELETE * FROM tblMaster_Import;"
MsgBox "Please WAIT while we process this request"
mypath = "C:\Master\"
ChDir (mypath)
myfile = Dir(mypath & "*.xls")
Do While myfile <> ""
If myfile Like "*.xls" Then
'this will import ALL the excel files
'(one at a time, but automatically) in this folder.
' Make sure that's what you want.
'DoCmd.TransferSpreadsheet acImport, 8, "tblMasterCard_Import", mypath & myfile
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel8, "tblMaster_Import", mypath & myfile, 1
Set rs = CurrentDb.OpenRecordset("FilesDownloaded")
rs.AddNew
rs.Fields("Filename").Value = myfile
rs.Update
rs.Close
Set rs = Nothing
End If
myfile = Dir()
Loop
'append data to tblAll (risk of duplicates at this point)
DoCmd.RunSQL "INSERT INTO tblAll SELECT tblMaster_Import.* FROM tblMaster_Import;"
DoCmd.OpenQuery "qryUpdateDateField", acViewNormal
''this code will apend to an existing table and runs the risk of doubling data.
DoCmd.SetWarnings (True)
MsgBox "Your upload is complete"
End Function
A possible solution is to make a query, which gives you the result of lines in the table FilesDownloaded, with condition that the name is equal to your file. Something like this:
countString = "SELECT COUNT(*) FROM [FilesDownloaded] WHERE [NAMEOFCOL] = " & myFile
Then run the query, and if the result is more than 1, you obviously have it.

Import append vba MS-Access

Need help with the Access VBA
I am trying to import spreadsheets from excel into access. Once it is imported it will be appended into a different table and the import table will be deleted so that i can then import a new spreadsheet. However i have error when appending i need to append the entire table because i will not always know the header from the spreadsheet and how many their are.
Please Help. Below is the code i have been work on.
Module-
Option Compare Database
Function selectFile()
Dim fd As FileDialog, fileName As String
On Error GoTo ErrorHandler
Set fd = Application.FileDialog(msoFileDialogFilePicker)
fd.AllowMultiSelect = False
If fd.Show = True Then
If fd.SelectedItems(1) <> vbNullString Then
fileName = fd.SelectedItems(1)
End If
Else
'Exit code if no file is selected
End
End If
'Return Selected FileName
selectFile = fileName
Set fd = Nothing
Exit Function
ErrorHandler:
Set fd = Nothing
MsgBox "Error " & Err & ": " & Error(Err)
End Function
VBA -
Private Sub cmdImportNoDelete_Click()
'Unset warnings
DoCmd.SetWarnings False
'Import spreadsheet
DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel12Xml, "Import", selectFile, True
'SQL append statement, Needs to append entire table (must be able to append multiply table with different headers
DoCmd.RunSQL "INSERT INTO Tbl_ImportDump * FROM Import"
'SQL delete Table
DoCmd.DeleteObject acTable, "Import"
DoCmd.SetWarnings True
End Sub
Tbl_ImportDump must exist. If it doesn't already, create it:
DoCmd.RunSQL _
"CREATE TABLE Tbl_ImportDump (Col1 Col1DataType, Col2 Col2DataType, etc)"
Then after importing your spreadsheet (as table Import, as you've already done), get the column names in the Import table, and put these into the insert statement:
dim ImportCols as String: ImportCols = ""
dim ii as long
dim rs as Recordset
rs.OpenRecordset "SELECT TOP 1 * FROM Import"
for ii = 0 to 2
ImportCols = ImportCols & "," & rs.Fields(0).Name
next
importCols = mid(importCols, 2)
DoCmd.RunSQL _
"INSERT INTO Tbl_ImportDump (Col1, Col2, Col3) " & _
"SELECT " & importCols & " FROM Import"

How to update multiple tables in MS Access from multiple excel files?

Every month I have to UPDATE my MS Access db from approximately 30 excel files. They have all the same structure and format. I try to modify this code several times to update each tables in my db in once but i didn't succeed. (found in this forum)
I have 3 questions:
How is it possible to say to the vba code to look at this range in the excel template which correspond to this column in the MS Access db?
How can i make this vba code update all the tables in once based on the Primary Key?
Then, Is it possible to select the Folder where all these excel files are and the code will loop through all files?
Public Sub UpdatePriceList()
Dim cn As ADODB.Connection, rs As ADODB.Recordset
Dim sProduct As String, sVariety As String, cPrice As Variant
' connect to the Access database
Set cn = New ADODB.Connection
cn.Open "Provider=Microsoft.ACE.OLEDB.12.0; " & _
"Data Source=C:\Users\Gord\Desktop\Database1.accdb;"
' open a recordset
Set rs = New ADODB.Recordset
rs.Open "PriceList", cn, adOpenKeyset, adLockOptimistic, adCmdTable
Range("A2").Activate ' row 1 contains column headings
Do While Not IsEmpty(ActiveCell)
sProduct = ActiveCell.Value
sVariety = ActiveCell.Offset(0, 1).Value
cPrice = ActiveCell.Offset(0, 2).Value
rs.Filter = "product='" & sProduct & "' AND variety='" & sVariety & "'"
If rs.EOF Then
Debug.Print "No existing record - adding new..."
rs.Filter = ""
rs.AddNew
rs("product").Value = sProduct
rs("variety").Value = sVariety
Else
Debug.Print "Existing record found..."
End If
rs("price").Value = cPrice
rs.Update
Debug.Print "...record update complete."
ActiveCell.Offset(1, 0).Activate ' next cell down
Loop
rs.Close
Set rs = Nothing
cn.Close
Set cn = Nothing
End Sub
thank you in advance!
If you know how to deal with a single excel file and convert it to acess table and all the rest of file will be treated the same way. You can use a bot which will automate the task and do it quickfast. You just have to give it the "instruction" how to deal with single file and it will do the rest.
The instruction can be as simple as a notepad file with table containing the mouse positions and the stroken keyboards key necessary to deal with one file.
You may use "Autoit V3 script"

Import Unknown Field Order From Text

I've been assigned the task of importing about 180 csv files into an access 2007 database. These files have been put together over the years and will be put into 1 of 3 folders. I have not set up any data checks or restrictions to these tables (such as primary keys, validation rules, or relationships). That will be done once the data has been imported. The data contained in these files are from a survey which has changed over the years. This change has caused the fields to change. The order of them has changed or sometimes a field is there and sometimes it is not. I do have a list of all the fields possible though and what table each csv file should be imported to, and know that all these fields can be text.
Here is my problem: Not knowing what the order of the columns or if a column will exist, is it possible to run a function to import these text files into their relative tables by mapping each column in the text file to it's associated column in the access table?
Each text file has headers which is useful to see shat they actually are, but there is no text qualifier which can be very annoying when dealing with id codes consisting entirely of numbers. Below is what I've tried so far. It gets the file location from a function elsewhere, adds each filename in that location to a collection, then for each file in that collection it tries to import it into it's relative field.
'Get file names from the folder and store them in a collection
temp = Dir(location & "\*.*")
Do While temp <> ""
fileNames.Add temp
temp = Dir
Loop
'Go through each file in the collection and preccess it as needed
For Each temp2 In fileNames
If (temp2 Like "trip*") Then 'Import trip files
'Gets the data from a query 'DoCmd.RunSQL "SELECT * FROM [Text;FMT=Delimited;HDR=YES;IMEX=2;CharacterSet=437;DATABASE=" & location & "].[" & temp2 & "] As csv;"
DoCmd.TransferText acImportDelim, "Trips_Import", "tbl_Trips", location & "\" & temp2, -1
End If
If (temp2 Like "catch*") Then 'Import catch files
DoCmd.TransferText acImportDelim, "Catch_Import", "tbl_Catch", location & "\" & temp2, -1
End If
If (temp2 Like "size*") Then 'Import size files
DoCmd.TransferText acImportDelim, "Size_Import", "tbl_Size", location & "\" & temp2, -1
End If
Next temp2
You can create a SELECT * query for each CSV file and open the query as a recordset. Open another recordset for the destination table.
Then for each row in the CSV recordset, add a row to the destination recordset, loop through the CSV Fields collection, and add each CSV field value to the destination field with the same name.
This approach is independent of the order in which the fields appear in the CSV file. It also doesn't matter if the CSV file includes only a subset of the fields present in the destination table. As long as each CSV field also exists in the table, it should work (assuming compatible data types, the value satisfies validation rules/constraints, etc.).
Dim db As DAO.Database
Dim fld As DAO.Field
Dim rsDest As DAO.Recordset
Dim rsSrc As DAO.Recordset
Dim strSelect As String
Dim strTableName As String
Set db = CurrentDb
'Go through each file in the collection and preccess it as needed
For Each temp2 In fileNames
Select Case Left(temp2, 4)
Case "trip"
strTableName = "tbl_Trips"
Case "catc"
strTableName = "tbl_Catch"
Case "size"
strTableName = "tbl_Size"
Case Else
' what should happen here?
' this will trigger an error at OpenRecordset(strTableName) ...
strTableName = vbNullString
' figure out a better alternative
End Select
strSelect = "SELECT csv.* FROM " & _
"[Text;FMT=Delimited;HDR=YES;IMEX=2;CharacterSet=437;DATABASE=" & _
Location & "].[" & temp2 & "] As csv;"
Debug.Print strSelect
Set rsSrc = db.OpenRecordset(strSelect, dbOpenSnapshot)
Set rsDest = db.OpenRecordset(strTableName, dbOpenTable, dbAppendOnly)
With rsSrc
Do While Not .EOF
rsDest.AddNew
For Each fld In .Fields
rsDest.Fields(fld.Name).value = fld.value
Next
rsDest.Update
.MoveNext
Loop
.Close
End With
rsDest.Close
Next temp2
Note: This is a RBAR (row by agonizing row) approach, so the performance will be less than stellar. However, I presumed you will do this only once, so the performance hit will not be a deal-breaker. If you need a faster set-based approach instead, you can build and execute an "append query" for each CSV file. To do that, you would first need to get the CSV field names, and then build the appropriate INSERT INTO statement.