Would you recommend working with multiple programmers on an MS Access application?
One of our MS Access application has grown to the point where the number of changes (bug fixes) and new features can no longer be handled by one programmer in the requested time frame.
We are trying to introduce version control using the undocumented SaveAsText and LoadFromText procedures in VBA to make collaboration on this application possible. Unfortunately we have already run into problems loading modified forms and reports back into Access as a checksum is stored in every form text file.
Before putting time into building an import/export application to compile text files into an Access database, we would like to hear your recommendations.
I think you should avoid this path at all cost, and try and persuade management into redevelopment.
It's a bitter pill to swallow, but this is going to need to be redeveloped sooner or later, and you are just saving them time and money.
We were using Microsoft's own version control add-in for MS Access 2000/2002/2003 for about 5 years now, and I can't remember a single serious problem. Usability of this add-in barely deserves a "B", but it must be much, much more convenient than fiddling with any ad-hoc method involving manual or semi-manual exporting/importing of Access forms, modules, etc.
We were using VSS as a version control system all the time. No problems whatsoever. However, if you have some good reasons to avoid VSS, you may have some options:
The version control add-in that we were using does not require VSS. Theoretically it can be used with any version control system that implements Microsoft Source Code Control Interface (MSCCI). For example, when we had to let somebody work on this project remotely, we used SourceOffsite by SourceGear. Access version control add-in worked with this third-party product fairly well (not without some quirks, but well enough). So, if your favorite version control system complies with MSCCI, you could try to use it.
Now that Microsoft has this Team Foundation thingy, apparently there are other options to be used to integrate MS Access with version control. We did not explore this path, though. This article may be a good start for exploring it.
Hope this would be of some help. :-)
P.S. I am not a big fan of MS Access. In fact, I rather hate it as a platform for a user front-end. If I had a choice, I would run away from it yesterday. :-) However, I must admit that existence of this version control add-in is one of the few things that makes maintenance of our old Access+SQLServer project more or less tolerable. :-))
In addition to what I already said here, I should add that the whole system works very well. The comparison process takes less than 30 minutes a week, for a team of 3 programmers. So let's describe it a little bit.
We have basically 2 versions of our Access program:
The "Developer's version", with all the stuff in it.
We each begin to work with an identical version of our developer's edition. As each one modifies or add parts of the code, we have to run some comparison routine on a regular basis. To do so, we have an object-export routine to a common "comparison" folder. An object (module for example) is exported as a text file (saveAsText command, do not work with tables, see infra), it will be compared to the existing equivalent text files in the folder. If files are identical, there is no file exported. If files are different, the new module is exported with the developer's name as an addition to the file name (if modQueries.txt exists, then modQueries_philippe.txt is created...). Of course if there is no equivalent .txt file in the folder, it will be created at first export.
At the end of the period, we would get in our folder the following files
modQueries.txt, being the first "original", last common version of the module
modQueries_Philippe.txt, with Philippe's modifications
modQueries_Denise.txt, with Denise's modifications
As the module was not modified by other developers, their export did not lead to the creation of a specific modQueries_developersName.txt file
If for any reasons Denise exported many times her module, only the last version is in the comparison folder.
We can then compare (with a "text file" comparer) the different versions and create the "updated" version of the module. We have a screen giving us the number of objects in the comparison folder, number of version for each object, and it is even possible to open the file comparer directly from the developer's interface (We use "File Compare Tool" which has a command-line mode and can then be started directly from Access).
The forms compare issue is quite special, as one of our rules is to have no specific code in our forms (please see here for more details). Forms are then only for display, so usually we do not even compare them. We just make sure that each one of them is updated by only one person (which is quite logical).
The table compare issue (we have local tables) can be only made between mdb files. As we export one text file per module, we also export one mdb file per table. We have a small routine allowing us to identify table differences at the structure level or at the record level.
After each comparison procedure, a subroutine will use all the objects available ini the comparison folder and create a whole new clean mdb file from scratch. This is the new developer's version. Every developer can then copy it on his computer and continue his work.
Developer's versions do not have numbers, but contains last client version number.
The client version, with limited stuff, automatically distributed to users
Each developer has the possibility to build a "client" mdb for final users. This mdb is created from scratch, in a way quite similar to our developer's version, but not all objects are exported. Some specific switches are turned off (special keys, access to code, etc). This mdb holds a version number as a property. The version number is used to build the name of the mdb file.
At production time, this mdb file is zipped and placed in a specific "distribution" folder. Each time a user starts the app, it will automatically check this folder to see if a new version is available. If yes, the client mdb file is updated from the distribution folder, and the app is restarted.
This distribution folder is replicated at night time with our overseas agencies. Users abroad will then be able to install the new version on the following day.
Following the direction provided by Yarik we settled on continuing developing in Access using the Access Add-in Source Code Control, the SVN SCC API Plugin by PushOk Software and Subversion. This stack provides us with seamless Access integration, full-backup and restore and an open version control system.
We had to install a hotfix to Access 2003 and make sure the default database file type matched our database file type to make it work.
We will continue to update this answer with our findings.
Have look at this thread:
How do you use version control with Access development?
Sounds like a terribly painful way to do team development. If you have any options for porting to another environment like VS2008 that would be my recommendation.
There is no easy way to work on Access as a team and even version control might be a bit tricky.
Related
I save the ms access file form in the network where is accessible by everyone. Everyone has a link on their desktop to the file in the network. Every time I change the file I have to ensure every one has the ms access file closed.
I really don't want to go looking everywhere to check if the app is closed. I've also set the file to read only, but no luck...
The basic concept for MOST software in the market places is you have two parts to application:
Word documents (data) + the word program executable
Excel document (data) + the word program executable.
So for 20 years in your company, how do you deploy software?
Answer:
You install the program on EACH computer.
So you are talking about a SOFTWARE program that you developed. Just because you use c++, vb.net or Access to CREATE SOFTWARE, then you don't break this rule. There is a difference between a data file a document and COMPUTER PROGRAM.
A COMPUTER program has code, forms and user interface. So if your computer support people don't know the difference between a document and a computer program then this would be the source of your problems.
As a result, you don't allow multiple people into the one program. I mean, if one person in your building has trouble with word does everyone go home?
And while you work on the NEXT great version of your software, users should not care.
So, this means:
Like most of your software, you deploy it to each user's workstation. This will allow you to work on the next great version of your software.
When you are done your changes and added new code and new features you then:
Compile the program into an executable (in Access, this is an mde, or now an accDE. You then deploy this to each users workstation.
The above approach thus allows you to work on the NEXT GREAT version of your software. You can add to your software some update code, or even just adopt a logon script that copies down the next great version of this software to each user's desktop.
The result is:
You don't care if users are in their current version of their software, since you are working on that copy which is not locked.
So, you should be distribution a compiled version of your software, and like most of your software you REALLY WANT to distribute that software to each user's workstation.
Last but not least:
Programs like Word, or Excel have what is called re-entrant code. So if you are working in a terminal server environment, then those office programs tolerate multiple users on that server, but your program created with Access DOES NOT. So even in this case, each logged on user is to receive their OWN SEPARATE copy of the COMPILED program you created.
So how c++ works, vb.net, VB6 or in this case Access works is the SAME for most programs:
You distribute that program to EACH user's desktop, and thus the issue of you working on the next great version is a non-issue. You simply have to setup and adopt a means to check for a new version. That means you distribute a program update, not a single form update. Even in the case of changing come code, or modify a report, that will cause you to like MOST software to have to issue a new version.
So, if you issue a new version of your software you not have the problem you outline. Those users don't have to get out of their current version since you are working on a copy of the program.
You thus need to adopt some system and code that will allow you to roll out that update to each user.
I explain in details not only should you run a split database with the data and tables separate (as you have), but ALSO why you distribute a compiled edition and ALSO why EACH desktop should receive a copy of this new program here:
http://www.kallal.ca/Articles/split/index.htm
So the ONLY basic concept you need to grasp here is that in the computer and software industry we have a concept of a computer program. Once you are able to grasp that you are creating a computer program, then you can NOW grasp the concept that such programs AFTER you create such a program or AFTER you modify or create a new version of that program, you THEN distribute that NEW program to your user base. This is quite much how all software works, and Access is no different in this regards.
You don't try to include one form into the production software, but add the new form, modify code and then COMPILE the software into an executable. In fact, using a compiled Access application is strongly recommend since un-handled errors NEVER re-set global variables and as a result your software becomes far more reliable and that even includes code without error handling since local and global vars are never re-set.
One solution would be to maintain a copy of all forms, reports, modules, queries, etc (front-end code) in a separate database from the tables. You can then link the tables to a 2nd database (back-end) in a shared network Access DB. The front-end database would be deployed to each workstation, while the back-end database is shared. Obviously, this introduces the need to update each workstation, but it means that when you need to roll out new front-end logic, you don't have to go around kicking everybody out at once.
Otherwise, sadly, you need to acquire an exclusive lock on the database, and the only way to do that, is to be the only one with it open.
With the help of a script, you could automate pulling the latest "client". You could just ask all users to exit the app and double-click a batch file which copies the front-end from some network location.
David Fenton recently mentioned in another thread that
"The only proper place for any Access app (since Windows 2000, in fact) is the folder the %AppData% environment variable points to."
I greatly respect David's knowledge, especially in all matters relating to Access, but I'm confused by this statement.
What is the advantage of following this advice, especially in an environment where you are going to have multiple people using the same computer to access your app?
Won't installing to this folder only install the app for one user? And if this is true, won't installing your app multiple times leave multiple, separate copies of your app on the machine? Hard drive space is cheap these days, but I still don't want a front end file and other supporting files (graphics, Word and Excel templates, etc.) copied multiple times onto a machine when one copy will do.
What are your thoughts? Am I missing something key to understanding David's advice?
Yes, this is an issue but the only way around it is, assuming the IT admins allow it, to create a folder in the root of C drive and install the Access FE database file in that folder. That said I'd stil use the Application Data folder even if files are duplicated. As you state hard drives are cheap.
This assumes you don't mean a Terminal Server/Citrix system where users are simultaneously logged into the system.
First off, this is an issue only for a workstation that has multiple users logging on to it. That's pretty uncommon, isn't it?
Second, you admit there's no issue with disk space, so the only real issue is keeping the front end up-to-date, and that issue is really completely orthogonal to the question of where the front end is being stored.
That issue can be addressed by using any of a number of solutions that automatically copy a new version of the front end when the user opens it (if needed). Tony Toews's Auto FE Updater is the best solution I know of. It's quite versatile and easy to use, and Tony's constantly improving it.
So, in short, I don't think there's any issue here at all.
If everything is always the same for every user on a given machine, then multiple copies of a file may not be such a good idea. But when that one exception occurs, you've painted yourself into a corner. They may need a different template version for example.
You seem to be in a rare situation for an Access developer.
You're running into a bit of an issue here, because you're thinking about the environment variable name %appdata%. That variable stores the directory returned by SHGetSpecialFolderPath(CSIDL_APPDATA).
What you're looking for is the directory returned by SHGetSpecialFolderPath(CSIDL_COMMON_APPDATA). There's no environment variable for that directory. This directory is (as the name indicates) common to all users.
The advantage of David's method is that the Access data is protected by NTFS access rights, when it's in CSIDL_APPDATA. A user can only delete his copy. In CSIDL_COMMON_APPDATA, anyone can delete the single shared copy.
It's probably always best to put these advice and tips into perspective. The assumption being made here is if your application is going to be utilized in a multi user mode (that means more than one user in the application of the same time), then it's pretty much assumed that your applications going to be split into two parts. The so called application part (front end), and then the data file only part, or so called backend part.
So, you have a FE and a BE.
In this environment, each individual user within your office will have their own copy of the application placed on their workstation. The BE (data file) is thus assumed to be placed on some share folder on a server.
In the case we're not going to have multiple users running this application, or the application is not really under development, then you really don't need to split your application into two parts. However if you split your application, it means all of your users can safely work and utilize your application while you work on a copy of the next great version of this application release. Without a split environment, you really can't have any workable development cycle.
It is a long time and honored suggestion that if you're going to use access in a multi user environments, each individual user must have a copy of the front end application placed on each individual computer. If you avoid this suggestion, the end result is instability in the general operation of your application.
I have an article here that explains on a conceptual level and doesn't just tell you two split your application, but explains well why you should split your application:
http://www.members.shaw.ca/AlbertKallal/Articles/split/index.htm
I've been tasked with finding a suitable tool to decompile MDE files. The MDEs were created by staff who have since left (familar story eh?) and we do not have access to the originally MDB files.
The reason we need access to the original code is that the data source is changing (the backend as well as some of the table and queries) and we need a way to update queries. An example of a change, in a SELECT statement where is the WHERE clause looks for zero as a string ("0") rather than an integer.
I'm aware that unless you use the services of people like EverythingAccess.com its unlikely you will ever get the source code back.
My main query is to ask for alternative methods to decompiling code. An example of the kinds of methods I'm thinking about is to spy on the traffic between the app the the ODBC DSN using tcpdump. I might then be able to write code to translate the data source queries between the old and new systems.
Ideally I'd prefer a solution that is application centric rather than one that analyses all network traffic.
I should add one caveat, no doubt most of you are thinking the best solution is to rewrite the code, based on its perceived functionality. This is the option we're not considering (at the moment).
Queries and linked tables can be changed in an MDE.
Alternatively see Converting a Microsoft Access MDE file back into an MDB file. Section 2 for some tips
I see you're already aware of MDE to MDB Conversion Service for Microsoft Access Wayne has stated he will require proof that you own the code.
i’m wondering if it’s possible to load code from an external file to be used in ms access. this is important to allow easy development using version control—.mdb files are impossible to version control.
See How do you use version control with Access development? for an excellent solution. I modified the vb scripts slightly to allow exporting and importing of queries, as well as the other types.
I use these scripts along with Mercurial to do my version control. Using Access 2002 this has been very reliable for me. I restored a previous changeset and rebuilt the MDB and it seemed to work with no problems. Also, remarkably few resources required. One of my projects has 12 changesets committed that take up a total of 16 MB in the repository, and the base mdb is about 10 MB itself. I highly recommend this approach.
You could write a small import/export tool (in the app itself, or in a separate MDB) which uses the Application.SaveAsText and Application.LoadFromText methods. These are undocumented, but once you've typed the method name into the editor it will prompt you with the argument types, which are: ObjectType As AcObjectType, ObjectName As String, FileName As String.
You might want to consider library database, i.e., and external MDB or MDE that has code that can be used in any MDB/MDE. If you're using an MDE for the front end, then it will have to be recompiled each time the library changes, which can be a real pain if you're trying to use a shared library MDE. This can be done with references, or by Application.Run, the same way you can call functions in the Access wizard database, e.g., the Zoom box:
Application.Run("UTILITY.BuilderZoom", "MyForm", "MyControl", "Values")
This is how you can launch the "zoom box" that you get in the Access UI when you hit Shift-F2 while in a textbox.
What it's doing is running the BuilderZoom function in the UTILITY database. There is no path specified because it's in the Access folder. If you have your library elsewhere, you'd have to supply the path to Application.Run.
I have created an MS Access 2003 application, set up as a split front-end/back-end configuration, with a user group of about five people. The front end .mdb sits on a network file server, and it contains all the queries, forms, reports, and VBA code, plus links to all the tables in the back end .mdb and some links to ODBC data sources like an AS/400. The back end sits on the same network file server, and it just has the table data in it.
This was working well until I "went live" and my handful of users started coming up with enhancement requests, bug reports, etc. I have been rolling out new code by developing/testing in my own copy of the front-end .mdb in another network folder (which is linked to the same back-end .mdb), then posting my completed file in a "come-and-get-it" folder, alerting the users, and they go copy/paste the new front-end file to their own folders on the network. This way, each user can update their front end when they're at a 'stopping point' without having to boot everyone out at once.
I've found that when I'm developing now, sometimes Access becomes extremely slow. Like, when I am developing a form and attempt to click a drop-down on the properties box, the drop-down arrow will push in, but it will take a few seconds before the list of options appears. Or there's tons of lag in selecting & moving controls on a form. Or lots of keyboard lag.
Then, at other times, there's no lag at all.
I'm wondering if it's because I'm linked to the same back end as the other users. I did make a reasonable effort to set up the queries, forms, reports etc. with minimal record locking, if any at all, depending on the need. But I may have missed something, or perhaps there is some other performance issue I need to address.
But I'm wondering if there is an even better way for me to set up my own development back-end .mdb, so I can be testing my code on "safe" data instead of the same live data as the rest of the users. I'm afraid that it's only a matter of time before I corrupt some data, probably at the worst possible moment.
Obviously, I could just set up a separate back-end .mdb and manually reconfigure the table links in the front end every time, using the Linked Table Manager. But I'm hoping there is a more elegant solution than that.
And I'm wondering if there are any other performance issues I should be considering in this multi-user, split database configuration.
EDIT: I should have added that I'm stuck with MS Access (not MS-SQL or any other "real" back end); for more details see my comment to this post.
If all your users are sharing the front end, that's THE WRONG CONFIGURATION.
Each user should have an individual copy of the front end. Sharing a front end is guaranteed to lead to frequent corruption of the shared front end, as well as odd corruptions of forms and modules in the front end.
It's not clear to me how you could be developing in the same copy of the front end that the end users are using, since starting with A2000, that is prohibited (because of the "monolithic save model," where the entire VBA project is stored in a single BLOB field in a single record in one of the system tables).
I really don't think the problems are caused by using the production data (though it's likely not a good idea to develop against production data, as others have said). I think they are caused by poor coding practices and lack of maintainance of your front end code.
turn off COMPILE ON DEMAND in the VBE options.
make sure you require OPTION EXPLICIT.
compile your code frequently, after every few lines of code -- to make this easy, add the COMPILE button to your VBE toolbar (while I'm at it, I also add the CALL STACK button).
periodically make a backup of your front end and decompile and recompile the code. This is accomplished by launching Access with the /decompile switch, opening your front end, closing Access, opening your front end with Access (with the SHIFT key held down to bypass the startup code), then compacting the decompiled front end (with the SHIFT key held down), then compiling the whole project and compacting one last time. You should do this before any major code release.
A few other thoughts:
you don't say if it's a Windows server. Linux servers accessed over SAMBA have exhibited problems in the past (though some people swear by them and say they're vastly faster than Windows servers), and historically Novell servers have needed to have settings tweaked to enable Jet files to be reliably edited. There are also some settings (like OPLOCKS) that can be adjusted on a Windows server to make things work better.
store your Jet MDBs in shares with short paths. \Server\Data\MyProject\MyReallyLongFolderName\Access\Databases\ is going to be much slower reading data than \Server\Databases. This really makes a huge difference.
linked tables store metadata that can become outdated. There are two easy steps and one drastic one to be taken to fix it. First, compact the back end, and then compact the front end. That's the easy one. If that doesn't help, completely delete the links and recreate them from scratch.
you might also consider distributing an MDE to your end users instead of an MDB, as it cannot uncompile (which an MDB can).
see Tony Toews's Performance FAQ for other generalized performance information.
1) Relink Access tables from code
http://www.mvps.org/access/tables/tbl0009.htm
Once I'm ready to publish a new MDE to the users I relink the tables, make the MDE and copy the MDE to the server.
2) I specifically created the free Auto FE Updater utility so that I could make changes to the FE MDE as often as I wanted and be quite confident that the next time someone went to run the app that it would pull in the latest version. For more info on the errors or the Auto FE Updater utility see the free Auto FE Updater utility at http://www.granite.ab.ca/access/autofe.htm at my website to keep the FE on each PC up to date.
3) Now when working on site at a clients I make the updates to the table structure after hours when everyone is out of the system. See HOW TO: Detect User Idle Time or Inactivity in Access 2000 (Q210297) http://support.microsoft.com/?kbid=210297 ACC: How to Detect User Idle Time or Inactivity (Q128814) http://support.microsoft.com/?kbid=128814
However we found that the code which runs on the timer event must be disabled for the programmers. Otherwise weird things start happening when you're editing code.
Also print preview would sometimes not allow the users to run a menu item to export the report to Excel or others. So you had to right click on the Previewed report to get some type of internal focus back on the report so they could then export it. This was also helped by extending the timer to five minutes.
The downside to extending the timer to five minutes was if a person stays in the same form and at the same control for considerable parts of the day, ie someone doing the same inquiries, the routine didn't realize that they had actually done something. I'll be putting in some logic sometime to reset this timer whenever they do something in the program.
4) In reference to another person commenting about scripts and such to update the schema see Compare'Em http://home.gci.net/~mike-noel/CompareEM-LITE/CompareEM.htm. While it has its quirks it does create the VBA code to update tables, fields, indexes and relationships.
Use VBA to unlink and re-link your tables to the new target when switching from dev to prod. It's been to many years for me to remember the syntax--I just know the function was simple to write.
Or use MS-Access to talk to MS-Access through ODBC, or some other data connection that lives outside of the client mdb.
As with all file base databases, you will eventually run into problems with peak usage or when you go over a small magical number somewhere between 2 and 30.
Also, Access tends to corrupt frequently, so backup, compact and repair need to be done on an frequent basis. 3rd party tools used to exist to automate this task.
As far as performance goes, the data is being processed client side, so you might want to use something like netmeter to watch how much data is going over the wire. The same principle about indexing and avoiding table scans apply to file base dbs as well.
Many good suggestions from other people. Here's my 2 millicents worth. My backend data is on server accessed through a Drive mapping. In my case, the Y drive. Production users get the mapping through a login script using active directory. Then the following scenarios are easily done by batch file:
Develop against local computer by doing a subst command in a batch file
run reports against last nights data by pointing Y to the backup server (read only)
run reports against end of month data by pointing to the right directory
test against specialized scenarios by keeping a special directory
In my environment (average 5 simultaneous users, 1000's of rows, not 10,000's.) corruption has occurred, but it's rare and manageable. Only once in the last several years have we resorted to the previous days backup. We use SQL Server for our higher volume stuff, but it's not as convenient to develop against, probably because we don't have a SQL admin on site.
You might also find some of the answers to this question (how to extract schemas from access) to be useful as well. Once you've extracted a schema using one of the techniques that were suggested you gain a whole range of new options like the ability to use source control on the schemas, as well as being able to easily build "clean" testing environments.
Edit to respond to comment:
There's no easy way to source control an Access database in it's native format, but schema files are just text files like any other. Hence, you can check them in and out of the source control software of your choice for easy version control/rollbacks.
Or course, it relies on you having a series of scripts set up to re-build your database from the schema. Once you do, it's normally fairly trivial to create an option/alternative version that rebuilds it in a different location, allowing you to build test environments from any previous committed version of the schema. I hope that clarifies a bit!
If you want to update the back end MDB schema automatically when you release a new FE to the clients then see Compare'Em http://home.gci.net/~mike-noel/CompareEM-LITE/CompareEM.htm will happily generate the VBA code need to recreate an MDB. Or the code to create the differences between two MDBs so you can do a version upgrade of the already existing BE MDB. It's a bit quirky but works.
I use it all the time.
You need to understand that a shared mdb file for the data is not a robust solution. Microsoft would suggest that SQL Server or some other server based database would be a far better solution and would allow you to use the same access front end. The migration wizard would help you make the changeover if you wanted to go that way.
As another uses pointed out, corruption will occur. It is simply a question of how often, not if.
To understand the performance issues you need to understand that to the server the mdb file with the data in it is simply that, a file. Since no code runs on the server, the server does not understand transactions, record locking etc. It simply knows that there is a file that a bunch of people are trying to read and write simultaniously.
With a database system such as SQL Server, Oracle, DB2. MySQL etc. the database program runs on the server and looks to the server like a single program accessing the database file. It is the database program (running on the server) that handles record locking, transactions, concurrency, logging, data backup/recovery and all the other nice things one wants from a database.
Since a database program designed to run on the server is designed to do that and only that, it can do it far better and more efficently that a program like Access reading an writing a shared file (mdb).
There are two rules for developing against live data
The first rule is . . . never develop
against live data. Not ever.
The second rule is . . .never develop
against live data. Not ever.
You can programatically change the bindings for linked tables, so you can write a macro to change your links when you're deploying a new version.
The application is slow because it's MS Access, and it doesn't like many concurrent users (where many is any number > 1).