I have a broad question that I would like some advice on.
Currently, I have a number of databases in a shared location on our company's network. When at the office, these can be accessed quickly (but through VPN, it's slow).
We have multiple locations around the country and can access the drives of each location to pull up excel sheets, pdf's, etc. The problem is, Access applications are Extremely slow when accessing another location's shared drive.
Is there anything that can be done to increase performance other than migrating to SQL server and using a web-based app? Just looking for general advice here.
The reason why the VPN is slow is well because your VPN is likely 50 to 100 times slower than your LAN (local area network). In effect you asking the following question:
Why does it take longer to walk to the store then to drive to the store?
Answer:
because you going slower along the way (not really rocket science here).
A few solutions are:
Consider moving the back end data part to SQL server. So while ADP's are being depreciated, this in no way affects the great choice and suggestion to keep the Access application as a front end and move the tables (data) to SQL server.
If you only need a few forms, then Access 2010 (and 2013) does now support web publishing. Here is a video of an Access application of mine, and note how at half way point I switch to running the application in a standard browser:
http://www.youtube.com/watch?v=AU4mH0jPntI
As noted, another great solution is to use some type of remote desktop solution. I explain why your connection is slow and give some suggestions for using Access on a WAN here:
http://www.kallal.ca//Wan/Wans.htm
So the basic issue here is your VPN is too slow and is far slower then your LAN.
You cannot increase performance over slow connections of an Access Database. There is no server serving records to the client, so Access has to download the whole dataset to do operations on it. You don't have to have a we-based app, though.
You can try to convert your .mdb to a .adp, or access data project. If successful, you will still use Access as the client, but all your data would be migrated to SQL Server. Queries on this data will be handled by the server, and you will see drastic performance improvements over slow connections.
Related
I am new to a new role at a company where they are using MS Access with a MySQL db which is running in server that's physically in our office behind our private network. I have been hired to develop an entire new application to bring the company up to modern standards. As we move features/modules to the new Angular/NodeJs App I am writing, users still need to utilize the UI provided by MS Access to the new production database that will be on AWS Lightsail.
However, when I change the configurations of Ms Access, OBDC connections to point to the AWS Lightsail MySQL Db, everything(reports especially) in the MS Access UI becomes slower than when it was being pointed to the MySQL Db here in office in-network.
I am going to the "Linked Table Manager" and changing the "Connection String".
Somewhere I read I should make sure SSLMODE is disabled to remove any performance issues.
DSN=AWS_Dev;DATABASE=ECSDataTables;PORT=3306;SERVER=IP_ADDRESS;SSLMODE=DISABLED;
I went through the normal "ODBC Data Source Administrator" in Windows and added the MySQL AWS host, user/pass as normal.
I have done extensive research and have found several sources, but none are really helping.
I have been asked not to spend too much time trying to fix/optimize anything in MS Access as my focus should be on the new application, but it's hard to believe that a simple switch of MySQL database can have such impact. In the new Angular/NodeJs application, everything runs very fast, so I know it's not the AWS MySQL db or anything.
Am I missing something, any configurations I should be doing in Ms Access? I have not used VB in about a decade, so I am hoping something can be done without the need of too much technical background in this matter.
Thank You.
Well, the issue is that your local area network (LAN) is about 10 times, or more faster then your internet connection.
Your low cost office network is very likely to be a 1 gig bit network. (100 base T is rare).
However, your internet high speed connection is likely say 10 mbits. So, you going from 1000 to 10 - that is 100 times slower. So, 3 seconds now becomes 300 seconds.
I mean, with such a slower connection speed, then no surprise should exist here.
What you can do is for any report that is a complex join of client side sql is convert the sql query to a server side view, link to that view. Now use that view as the base source for the report. And of course existing VBA filers that you always use (right???) to launch a report will now only pull the data it needs down the network pipe. Access reports (or forms) only pull down what you ask - not the whole table. So, any filter you have (use the where clause of the open report command) will be respected. So, you either have to pull less data, or simply find something with a similar speed rating as your local area network (and such high speed internet is rare).
The LAN vs WAN concept and speed issue is outlined in this article:
http://www.kallal.ca//Wan/Wans.html
While the above article is very old, the speed differences of the internet are about 10x faster today, but so is the local area that's gone from 100 baseT to 1 gig bit base.
So, things are slower because you are working with a VASTLY slower connection speed. Slower is slower!!!
Edit
While as noted, access will only pull what you ask, the case where access client does a poor job is sql queries that involve multiple tables - often the client will mess up what it sends server side. As noted, the solution in this case is to adopt views server side. This means you move the client side query that drives the report to a view, and link to that view. You not gain much performance for a single table query, but for any report based on complex (multi-table joins), then using a view will force the sql and "join work" to occur sql server side, and this can result in huge performance gains.
Well this is a case where limited knowledge just produces worst results than the expected ones.
Over the years top DBAs just "hate" Ms Access... they just see only problems,issues you name it ...the end sentence is "switch to a real Database engine".
Well this has created a faulty assumption that MsSQL, MySQL,Oracle, PostGreSQL and the rest database engines are somewhat a "magic pill"...you just switch the BE to one of the above DBE and all your problems will get resolved...just like that.
DBE --Database Engine (if you would like to call somewhat else feel free)
WRONG
Ms Access follows a different philosophy from the DBE and it does its job damn well given all its shortcoming and the major fact that is a file based DBE.
Switching to another DBE will give amazing performance IF and ONLY IF you respect the fact that you are not working with Access ....just don't treat e.g. MySQL as your file repository and DON'T just link the tables and expect everything to go well...
Want to keep blaming Access ...just switch over to another platform (.NET,PHP,Js , Java...make your pick) ...and do a small application that pulls ALL of your data in a single go like you do with Access . it will certainly crash or go Not responding...
So stop blaming Access ...start reading on how to make the most of two worlds and i am pretty sure that the results will amaze you....but again i must stress out that this is not a "magic pill" solution ...it involves a LOT of work ...planning,data manipulation,normalization,code changes and above all change of philosophy..
I would recommend starting the journey by picking this book : https://www.amazon.com/Microsoft-Access-Developers-Guide-Server/dp/0672319446 ( i don't want complains about its Old and MsSQL ...just read first and complain later)
Also take a look at an old benchmark alike video i made some years ago : https://www.linkedin.com/posts/tsgiannis_a-small-demo-of-connecting-ms-access-fe-to-activity-6392696633531858944-dsuU
Last but not least....years ago i was making some tests to see what the "magic pill" would do to my company's applications....the simplest test of all...
A simple table with few fields but with around 8 millions records...just display it
Access BE (local)--> It would run in 1-2 seconds...that's fast
Access BE (Network share)--> It would run in a few seconds...not so fast but it was usable
MSSQL BE (linked table)--> somethimes it get the results sometimes it wouldn't....slow...really slow ..like you make a coffee and go for a small walk.
MySQL BE (linked table)--> it never finished...timeout of "Not Responding"
PostGreSQL BE (linked table)--> it never finished...timeout of "Not Responding"
So stop blaming Access...start working and get amazed....
I am developing an application which uses as a back-end an MS Access database (.mdb, not my decision). Recently I came across someone suggesting that using JET engine over WAN is not really a good idea, with a high risk of data corruption. Since my application should be doing just that (connecting to database on NAS (EDIT: not NAS, shared shared network drive), I got worried. It is really that risky? If so, is there any work around or is an MS Access database just unusable for that kind of application?
EDIT
The front end is .NET windows desktop application in C# (WPF). The system does not have many users, max 10. Most of the time they will approach the database from LAN and 99% of writing to the database will be done within the LAN (from the area of the company). However there are some cases where they will connect to the NAS (EDIT: not NAS, shared shared network drive) from outside the company via network (from their home).
If you have a 100 Mb/s fibre, it will be OK, but if your line is, say, an xDSL line, it is generally an absolute no-no.
Convince the powers that be to move the backend to a server engine like SQL Server where the Express version is free.
The scenario you describe is not a good fit for having an Access database as the back-end. The WAN users could very well find the application slow, but the NAS is the real cause for concern regarding corruption, and that would affect both LAN and WAN users.
Many (most?) NAS devices run on Linux and use Samba to provide Windows file-sharing services. The Access Database Engine apparently uses some low-level features of "real" Windows file sharing that Samba does not always fully implement (ref: here).
In fact, the only time I've seen repeated corruption problems with a shared Access back-end (and a properly distributed front-end) was when a client moved their file shares from an older Windows server to a newer NAS device. The Access application continued to work for the most part, but every few months they would find that the primary keys of some tables would disappear after they did a Compact and Repair on the back-end database file. That never happened while their file share was on the Windows server.
Splitting a front-end from a back-end removes the majority of the risk of corruption. Of course, with Access there's always the possibility and if you're looking for something that reduces the risk to close to nil then you might want to consider SQL Server or MySQL. However, using Access is fine as long as you take proper precautions.
For example, you might want to look into record-locking on tables that will get edited, to prevent multiple simultaneous writes. Backing up your DB on a regular basis is always good, too.
So, we want to move out from Air (Adobe stopping support and really bad implementation for the sqlite api, among other things).
I want to make 3 things:
Connect with a flash (not web) application to a local mysql database.
Connect with a falsh (not web) application to a remote mysql database.
Connect with a flash (web) application with a remote mysql database.
All of this can be done without any problem, however:
1 and 2 can be done (WITHOUT using a webserver) using for example this:
http://code.google.com/p/assql/
3 can be done using also the above one as far as I understand.
Question are:
if you can connect with socket wit mysql server, why use a web server (for example with php) to connect like a inter connectioN? why not connnect directly?
I have done this a lot of times, using AMFPHP for example, but wouldn't be faster going directly?
In the case of accessing local machine, it will be a more simple deploy application that only require the flash application + mysql server, not need to also instal a web server.
Is this assumption correct?
Thanks a lot in advance.
The necessity of separate layer of data access usually stems from the way people build applications, the layered architecture, the distribution of the workload etc. SQL server usually don't provide very robust API for user management, session management etc. so one would use an intermediate layer between the database and the client application so that that layer could handle the issues not related directly to storing the data. Security plays a significant role here too. There are other concerns as well, as, for example, some times you would like to close all access to the database for maintenance reasons, but if you don't have any intermediate layer to notify the user about your intention, you'd leave them wondering about whether your application is still alive. The data access layer can also do a lot of caching, actually saving your trips to the database, you would have to make from client (of course, the client can do that too, but ymmv).
However, in some simple cases, having an intermediate layer is an overhead. More yet, I'd say that if you can, do it without an intermediate layer - less code makes better programs, but all chances are for that you will find yourself needing that layer for one reason or another.
Because connecting remotely over the internet poses huge huge huge security problems. You should never deploy an application that connects over the internet to a database directly. That's why AIR and Flex doesn't have remote Mysql Drivers because they should never be used except for building development type tools. And, even if you did build a tool that could connect directly, any descent network admin is going to block access to the database from anywhere outside the DMZ and internal network.
First in order your your application to connect to the database the database port has to exposed to the world. That means I won't have to hack your application to get your data. I just need to hack your database, and I can cut you out of the problem entirely because you were stupid enough to leave your database port open to me.
Second most databases don't encrypt credentials or data traveling over the wire. While most databases support SSL connections most people don't turn it on because applications want super fast data access and they don't want to pay for SSL encryption overhead blah blah blah. Furthermore, most applications sit in the DMZ and their database is behind a firewall so between the server and the database is unlikely something could be eavesdropping on their conversation. However, if you connected directly from an AIR app to the database it would be very easy to insert myself in the middle and watch the traffic coming out of your database because your not using SSL.
There are a whole host of problems doing what you are suggesting around privacy and data integrity that you can't guarantee by allowing a RIA direct access to the database its using.
Then there are some smaller nagging issues like if you want to do modern features like publishing reports to a central server so users don't have to install your software to see them, sending out email, social features, web service integration, cloud storage, collaboration or real time messaging etc you don't get if you don't use a web application. Middleware also gives you control over your database so you can pool connections to handle larger load. Using a web application brings more to the table than just security.
I am developing a database with about 10 tables in it. Basically it will be used in 2 or 3 distant geographical locations (let's call them A,B and C). The desired work flow will be as follows:
A,B and C should always have the same database. So when A does any changes he should be able to send those changes over to B and C. Emailing the entire mdb file doesnt make sense since its 15+mb in size. So I would like to send the new additional records and changes only to B and C. The changes B and C make should also be reflected to the other repective parties. How can I do this?
I have a few ideas in mind but cont know how to implement it.
solution 'A' - export the data tables only into a xls file and email that. But the importing of the tables into the mdb file could be a bit complex right? and the xls is file will also become bigger and bigger with time.
solution 'B' - try extract just the changes and email only the new parts? (but how to extract just those)
Solution 'C' - find some way of syncing all users onto the same database(storage) location. I was thinking of a front/back end splitting solution by storing the tables in a shared drive in the parent company's server (which is also overseas). But the network connection between locations is very slow, and I dont know how much bandwidth is needed for this.
Any recomendations would be most welcome!
In regard to sources for information on replication, start with my Jet Replication Wiki.
But I would never recommend Jet replication for your scenario. The only environment where I currently recommend it (and I've been doing replicated apps since 1997 and still have several in production use) is for supporting laptop users who have to work with live data in the field disconnected from any network, and return to the home office and synch direct with the mother ship.
The easiest solutions with an Access application would be hosting the app on Windows Terminal Server/Citrix and the users would run it over a Remote Desktop Connection, or using Sharepoint. The Terminal Server/Citrix solution has no accomodation for disconnected users, but Sharepoint can accomodate offline usage and synch changes when connected. Access 2010 and Sharepoint 2010 provide a host of new features, including better schema design, the equivalent of triggers and greatly improved peformance for large Sharepoint lists, so it's a no-brainer to me that if you choose Sharepoint you'd want to use A2010 and Sharepoint 2010.
While it's possible to do what you want with Jet Replication, it requires a lot of setup on the server and client ends, and is relatively fragile (not in terms of data integrity if you're using indirect replication (as you should), but in terms of network reliability) -- there are too many moving parts and too many failure points.
Windows Terminal Server/Citrix is by far the simplest, with the fewest moving parts and completely centralized administration, and works very well for a relatively small investment.
Sharepoint is more complicated than WTS/Citrix, but is less complex and more centralized than a Jet Replication solution.
If it were me, I'd probably go with WTS/Citrix if there was no need for disconnected usage, but I'd be salivating over trying out A2010/Sharepoint 2010. If there was a need for disconnected usage, then I'd definitely go the Sharepoint route.
You want to use "Jet Replication". See
MSDN Search for jro at http://social.msdn.microsoft.com/Search/en-US?query=jro&ac=8
MSDN Search for access replication at http://social.msdn.microsoft.com/Search/en-US?query=access%20replication&ac=3
It's been some time since I did it, but the indirect method of replication worked well for me in a similar situation.
It takes something to set up. The documentation used to be appalling for it, but I found articles written by Michael Kaplan (aka Michka) that walked me through how to do it.
If your final environment is going to be fairly stable, then use Access the whole way. If not, then I'd urge you to take HansUp's advice and go with SQL Server or SharePoint.
Do note: if you're working in Access 2007 or later, replication is not directly supported, and you'll have to roll-your-own bits and pieces. If you're using an earlier installation, you'll be fine, but allow time for some head-scratching.
I have a product designed to be a desktop product using MS Access file as a DB.
Now, some users need to install it in a few PCs (let's say 2 or 3) and SHARE the database.
I thought to place the MS Access file in a shared folder and access it from the PC, but... the JET Engine is designed for multiple user access?
Any tips or things to be aware of doing this?
EDIT:
The app is a .net one, using the database as storage (not using the database as frontend)
There is so much misinformation in the answers in this thread that I don't know where to start. I just spent 4 points in reputation voting down the answers with misleading and wrong information in them.
the Jet database engine (which is all that's involved here, as the OP clarified with an edit) is by default multi-user -- it was built from the ground up to be that way.
sharing a Jet data store is very reliable when the network is not substandard. This means not a WAN and not wireless, because the bandwidth has to be sufficient for Jet to maintain the LDB file (for multi-user locking), which means a ping by your local PC's instance of the Jet database engine once per second (with default settings), and because Jet can't recover from a dropped connection (which is quite common in a wireless environment).
the situation where Access falls down is when a front-end Access application MDB is shared (which is not the case for this poster). The reason it fails is because you're sharing things that can't be reliably shared and have no reason to be shared. Because of the way Access objects are stored in an MDB file (the entire Access project is stored in a single BLOB field in one record in one of the system tables), it's very prone to corruption if multiple users open it. In my estimation, sharing an Access front end (or an unsplit MDB with the tables and forms/reports/etc. all in one MDB) is the source for 99.99% of corruptions of Access/Jet files.
My basic answer to the OP's question is that, yes, Jet would be a great data store for an app of that size. However, if there's any possibility at all for the user population to grow above 25, then it might be better to start off from scratch with a database engine that is more robust at higher user populations.
It's perfectly feasible to do this; but you MUST split the database into a front end (with forms, queries, code) and a back end (data only). Every user has to have the front end on their own computer, linking to the shared back end.
It will be slow as Jet generates a ton of network traffic. Microsoft is also gradually deprecating Access as a development tool. Access 2007, for instance, has a far less sophisticated security model than Access 2003.
As a long time Access developer I am gradually moving away from Access.
Don't do it... the Jet database claims to be able to support multiple users, but it is incredibly easy to use the upsizing wizard to convert your Access file to a Sql Express database. That database file could EASILY become locked by a user or admin, and all of your users would be unable to use the database.
... and Sql Express is free. Your upgrade path from there to a full instance of Sql Server or some other commercial database is simple.
With 2 or 3 users on a reliable local network you should be fine, as long as you back the network drive up often.
Avoid any bit/bool fields in your tables - Jet has some nasty corruption issues with multiple access to them.
Also bear in mind that all locking in Access is optimistic: you will get dirty reads occasionally.
MS Access is designed for small office scenarios like this: non-critical light office use that you can set up with the minimum of programming.
Expect the data file to get corrupted every now and then - back up regularly.
The ACE/Jet engine is a great piece of software but, while it was designed to support multiple users, actually supporting multiple users in practise is not one of its strong points. The last straw for me is where then removed user level security (ULS) from the engine: I suppose I can imagine a simple database situation where all users will have the same privileges (i.e. admin access to all database objects) but IMO that is not supporting multiple users well, as compared with, say, MS SQL Server.
Yes, it supports access by multiple (that is, a small, workgroup-sized, number) of users over a network file share. However, the file share architecture is simply not ideal for supporting simultaneous writing to a file by multiple users. A client/server database system (SQL Server, etc.) generally provides better performance, security, and reliability.
As a sysadmin, please don't use Access for anything multi-user. Do what Jeff Fritz suggests and use a database that is designed for multi-user access. You may think that your little app is only going to be shared between a few people, but I guarantee you that it'll have a hundred users and fifty new features by the end of the year. And if those are all Access, rather than VB/SQL Express, your Ops people will break into your house one night and slit your throat.
Access isn't a client-server app, and provides very little in the way of backup/restore, or any automation whatsoever. Not to mention the interface and the DB are very tightly coupled... so if you ever want to turn this into a web app, or make any serious changes, your world will be filled with pain.
It's been done so many times by so many generic software engineers where we've seen a .mdb go corrupt in a multi user situation. If so many experienced specialist Access developers can get it right, as I'm inclined to believe, then we generalists must be doing something wrong and that something must be fairly fundamental yet non-obvious for so many of us to run away from the thing screaming 'Never again!' So if you consider yourself to be a experienced specialist Access developer (or you know how to find one) then go for it. But if you are a generalist or casual user looking for a lightweight back end then I suggest you look elsewhere (SQL Server is good IMO).
If your users can wait twice as long for an application with half of the features they want, then don't use Access.
Jet does not have the sophisticated lock logic required to support multi-user scenarios. You can get away with using it if your application is mostly reads and low-contention.
I've seen websites support many users, but I would recommend SQL Express unless you have a compelling reason to choose Jet.
I can tell you from painful experience that Jet 3/3.5 was not reliable. I saw it crash frequently under light load and when there were crashes you risked data corruption. It used to be extremely sensitive to any power problems, any client crashing against it (even the UI linked to the mdb), and any LAN problems. More recent versions of Jet might be better but switching to Sql Server is clearly the way to go in my opinion for anything other than trivial data entry with a small number of users. Sql Express is free and you don't really lose anything, especially if you're UI is in .Net, rather than Access.
EDIT: Microsoft doesn't think you should rely on Jet 4 either.
from: http://support.microsoft.com/kb/303528
Microsoft Jet is not intended for use with high-stress server applications, high-concurrency server applications, or 24 hours a day, seven days a week server applications. This includes server applications, such as Web applications, commerce applications, transactional applications, and messaging server applications. For these types of applications, the best solution is to switch to a true client/server-based database system, such as Microsoft Data Engine (MSDE) or Microsoft SQL Server. When you use Microsoft Jet in high-stress applications such as Microsoft Internet Information Server (IIS), you may experience any one of the following problems:
Database corruption
Stability issues, such as IIS crashing or locking up
Sudden failure or persistent failure of the driver to connect to a valid database that requires re-starting the IIS service
just check whether the db lock file (like .ldb) is there or not. If it is there, somebody is accessing that file. If it is not there, at present there is no one accessing that file and you may proceed. Otherwise, wait for when that file (.ldb) is no longer existing.
If you use a Terminal Server, the performance is real good. We have more solutions up to 50 Users at one Access mdb. Development is real fast and deployment easy.
Problems:
everybody can copy data mdb
no access rights
limited store procedures
optimize (compress and repair) only possible with no use data Database
limit to 2 GB!