This may seem like an obvious question for those that use relational databases frequently, however, I'm only just getting started with them - in my case, ms-Access.
What I was wondering was when people talk about 'data manipulation' they frequently mean amended errors, cleaning up data in some form or another and generally updating a database in someway. I have a long history with using excel, and as such it feels quite intuitive for me to do things using this software. However, when you start incorporating a data management system like Access does one tend to manipulate data directly in Access- or for example, does one export data to excel, do what is required to the data and then export it back into the data base (this with my limited knowledge of Access seems like the obvious choice).
Thanks for the advice - furthermore if anyone has any good recommendations to up my 'Access game' quickly (possibly tutorials) that would be amazing.
I have been through the same path you're taking now, and have had the same question before. It seemed like a struggle to relearn how to do a simple task in Excel using only Access. However, Access (or any SQL-based Management Studio) gives you a more robust system to work with.
For example, if you're cleaning data in Excel using VLOOKUPs to insert the proper values from a separate worksheet, it can be easy to write the equation but leaves you vulnerable to copying bad data (one row is an integer value instead of a string). With Access, there are better controls over the manipulation of data that will throw errors when it runs into unexpected values like that.
Furthermore, Access (or SQL in general) is much better at blending multiple datasets, validating data, and maintaining data integrity. Excel can do most of these tasks, but the processes are much more labor intensive.
Having said all of that, there are still situations where I might dump the results of a query into excel to verify calculations were done appropriately, sorting and filtering can sometimes be faster, and quick analyses can be easier to do in Excel, but that could be a function of my inexperience.
The steps I took in developing my Access knowledge was stepping back and learning about database design. I think that's the largest gap between Excel and Access: the relational model. After understanding the basics of relational databases, I suggest learning some SQL. I decided to step back again and learn T-SQL (slightly different syntax than the variation Access uses: Access-SQL, but is more mobile). If you can learn to write good queries with Access's GUI first, it makes it easier to apply SQL knowledge by transferring the GUI understanding to the outputted SQL code in Access.
Access is also far better with front-end development than Excel. Once you understand how the back-end works, the front-end is much easier to pick up. From there, you may consider learning some VBA to achieve more advanced solutions. Luckily, VBA also applies to Excel, so by that time you should be proficient at swapping back and forth between platforms, depending on the project.
For me, I used "SQL Queries For Mere Mortals" as my guide/tutorial for tackling database design and SQL understanding.
Related
I am currently doing a project in Ruby on Rails and I have been presented with a dilemma.
The dilemma is that the users of my system will be uploading an excel spreadsheet. The issue is should I just read straight from this excel spreadsheet into my front-end or should I load this spreadsheet into my MySQL database and then to my front-end.
I have asked numerous people about this issue and have researched on-line to no avail.
Any help would be much appreciated.
The Excel file is not a database. If you need to allow it as source input, parse it, copy the data into a real database and connect to it.
The database is more flexible and efficient for querying and processing information.
I can think of two benefits, or rather options, of having them upload the excel spreadsheet for processing by your back end.
1) would be for your tracking purposes (who sent what and here is what the back-end did with it...). In fact consider that other formats/versions could be introduced, would it be important to keep them to identify what went wrong? "How can we handle this new format"?
2) On the other side, the front-end way that is, you offload processing from the back-end, but that means that the browser app could get fairly complex and depending on your excel, that is if it has many relationships, sending that data up to the server could be complex. However if is simply a flat spreadsheet, say simple rows without totals/tax calc/..., then it might be an advantage of loading it into the browser and then sending these rows up to the server if offloading processing is of any importance.
However point 2 really is diluted by point 1, which to me would be of greater importance for future migration of this service. So I personally would choose uploading it and processing on the back end.
Update
As you clarified in the comments, if you are asking about the use of Excel on the backend as a database? I would agree with Simone Carletti's answer here. Maybe just add a real database gives you much more flexibility, more tools and, more performance. This difference is loading a file, parsing it into some structure, then saving it (unless you are using some .NET framework and even if, the Database (MySQL, MongoDB...) would give you much more flexibility in structuring and querying, over the headache of managing with the speed of DB connections. You might just want to write a sample in both to evaluate, the DB solution will probably win you over.
I am trying to build a user interface which will allow users to choose what kind of information they want, then based on these selected conditions, I need to query an access database and retrieve the corresponding data for future calculation. During calculation, there are two additional tables that I will query from. Finally, I need to display the calculated results to users. Now I have some questions:
Which one will be easier if I use excel as a front-end to users and retrieve querying data from Access to excel vs. I use access forms as a front-end to users and directly work with access to query data?
Does access forms can perform functions like allow users to select options from a drop down list? and once options selected, Access will query the target info?
Can Access perform intermediate calculation for large amount records? Will it get very slow? Compared to excel, which one is better in terms of calculation?
I never use Access before and just know a little about Excel VBA, not totally familiar with it, not to mention object models. So, in this case, which way is time-saving for me? or they are probably the same time-consuming?
Anything else that I should be aware of?
Thanks so much!
Bing
Definitely Access - it is specifically made for working with, querying, and reporting on data. The language of VBA that you have experience with in Excel is the exact same language used for coding in MS Access!
Access, no question!
Absolutely. Of course it doesn't magically know this as your requirement. You have to drop the combo box on the form yourself, and then in the OnChange event of the combo box, alter a query statement that displays results to filter based on the user's selection.
Sure. Make a temp table (a table that is only used for the calculation), populate the records with INSERT statements, apply intermediate calculations with UPDATE statements, and display the results. Access is built for this kind of thing and will most likely be faster than Excel.
Access uses VBA as well. There may be a larger learning curve to switch, but if you are familiar with the user interface objects in Excel such as Button, Combo Box, etc., they will be very similar and yet more intuitive in Access (Because that's what Access was made for)
Access has some quirks. Google a few tutorials and get some background on the ways Access is traditionally used for your kinds of scenarios and it will go a long way toward streamlining your development.
Start with the simplest model that will accomplish what your requirements are, and implement it. Then work through the quirks and bugs that you find with Google as your friend, and you should be able to get something solid pretty quickly! As you get into the experience, feel free to post new questions with specific problems you find along the way.
Best Regards,
We are currently having an OLTP sql server 2005 database for our project. We are planning to build a separate reporting database(de-normalized) so that we can take the load off from our OLTP DB. I'm not quite sure which is the best approach to sync these databases. We are not looking for a real-time system though. Is SSIS a good option? I'm completely new to SSIS, so not sure about the feasibility. Kindly provide your inputs.
Everyone has there own opinion of SSIS. But I have used it for years for datamarts and my current environment which is a full BI installation. I personally love its capabilities to move data and it still is holding the world record for moving 1.13 terabytes in under 30 minutes.
As for setup we use log shipping from our transactional DB to populate a 2nd box. Then use SSIS to de-normalize and warehouse the data. The community for SSIS is also very large and there are tons of free training and helpful resources online.
We build our data warehouse using SSIS from which we run reports. Its a big learning curve and the errors it throws aren't particularly useful, and it helps to be good at SQL, rather than treating it as a 'row by row transfer' - what I mean is you should be creating set based queries in sql command tasks rather than using lots of SSIS component and dataflow tasks.
Understand that every warehouse is difference and you need to decide how to do it best. This link may give you some good idea's.
How we implement ours (we have a postgres backend and use PGNP provider, and making use of linked servers could make your life easier ):
First of all you need to have a time-stamp column in each table so you can when it was last changed.
Then write a query that selects the data that has changed since you last ran the package (using an audit table would help) and get that data into a staging table. We run this as a dataflow task as (using postgres) we don't have any other choice, although you may be able to make use of a normal reference to another database (dbname.schemaname.tablename or somthing like that) or use a linked server query. Either way the idea is the same. You end up with data that has change since your query.
We then update (based on id) the data that already exists then insert the new data (by left joining the table to find out what doesn't already exist in the current warehouse).
So now we have one denormalised table that show in this case jobs per day. From this we calculate other tables based on aggregated values from this one.
Hope that helps, here are some good links that I found useful:
Choosing .Net or SSIS
SSIS Talk
Package Configurations
Improving the Performance of the Data Flow
Trnsformations
Custom Logging / Good Blog
I've been tasked with finding a suitable tool to decompile MDE files. The MDEs were created by staff who have since left (familar story eh?) and we do not have access to the originally MDB files.
The reason we need access to the original code is that the data source is changing (the backend as well as some of the table and queries) and we need a way to update queries. An example of a change, in a SELECT statement where is the WHERE clause looks for zero as a string ("0") rather than an integer.
I'm aware that unless you use the services of people like EverythingAccess.com its unlikely you will ever get the source code back.
My main query is to ask for alternative methods to decompiling code. An example of the kinds of methods I'm thinking about is to spy on the traffic between the app the the ODBC DSN using tcpdump. I might then be able to write code to translate the data source queries between the old and new systems.
Ideally I'd prefer a solution that is application centric rather than one that analyses all network traffic.
I should add one caveat, no doubt most of you are thinking the best solution is to rewrite the code, based on its perceived functionality. This is the option we're not considering (at the moment).
Queries and linked tables can be changed in an MDE.
Alternatively see Converting a Microsoft Access MDE file back into an MDB file. Section 2 for some tips
I see you're already aware of MDE to MDB Conversion Service for Microsoft Access Wayne has stated he will require proof that you own the code.
I'm trying to split a database into two pieces -- a backend that updates automatically, and a front-end that allows searching and adding/editing comments. The data in the source database is pulled together from multiple tables into a pair of queries, and I want to use these queries as the source of the current database.
Access 2007 supports splitting a database into multiple pieces, but not in the way I'm looking for. It keeps the tables in the source database and puts all the forms, queries, reports, and macros into the new database. The tables and queries are already in the back-end, and this new database should just provide a good GUI to the end-user.
Access 2007 also supports linked tables, but these can only use a table as a source, not a query object.
I was thinking that the best way to do this would be to do a SQL query along the lines of
SELECT * FROM SourceQuery IN "C:\Path\To\ExternalDB.accdb";
Is what I'm working towards even possible, and would this be the best way to do it?
Since its still relatively early in the project, rearchitecting the database isn't out of the question, but is something I'd prefer to avoid.
You described the usual Access BE-FE division correctly: only tables in the back-end. I'm aware not all DB programs do it that way, but this is Access and my approach would be to honor the usual division. (And you hardly have a choice in that you can't "link to a query" in Access.)
Reviewing your comment ('There is a specific reason ...'), I think this would possibly mean
adding a few more tables to the back-end, essentially buckets (import-data in ready form; export 1; export 2) that allow all users to get to consistent processed data;
making a small admin FE that sits next to the BE and stores your modules, queries for export, and export routines; and
having some redundant queries on the user FE. This is vexing in my own work. I just try to design sturdy stable "building block" queries in those roles, and keep their number to a minimum.
Hope I'm understanding you correctly, but the most sensible solution would be to link the tables in the backend DB and copy the queries to the UI database. Those queries would still be able to access the uderlying tables (via the linked tables) without issues and would be accessible through normal means to your forms and VBA code.
Is there a particular reason you don't want the queries in the UI database?