I have a large MS Access database, and I want to get rid of all the records that aren't relevant to my city. I could make a series of queries, but that would be tedious. And I don't know an efficient way to get just the data from the queries into another database, or into Excel. I only know tedious ways.
In addition to tedious queries, you can write tedious VBA code.
Loop the recordset and validate each record. If it fails, delete the record and move to the next.
Even though tedious, it is more fun than writing SQL.
Related
I'm trying to write some VBA code that will export an MS Access query.
I'd like it to be run it all from within Excel to source from a database I don't understand and don't want to tamper with. I just know I have to run a query and export it.
That's easy enough if the query doesn't rely on a user-input value to run (EG beginning/end dates) but if it needs input I can't find a good way to make that be set automatically.
I've seen some clever options here:
https://bytes.com/topic/access/answers/466248-can-i-pass-parameters-macro-query
A) Putting the SQL underlying the query within the VBA, but that can get very unwieldy, especially with big queries that I'm unfamiliar with. And TBH I don't quite understand the coding behind it- but that's just me being lazy...
B) Use a form on which to record the desired value- again unwieldy and also I can't change the query code.
But I'd rather something along the lines of: cmdRun(QueryX,PARAM1,PARAM2). Is that wishful thinking?
I am coding a small app and have a question. i am deciding if storing data in JSON or mysql row is best for my scenario.
I am coding an app that may have lots of page hits, and because of that, i am thinking to store JSON encoded array into a column VS mysql rows. One query will be faster to execute VS 2.
The problem i am trying to figure out is i need to delete part of the JSON encoded array, but doing that means upon delete request, i will have to get the entire data, JSON decode and unset the object then update the row again. VS mysql delete row.
Is there a way that maybe i don't know of that can make this much easier to handle?
It probably depends on details you're not providing.
Do you have to use MySQL? If you're fetching a JSON object, and then modifying it and storing it back again, MongoDB seems faster for that use case.
One query will be faster to execute VS 2.
You don't need more than one query to return several rows; the query might return more rows, but looping over results and serialising/deserialising JSONs are both negligible costs compared to other things you will have to do on your site. Don't think too much into this.
As a rule of thumb, on a relational database, try to normalise the data until you see performance issues. If you're set to use MySQL, you probably want many rows. As your dataset grows, the most straightforward way to improve query performance will be to add indexes, and you won't be able to do that on a JSON blob.
Basically, I have three tables that contain all the data that I want, but I am having to do some crazy JOIN and WHERE statements that are not working for me. I have finally resorted to using temporary tables, but I was wondering if there is a more long term solution.
The Situation: We Pull large amounts of data via SOAP to our database, we have no control on how the data is organized, combined together, labeled and etc., we need to split it up the best we can so that it can eventually become useful to us.
What I am asking is how do the pro's or etc. "Prep" data so that it can either be finally inserted into a usefully table via other tables quickly, and how does it stay updated with the flow of new data coming in? What is the terminology? What should I research?
Thanks in advance!
The terminology I use for that is for preparing the data and getting ready for insertion is "staging" the data. We typically insert/update rows into temporary staging tables.
We massage and tweak the data in the staging tables, assigning foreign keys, fixing malformed data, splitting big multiuse fields into individual fields, and so on, to get the data cleaned up BEFORE the rows are inserted into the actual target tables.
(I don't know that this is a standard terminology, others may refer to it differently.)
FOLLOWUP
To improve query performance for complex data, we sometimes store pre-joined and pre-calculated results. Basically, we populate a table with "query ready" results, to make for much simpler queries of historic data. The big drawback to this is that we now have redundant data, which can become "out-of-sync" with the operational data. We use a scheduled (nightly) process to re-populate these tables.
(I'm unsure of the industry standard jargon for these types of tables.)
In terms of researching this, these approaches are going to be described in articles/white papers on "data warehouse" and "data mart". And that's almost always described as "ETL" The three major steps: Extract - Transform - Load. There's also a lot of noise in the industry press about "data mining" and "big data".
I have to produce a pretty complex report. Without a doubt someone will ask me for the individual records that make up the counts, totals etc.
Im new to mySql. I have tried and found out that I can store Sql statements as data of type Text without any real problems.
Being able to do it, however begs the main question:
Is this a good idea? Is there anything here that will hit me with a "Got ya!"
(Its always what you don't know that gets you, never what you do know)
The report will return Sales Count: 5
I would Live to store "SELECT * FROM Sales WHERE Producer_ID = 34" so I can quickly get to the 34 records that make up the 5 Sales count
Storing SQL queries as data in a SQL DB is sometimes labelled dynamic SQL queries. It is prone to injection issues, and should not be undertaken lightly. I recommend not doing so unless you have a very good reason to do so.
I'm trying to figure out how MS Access is handling open database objects when I build and run reports.
I have dozens of queries in each report and, since my database is normalized, each query is looking at many different tables. However, I always seem to reach the point where I have so many database objects open, that Access won't let me include any more queries in the report. My idea of how the reports are being generated is probably naive, but I just assumed Access would run a query, place the data in the reports, close the query and all associated objects, and repeat. But this doesn't seem to be the case.
The only reason I even care is because I am trying to speed up my reports. I read somewhere that you shouldn't build your tables to make querying easy, but in an effort to avoid the 7 or 8 minute time-void that happens every time I try to run a report, I have re-constructed the tables so the queries can use less objects when generating the data. The tables are still normalized to an extent, but I guess the only down-side is I am technically storing calculations in the table. For example, I have to report the current age of our customer base, and instead of grouping them with a simple switch statement (which requires me to query a separate table that holds the birthday), I have placed the age ranges (18-22, 23-27, etc) directly in the table.
Anyways, if someone could enlighten me about how access is closing the database objects, maybe I can pinpoint why my reports are so slow.
Thanks
Supposing that you really have (correct) normalised tables, and the tables are really impossible to reduce, and you really need to append all those subreports in your main report, you should create temporary tables to set as your report's DataSource. It will be a little bit boring, but you can create a step where your queries are executed and generates one or more temporary tables. When these tables is created with the filtered data which you desire in the report, you should open the report much more quickly than before. After the report's exhibition/print you should drop the temp tables or mantain them just until you need those specific data, like a buffer.
Otherwise, I suggest you to consider the entyre design of your DB, if it is possible.