So I made a little mistake when I initially setup on of my Access tables and need help fixing it (as easily as possible).
I have one table called Locations (small table, 10s of rows) and another called Encounter Data (large table, 100,000s rows). The Encounter Data has a lookup column that points to Locations.
My mistake I made was to store the location name and not its primary key in the Encounter Data column. Now, obviously, if I make a change in the Locations table to the name of a particular item, the lookup in Encounter Data fails. What I should have down was store the Primary Key (then I can edit all the Locations information freely).
I am to fix that.
But I am not sure how to go about inserting the correct ID in a new column in place of the name.
My thought was to simply add a new column, create the lookup correctly (store the primary key), and then to either: [1] sort my Encounter Data by location and do a quick-drag/auto-fill like I would in excel to match up the correct IDs with the stored names or [2] do a find/replace ... only find a value in LocationWhoopsColumn and enter the correct ID in the LocationCorrectColumn.
But neither of these methods seems feasible. I have very few location but a lot of rows to fix ... how do I do this quickly? Manually changing row-by-row would take hours if not days.
Thanks!~
Create the new column LocationId in Encounter Data and then run
UPDATE [Encounter Data] a INNER JOIN Locations b ON a.LocationName = b.Name
SET a.LocationId = b.id
check the results and then drop column LocationName when satisfied.
"... find a value in LocationWhoopsColumn and enter the correct ID in the LocationCorrectColumn"
Use DLookup to retrieve the primary key value from Locations based on matching the location name field to LocationWhoopsColumn.
UPDATE YourBigTable
SET LocationCorrectColumn =
DLookup(
"pkey",
"Locations",
"location_name ='" & LocationWhoopsColumn & "'");
I guessed "pkey" and "location_name" as Locations field names; replace with your real names.
That UPDATE may not be fast, but hopefully that won't be a deal breaker since you need do this one time only.
Related
I have 1.7 million records in an access table sorted A to Z. the records are not unique and there are repeated records. I want to make them unique based on their frequency. if a record has been repeated 4 times I want the first one to get "-1" at the end of the record value, the second record get "-2" and so on. in this way similar records will become unique. all similar record are beside each other because of sorting. in excel I do this task by an If function (if this cell value<>the cell value above then "1" else above repeat number plus 1) but in access I don't know what to do (I'm a beginner).
finally I want to add a column to original table which is (original record value - repeat number).
I appreciate your help
Note about sort order:
Sort order in a relational database is not concrete like in a spreadsheet. There is no concept of rows being "next to each other", unless in context of an index. An index is largely a tool for the database to handle the data more efficiently (and to aid in defining uniqueness). The order itself is still largely dynamic because the order of a particular query can be specified differently from the index (or from storage order) and this does not change how the data is actually stored. Being "next to each other" is essentially a useless concept in SQL queries, unless you mean "next to each other numerically", for instance with an AutoNumber field or with the "repeat numbers" you want to add. Unlike in a spreadsheet, you cannot refer to the row "just above this row" or the "row offset by 2 from the 'current' row".
Solution
Regardless of whether or not you will use the AutoNumber column later, add a Long Integer AutoNumber column anyway. This column is named [ID] in the example code. Why? Because until you add something to allow the database to differentiate between the rows, there is technically no way using standard SQL to reliably reference individual duplicates since there is no way to distinguish individual rows. Even though you say that there are other differentiating columns, your own description rules out using them as a reliable key in referring to specific rows. (Even without such a differentiating column, Access can technically distinguish between rows. Iterating through a DAO.Recordset object in VBA would work, but perhaps not very elegant / efficient.)
Also add a new integer column for counting repeats, which below is named [DupeIndex]. A separate field is preferred (necessary?) because it allows continued reference to the original, unaltered duplicate values. If the reference number were directly updated, it would no longer match other fields and so would not be easily detected as a duplicate anymore. The following solution relies on grouping of ALL duplicate values, even those already "marked" with a [DupeIndex] number.
You should also realize that in comparing different data sets, that having separate fields allows more flexibility in matching the data. Having the values appended to the reference number complicates comparison, since you likely not only want to compare rows with the same duplication index, rather you will want to compare all possible combinations. For example, comparing records 123-1 in one set to 123-4 in another... how do you select such rows in an automated fashion? You don't want to have to manually code all combinations, but that's what you'll end up doing if you don't keep them separate like {123,1} and {123,4}.
Create and save this as a named query [Duplicates]. This query is referenced by later queries. It could instead be embedded as a sub query, but my preferences is to use saved queries for easier visualization and debugging in Access:
SELECT Data.RefNo, Count(Data.ID) AS Dupes, Max(Data.DupeIndex) AS IndexMax
FROM Data
GROUP BY Data.RefNo
HAVING Count(Data.ID) > 1
Execute the following to create a temporary table with new duplicate index values:
SELECT D1.ID, D1.RefNo,
IIf([Duplicates].[IndexMax] Is Null,0,[Duplicates].[IndexMax])
+ 1
+ (SELECT Count(D2.ID) FROM Data As D2
WHERE D2.[RefNo]=[D1].[RefNo]
And [D2].[DupeIndex] Is Null
And [D2].[ID]<[D1].[ID]) AS NewIndex
INTO TempIndices
FROM Data AS D1 INNER JOIN Duplicates ON D1.RefNo = Duplicates.RefNo
WHERE (D1.DupeIndex Is Null);
Execute the update query to set the new duplicate index values:
UPDATE Data
INNER JOIN TempIndices ON Data.ID = TempIndices.ID
SET Data.DupeIndex = [NewIndex]
Optionally remove the AutoNumber field and now assign the combined [RefNo] and new [DupeIndex] as primary key. The temporary table can also be deleted.
Comments about the queries:
Solution assume that [DupeIndex] is Null for unprocessed duplicates.
Solution correctly handles existing duplicate index numbers, only updating duplicate rows without an unique index.
Access has rather strict conditions for UPDATE queries, namely that updates are not based on circular references and/or that that joins will not produce multiple updates for the same row, etc. The temporary table is necessary in this case, since the query determining new index values refers multiple times in sub queries to the very column that is being updated. (If the update is attempted using joins on the subqueries, for example, Access complains that Operation must use an updatable query.)
TL;DR: Is this design correct and how should I query it?
Let's say we have history tables for city and address designed like this:
CREATE TABLE city_history (
id BIGINT UNSIGNED NOT NULL PRIMARY KEY,
name VARCHAR(128) NOT NULL,
history_at DATETIME NOT NULL,
obj_id INT UNSIGNED NOT NULL
);
CREATE TABLE address_history (
id BIGINT UNSIGNED NOT NULL PRIMARY KEY,
city_id INT NULL,
building_no VARCHAR(10) NULL,
history_at DATETIME NOT NULL,
obj_id INT UNSIGNED NOT NULL
);
Original tables are pretty much the same except for history_id and obj_id (city: id, name; address: id, city_id, building_no). There's also a foreign key relation between city and address (city_id).
History tables are populated on every change of the original entry (create, update, delete) with the exact state of the entry at given time.
obj_id holds id of original object - no foreign key, because original entry can be deleted and history entries can't. history_at is the time of creation of history entry.
History entries are created for every table independently - change in city name creates city_history entry but does not create address_history entry.
So to see what was the state of the whole address with city (e.g. on printed documents) at any T1 point in time, we take from both history tables most recent entries for given obj_id created before T1, right?
With this design in theory we should be able to see the state of signle address with city at any given point of time. Could anyone help me create such a query for given address id and time? Please note that there could be multiple records with the same exact timestamp.
There is also a need to create a report for showing every change of state of given address in given time period with entries like "city_name, building_no, changed_at". Is it something that can be created with SQL query? Performance doesn't matter here so much, such reports won't be generated so often.
The above report will probably be needed in an interactive version where user can filter results e.g. by city name or building number. Is it still possible to do in SQL?
In reality address table and address_history table have 4 more foreign keys that should be joined in report (street, zip code, etc.). Wouldn't the query be like ten pages long to provide all the needed functionality?
I've tried to build some queries, play with greatest-n-per-group, but I don't think I'm getting anywhere with this. Is this design really OK for my use cases (if so, can you please provide some queries for me to play with to get where I want?)? Or should I rethink the whole design?
Any help appreciated.
(My answer copied from here, since that question never marked an answer as accepted.)
My normal "pattern" in (very)pseudo code:
Table A: a_id (PK), a_stuff
Table A_history: a_history_id (PK), a_id(FK referencing A.a_id), valid_from, valid_to, a_stuff
Triggers on A:
On insert: insert values into A_history with valid_from = now, and valid_to = null.
On update: set valid_to = now for last history record of a_id; and do the same insert from the "on insert" trigger with the updated values for the row.
On delete: set valid_to = now for last history record of a_id.
In this scenario, you'd query history with "x >= from and x < to" (not BETWEEN as the a previous record's "from" value should match the next's to "value").
Additionally, this pattern also makes "change log" reports easier.
Without a table dedicated to change logging, the relevant records can be found just by SELECT * FROM A_history WHERE valid_from BETWEEN [reporting interval] OR valid_to BETWEEN [reporting interval].
If there is a central change log table, the triggers can just be modified to include log entry inserts as well. (Unless log entries include "meta" data such as reason for change, who changed, etc... obviously).
Note: This pattern can be implemented without triggers. Using a stored procedure, or even just multiple queries in code, can actually negate the need for the non-history table.
The history table's "a_id" would need to be replaced with whatever uniquely identifies the record normally though; it could still be an id value, but these values would need synthesized when inserting, and known when updating/deleting.
Queries:
(if not new) UPDATE the most recent entry's valid_to.
(if not deleting) INSERT new entry
This is a very "traditional" Problem, when it comes down to versioning (or monitoring) of changes to a certain row.
There are various "solutions", each having its own drawback and advantage.
The following "statements" are a result of my expericence, they are neither perfect, nor do I claim they are the "only ones"!
1.) Creating a "history table": That's the worst Idea of all. You would always need to take into account which table you need to query, depending on DATA that should be queried. That's a "Chicken-Egg" Problem...
2.) Using ONE Table with ONE (increasing) "Revision" Number: That's a better approach, but it will get "hard" to query: Determining the "most recent row" per "id" is very costly no matter which aproach is used.
My personal expierence is, that following the pattern of a "double linked List" ist the best to solve this, when it comes down to Millions of records:
3.) Maintain two columns among every entity, let's say prev_version_id and next_version_id. prev_version_id points to NULL, if there is no previous version. next_version_id points to NULL if there is no later version.
This approach would require you to ALWAYS perform two actions upon an update:
Create the new row
Update the old rows reference (next_version_id) to the just insterted row.
However, when your database has grown to something like 100 Million Rows, you will be very happy that you have choosen this path:
Querying the "Oldest" Version is as simple as querying where ISNULL(prev_version_id) and entity_id = 5
Querying the "Latest" Version is as simple as querying where ISNULL(next_version_id) and entity_id = 5
Getting a full version history will just target the entity_id=5 of the data-table, sortable by either prev_version_id or next_version_id.
The very often neglected fact: The first two queries will also work to get a list of ALL first versions or of ALL recent versions of an entity - in about NO TIME! (Don't underestimate how "costly" it can be do determine the most recent version of an entity otherwise! Believe me, when "testing" everything seems equaly fine, but the real struggle starts when live-data with millions of records is used.)
cheers,
dognose
Good Morning All,
I'm having problem pulling the data I need from a SQL Backend and keeping it up to date.
I've got two tables, that hold the data at need. At one Point they were split due to a software update we received. First Table dbo_PT_NC Second Table dbo_PT_Task
Primary key of PT_NC is the "NCR" Field, The Task Table has its own Unique ID, But the PT_Task.TaskTypeID field is linked to the "NCR" field
SELECT dbo_PT_Task.TaskTypeID,
dbo_PT_NC.NCR,
dbo_PT_NC.NCR_Date,
dbo_PT_NC.NC_type,
dbo_PT_NC.Customer,
dbo_PT_NC.Material,
dbo_PT_NC.Rev,
dbo_PT_NC.Qty_rejected,
dbo_PT_Task.TaskType,
dbo_PT_Task.Notes AS dbo_PT_Task_Notes,
dbo_PT_NC.Origin,
dbo_PT_NC.Origin_ref,
dbo_PT_NC.Origin_cause,
dbo_PT_NC.Origin_category
FROM dbo_PT_NC INNER JOIN dbo_PT_Task ON dbo_PT_NC.[NCR] = dbo_PT_Task.[TaskTypeID]
WHERE (((dbo_PT_NC.NCR_Date)>=#1/1/2016#) AND ((dbo_PT_Task.TaskSubType)="Origination"))
ORDER BY dbo_PT_NC.NCR_Date, dbo_PT_NC.Customer;
After I have this data pulled and put into a Snapshot (I do not want the Live Data to be accessible by the front end users) I'll be adding columns for a Weak Point Management System we are implementing, Fields Such as:
Scrap Code (lookup field to another table i've built inside excel)
Containment, Root Cause, Plan, Do, Check, and Act, all of which Should most likely be Memo Fields (As characters may break 255)
Date Completed A date the process was complete
This table (The data i've snapshotted and the new fields added) will need to be updated with New or Changed Records from the SQL Backend i've previously connected to.
UPDATE
Big thanks to Andre.. Got it working, Sample code below (i've added more update fields since)
UPDATE tblWeakPointMaster, dbo_PT_NC INNER JOIN dbo_PT_Task ON dbo_PT_NC.NCR = dbo_PT_Task.TaskTypeID
SET tblWeakPointMaster.Qty_rejected = [dbo_PT_NC].[Qty_rejected],
tblWeakPointMaster.dbo_PT_Task_Notes = [dbo_PT_Task].[Notes],
tblWeakPointMaster.Material = [dbo_PT_NC].[Material],
tblWeakPointMaster.Rev = [dbo_PT_NC].[Rev],
tblWeakPointMaster.NC_type = [dbo_PT_NC].[NC_type]
WHERE (((tblWeakPointMaster.NCR)=dbo_PT_NC.NCR) And ((tblWeakPointMaster.TaskID)=dbo_PT_Task.TaskID));
I assume there is a 1:n relation between PT_NC and PT_Task?
Then you should include both primary keys in the import SELECT.
Either use them as composite primary key in the Access tables instead of the new KEY column. Or if that is impractical because other tables are linking to tblWeakPointMaster, you can also keep that primary key.
But in any case, these two columns form the JOIN between tblWeakPointMaster and tblWeakPointUpdates.
All other columns can be used to update tblWeakPointMaster from tblWeakPointUpdates (assuming they can be edited in the original database).
Edit: if you don't use them as composite primary key, you need to create an unique index on the combination, or the JOIN will not be updateable, I think.
Maybe my way of thought is wrong, so first off I will try to explain the situation in words.
I have a Database with about 40000 Titles, some of them are related to each other. This Relations are saved in a different database in a simple a b type manner. The problem is that the system was changed so there is a special row in the main database which contains the old id. And the b value is also the old id... However now I want to replace the b value with the new auto increment id given by the database for all the entries in the relations table. So I thought that I need to create a new table, copy the contents but replacing the b value with the new id...
CREATE TABLE list_relations_new LIKE list_listitem_relations;
INSERT list_relations_new
SELECT lr.a, li.id, lr.typeID
FROM list_listitems li
LEFT JOIN list_listitem_relations lr
ON li.oldID = lr.b
Executing this query doesn't give any errors, but also doesn't even create a new table? I hope somebody can help me to sort this out...
I have a fairly simple database that I inherited. For the purposes of this question, there are two tables:
Mastertable and Providertable.
Mastertable references Providertable thruogh provid, which is a FK to Providertable PK (provid).
So it looks like this:
Mastertable:
acct (PK)
(other fields)
provid (FK)
Providertable
provid (PK)
provname
provspecialty
Simple right? However, the Mastertable!provid field is actually a lookup table which displays Providertable!provname but stores provid. There is a form the users use to populate the Mastertable, and it has this lookup field shown.
The users now want to show the provider specialty based on what they select as the provid. I can't figure this out to save my life. I'm pretty well versed in SQL, having written many stored procedures and created a few db apps using .NET, but this is quite challenging. I tried creating a lookup field called provspeciality, but that's not what they want. I tried changing the "OnUpdate" event for the lookup field to point the Provider Specialty label to the right thing.
Right now, I can't even get a simple select going that joins the two tables since they are using this lookup field as the FK and Access I guess can't understand it. Any help appreciated.
Since the Mastertable provid field is a lookup type, the displayed value is the lookup value rather than the value which is actually stored in the field. This query will show you the stored provid values.
SELECT acct, provid
FROM Mastertable;
And I think you should be able to retrieve the matching provider specialties with a query similar to this:
SELECT m.provid, p.provname, p.provspecialty
FROM
Mastertable AS m
INNER JOIN Providertable AS p
ON p.provid = m.provid;
You may even be able to use that query as the Row Source for a combo or list box on your form. Make provid the bound column. You may wish to set the provid column width to 0" so it is not actually displayed in the control, but still stored in Mastertable.
I think you should modify the table to make provid a normal (numeric?) field instead of a lookup. Fortunately you indicated this is "fairly simple database", so that will hopefully limit the amount of additional changes you need to be compatible with the redesigned table. Good luck.
I created an "On Change" event for the form field "provid" which is a lookup field that displays the provider name while putting the provider id into the master table as a FK. However apparently Access is not able to do lookups based on this field (or I am doing something wrong) using queries - as shown above in comments. What I did is use this as the event code. Important, you must enable macros for this to work!
Private Sub provid_Change()
Me.txtProviderSpecialty = DLookup("Provspecialty", "Providertable", "provid = " & Me.provid.Value)
End Sub