I'm in the process of creating a website and would like some advise on my database schema as I don't have very much experience in that field.
Say I had a site where 2 products were for sale, product1 and product2.
When a user buys either product they get a user account with a key to access their account. A user can download their product from their account page. That user can then at a later date purchase the remaining product and the product gets added to their account allowing them to download both product1 and product2 using the key given to them when purchasing the first product.
Payment information (payment method, transaction id, timestamp, etc) will also be stored.
I currently have my schema planned as so:
I feel that the way I'm doing it is not optimal and there are better ways to organize it. Any advise?
Hope you can help and thanks in advance :)
It is a good idea to start off identifying the objects your schema will store. It looks like candidate components might be
user
(user data)
product
purchase
These entities will be the tables you start out with. Add relationships appropriately.
Identifying the entities in your schema will (hopefully) keep you from outwardly expanding tables. You don't want to have to revise your whole schema for example every time you add a product. Similarly, you would not want to add a new table each time.
Once you've identified these, start putting them in normal form. 3rd normal form is generally recommended. You will likely find, as you have indicated in your schema, there is much redundant data that would fall under the purchase entity that separating it out to another table makes for a more maintainable schema.
This will eliminate redundant data in your tables which might later fall out of synch and help to identify where some data is out of place.
Related
What is the best-practice for maintaining the integrity of linked data entities on update?
My scenario
I have two entities "Client and
Invoice". [client is definition and
Invoice is transaction].
After issuing many invoices to the
client it happens that the client
information needs to be changed
e.g. "his billing address/location
changed or business name ... etc".
It's normal that the users must be
able to update the client
information to keep the integrity of
the data in the system.
In the invoice "transaction entity"
I don't store just the client id but
also all the client information related to the
invoice like "client name, address,
contact", and that's well known
approach for storing data in
transaction entities.
If the user created a new invoice the
new client information will be
stored in the invoice record along
with the same client-id (very
obvious!).
My Questions
Is it okay to bind the data entities
"clients" from different locations
for the Insert and the update?
[Explanation: if I followed the
approach from step 1-4 I have to
bind the client entity from the
client table in case of creating new
invoice but in case of
updating/printing the invoice I have
to bind the client entity from the
invoice table otherwise the data
won't be consistent or integer...So
how I can keep the data integrity
without creating spaghetti code in
the DAL to handle this custom
requirements of data binding??]
I passed through a system that was
saving all previous versions of an
entity data before the update
"keeping history of all versions".
If I want to use the same method to
avoid the custom binding how I can
do this in term of database design
"Using MYSQL"? [Explanation: some
invoices created with version 1.0 of
the client then the client info
updated and its version became 1.1
and new invoices created with last
version...So is it good to follow
this methodology? and how I should
design my entities/tables to fulfil the requirements of entity
versioning and binding?
Please provide any book or reference
that can kick me in the right
direction?
Thanks,
What you need to do is leave the table the way it is. You are correct, you should be storing the customer information in the invoice for history of where the items were shipped to. When it changes, you should NOT update this information except for any invoices which have not yet been shipped. To maintain this type of information, you need a trigger on the customer table that looks for invoices that have not been shippe and updates those addresses automatically.
If you want to save historical versions of the client information, the correct process is to create an audit table and populate it through a trigger.
Data integrity in this case is simply through a foreign key to the customer id. The id itself should not ever change or be allowed to change by the user and should be a surrogate number such as an integer. Becasue you should not be changing the address information in the actual invoice (unless it has not been shipped in which case you had better change it or the product will be shipped to the wrong place), this is sufficent to maintain data integrity. This also allows you to see where the stuff was actually shipped but still look up the current info about the client through the use of the foreign key.
If you have clients that change (compaies bought by other companies), you can either run a process onthe server to update the customer id of old records or create a table structure that show which client ids belong to a current parent id. The first is easier to do if you aren;t talking about changing millions of records.
"This is a business case where data mnust be denormalized to preserve historical records of what was shipped where. His design is not incorrect."
Sorry for adding this as a new response, but the "add comment" button still doesn't show.
"His design" is indeed not incorrect ... because it is normalized !!!
It is normalized because it is not at all times true that the address corresponding to an invoice functionally depends on the customer ID exclusively.
So : normalization, yes I do think so. Not that normalization is the only issue involved here.
I'm not completely clear on what you are getting at, but I think you want to read up on normalization, available in many books on relational databases and SQL. I think what you will end up with is two tables connected by a foreign key, but perhaps some soul-searching per previous sentence will help you clarify your thoughts.
The title is somewhat hard to understand, so here is the explanation:
I am building a system, that deals with retail transactions. Meaning - purchases. I have a database with products, where each product has an ID, that is also known to the POS system. When a customer makes a purchase, the data is sent to the back-end for parsing, and is saved. Now everything is fine and dandy, until there are changes to the products name, since my client wants to see the name of the product, as it was purchased then.
How do I save this data, while also keeping a nice, normal-formed database?
Solutions I could think of are:
De-normalization, where we correlate the incoming data with the info we have in the database, and then save only the final text values, not id's.
Versioning, where we keep multiple versions of every product, and save the transactions with the id of the products version, when it came in. The problem with this one is, that as our retail store chain grows, and there are more and more changes happening to the products, the complexity of the whole product will greatly increase.
Any thoughts on this?
This is called a slowly changing dimension.
Either solution that you mention works. My preference is the second, versioning. I would have a product table that has an effdate and enddate on the record. You can easily find the current record (where enddate is null) or the record at any point in time.
The first method always strikes me as more "quick-and-dirty", but it also works. It just gets cumbersome when you have more fields and more objects you are trying to track. It does, in general though, win on performance.
If the name has to be the name as it was originally, the easiest, simplest and most reliable way to do that is to save the name of the product in the invoice line item record.
You should still link to the product with a ProductID, of course.
If you want to keep a history of name changes, you can do that in a separate table if you wish:
ProductNameID
ProductID
Date
Description
And store a ProductNameID with the invoice line item.
So I have this application that I'm drawing up and I start to think about my users. Well, My initial thought was to create a table for each group type. I've been thinking this over though and I'm not sure that this is the best way.
Example:
// Users
Users [id, name, email, age, etc]
// User Groups
Player [id, years playing, etc]
Ref [id, certified, etc]
Manufacturer Rep [id, years employed, etc]
So everyone would be making an account, but each user would have a different group. They can also be in multiple different groups. Each group has it's own list of different columns. So what is the best way to do this? Lets say I have 5 groups. Do I need 8 tables + a relational table connecting each one to the user table?
I just want to be sure that this is the best way to organize it before I build it.
Edit:
A player would have columns regarding the gear that they use to play, the teams they've played with, events they've gone to.
A ref would have info regarding the certifications they have and the events they've reffed.
Manufacturer reps would have info regarding their position within the company they rep.
A parent would have information regarding how long they've been involved with the sport, perhaps relations with the users they are parent of.
Just as an example.
Edit 2:
**Player Table
id
user id
started date
stopped date
rank
**Ref Table
id
user id
started date
stopped date
is certified
certified by
verified
**Photographer / Videographer / News Reporter Table
id
user id
started date
stopped date
worked under name
website / channel link
about
verified
**Tournament / Big Game Rep Table
id
user id
started date
stopped date
position
tourney id
verified
**Store / Field / Manufacturer Rep Table
id
user id
started date
stopped date
position
store / field / man. id
verified
This is what I planned out so far. I'm still new to this so I could be doing it completely wrong. And it's only five groups. It was more until I condensed it some.
Although I find it weird having so many entities which are different from each other, but I will ignore this and get to the question.
It depends on the group criteria you need, in the case you described where each group has its own columns and information I guess your design is a good one, especially if you need the information in a readable form in the database. If you need all groups in a single table you will have to save the group relevant information in a kind of object, either a blob, XML string or any other form, but then you will lose the ability to filter on these criteria using the database.
In a relational Database I would do it using the design you described.
The design of your tables greatly depends on the requirements of your software.
E.g. your description of users led me in a wrong direction, I was at first thinking about a "normal" user of a software. Basically name, login-information and stuff like that. This I would never split over different tables as it really makes tasks like login, session handling, ... really complicated.
Another point which surprised me, was that you want to store the equipment in columns of those user's tables. Usually the relationship between a person and his equipment is not 1 to 1 and in most cases the amount of different equipment varies. Thus you usually have a relationship between users and their equipment (1:n). Thus you would design an equipment table and there refer to the owner's user id.
But after you have an idea of which data you have in your application and which relationships exist between your data, the design of the tables and so on is rather straitforward.
The good news is, that your data model and database design will develop over time. Try to start with a basic model, covering the majority of your use cases. Then slowly add more use cases / aspects.
As long as you are in the stage of planning and early implementation phasis, it is rather easy to change your database design.
We have a requirement in our application where we need to store references for later access.
Example: A user can commit an invoice at a time and all references(customer address, calculated amount of money, product descriptions) which this invoice contains and calculations should be stored over time.
We need to hold the references somehow but what if the e.g. the product name changes? So somehow we need to copy everything so its documented for later and not affected by changes in future. Even when products are deleted, they need to reviewed later when the invoice is stored.
What is the best practise here regarding database design? Even what is the most flexible approach e.g. when the user want to edit his invoice later and restore it from the db?
Thank you!
Here is one way to do it:
Essentially, we never modify or delete the existing data. We "modify" it by creating a new version. We "delete" it by setting the DELETED flag.
For example:
If product changes the price, we insert a new row into PRODUCT_VERSION while old orders are kept connected to the old PRODUCT_VERSION and the old price.
When buyer changes the address, we simply insert a new row in CUSTOMER_VERSION and link new orders to that, while keeping the old orders linked to the old version.
If product is deleted, we don't really delete it - we simply set the PRODUCT.DELETED flag, so all the orders historically made for that product stay in the database.
If customer is deleted (e.g. because (s)he requested to be unregistered), set the CUSTOMER.DELETED flag.
Caveats:
If product name needs to be unique, that can't be enforced declaratively in the model above. You'll either need to "promote" the NAME from PRODUCT_VERSION to PRODUCT, make it a key there and give-up ability to "evolve" product's name, or enforce uniqueness on only latest PRODUCT_VER (probably through triggers).
There is a potential problem with the customer's privacy. If a customer is deleted from the system, it may be desirable to physically remove its data from the database and just setting CUSTOMER.DELETED won't do that. If that's a concern, either blank-out the privacy-sensitive data in all the customer's versions, or alternatively disconnect existing orders from the real customer and reconnect them to a special "anonymous" customer, then physically delete all the customer versions.
This model uses a lot of identifying relationships. This leads to "fat" foreign keys and could be a bit of a storage problem since MySQL doesn't support leading-edge index compression (unlike, say, Oracle), but on the other hand InnoDB always clusters the data on PK and this clustering can be beneficial for performance. Also, JOINs are less necessary.
Equivalent model with non-identifying relationships and surrogate keys would look like this:
You could add a column in the product table indicating whether or not it is being sold. Then when the product is "deleted" you just set the flag so that it is no longer available as a new product, but you retain the data for future lookups.
To deal with name changes, you should be using ID's to refer to products rather than using the name directly.
You've opened up an eternal debate between the purist and practical approach.
From a normalization standpoint of your database, you "should" keep all the relevant data. In other words, say a product name changes, save the date of the change so that you could go back in time and rebuild your invoice with that product name, and all other data as it existed that day.
A "de"normalized approach is to view that invoice as a "moment in time", recording in the relevant tables data as it actually was that day. This approach lets you pull up that invoice without any dependancies at all, but you could never recreate that invoice from scratch.
The problem you're facing is, as I'm sure you know, a result of Database Normalization. One of the approaches to resolve this can be taken from Business Intelligence techniques - archiving the data ina de-normalized state in a Data Warehouse.
Normalized data:
Orders table
OrderId
CustomerId
Customers Table
CustomerId
Firstname
etc
Items table
ItemId
Itemname
ItemPrice
OrderDetails Table
ItemDetailId
OrderId
ItemId
ItemQty
etc
When queried and stored de-normalized, the data warehouse table looks like
OrderId
CustomerId
CustomerName
CustomerAddress
(other Customer Fields)
ItemDetailId
ItemId
ItemName
ItemPrice
(Other OrderDetail and Item Fields)
Typically, there is either some sort of scheduled job that pulls data from the normalized datas into the Data Warehouse on a scheduled basis, OR if your design allows, it could be done when an order reaches a certain status. (Such as shipped) It could be that the records are stored at each change of status (with a field called OrderStatus tacking the current status), so the fully de-normalized data is available for each step of the oprder/fulfillment process. When and how to archive the data into the warehouse will vary based on your needs.
There is a lot of overhead involved in the above, but the other common approach I'm aware of carries even MORE overhead.
The other approach would be to make the tables read-only. If a customer wants to change their address, you don't edit their existing address, you insert a new record.
So if my address is AddressId 12 when I first order on your site in Jamnuary, then I move on July 4, I get a new AddressId tied to my account. (Say AddressId 123123 because your site is very successful and has attracted a ton of customers.)
Orders I palced before July 4 would have AddressId 12 associated with them, and orders placed on or after July 4 have AddressId 123123.
Repeat that pattern with every table that needs to retain historical data.
I do have a third approach, but searching it is difficult. I use this in one app only, and it actually works out pretty well in this single instance, which had some pretty specific business needs for reconstructing the data exactly as it was at a specific point in time. I wouldn't use it unless I had similar business needs.
At a specific status, serialize the data into an Xml document, or some other document you can use to reconstruct the data. This allows you to save the data as it was at the time it was serialized, retaining original table structure and relaitons.
When you have time-sensitive data, you use things like the product and Customer tables as lookup tables and store the information directly in your Orders/orderdetails tables.
So the order table might contain the customer name and address, the details woudl contain all relevant information about the produtct including especially price(you never want to rely on the product table for price information beyond the intial lookup at teh time of the order).
This is NOT denormalizing, the data changes over time but you need the historical value, so you must store it at the time the record is created or you will lose data intergrity. You don't want your financial reports to suddenly indicate you sold 30% more last year because you have price updates. That's not what you sold.
What is the best-practice for maintaining the integrity of linked data entities on update?
My scenario
I have two entities "Client and
Invoice". [client is definition and
Invoice is transaction].
After issuing many invoices to the
client it happens that the client
information needs to be changed
e.g. "his billing address/location
changed or business name ... etc".
It's normal that the users must be
able to update the client
information to keep the integrity of
the data in the system.
In the invoice "transaction entity"
I don't store just the client id but
also all the client information related to the
invoice like "client name, address,
contact", and that's well known
approach for storing data in
transaction entities.
If the user created a new invoice the
new client information will be
stored in the invoice record along
with the same client-id (very
obvious!).
My Questions
Is it okay to bind the data entities
"clients" from different locations
for the Insert and the update?
[Explanation: if I followed the
approach from step 1-4 I have to
bind the client entity from the
client table in case of creating new
invoice but in case of
updating/printing the invoice I have
to bind the client entity from the
invoice table otherwise the data
won't be consistent or integer...So
how I can keep the data integrity
without creating spaghetti code in
the DAL to handle this custom
requirements of data binding??]
I passed through a system that was
saving all previous versions of an
entity data before the update
"keeping history of all versions".
If I want to use the same method to
avoid the custom binding how I can
do this in term of database design
"Using MYSQL"? [Explanation: some
invoices created with version 1.0 of
the client then the client info
updated and its version became 1.1
and new invoices created with last
version...So is it good to follow
this methodology? and how I should
design my entities/tables to fulfil the requirements of entity
versioning and binding?
Please provide any book or reference
that can kick me in the right
direction?
Thanks,
What you need to do is leave the table the way it is. You are correct, you should be storing the customer information in the invoice for history of where the items were shipped to. When it changes, you should NOT update this information except for any invoices which have not yet been shipped. To maintain this type of information, you need a trigger on the customer table that looks for invoices that have not been shippe and updates those addresses automatically.
If you want to save historical versions of the client information, the correct process is to create an audit table and populate it through a trigger.
Data integrity in this case is simply through a foreign key to the customer id. The id itself should not ever change or be allowed to change by the user and should be a surrogate number such as an integer. Becasue you should not be changing the address information in the actual invoice (unless it has not been shipped in which case you had better change it or the product will be shipped to the wrong place), this is sufficent to maintain data integrity. This also allows you to see where the stuff was actually shipped but still look up the current info about the client through the use of the foreign key.
If you have clients that change (compaies bought by other companies), you can either run a process onthe server to update the customer id of old records or create a table structure that show which client ids belong to a current parent id. The first is easier to do if you aren;t talking about changing millions of records.
"This is a business case where data mnust be denormalized to preserve historical records of what was shipped where. His design is not incorrect."
Sorry for adding this as a new response, but the "add comment" button still doesn't show.
"His design" is indeed not incorrect ... because it is normalized !!!
It is normalized because it is not at all times true that the address corresponding to an invoice functionally depends on the customer ID exclusively.
So : normalization, yes I do think so. Not that normalization is the only issue involved here.
I'm not completely clear on what you are getting at, but I think you want to read up on normalization, available in many books on relational databases and SQL. I think what you will end up with is two tables connected by a foreign key, but perhaps some soul-searching per previous sentence will help you clarify your thoughts.