Database Architecture for logging - mysql

This is something that has bothered me for a long time and i still have been unable to find an answer.
I have a huge system with alot of different features. What is common for this system is of course that my users can
create, update, read & delete
different parts of my system.
For simple reasons lets say i have an application that has the following features:
Document administration
Video administration
User administration
Salery administration
(Please do note i took these at random just to prove a point that all of these would have their own separate tables and does not necessarily be connected).
Now i wish to create some sort of logging system. So that when ever someone either create,update or delete an entity it will be recorded.
Now as far as i can see i can do this two ways.
1.
Create a logging table for each of the 4 features that is in my system. However with this method i am required to create a logging table for each new feature i add to the system. i would also have to combine data from X number of tables if i wish to create a log which potentially could be a huge task!
2.
i could create something like the following:
However once again i would have to add a col for each new feature i will add.
So my question is what is the best way for creating logging database architecture
Or is there an easier way?

Instead of one target_xx for each feature, you could do it this way:
target_id | target_type
1 video
4 document
5 user
2 user
or even better. A table with target types and insert only the respective id's on target_type
Something like this:

if you want to capture for each table creation and update date, i would just use the default and the update event from mysql. You can define the fields like this for a table:
ALTER TABLE table
ADD COLUMN CreateDate Datetime DEFAULT CURRENT_TIMESTAMP,
ADD COLUMN LastModifiedDate Datetime ON UPDATE CURRENT_TIMESTAMP;
You can add these 2 fields in all tables. If you want to use one central table for logging (which might be more difficult to manage, because you always need to create joins, maybe also worse performance), then I would work with triggers.

Related

How to design a MySQL table that tracks the Status of each Asset, as well as every old Status?

I would like to create a table that tracks the status of each asset as well as each past status. Basically I want to keep a log of all status changes.
Do I create a timestamp for each updated status and have every update be its own separate row, linked back to the asset through the assetid? Then sort by the timestamp to get these statuses in order? I can see this table getting unwieldy if there are tons of rows for each asset and the table grows linearly over time.
This is for a MySQL database.
Here is an example of how I have designed a database table to track/log purposes.
Columns:
auto increment pk (if you don't have better pk)
timestamp
tracked object id (asset_id in your case)
event type (probably you don’t need but this is explained below)
content (this can be also named status in your case)
My example is very simplified but the main idea is to insert each record into own row. You can create a table with proper primary keys or indexes to have a good search performance.
Using the structure you should be able to search by asset, by status, or get latest changes etc. The structure depends on your needs so usually I have modified it to support the need.
Don’t care too much about the event -columns. I just put it here because most of the implementations are based on event sourcing. Here is a link to one article that could explain it: http://scottlobdell.me/2017/01/practical-implementation-event-sourcing-mysql/
I suggest that you could read more about that event sourcing that if the design could work in your case. Look only the database example because that is similar like in my example.
In the results, you should have a journal of status changes. Then it depends on your code how to handle/read data and show results.
About the linear growth… I would say it is not a big problem. Of course, if you have more information what “tons of rows” means, then ask. I have not seen any scaling problems. The same structure works very well with relational or with NoSQL databases. Mysql also has features to optimize that kind of structure if the size of data will be a problem.

How to handle database design with time stamps and features?

I'm designing a database in MySQL, and have a couple of questions about how tables and columns ought to be defined.
Right now, I have one entity (let's call it Entity1) that will have several (5-10) features that may change with time. I will also need to be able to add features as time progresses. There are 2 designs I'm considering:
1: 1 table with Entity1 info, 1 table with a timestamp and then all of the features at that given timestamp in every row. This will make it easier to enforce the connections between all of the features.
2: 1 table with Entity1 info, and 1 table for each feature, with each having a timestamp of the addition of that row. This will reduce the amount of redundant data I store in my database.
Are either of these designs ideal? Is there some database literature I should look at? I'm a total noob with database design.
Thanks
Consider creating 2 tables: entity table and feature table.
entity (id, ...)
feature(id, entity_id_foreign_key, feature_name, created_timestamp, feature_value)
It will work if feature values are all equal type. Good thing is that you don't ever need to touch the schema if you want to add or remove features, so no migrations.

What are best practices for creating table for "log of changes" in Database Table?

We are updating table XYZ have following fields:
First Name|Middle Name|Last Name|Address|DOB|Country|County|(etc.)
Initially, we are calling some web service which is sending updated information for a row in XYZ like either update first name or DOB update or both or all or none.
Now there is requirement to create a log table in database which store summary of old records and changes done to XYZ. Every affected row should be reported.
Is it good to create similar fields in new table say ABC:
First Name|Middle Name|Last Name|Address|DOB|Country|County|Update_Date
with additional field called "Update_datetime"
Now each time service called we will select values from previous row i.e from XYZ and update the same to ABC with update date.
What are loopholes in this practice? What other better practices can be followed?
Is there a requirement for a log table or a requirement for a proper history?
Oracle has history functionality Out of the box
I doubt MySQL does - you may have to do a different way.
The pros of Oracle is that it will not fail - it's a core feature. The cons of hand rolled is, well, it's hand rolled. Lots of SPs, triggers or other nastiness that people can deliberately or inadvertently bypass.
I echo the need to know what the requirements are behind this. Is it to be human readable (auditing, debugging etc.) or machine readable (e.g. event sourcing architectural pattern)? How often will you need to go back to look at previous versions? How often do things change?
If it's event sourcing, then there are a few answers around on Stack Overflow about that, e.g. Using an RDBMS as event sourcing storage and best event sourcing db strategy. For more of an introduction, there's e.g. a Martin Fowler video.
There are also SO answers on logging changes in MySQL and Using MySQL triggers to log all table changes to a secondary table and an alternative approach (using 1 table, but adding sort-of version numbers to show each record's validity).

Creating a table with specific field parameters

I struggling to create a table that sets table parameters as well as creating the columns.
I am using MySQL server.
I require that the table meets the following criteria:
The table should be Called CUSTOMER with the columns CUST, LOCX, LOCY.
The column CUST will be a 1 up serial starting 1001 and will be the primary key.
LOCX and LOCY will contain X and Y Integers no greater than +-11, and will be foreign keys to other tables.
For info: I then intend to add my data to the table using the INSERT INTO function in a separate query that I already have.
Any direction on the construction of a query to create a table meeting the requirements above will be greatly appreciated
you can create a new table with a MySQL-GUI if you have problems with that.
These GUI-tools usually provide a New-Table button that also allows you to define your table without writing any code. They are often limited but should be more than sufficient for your needs. there are 1-month trial versions for paid versions and even completely free GUIs so you don't have to buy anything.
after that use the following code to retrieve "perfect" SQL from MySQL:
show create table your_schema_name.your_table_name
do that a few times and study the code. Soon you will be able to write create-table statements and include more complex column definitions on your own. It will also be easier to understand the MySQL Documentation which can be confusing and somehow intimidating with its completeness for beginners.

Mysql database design

currently Im working on a project that, at first glance, will require many tables in a database. Most of the tables are fairly straightforward however I do have an issue. One of the tables will be a list of members for the website, things like username, password, contact info, bio, education, etc will be included. This is a simple design, however, there is also a need for each member to have their availability entered and store in the database as well. Availability is defined as a date and time range. Like available on 4/5/2011 from 1pm to 6pm EST, or NOT available every friday after 8pm EST. For a single user, this could be a table on its own, but for many users, Im not sure how to go about organizing the data in a manageable fashion. First thought would be to have code to create a table for each user, but that could mean alot of tables in the database in addition to the few I have for other site functions. Logically i could use the username appended to Avail_ or something for the table name ie: Avail_UserBob and then query that as needed. But im curious if anyone can think of a better option than having the potential of hundreds of tables in a single database.
edit
So general agreement would be to have a table for members, unique key being ID for instance. Then have a second table for availability (date, start time, end time, boolean for available or not, and id of member this applies to). Django might sound nice and work well, but i dont have the time to spend learning another framework while working on this project. The 2 table method seems plausable but Im worried about the extra coding required for features that will utilize the availability times to A) build a calender like page to add, edit, or remove entered values, and B) match availabilities with entries from another table that lists games. While I might have more coding, I can live with that as long as the database is sound, functional, and not so messy. Thanks for the input guys.
Not to sound like a troll, but you should take a look into using a web framework to build most of this for you. I'd suggest taking a look at Django. With it you can define the type of fields you wish to store (and how they relate) and Django builds all the SQL statements to make it so. You get a nice admin interface for free so staff can login and add/edit/etc.
You also don't have to worry about building the login/auth/change password, etc. forms. all that session stuff is taken care of by Django. You get to focus on what makes your project/app unique.
And it allow you to build your project really, really fast.
djangoproject.org
I don't have any other framework suggestions that meet your needs. I do... but I think Django will fit the bill.
Create a table to store users. Use its primary key as foreign key in other tables.
The databases are written to hold many many rows in a table. There are not optimized for table creation. So it is not a good idea to create a new table for each user. Instead give each user an unique identifier and put the availability in a separate table. Provide an additional flag to make an entry valid or invalid.
Create a table of users; then create a table of availabilities per user. Don't try to cram availabilities into the user table: that will guarantee giant grief for you later on; and you'll find you have to create an availabilities table then.
Google database normalization to get an idea why.
Take it as truth from one who has suffered such self-inflicted grief :-)