Once a record is entered in a particular table, I think I don't ever want it to be deleted. The records are there to keep track of things for historical purposes. I'm using MySQL, InnoDB. Is there any way I can protect that table from accidental deletes by the application layer? If so, is this a good approach?
If you can set the permission for your user(s), don't give the delete permission. There're situations where this practice is a "must be" like the tables for statistics purpose. Thus, if your table is used to achieve one of this goal, it's a good approach.
I use a trigger that detects the deletion and does some illegal stuff so the whole operation fails. For example like this:
CREATE TRIGGER protect_delete before delete ON protected_table
FOR EACH ROW UPDATE non_existing_table SET non_existing_column=1;
So when someone will attempt a delete operation - the whole statement will fail. You also might use better names for non_existing_column and non_existing_table.
E.g it is possible to obtain an error message like this:
ERROR 1146 (42S02): Table 'database.delete_restricted_on_tableX'
doesn't exist
EDIT: also it is possible to create even better fail messages, please check here http://www.brokenbuild.com/blog/2006/08/15/mysql-triggers-how-do-you-abort-an-insert-update-or-delete-with-a-trigger/
One other option is switch to the ARCHIVE engine for historical tables.
This will prevent any DELETE or UPDATE actions on the table, and compress the data. One (major) downside to this approach is that you cannot index the columns on the table.
I think you should implement this logic in your application layer and insert a column where you put a no-delete flag.
Another idea would be to exclude the delete access for the db user
You may want to write a trigger that detects the deletion, then reinserts the record, but there may be some issues with that, so you can also just add a foreign key constraint that will prevent the deletion.
For some discussions on this you can look at: http://rpbouman.blogspot.com/2011/10/mysql-hacks-preventing-deletion-of.html.
Related
In MySQL, which is a better practice? Always use "CREATE TABLE IF NOT EXISTS", or check first the existence of the table using "SHOW TABLES LIKE" before making the table?
I have to regularly save a page where the table for it may or may not be there (sometimes, it is deliberately deleted when not in use). Previously, I used to do "SHOW TABLES LIKE" to check if that table exists before I insert new entries. But I changed it to "CREATE TABLE IF NOT EXISTS". Either way, I just do a "INSERT .. ON DUPLICATE UPDATE" to add new or update existing data.
I don't know how to benchmark this, which is why I am asking.
Performance isn't critical with these operations.
The key aspect is race conditions. If you use CREATE TABLE IF NOT EXISTS you know it will happen or it won't. If two threads happen to be doing this statement one will succeed and the other won't be negatively affected.
If a SHOW TABLE LIKE was used in both threads, both could detect the table didn't exist, and upon trying to create the table, one would fail.
So use CREATE TABLE IF NOT EXISTS to mitigate race conditions. Also in general is better to use a database provided feature than roll your own.
CREATE TABLE IF NOT EXISTS is option provided by MySQL and good to use. If table will not exist this statement will create else will skip.
other end, if you first check table existence by show table like then either skip or create as per result of condition/check. Ultimately you are increasing one step or runtime of you script or program for same functionality which you can achieve in single step.
I have several databases that are used by several applications (one of which is our own, the others we have no control over in what they do).
Out software has to know when the database has last been changed. For reasons I won't get into to keep this short we decided that going with a new table per database that has a singular field: last_changed_on that has a GetDate() as a value. This way our own software can check when it was last changed and check it to the date it has stored for said database and do things if the date is newer than what is stored in-memory.
After doing some research we decided that working with Triggers was the way to go, but from what I could find online, triggers look at specific columns that you set for Updates.
What I'd like to know is if there is a way to automate the process or just have a trigger that happens whenever anything happens insert, update, remove wise?
So I am looking for something like this:
CREATE TRIGGER LastModifiedTrigger
ON [dbo].[anytable]
AFTER INSERT, UPDATE, DELETE
AS
INSERT INTO dbo.LastModifiedTable (last_modified_on) VALUES (CURRENT_TIMESTAMP)
I know that the above example isn't a correct trigger, I'm rather new to them so I was unsure on how to word it.
It might be interesting to note that I can have my own software run several queries creating the queries automatically for each table and each column, but I'd rather avoid to do that as keeping track of all those triggers will be a pain in the long run.
I'd prefer to have a little triggers per database as possible, if only by not having to make a trigger for each individual column name.
Edit: To clarify: I am trying to avoid having to create an automated script that goes and scans every table, and sequentially every column of every table, to create a trigger to see if something is changed there. My biggest issue at the moment is the trigger behavior on updates, but I'm hoping to avoid having to specify tables as well for insert and delete
Edit 2: To avoid future confusion, I'm looking for a solution to this problem for both SQL Server (MS SQL/T SQL) and MySQL
Edit 3: Turns out that I read the documentation very wrongly and (at least on MySql) the trigger activates on any given updated column without having to define a specific one. Regardless, I'm still wondering if there is a way to just have less triggers than having one for each table in a database. (i.e. 1 for any type of update(), 1 for any type of insert(), and 1 for any type of delete()
EDIT 4: Forgot that the argument for overwriting 1 field will come with performance issues, I've considered this and I'm now working with multiple rows. I've also handled the creating of 3 triggers (insert(), update(), and delete()) for each database through my software's code, I really wished this could've been avoided, but it cannot.
Solution
After a bunch more digging on the internet and keep finding opposite results of what I was looking for, and a bunch of trial and error, I found a solution.
First and foremost: having triggers not being dependent on a table (aka, the trigger activates for every table is impossible, it cannot be done, which is too bad, it would've been nice to keep this out of the program code, but nothing I can do about it.
Second: the issue for updates on not being column specific was an error due to my part for searching for triggers not being dependent on specific columns only giving me examples for triggers that are.
The following solution works for MySql, I have yet to test this on SQL Server, but I expect it to not be too different.
CREATE TRIGGER [tablename]_last_modified_insert
AFTER INSERT/UPDATE/DELETE ON [db].[tablename]
FOR EACH ROW
BEGIN
INSERT INTO [db].last_modified(last_modified_on)
VALUES(current_timestamp())
END
As for dynamically creating these triggers, the following show how I get it to work:
First Query:
SHOW TABLES
I run the above query to get all the tables in the database, exclude the last_modified I made myself, and loop through all of them, creating 3 triggers for each.
A big thank you to Arvo and T2PS for their replies, their comments helped by pointing me in the right direction and writing up the solution.
You're slightly off in the assumption that SQL Server triggers are per-column; the CREATE TRIGGER syntax binds the trigger to the named table for the specified operations. The trigger will be called with two logical tables in scope (inserted & deleted) that contain the rows modified by the operation that caused the trigger to fire; if you wanted to check for specific columns' values or changes, then the trigger logic would need to operate against those logical tables.
If you take this approach, you will need to create a trigger for each table you wish to monitor in this fashion; we've had a similar need to track changes (at a more granular level), we didn't find a "pseudotable" that corresponds to all tables in a schema/database. You should also be aware that locking semantics will come into play by doing this, as you will have triggers from multiple tables all targeting the same row for an update as part of separate operations -- depending on the concurrency model in effect, you could be looking at performance consequences by doing so if you expect multiple DML queries to operate concurrently against your database.
I would suggest checking Arvo's commented link above for suitability instead; querying system views is more likely to avoid contention (and other performance-related) issues from using triggers in your scenario.
After a bunch more digging on the internet and keep finding opposite results of what I was looking for, and a bunch of trial and error, I found a solution.
First and foremost: having triggers not being dependent on a table (aka, the trigger activates for every table is impossible, it cannot be done, which is too bad, it would've been nice to keep this out of the program code, but nothing I can do about it.
Second: the issue for updates on not being column specific was an error due to my part for searching for triggers not being dependent on specific columns only giving me examples for triggers that are.
The following solution works for MySQL, I have yet to test this on SQL Server, but I expect it to not be too different.
CREATE TRIGGER [tablename]_last_modified_insert
AFTER INSERT/UPDATE/DELETE ON [db].[tablename]
FOR EACH ROW
BEGIN
INSERT INTO [db].last_modified(last_modified_on)
VALUES(current_timestamp())
END
As for dynamically creating these triggers, the following show how I get it to work:
First Query:
SHOW TABLES
I run the above query to get all the tables in the database, exclude the last_modified I made myself, and loop through all of them, creating 3 triggers for each.
Perhaps you could use Audit for SQL Server:
CREATE SERVER AUDIT [ServerAuditName]
TO FILE
(
FILEPATH = N'C:\Program Files......'
)
ALTER SERVER AUDIT [ServerAuditName] WITH (STATE=ON)
GO
CREATE DATABASE AUDIT SPECIFICATION [mySpec]
FOR SERVER AUDIT [ServerAuditName]
ADD (INSERT, UPDATE, DELETE ON DATABASE::databasename BY [public])
WITH (STATE=ON)
GO
Then you can query for changes:
SELECT *
FROM sys.fn_get_audit_file ('C:\Program Files......',default,default);
GO
I am trying to restore a DB using an SQL script, but things foreign key constraints get in the way
I am taking a MySQL DB and bringing it over to PostgreSQL.
Since the MySQL create table syntax ended up being quite different, I took another PostgreSQL DB with the same schema, but different data and restored the schema only, from that.
In other words, I now have a database with tables, constraints, sequences and all that shnaz but no data inside.
So, it's is time to restore data.
I take a backup of the MySQL DB with phpMyAdmin (data only) as an SQL script (pgAdmin does not seem to accept zip or gzip files for some reason) and run the SQL script.
Now, this is where the problems start to happen, it's only natural, I am going from MySQL to PostgreSQL, so syntax errors are bound to happen.
But, there are other non syntax related problems to, like this one:
ERROR: insert or update on table "_account" violates foreign key constraint "fk_1_account"
DETAIL: Key (accountid)=(2) is not present in table "_entity".
So, yeah, basically, a foreign constraint exists, the query is trying to insert data into the _account table, but the corresponding data has not been inserted into the _entity table yet.
How do I get around that? Is there a way to make pgAdmin3/PostgreSQL disable ALL OF the constraints, insert the data, and then re-enable the constraints?
A syntax related error I encountered, was this one:
INSERT INTO _accounttype_seq (id) VALUES (11);
The PostgreSQL equivalent of that statement (if I am correct) is
ALTER SEQUENCE _accounttype_seq INCREMENT BY 11;
But, it's a bit of a pain to run through the whole script and change all 200+ Sequence insert statements. So, I am being lazy here, but is there an easier way to deal with the sequences as well?
Or, do you guys have any suggestions for a different set of tools to make this easier?
Thanks for your time, have a good day.
Do not try to get around the foreign key constraints. That is the way to make sure the data is bad.
First look at the constraints and make sure you are inserting to the tables in the correct order. If _entity is parent of "_account, then it should be populated first.
Next you need to have the script move any failing records to an exception table. Then you can look at them and see what the data integrity issues is and if you need to throw the records away permanently or try to figure out what the missing parent value should be. If it is critical data such as orders where the customer no longer exists (possible in any system that didn't have correct fks to begin with) and you must keep the record and cannot determine what the parent value should have been, you can create an 'Unknown" record in the customer table and assign all bad orders to that customer id.
And manually changing the alter sequences shouldn't take long even if it is boring. There wil be plently of other things you need to handle manually in a conversion of this type.
I would try to find a data import tool for PostgreSQL - I live in SQL server world where I would use SSIS but you need the equivalent of SSIS for the PostgreSQL world.
Aparently the foreign keys weren't actually enforced in MySQL (maybe because of using MyISAM) or the generated SQL just does it in the wrong order.
If it's "only" the wrong order, I see two possible solutions:
edit the generated script and either move all FK definitions to the end of the script
Edit the definition of each FK constraint and set them all to initially deferred. Then run the script as one single transaction with only on commit at the very end.
Edit (because this is too much to be put as a comment)
Using SET CONSTRAINTS ALL DEFERRED will only work if the constraints have been created with the option DEFERRABLE.
To run everything in one single transaction, you have to make sure you have turned autocommit off. Then simply run the INSERTs and at the very end issue a COMMIT. A ; will only commit if you have autocommit on.
If you want to be independent of the autocommit setting, then start your script with [BEGIN][1] and make sure there is only a single COMMIT at the very end.
BEGIN DEFERRABLE
INSERT INTO table_one ... ;
INSERT INTO table_two ... ;
.....
COMMIT;
I need the sample program in Java for keeping the history of table if user inserted, updated and deleted on that table. Can anybody help in this?
Thanks in advance.
If you are working with Hibernate you can use Envers to solve this problem.
You have two options for this:
Let the database handle this automatically using triggers. I don't know what database you're using but all of them support triggers that you can use for this.
Write code in your program that does something similar when inserting, updating and deleting a user.
Personally, I prefer the first option. It probably requires less maintenance. There may be multiple places where you update a user, all those places need the code to update the other table. Besides, in the database you have more options for specifying required values and integrity constraints.
Well, we normally have our own history tables which (mostly) look like the original table. Since most of our tables already have the creation date, modification date and the respective users, all we need to do is copy the dataset from the live table to the history table with a creation date of now().
We're using Hibernate so this could be done in an interceptor, but there may be other options as well, e.g. some database trigger executing a script, etc.
How is this a Java question?
This should be moved in Database section.
You need to create a history table. Then create database triggers on the original table for "create or replace trigger before insert or update or delete on table for each row ...."
I think this can be achieved by creating a trigger in the sql-server.
you can create the TRIGGER as follows:
Syntax:
CREATE TRIGGER trigger_name
{BEFORE | AFTER } {INSERT | UPDATE |
DELETE } ON table_name FOR EACH ROW
triggered_statement
you'll have to create 2 triggers one for before the operation is performed and another after the operation is performed.
otherwise it can be achieved through code also but it would be a bit tedious for the code to handle in case of batch processes.
You should try using triggers. You can have a separate table (exact replica of your table of which you need to maintain history) .
This table will then be updated by trigger after every insert/update/delete on your main table.
Then you can write your java code to get these changes from the second history table.
I think you can use the redo log of your underlying database to keep track of the operation performed. Is there any particular reason to go for the program?
You could try creating say a List of the objects from the table (Assuming you have objects for the data). Which will allow you to loop through the list and compare to the current data in the table? You will then be able to see if any changes occurred.
You can even create another list with a object that contains an enumerator that gives you the action (DELETE, UPDATE, CREATE) along with the new data.
Haven't done this before, just a idea.
Like #Ashish mentioned, triggers can be used to insert into a seperate table - this is commonly referred as Audit-Trail table or audit log table.
Below are columns generally defined in such audit trail table : 'Action' (insert,update,delete) , tablename (table into which it was inserted/deleted/updated), key (primary key of that table on need basis) , timestamp (the time at which this action was done)
It is better to audit-log after the entire transaction is through. If not, in case of exception being passed back to code-side, seperate call to update audit tables will be needed. Hope this helps.
If you are talking about db tables you may use either triggers in db or add some extra code within your application - probably using aspects. If you are using JPA you may use entity listeners or perform some extra logic adding some aspect to your DAO object and apply specific aspect to all DAOs which perform CRUD on entities that needs to sustain historical data. If your DAO object is stateless bean you may use Interceptor to achive that in other case use java proxy functionality, cglib or other lib that may provide aspect functionality for you. If you are using Spring instead of EJB you may advise your DAOs within application context config file.
Triggers are not suggestable, when I stored my audit data in file else I didn't use the database...my suggestion is create table "AUDIT" and write java code with help of servlets and store the data in file or DB or another DB also ...
If you got 100 000 users, is MySQL executing one SQL query at the time?
Because in my PHP code I check if a certain row exists; if it doesn't it creates one. If it does, it just updates the row counter.
It crossed my mind that perhaps 100 users are checking if the row exists at the same time, and when it doesn't they all create one row each.
If MySQL is handling them sequentially I know that it won't be an issue, then one user will check if it exists, if not, create it. The other user will check if it exists, and since that's the case, it just updates the counter.
But if they all check if it exists at the same time and let's say it doesn't, then they all create one row and the whole table structure will fail.
Would be great if someone could shed some light on this topic.
Use a UNIQUE constraint or, if viable, make the primary key one of your data items and the SQL server will prevent duplicate rows from being created. You can even use the "ON DUPLICATE KEY UPDATE ..." syntax to specify the alternate operation if the row already exists.
From your comments, it sounds like you could use the user_id as your primary key, in which case, you'd be able to use something like this:
INSERT INTO usercounts (user_id,usercount)
VALUES (id-goes-here,1)
ON DUPLICATE KEY UPDATE usercount=usercount+1;
If you put the check and insert into a transaction then you can avoid this problem. This way, the check and create will be run as one one query and there shouldn't be any confusion