I am working with Hibernate 3.6.4 and MySQL.
I have a table with unique constraints on four columns and 3 other columns. When the UI application create new instances of the corresponding Object it may create it with those four properties with values already in the table. the result, upon save, is, of course JDBC Exception of duplicate entry.
Is there a way to tell Hibernate to not insert new entry but update the rest of the three columns or upon each save I need to manually query the DB to see if exist and update accordingly?
Thanks.
The clean and database independent approach for this problem is to first check if such an instance exists and depending on that do an insert or update in your application logic.
That said, there might be a way to take advantage of the MySQL INSERT ... ON DUPLICATE KEY UPDATE feature documented here. In this case you must specify a custom SQL INSERT statement for your entity like described in this related question. But if this works depends on the way your entity IDs are generated to begin with. Take a look at this blog article concerning this issue.
Generally, you must deal with every aspect of the problem that Hibernate thinks a transient instance is persisted, when in fact a persistent instance is updated. This might be an issue with generated entity IDs, other generated entity values, entity versions, concurrency, expected insert/update row count, 2nd level and query cache, etc.
So, I think while this would be a nice thing to experiment with I would definitely not use this feature in a production application.
You must indeed explicitely get the entity with the four unique values, and then update it if it exists or create a new one if it does not. There is no way around that.
BTW, note that even with such a mechanism, you might end up with exceptions if two transactions get the entity concurrently, find that it doesn't exist, and both try to create a new one.
Related
I have several databases that are used by several applications (one of which is our own, the others we have no control over in what they do).
Out software has to know when the database has last been changed. For reasons I won't get into to keep this short we decided that going with a new table per database that has a singular field: last_changed_on that has a GetDate() as a value. This way our own software can check when it was last changed and check it to the date it has stored for said database and do things if the date is newer than what is stored in-memory.
After doing some research we decided that working with Triggers was the way to go, but from what I could find online, triggers look at specific columns that you set for Updates.
What I'd like to know is if there is a way to automate the process or just have a trigger that happens whenever anything happens insert, update, remove wise?
So I am looking for something like this:
CREATE TRIGGER LastModifiedTrigger
ON [dbo].[anytable]
AFTER INSERT, UPDATE, DELETE
AS
INSERT INTO dbo.LastModifiedTable (last_modified_on) VALUES (CURRENT_TIMESTAMP)
I know that the above example isn't a correct trigger, I'm rather new to them so I was unsure on how to word it.
It might be interesting to note that I can have my own software run several queries creating the queries automatically for each table and each column, but I'd rather avoid to do that as keeping track of all those triggers will be a pain in the long run.
I'd prefer to have a little triggers per database as possible, if only by not having to make a trigger for each individual column name.
Edit: To clarify: I am trying to avoid having to create an automated script that goes and scans every table, and sequentially every column of every table, to create a trigger to see if something is changed there. My biggest issue at the moment is the trigger behavior on updates, but I'm hoping to avoid having to specify tables as well for insert and delete
Edit 2: To avoid future confusion, I'm looking for a solution to this problem for both SQL Server (MS SQL/T SQL) and MySQL
Edit 3: Turns out that I read the documentation very wrongly and (at least on MySql) the trigger activates on any given updated column without having to define a specific one. Regardless, I'm still wondering if there is a way to just have less triggers than having one for each table in a database. (i.e. 1 for any type of update(), 1 for any type of insert(), and 1 for any type of delete()
EDIT 4: Forgot that the argument for overwriting 1 field will come with performance issues, I've considered this and I'm now working with multiple rows. I've also handled the creating of 3 triggers (insert(), update(), and delete()) for each database through my software's code, I really wished this could've been avoided, but it cannot.
Solution
After a bunch more digging on the internet and keep finding opposite results of what I was looking for, and a bunch of trial and error, I found a solution.
First and foremost: having triggers not being dependent on a table (aka, the trigger activates for every table is impossible, it cannot be done, which is too bad, it would've been nice to keep this out of the program code, but nothing I can do about it.
Second: the issue for updates on not being column specific was an error due to my part for searching for triggers not being dependent on specific columns only giving me examples for triggers that are.
The following solution works for MySql, I have yet to test this on SQL Server, but I expect it to not be too different.
CREATE TRIGGER [tablename]_last_modified_insert
AFTER INSERT/UPDATE/DELETE ON [db].[tablename]
FOR EACH ROW
BEGIN
INSERT INTO [db].last_modified(last_modified_on)
VALUES(current_timestamp())
END
As for dynamically creating these triggers, the following show how I get it to work:
First Query:
SHOW TABLES
I run the above query to get all the tables in the database, exclude the last_modified I made myself, and loop through all of them, creating 3 triggers for each.
A big thank you to Arvo and T2PS for their replies, their comments helped by pointing me in the right direction and writing up the solution.
You're slightly off in the assumption that SQL Server triggers are per-column; the CREATE TRIGGER syntax binds the trigger to the named table for the specified operations. The trigger will be called with two logical tables in scope (inserted & deleted) that contain the rows modified by the operation that caused the trigger to fire; if you wanted to check for specific columns' values or changes, then the trigger logic would need to operate against those logical tables.
If you take this approach, you will need to create a trigger for each table you wish to monitor in this fashion; we've had a similar need to track changes (at a more granular level), we didn't find a "pseudotable" that corresponds to all tables in a schema/database. You should also be aware that locking semantics will come into play by doing this, as you will have triggers from multiple tables all targeting the same row for an update as part of separate operations -- depending on the concurrency model in effect, you could be looking at performance consequences by doing so if you expect multiple DML queries to operate concurrently against your database.
I would suggest checking Arvo's commented link above for suitability instead; querying system views is more likely to avoid contention (and other performance-related) issues from using triggers in your scenario.
After a bunch more digging on the internet and keep finding opposite results of what I was looking for, and a bunch of trial and error, I found a solution.
First and foremost: having triggers not being dependent on a table (aka, the trigger activates for every table is impossible, it cannot be done, which is too bad, it would've been nice to keep this out of the program code, but nothing I can do about it.
Second: the issue for updates on not being column specific was an error due to my part for searching for triggers not being dependent on specific columns only giving me examples for triggers that are.
The following solution works for MySQL, I have yet to test this on SQL Server, but I expect it to not be too different.
CREATE TRIGGER [tablename]_last_modified_insert
AFTER INSERT/UPDATE/DELETE ON [db].[tablename]
FOR EACH ROW
BEGIN
INSERT INTO [db].last_modified(last_modified_on)
VALUES(current_timestamp())
END
As for dynamically creating these triggers, the following show how I get it to work:
First Query:
SHOW TABLES
I run the above query to get all the tables in the database, exclude the last_modified I made myself, and loop through all of them, creating 3 triggers for each.
Perhaps you could use Audit for SQL Server:
CREATE SERVER AUDIT [ServerAuditName]
TO FILE
(
FILEPATH = N'C:\Program Files......'
)
ALTER SERVER AUDIT [ServerAuditName] WITH (STATE=ON)
GO
CREATE DATABASE AUDIT SPECIFICATION [mySpec]
FOR SERVER AUDIT [ServerAuditName]
ADD (INSERT, UPDATE, DELETE ON DATABASE::databasename BY [public])
WITH (STATE=ON)
GO
Then you can query for changes:
SELECT *
FROM sys.fn_get_audit_file ('C:\Program Files......',default,default);
GO
Consider this code. This code inserts the row to the database if it is not found, then only updates it if the row is found. The updateNode() method gives the entity some values based on the user input, so I called it in both cases.
session.beginTransaction();
node = (Node)session.createQuery("from Node").uniqueResult();
if (node == null) {
node = new Node();
updateNode();
session.save(node);
} else {
updateNode();
}
session.getTransaction.commit();
Is there a better way of checking if rows exist in the table aside from using queries?
Is the cat alive or dead? You don't know without checking it, so you'll have to execute the query in the database.
I assume your question is about to avoid writing such a query manually, but rather letting Hibernate do it itself. Then you may want to look at querying by example/prototype.
Regardless of the approach taken, keep concurrency issues in mind though; you may want to apply some unique constraints and/or optimistic/pessimistic locks.
The only way to find out if something already exists in the database is to query it. However you do not need a separate query. You need only one query and not two thanks to mysql's INSERT... ON DUPLICATE KEY UPDATE feature. And it doesn't need any additional java coding either.
If you want to do this with hibernate it will have to be a custom query and you will need to return the inserted row id with LAST_INSERT_ID in your query.
Is it possible to use any sort of logic in MySQL without using any procedures? My web hosting does not let me create any procedures so I'm looking for a workaround.
The type of thing I want to do is only add an item to a table if it doesn't already exist. Or add a column to a table if it's not already there. There are some operations that can be done such as CREATE TABLE IF NOT EXISTS and so on, but some operations I require do not have such luxuries :(
I realised late on that my lovely procs won't work and so I tried writing IF/ELSE logic as top-level queries, but for MySQL, IF ELSE blocks only seem to work inside functions/procs and not at the global scope.
Any workarounds greatfully received - I've already asked the hosting to grant me privileges to create procedures but no reply as yet...
I suppose you don't have access to the INFORMATION_SCHEMA either. You can possibly find solutions but it would be better, in my oninion, to:
Change your hosting provider. Seriously. Pay more - if needed - for a MySQL instance that you can configure to your needs. You only have a crippled DBMS if you are not allowed to create procedures and functions.
Posible workarounds for the specific task: You want to add a column if it doesn't exist.
1) Just ALTER TABLE and add the column. If it already exists, you'll get an error. You can catch that error, in your application.
2) (If you have no access to the INFORMATION_SCHEMA) maintain a version of the schema, for your database.
The best solution that I can think of would be to use an additional language with SQL. For example, you can run a query for a specific record, and based on the response that you get, you can conditionally run an INSERT statement.
For inserting a table if it doesn't exist, try using the SHOW TABLES statement and testing whether or not a name exists in the result set.
MySQL supports INSERT IGNORE. and INSERT ... ON DUPLICATE KEY UPDATE.
The following will insert a new row, but only if there is no existing row with id=10. (This assumes that id is defined as a unique or primary key).
INSERT IGNORE INTO my_table (id, col1, col2) values (10, "abc", "def");
The following will insert a new row, but if there is an existing row with id=10 (again, assuming id is unique or primary), the existing row will be updated to hold the new values, instead of inserting a new row.
INSERT INTO my_table (id, col1, col2) values (10, "abc", "def")
ON DUPLICATE KEY UPDATE col1=VALUES(col1), col2=VALUES(col2)
Also, CREATE TABLE supports the IF NOT EXISTS modifier. So you can do something like:
CREATE TABLE IF NOT EXISTS my_table ...
There are many other similar options and modifiers available in MySQL. Check the docs for more.
Originally I created a big script to create or update the database schema, to make it easier to deploy database changes from my local machine to the server.
My script was doing a lot of "if table 'abc' exists and it doesn't have a FK constraint called 'blah'" then create an FK constraint called 'blah' on table 'abc'... and so on.
I now realise it's not actually necessary to check whether a table has a certain column or constraint etc, because I can just maintain a schema-versioning system, and query the DB schema-version when my app starts, or when I navigate to a certain page.
e.g. let's say I want to add a new column to a table. It works like this:
Add a new migration script to the app code, containing the SQL required to add the column to the existing table
Increment the app's schema-version by 1
On app startup, the app queries the DB for the DB's schema-version
If DB schema-version < app schema-version, execute the SQL migration scripts between the two schema-versions, and then update the DB schema-version to be the same as the app
e.g. if the DB's schema-version is 5 and the app version is 8, the app will apply migration scripts 5-6, 6-7 and 7-8 to the DB. These can just be run without having to check anything on the DB side.
The app is therefore solely responsible for updating the DB schema and there's no need for me to ever have to execute schema change scripts on the local or remote DB.
I think it's a better system than the one I was trying to implement for my question.
I am trying to restore a DB using an SQL script, but things foreign key constraints get in the way
I am taking a MySQL DB and bringing it over to PostgreSQL.
Since the MySQL create table syntax ended up being quite different, I took another PostgreSQL DB with the same schema, but different data and restored the schema only, from that.
In other words, I now have a database with tables, constraints, sequences and all that shnaz but no data inside.
So, it's is time to restore data.
I take a backup of the MySQL DB with phpMyAdmin (data only) as an SQL script (pgAdmin does not seem to accept zip or gzip files for some reason) and run the SQL script.
Now, this is where the problems start to happen, it's only natural, I am going from MySQL to PostgreSQL, so syntax errors are bound to happen.
But, there are other non syntax related problems to, like this one:
ERROR: insert or update on table "_account" violates foreign key constraint "fk_1_account"
DETAIL: Key (accountid)=(2) is not present in table "_entity".
So, yeah, basically, a foreign constraint exists, the query is trying to insert data into the _account table, but the corresponding data has not been inserted into the _entity table yet.
How do I get around that? Is there a way to make pgAdmin3/PostgreSQL disable ALL OF the constraints, insert the data, and then re-enable the constraints?
A syntax related error I encountered, was this one:
INSERT INTO _accounttype_seq (id) VALUES (11);
The PostgreSQL equivalent of that statement (if I am correct) is
ALTER SEQUENCE _accounttype_seq INCREMENT BY 11;
But, it's a bit of a pain to run through the whole script and change all 200+ Sequence insert statements. So, I am being lazy here, but is there an easier way to deal with the sequences as well?
Or, do you guys have any suggestions for a different set of tools to make this easier?
Thanks for your time, have a good day.
Do not try to get around the foreign key constraints. That is the way to make sure the data is bad.
First look at the constraints and make sure you are inserting to the tables in the correct order. If _entity is parent of "_account, then it should be populated first.
Next you need to have the script move any failing records to an exception table. Then you can look at them and see what the data integrity issues is and if you need to throw the records away permanently or try to figure out what the missing parent value should be. If it is critical data such as orders where the customer no longer exists (possible in any system that didn't have correct fks to begin with) and you must keep the record and cannot determine what the parent value should have been, you can create an 'Unknown" record in the customer table and assign all bad orders to that customer id.
And manually changing the alter sequences shouldn't take long even if it is boring. There wil be plently of other things you need to handle manually in a conversion of this type.
I would try to find a data import tool for PostgreSQL - I live in SQL server world where I would use SSIS but you need the equivalent of SSIS for the PostgreSQL world.
Aparently the foreign keys weren't actually enforced in MySQL (maybe because of using MyISAM) or the generated SQL just does it in the wrong order.
If it's "only" the wrong order, I see two possible solutions:
edit the generated script and either move all FK definitions to the end of the script
Edit the definition of each FK constraint and set them all to initially deferred. Then run the script as one single transaction with only on commit at the very end.
Edit (because this is too much to be put as a comment)
Using SET CONSTRAINTS ALL DEFERRED will only work if the constraints have been created with the option DEFERRABLE.
To run everything in one single transaction, you have to make sure you have turned autocommit off. Then simply run the INSERTs and at the very end issue a COMMIT. A ; will only commit if you have autocommit on.
If you want to be independent of the autocommit setting, then start your script with [BEGIN][1] and make sure there is only a single COMMIT at the very end.
BEGIN DEFERRABLE
INSERT INTO table_one ... ;
INSERT INTO table_two ... ;
.....
COMMIT;
I need the sample program in Java for keeping the history of table if user inserted, updated and deleted on that table. Can anybody help in this?
Thanks in advance.
If you are working with Hibernate you can use Envers to solve this problem.
You have two options for this:
Let the database handle this automatically using triggers. I don't know what database you're using but all of them support triggers that you can use for this.
Write code in your program that does something similar when inserting, updating and deleting a user.
Personally, I prefer the first option. It probably requires less maintenance. There may be multiple places where you update a user, all those places need the code to update the other table. Besides, in the database you have more options for specifying required values and integrity constraints.
Well, we normally have our own history tables which (mostly) look like the original table. Since most of our tables already have the creation date, modification date and the respective users, all we need to do is copy the dataset from the live table to the history table with a creation date of now().
We're using Hibernate so this could be done in an interceptor, but there may be other options as well, e.g. some database trigger executing a script, etc.
How is this a Java question?
This should be moved in Database section.
You need to create a history table. Then create database triggers on the original table for "create or replace trigger before insert or update or delete on table for each row ...."
I think this can be achieved by creating a trigger in the sql-server.
you can create the TRIGGER as follows:
Syntax:
CREATE TRIGGER trigger_name
{BEFORE | AFTER } {INSERT | UPDATE |
DELETE } ON table_name FOR EACH ROW
triggered_statement
you'll have to create 2 triggers one for before the operation is performed and another after the operation is performed.
otherwise it can be achieved through code also but it would be a bit tedious for the code to handle in case of batch processes.
You should try using triggers. You can have a separate table (exact replica of your table of which you need to maintain history) .
This table will then be updated by trigger after every insert/update/delete on your main table.
Then you can write your java code to get these changes from the second history table.
I think you can use the redo log of your underlying database to keep track of the operation performed. Is there any particular reason to go for the program?
You could try creating say a List of the objects from the table (Assuming you have objects for the data). Which will allow you to loop through the list and compare to the current data in the table? You will then be able to see if any changes occurred.
You can even create another list with a object that contains an enumerator that gives you the action (DELETE, UPDATE, CREATE) along with the new data.
Haven't done this before, just a idea.
Like #Ashish mentioned, triggers can be used to insert into a seperate table - this is commonly referred as Audit-Trail table or audit log table.
Below are columns generally defined in such audit trail table : 'Action' (insert,update,delete) , tablename (table into which it was inserted/deleted/updated), key (primary key of that table on need basis) , timestamp (the time at which this action was done)
It is better to audit-log after the entire transaction is through. If not, in case of exception being passed back to code-side, seperate call to update audit tables will be needed. Hope this helps.
If you are talking about db tables you may use either triggers in db or add some extra code within your application - probably using aspects. If you are using JPA you may use entity listeners or perform some extra logic adding some aspect to your DAO object and apply specific aspect to all DAOs which perform CRUD on entities that needs to sustain historical data. If your DAO object is stateless bean you may use Interceptor to achive that in other case use java proxy functionality, cglib or other lib that may provide aspect functionality for you. If you are using Spring instead of EJB you may advise your DAOs within application context config file.
Triggers are not suggestable, when I stored my audit data in file else I didn't use the database...my suggestion is create table "AUDIT" and write java code with help of servlets and store the data in file or DB or another DB also ...