Deploy Sql Changes to Customer Databases (altered tables, functions,...) - mysql

Question

I am developing an application that uses a database (either PostgreSQL, MySQL, Oracle or MSSQL) on the customers system.
Therefore I need to perform database updates with each new version.
I am currently in the concept phase and have nothing running in production.
All the DDL statements are in script files.
The structure looks like this:
tables\employees.sql
customers.sql
orders.sql
Those scripts are also in version control and can be used to build the database from stretch.
Of course there will be changes sometime in the future to those tables.
For example table employees gets created like this:
CREATE TABLE if not exists employees
(
EmployeeId serial,
FirstName text,
PRIMARY KEY (EmployeeId)
);
And in a future release that table gets extended:
ALTER TABLE employees ADD COLUMN address varchar(30);
On my research I found this example: https://stackoverflow.com/posts/115422/revisions.
A version number gets used to perform specific changes.
I like that concept and my idea is to implement something similar.
But instead of a system version number I was thinking about introducing a version for each table.
When creating the employee table it gets the Version number 1. With each change on that table the version number get increased by 1. After adding the address column (alter statement above) the table version would be 2.
Each table change would happen in a nested transaction like this:
BEGIN TRANSACTION;
UPDATE employees SET Version = 2;
ALTER TABLE employees
ALTER TABLE employees ADD COLUMN address varchar(30);
END TRANSACTION;
If the table version is lower than the current table version the transaction would be rolled back.
The implentation of that logic is yet to be done.
The benefit would be that all changes on a table are inside the table's script file itself and the initial statement is always up to date.
For example when first creating the employee table it would look like this:
employees.sql
CREATE TABLE if not exists employees
(
EmployeeId serial,
FirstName text,
Version int default 1 not null,
PRIMARY KEY (EmployeeId)
);
After some changes it looks like this:
employees.sql
CREATE TABLE if not exists employees
(
EmployeeId serial,
FirstName varchar(100),
address varchar(80),
Version int default 3 not null, -- notice the 3
PRIMARY KEY (EmployeeId)
);
-- First Change
BEGIN TRANSACTION;
UPDATE employees SET Version = 2;
ALTER TABLE employees
ALTER TABLE employees ADD COLUMN address varchar(30);
END TRANSACTION;
-- Second Change
BEGIN TRANSACTION;
UPDATE employees SET Version = 3;
ALTER TABLE employees
ALTER COLUMN address TYPE varchar(80),
ALTER COLUMN FirstName TYPE varchar(100);
END TRANSACTION;
Is that concept acceptable or am I reinventing the wheel here?

Answer 1

I think setting the version number per table is overkill. Also, it complicates managing the DB and the application. I suggest you add a new table for DB_VersionNumber and add one row in this table for each upgrade. What I have been doing is this:
1) Create a table in DB for database versions (steps)
2) Create a SP that checks this table and runs a DB upgrade step if it does not exist in the table, otherwise the step is skipped.
3) For each and every DB change, add a step in the upgrade script file (which you have already created and added to the source control).
Here is the table and the SP:
IF OBJECT_ID (N'DB_Version', N'U') IS NULL
Begin
CREATE TABLE [DB_Version](
[VersionNumber] [decimal](18, 2) NOT NULL,
[CommitTimestamp] [smalldatetime] NOT NULL
) ON [PRIMARY]
ALTER TABLE DB_Version
ADD CONSTRAINT UQ_VersionNumber UNIQUE (VersionNumber);
End
IF OBJECT_ID ( 'NewDBStep', 'P' ) IS NULL
begin
Exec ('
-- ============================================
-- Description: Applies a new DB upgrade step to the current DB
-- =============================================
CREATE PROCEDURE NewDBStep
#dbVersion [decimal](18, 2),
#script varchar (max)
AS
BEGIN
If not exists (select 1 from DB_Version Where VersionNumber = #dbVersion)
Begin
-- SET NOCOUNT ON added to prevent extra result sets from
-- interfering with SELECT statements.
SET NOCOUNT ON;
BEGIN TRY
Begin tran
Exec (#script)
Insert into DB_Version (VersionNumber, CommitTimestamp) Values (#dbVersion, CURRENT_TIMESTAMP);
Commit tran
Print ''Applied upgrade step '' + Cast ( #dbVersion as nvarchar(20))
END TRY
BEGIN CATCH
Rollback tran
Print ''Failed to apply step '' + Cast ( #dbVersion as nvarchar(20))
Select ERROR_NUMBER() AS ErrorNumber
,ERROR_SEVERITY() AS ErrorSeverity
,ERROR_STATE() AS ErrorState
,ERROR_PROCEDURE() AS ErrorProcedure
,ERROR_LINE() AS ErrorLine
,ERROR_MESSAGE() AS ErrorMessage;
END CATCH
End
END ') ;
End
Then, apply your upgrades by calling the SP (the key is that you have to assign a unique step number to each upgrade script:
---------------- Add the new steps here
-- Step: 0.01
-- Adding the MyTableName table if it does not exist.
Exec NewDBStep 0.01, '
IF OBJECT_ID (N''MyTableName'', N''U'') IS NULL
Begin
CREATE TABLE [MyTableName](
[Id] [int] IDENTITY(1,1) NOT NULL,
[UserType] [nvarchar](20) NULL,
PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON) ON [PRIMARY]
) ON [PRIMARY]
End
'
Exec NewDBStep 1.00, '
-- Some other DDL script
'