I don't seem to be able to execute a type modification on a Mysql 5.7.10 instance (it works with H2 though).
Here are the changeset steps that are involved with the regarded field:
Creation:
<column name="last_modify_time" type="bigint">
<constraints nullable="false" />
</column>
Modification:
<modifyDataType tableName="USER" columnName="last_modify_time" newDataType="timestamp" />
the error msg in Mysql is
Invalid default value for 'last_modify_time' [Failed SQL: ALTER TABLE USER MODIFY last_modify_time timestamp]
Manually modifying the the request to the following works:
ALTER TABLE USER MODIFY last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
I don't really understand why Mysql needs the default value. Maybe it's an edge case with the version 5.7.10 (using the default configuration options).
In any case Liquibase should be able to handle it.
I've tried to add/remove default value prior to the modifyDataType, without success.
when you are modifying type of a column using liquibase which is non nullable ... you have to first make it nullable and then modify it use safe modify
It turns out that an exactly identical modification on a field named create_time, placed right before, didn't cause any problem. Swapping the two changsets order did solve the problem.
<changeSet id="11" author="author">
<comment>rename last_modify_time to last_modified_date and change type to timestamp</comment>
<modifyDataType tableName="USER" columnName="last_modify_time" newDataType="timestamp" />
<renameColumn tableName="USER" oldColumnName="last_modify_time" newColumnName="last_modified_date" columnDataType="timestamp" />
</changeSet>
<changeSet id="12" author="author">
<comment>rename create_time to created_date and change type to timestamp</comment>
<modifyDataType tableName="USER" columnName="create_time" newDataType="timestamp" />
<renameColumn tableName="USER" oldColumnName="create_time" newColumnName="created_date" columnDataType="timestamp"/>
</changeSet>
I still can't explain what happend, I'm happy enough to get it working. If someone wants to reproduce the error I'd be happy to help.
Related
I'm just getting into BIML and have written some Scripts to creat a few DTSX-Packages. In general the most things are working. But one thing makes me crazy.
I have an ODBC-Source (PostgreSQL). From there I'm getting data out of a table using an ODBC-Source. The table has a text-Column (Name of the column is "description"). I cast this column to varchar(4000) in the query in the ODBC-Source (I know that there will be truncation, but it's ok). If I do this manually in Visual Studio the Advanced Editor of the ODBC-Source is showing "Unicode string [DT_WSTR]" with a Length of 4000 both for the External and the Output-Column. So there everything is fine. But if I do the same things with BIML and generate the SSIS-Package the External-Column will still say "Unicode string [DT_WSTR]" with a Length of 4000, but the Output-Column is telling "Unicode text stream [DT_NTEXT]". So the mapping done by BIML differs from the Mapping done by SSIS (manually). This is causing two things (warnings):
A Warning that metadata has changed and should be synced
And a Warning that the Source uses LOB-Columns and is set to Row by Row-Fetch..
Both warnings are not cool. But the second one also causes a drasticaly degredation in Performance! If I set the cast to varchar(255) the Mapping is fine (External- and Output-Column is then "Unicode string [DT_WSTR]" with a Length of 255). But as soon as I go higher, like varchar(256) it's again treated as [DT_NTEXT] in the Output.
Is there anything I can do about this? I invested days in the Evaluation of BIML and find many things an increase in Quality of Life, but this issue is killing it. It defeats the purpose of BIML if I have to correct the Errors of BIML manually after every Build.
Does anyone know how I can solve this Issue? A correct automatic Mapping between External- and Output-Columns would be great, but at least the option to define the Mapping myself would be ok.
Any Help is appreciated!
Greetings
Marco
Edit As requested a Minimal Example for better understanding:
The column in the ODBC Source (Postegres) has the type "text" (Columnname: description)
I select it in a ODBC-Source with this Query (DirectInput):
SELECT description::varchar(4000) from mySourceTable
The ODBC-Source in Biml looks like this:
<OdbcSource Name="mySource" Connection="mySourceConnection"> <DirectInput>SELECT description::varchar(4000) from mySourceTable</DirectInput></OdbcSource>
If I now generate the dtsx-Package the ODBC-Source throws the above mentioned warnings with the above mentioned Datatypes for External and Output-Column
As mentioned in the comment before I got an answer from another direction:
You have to use DataflowOverrides in the ODBC-Source in BIML. For my example you have to do something like this:
`<OdbcSource Name="mySource" Connection="mySourceConnection">
<DirectInput>SELECT description::varchar(4000) from mySourceTable</DirectInput>
<DataflowOverrides>
<OutputPath OutputPathName="Output">
<Columns>
<Column ColumnName="description" SsisDataTypeOverride="DT_WSTR" DataType="String" Length="4000" />
</Columns>
</OutputPath>
<OutputPath OutputPathName="Error">
<Columns>
<Column ColumnName="description" SsisDataTypeOverride="DT_WSTR" DataType="String" Length="4000" />
</Columns>
</OutputPath>
</DataflowOverrides>
</OdbcSource>`
You won't have to do the Overrides for all columns, only for the ones you have mapping-Issues with.
Hope this solution can help anyone who passes by.
Cheers
I am trying out Liquibase in view of trying to automate Database Deployments and get some structure around Database changes.
Here is my Changeset which i wanted to evaluate for auto rollbacks. Now, i am trying to introduce an error in the foreign key constraint and was expecting that the rollback statements run after the error and it deletes the column from the customer table as well.
http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-2.0.xsd">
<preConditions>
<dbms type="mysql" />
</preConditions>
<changeSet id="002" author="Hrishi" failOnError="true"
runInTransaction="true">
<preConditions onError="HALT" onFail="HALT"
onFailMessage="The table gender doesnt exist. Precondition not met.">
<not>
<columnExists tableName="customer" columnName="gender_id" />
</not>
<and>
<not>
<foreignKeyConstraintExists foreignKeyName="fk_gender_customer"/>
</not>
</and>
</preConditions>
<addColumn tableName="customer" schemaName="sakila">
<column name="gender_id" type="smallint(1)" />
</addColumn>
<addForeignKeyConstraint constraintName="fk_gender_customer"
referencedTableName="gender" baseColumnNames="gender_id"
baseTableName="customer" referencedColumnNames="invoking_an_error" />
<rollback>
<dropColumn tableName="customer" columnName="gender_id" />
<dropForeignKeyConstraint baseTableName="customer"
constraintName="fk_gender_customer" />
</rollback>
</changeSet>
What really though does happen is that there is no entry in the DATABASECHANGELOG table. Which i expected, but then the column in the customer table still exists.
I did look at the help on the addColumn url on liquibase documentation and MySQL is supported.
http://www.liquibase.org/documentation/changes/add_column.html
Is my understanding correct that it should autorollback all the statements in the changeset?
EDIT : I think the problem in this specific case is that the ALTER table is a DDL statement. From the documentation of MySQL cannot-roll-back.
I'm not sure of the command you issued, but the changes listed in the <rollback> section of the changeset do not happen automatically when an error happens. When an error happens, liquibase tries to roll back the database transaction, but if some things have already been committed, you can get into an inconsistent state. The <rollback> tag is there so that you can manually roll back changes by using the liquibase rollback command.
I have a liquibase xml script. When I run it on Postgres I don't face any problem but when I run it for MYSQL it gives error when the structure is of the following type:-
<insert tableName="user_table">
<column name="id" valueComputed="(select max(id)+1 from user_table)"/>
<column name="name" value="someName"/>
</insert>
When the above script is executed for MYSQL it gives error:-
You can't specify target table 'user_table' for update in FROM clause.
I found a solution to this by using alias like this :-
<insert tableName="user_table">
<column name="id" valueComputed="(select max(id)+1 from (Select * from user_table) t)" />
<column name="name" value="someName"/>
</insert>
But there are thousands of entries like this. Is there any generic way of doing it so that I don't have to change the script at so many places. Thanks.
The easiest approach would be to just update the XML, either with an simple XML parser program or even a regexp search and replace in your text editor.
Alternately, you can override the standard liquibase logic to look for that particular valueComputed pattern and replace it. There are a couple points you could make the change at:
Override the liquibase.parser.core.xml.XMLCHangeLogSAXParser class, probably the parseToNode() method to search through the generated ParsedNode for valueComputed nodes
Override the liquibase.change.core.InsertDataChange class generateStatements() method or addColumn() method to replace valueComputed fields.
See http://liquibase.org/extensions for more info on writing extensions.
If I use a mysql database with utf-8 and InnoDB but liquibase causes an error because row size of a table is too large.
I have a released changeset which causes the error because it exceeds the row size.
<changeSet author="author" id="id">
<addColumn tableName="TABLE">
<column name="COLUMN" type="VARCHAR(5000)" /> <!-- Row size is too long -->
</addColumn>
</changeSet>
The solution for the problem with the row size is to change "VARCHAR(5000)" into "TEXT", but the problem is that this changeset has been executed on some systems so i cannot just change it here.
Is there any solution to tell liquibase that it should replace the changeset by another if not yet executed?
OK found a solution for this. It is possible to use validCheckSum to tell Liquibase that this changeset has been modified.
I just modified the changeset and added the tag like this.
<changeSet author="author" id="id">
<validCheckSum>oldChecksum</validCheckSum>
<validCheckSum>newChecksum</validCheckSum>
<addColumn tableName="TABLE">
<column name="COLUMN" type="TEXT" />
</addColumn>
</changeSet>
After that liquibase accepted the change set as executed so it was possible to make a new changeset which modifies the column.
<changeSet author="author" id="anotherId">
<modifyDataType tableName="TABLE" columnName="COLUMN" newDataType="TEXT" />
</changeSet>
I need to create a non-clustered index with INCLUDE columns (see the <create> tag below). Here's the mapping file:
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" namespace="MyApp" assembly="MyApp">
<class name="User" table="user" >
<id name="Id" type="Guid" column="user_id">
<generator class="guid.comb"/>
</id>
<property name="Name" column="name" not-null="true" />
<property name="Phone" column="phone" />
<property name="Zipcode" column="zipcode" />
</class>
<database-object>
<create>
CREATE NONCLUSTERED INDEX [IX_user_zipcode_id]
ON User (Zipcode)
INCLUDE (Name, Phone)
</create>
<drop>
DROP INDEX IX_user_zipcode_id
</drop>
<dialect-scope name="NHibernate.Dialect.MsSql2000Dialect"/>
<dialect-scope name="NHibernate.Dialect.MsSql2005Dialect"/>
<dialect-scope name="NHibernate.Dialect.MsSql2008Dialect"/>
</database-object>
</hibernate-mapping>
The problem I'm having is the index is not created at all. Nothing appears to be happening. This is my first time using <database-object> so I may be doing something wrong here.
I'm guessing INCLUDE is Sql Server specific which is why the dialect-scope is there. I know how to create a single and multi-column index, but this is not what I want. I want a single column index on zipcode and all other columns in the User table part of the INCLUDE clause of the query. Is there any way to create this type of index using the mapping file or some other way?
This is probably a long shot, but it would be nice to not have to specify every column but the indexed one in the INCLUDE part of the query... Instead to just let nhibernate add any new columns to the index that are added as properties to the mapping file.
So part of the problem was indeed my lack of understanding the database-object tag due mostly to poor documentation. From what I've gathered, the <create> and <drop> tags are only used when using SchemaExport like so:
Dim schemaExport As SchemaExport = New SchemaExport(NhibernateConfiguration)
schemaExport.Execute(False, True, False)
My app doesn't create the schema using that class. Instead it uses SchemaUpdate so the schema isn't blown away every time (the database may already exist on the users machine):
Dim schemaUpdate As SchemaUpdate = New SchemaUpdate(NhibernateConfiguration)
schemaUpdate.Execute(False, True)
That was the problem. The next logical question to ask is then how do you execute sql using SchemaUpdate. The answer... you can't. See this post: https://forum.hibernate.org/viewtopic.php?f=6&t=969584&view=next
Alas I am left to use raw sql. Maybe some day they will add an <update> tag.