I have a liquibase xml script. When I run it on Postgres I don't face any problem but when I run it for MYSQL it gives error when the structure is of the following type:-
<insert tableName="user_table">
<column name="id" valueComputed="(select max(id)+1 from user_table)"/>
<column name="name" value="someName"/>
</insert>
When the above script is executed for MYSQL it gives error:-
You can't specify target table 'user_table' for update in FROM clause.
I found a solution to this by using alias like this :-
<insert tableName="user_table">
<column name="id" valueComputed="(select max(id)+1 from (Select * from user_table) t)" />
<column name="name" value="someName"/>
</insert>
But there are thousands of entries like this. Is there any generic way of doing it so that I don't have to change the script at so many places. Thanks.
The easiest approach would be to just update the XML, either with an simple XML parser program or even a regexp search and replace in your text editor.
Alternately, you can override the standard liquibase logic to look for that particular valueComputed pattern and replace it. There are a couple points you could make the change at:
Override the liquibase.parser.core.xml.XMLCHangeLogSAXParser class, probably the parseToNode() method to search through the generated ParsedNode for valueComputed nodes
Override the liquibase.change.core.InsertDataChange class generateStatements() method or addColumn() method to replace valueComputed fields.
See http://liquibase.org/extensions for more info on writing extensions.
Related
I'm just getting into BIML and have written some Scripts to creat a few DTSX-Packages. In general the most things are working. But one thing makes me crazy.
I have an ODBC-Source (PostgreSQL). From there I'm getting data out of a table using an ODBC-Source. The table has a text-Column (Name of the column is "description"). I cast this column to varchar(4000) in the query in the ODBC-Source (I know that there will be truncation, but it's ok). If I do this manually in Visual Studio the Advanced Editor of the ODBC-Source is showing "Unicode string [DT_WSTR]" with a Length of 4000 both for the External and the Output-Column. So there everything is fine. But if I do the same things with BIML and generate the SSIS-Package the External-Column will still say "Unicode string [DT_WSTR]" with a Length of 4000, but the Output-Column is telling "Unicode text stream [DT_NTEXT]". So the mapping done by BIML differs from the Mapping done by SSIS (manually). This is causing two things (warnings):
A Warning that metadata has changed and should be synced
And a Warning that the Source uses LOB-Columns and is set to Row by Row-Fetch..
Both warnings are not cool. But the second one also causes a drasticaly degredation in Performance! If I set the cast to varchar(255) the Mapping is fine (External- and Output-Column is then "Unicode string [DT_WSTR]" with a Length of 255). But as soon as I go higher, like varchar(256) it's again treated as [DT_NTEXT] in the Output.
Is there anything I can do about this? I invested days in the Evaluation of BIML and find many things an increase in Quality of Life, but this issue is killing it. It defeats the purpose of BIML if I have to correct the Errors of BIML manually after every Build.
Does anyone know how I can solve this Issue? A correct automatic Mapping between External- and Output-Columns would be great, but at least the option to define the Mapping myself would be ok.
Any Help is appreciated!
Greetings
Marco
Edit As requested a Minimal Example for better understanding:
The column in the ODBC Source (Postegres) has the type "text" (Columnname: description)
I select it in a ODBC-Source with this Query (DirectInput):
SELECT description::varchar(4000) from mySourceTable
The ODBC-Source in Biml looks like this:
<OdbcSource Name="mySource" Connection="mySourceConnection"> <DirectInput>SELECT description::varchar(4000) from mySourceTable</DirectInput></OdbcSource>
If I now generate the dtsx-Package the ODBC-Source throws the above mentioned warnings with the above mentioned Datatypes for External and Output-Column
As mentioned in the comment before I got an answer from another direction:
You have to use DataflowOverrides in the ODBC-Source in BIML. For my example you have to do something like this:
`<OdbcSource Name="mySource" Connection="mySourceConnection">
<DirectInput>SELECT description::varchar(4000) from mySourceTable</DirectInput>
<DataflowOverrides>
<OutputPath OutputPathName="Output">
<Columns>
<Column ColumnName="description" SsisDataTypeOverride="DT_WSTR" DataType="String" Length="4000" />
</Columns>
</OutputPath>
<OutputPath OutputPathName="Error">
<Columns>
<Column ColumnName="description" SsisDataTypeOverride="DT_WSTR" DataType="String" Length="4000" />
</Columns>
</OutputPath>
</DataflowOverrides>
</OdbcSource>`
You won't have to do the Overrides for all columns, only for the ones you have mapping-Issues with.
Hope this solution can help anyone who passes by.
Cheers
I don't seem to be able to execute a type modification on a Mysql 5.7.10 instance (it works with H2 though).
Here are the changeset steps that are involved with the regarded field:
Creation:
<column name="last_modify_time" type="bigint">
<constraints nullable="false" />
</column>
Modification:
<modifyDataType tableName="USER" columnName="last_modify_time" newDataType="timestamp" />
the error msg in Mysql is
Invalid default value for 'last_modify_time' [Failed SQL: ALTER TABLE USER MODIFY last_modify_time timestamp]
Manually modifying the the request to the following works:
ALTER TABLE USER MODIFY last_modify_time timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP
I don't really understand why Mysql needs the default value. Maybe it's an edge case with the version 5.7.10 (using the default configuration options).
In any case Liquibase should be able to handle it.
I've tried to add/remove default value prior to the modifyDataType, without success.
when you are modifying type of a column using liquibase which is non nullable ... you have to first make it nullable and then modify it use safe modify
It turns out that an exactly identical modification on a field named create_time, placed right before, didn't cause any problem. Swapping the two changsets order did solve the problem.
<changeSet id="11" author="author">
<comment>rename last_modify_time to last_modified_date and change type to timestamp</comment>
<modifyDataType tableName="USER" columnName="last_modify_time" newDataType="timestamp" />
<renameColumn tableName="USER" oldColumnName="last_modify_time" newColumnName="last_modified_date" columnDataType="timestamp" />
</changeSet>
<changeSet id="12" author="author">
<comment>rename create_time to created_date and change type to timestamp</comment>
<modifyDataType tableName="USER" columnName="create_time" newDataType="timestamp" />
<renameColumn tableName="USER" oldColumnName="create_time" newColumnName="created_date" columnDataType="timestamp"/>
</changeSet>
I still can't explain what happend, I'm happy enough to get it working. If someone wants to reproduce the error I'd be happy to help.
I am working on a Joomla Module where images can be shown and be linked to a Joomla Article, Virtuemart product or an external link.
To have to option to select a Virtuemart Product, I added the following parameter to the module manifest xml:
<field
name="image_1_product"
type="sql"
default="10"
query="SELECT #__virtuemart_products.virtuemart_product_id, #__virtuemart_products_en_gb.product_name
FROM #__virtuemart_products, #__virtuemart_products_en_gb
WHERE #__virtuemart_products.virtuemart_product_id = #__virtuemart_products_en_gb.virtuemart_product_id
AND #__virtuemart_products.published = 1"
key_field="virtuemart_product_id"
value_field="product_name"
label="LABEL_PRODUCT"
description="DESC_PRODUCT">
</field>
This works just fine, when Virtuemart is installed. When Virtuemart is not installed, you get an error 'Unknown column' - which makes sense.
Because this is a Module Manifest Option, I can use only 1 query. Is there a way to run a query that checks if a table exists before it executes the select? Like:
IF EXISTS TABLE `table_a` (SELECT * FROM table_a)
If so, can it return a row like:
| id | value |
| 0 | Virtuemart not found |
Thank you!
I'm afraid its not possible with just single database query.
The only workaround that I can think of is to create a parameter helper for your module and use an addfieldpath to your module fieldset tag. With this solution you can write an method to do the query and create the output, then you can check if the table exists or not before creating your parameter.
for example:
<config>
<fields name="params">
<fieldset name="basic" addfieldpath="/modules/mod_mymodule/elements/">
<field name="image_1_product" type="product" default="10" label="LABEL_PRODUCT" description="DESC_PRODUCT"/>
</fieldset>
</fields>
</config>
now in /modules/mod_mymodule/elements/ create a PHP file named product.php and write your custom query and field descriptions as described in Joomla's documentation.
Joomla documentation is available for Custom parameter types and Creating custom template parameter types
I need to create a non-clustered index with INCLUDE columns (see the <create> tag below). Here's the mapping file:
<hibernate-mapping xmlns="urn:nhibernate-mapping-2.2" namespace="MyApp" assembly="MyApp">
<class name="User" table="user" >
<id name="Id" type="Guid" column="user_id">
<generator class="guid.comb"/>
</id>
<property name="Name" column="name" not-null="true" />
<property name="Phone" column="phone" />
<property name="Zipcode" column="zipcode" />
</class>
<database-object>
<create>
CREATE NONCLUSTERED INDEX [IX_user_zipcode_id]
ON User (Zipcode)
INCLUDE (Name, Phone)
</create>
<drop>
DROP INDEX IX_user_zipcode_id
</drop>
<dialect-scope name="NHibernate.Dialect.MsSql2000Dialect"/>
<dialect-scope name="NHibernate.Dialect.MsSql2005Dialect"/>
<dialect-scope name="NHibernate.Dialect.MsSql2008Dialect"/>
</database-object>
</hibernate-mapping>
The problem I'm having is the index is not created at all. Nothing appears to be happening. This is my first time using <database-object> so I may be doing something wrong here.
I'm guessing INCLUDE is Sql Server specific which is why the dialect-scope is there. I know how to create a single and multi-column index, but this is not what I want. I want a single column index on zipcode and all other columns in the User table part of the INCLUDE clause of the query. Is there any way to create this type of index using the mapping file or some other way?
This is probably a long shot, but it would be nice to not have to specify every column but the indexed one in the INCLUDE part of the query... Instead to just let nhibernate add any new columns to the index that are added as properties to the mapping file.
So part of the problem was indeed my lack of understanding the database-object tag due mostly to poor documentation. From what I've gathered, the <create> and <drop> tags are only used when using SchemaExport like so:
Dim schemaExport As SchemaExport = New SchemaExport(NhibernateConfiguration)
schemaExport.Execute(False, True, False)
My app doesn't create the schema using that class. Instead it uses SchemaUpdate so the schema isn't blown away every time (the database may already exist on the users machine):
Dim schemaUpdate As SchemaUpdate = New SchemaUpdate(NhibernateConfiguration)
schemaUpdate.Execute(False, True)
That was the problem. The next logical question to ask is then how do you execute sql using SchemaUpdate. The answer... you can't. See this post: https://forum.hibernate.org/viewtopic.php?f=6&t=969584&view=next
Alas I am left to use raw sql. Maybe some day they will add an <update> tag.
Using HBM files to map my types.
One of my classes uses bag of items called PartnerEnv. One of their fields is set to be the id which should be generated using increment. for some reason I am getting the following error:
could not fetch initial value for increment generator[SQL: SQL not available]
Inner details: "{"Invalid object name 'jj.dbo.Partners2Env'."}"
If I change the generation method to assigned everything is ok.
I will appreciate any help given!
Can you set your Id column on the PartnerEnv table (or whatever that table is called) to an Identity column and then use the following in the .hbm file for that class?
<id name="Id" type="Int32">
<column name="Id" />
<generator class="identity" />
</id>