how to make mysql database schema to be compatible with h2 database - mysql

I am currently using mysql as my database and use flyway to manage database schema. All my unit tests are running against mysql and they are running really slow with adding more unit tests. Now I want to change the database from mysql to h2 memory database in unit tests. Below is my setting for h2 db connection:
#Datasource
spring.datasource.url=jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE;DATABASE_TO_UPPER=true
spring.datasource.username=
spring.datasource.password=
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.default-transaction-isolation-level=1
When I run flywayMigrate, I got some sql errors. Below is one example, this sql is used to create a table on mysql but failed to run on h2.
CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8;
below is the error I got from h2. I don't know what wrong with my sql. Is there a way for h2 to accept mysql database schema?
Execution failed for task ':dbschema:flywayMigrate'.
> Error occurred while executing flywayMigrate
Migration V2016_02_26_12_59__create_file_storage.sql failed
-----------------------------------------------------------
SQL State : 42000
Error Code : 42000
Message : Syntax error in SQL statement "CREATE TABLE ""FILE_STORAGE"" (
""ID"" BIGINT(64) NOT NULL AUTO_INCREMENT,
""FILE_NAME"" VARCHAR(45) NULL,
PRIMARY KEY (""ID""))
DEFAULT CHARACTER[*] SET = UTF8 "; SQL statement:
CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8 [42000-190]
Location : db/migration/V2016_02_26_12_59__create_file_storage.sql (/Users/yzzhao/dev/cooltoo/cooltoo_backend/dbschema/build/resources/main/db/migration/V2016_02_26_12_59__create_file_storage.sql)
Line : 1
Statement : CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8
Syntax error in SQL statement "CREATE TABLE ""FILE_STORAGE"" (
""ID"" BIGINT(64) NOT NULL AUTO_INCREMENT,
""FILE_NAME"" VARCHAR(45) NULL,
PRIMARY KEY (""ID""))
DEFAULT CHARACTER[*] SET = UTF8 "; SQL statement:
CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8 [42000-190]
EDIT
I have hundreds of sql scripts which is running fine in mysql. So I don't want to change anything in these scripts. Is there a way to allow h2 accepts mysql script?

According to this description, you may try to use your H2 database in MySQL Compatibility Mode, by setting it in the connection string as MODE=MySQL. Here is exactly what is said about it:
To use the MySQL mode, use the database URL jdbc:h2:~/test;MODE=MySQL or the SQL statement SET MODE MySQL.
When inserting data, if a column is defined to be NOT NULL and NULL is inserted, then a 0 (or empty string, or the current timestamp for timestamp columns) value is used. Usually, this operation is not allowed and an exception is thrown.
Creating indexes in the CREATE TABLE statement is allowed using INDEX(..) or KEY(..). Example: create table test(id int primary key, name varchar(255), key idx_name(name));
Meta data calls return identifiers in lower case.
When converting a floating point number to an integer, the fractional digits are not truncated, but the value is rounded.
Concatenating NULL with another value results in the other value.
Text comparison in MySQL is case insensitive by default, while in H2 it is case sensitive (as in most other databases). H2 does support case insensitive text comparison, but it needs to be set separately, using SET IGNORECASE TRUE. This affects comparison using =, LIKE, REGEXP.

Your issue can be seen with your example
CREATE TABLE `file_storage`
(
'id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`)
)
DEFAULT CHARACTER SET = utf8;
The last line "DEFAULT CHARACTER SET = utf8" is setting a mySQL table option. H2 does not have such an option at either the table or schema level as it operates using Unicode at all times.
If you have a lot of SQL DDL statements that have been written over the years for MySQL you are likely to see a lot of such issues.

Related

How to set a default character set for a table in MySQL and MySQL Workbench?

I am forward engineering a MySQL database from an EER diagram in MySQL workbench, and am being shown the following error on the execution of the statement below:
ERROR: Error 1115: Unknown character set: 'DEFAULT'
CREATE TABLE IF NOT EXISTS `recipes_database_2`.`cleaned_string` (
`id` INT UNSIGNED NOT NULL AUTO_INCREMENT,
`string` VARCHAR(511) CHARACTER SET 'DEFAULT' NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `string_UNIQUE` (`string` ASC) VISIBLE)
ENGINE = InnoDB
AUTO_INCREMENT = 1
DEFAULT CHARACTER SET = DEFAULT
What can I change to fix this? I want the character set to be the one I set as the default for the database.
Just don't specify the CHARACTER SET at all, then the
database character set and collation are used
as one can read from the documentation.

MySQL: Expression of generated column contains disallowed function? CONCAT?

I have a table with a virtual generated column that concatenates five other columns (int and char) using CONCAT_WS(). This table contains 200-odd records and is never updated - it's just used as a lookup table. Recently, after months of untroubled processing, when I update records in a child table during which a SELECT is performed on this table, I sometimes see this error (ignore the "ITEM UPDATE FAILED" - that's me):
I am in development with a many changes every day, so it is impossible for me to determine if there is a correlating change. I have recently added "created" and "lastmodified" datetime fields to several tables with CURRENT_TIMESTAMP for DEFAULT or ON UPDATE, but not to this table.
Here's the table:
{EDIT} --- adding table definition:
CREATE TABLE `cpct_fixedfield` (
`id` int(11) UNSIGNED NOT NULL,
`name` varchar(256) NOT NULL,
`label` varchar(50) NOT NULL,
`field` int(11) NOT NULL,
`start` int(11) NOT NULL,
`rectype` int(11) NOT NULL ,
`mediatype` char(1) NOT NULL DEFAULT '' ,
`length` int(11) NOT NULL,
`userdefined` tinyint(1) NOT NULL,
`defaultval` varchar(5) NOT NULL,
`helpcode` varchar(10) NOT NULL,
`mandatory` varchar(2) NOT NULL ,
`idx` varchar(20) GENERATED ALWAYS AS (concat_ws('.',`field`,`rectype`,`mediatype`,`start`,`length`)) VIRTUAL NOT NULL)
ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;
The length of the data in field never exceeds 11chars. I can view the entire table in pma or Mysql Workbench and the virtual field materialises in all records without complaint, which suggests to me that there is nothing wrong with either the expression for the virtual column or the data in the columns that expression draws on.
The error occurs in several contexts when I am updating a child table. All the updates occur in Stored Procedures/Functions. One section of code that seems to trigger the error is this:
SET idxvar = CONCAT_WS(".", SUBSTRING(tmpfldkey,3,1), rectype, ptype, position, "%") COLLATE utf8mb4_general_ci;
SELECT id INTO ffid FROM cpct_fixedfield WHERE idx LIKE idxvar AND idx != "0.0..6.2";
All the variables involved are varchars or ints. utf8mb4_general_ci is used throughout the database.
I cannot find any reference in MYSQL documentation to CONCAT or CONCAT_WS being unsafe, and none of the columns referenced has a default using a non-deterministic function. All the other questions I can find in this forum and elsewhere about this error have arisen because of the use of non-deterministic functions like CURRENT_TIMESTAMP() in the virtual field, or a component of the field.
I replaced the SELECT on the table with a (large) CASE statement and all was well, and in fact, after I did this then reverted to the SELECT I had no errors for many hours. But it just happened again (so I'm back to the case statement).
I have run out of ideas - I'm hoping someone has some knowledge/experience that can help.
Thanks

mysql can not insert because no default value?

I have two tables with exactly the same schema. I can insert into one table but not another. The one that fails complains about no default value. Here's my create statement for the table
CREATE TABLE `t_product` (
`product_id` varchar(10) NOT NULL,
`prod_name` varchar(150) DEFAULT NULL,
`price` decimal(6,2) NOT NULL,
`prod_date` date NOT NULL,
`prod_meta` varchar(250) DEFAULT NULL,
`prod_key` varchar(250) DEFAULT NULL,
`prod_desc` varchar(150) DEFAULT NULL,
`prod_code` varchar(12) DEFAULT NULL,
`prod_price` decimal(6,2) NOT NULL,
`prod_on_promo` tinyint(1) unsigned NOT NULL,
`prod_promo_sdate` date DEFAULT NULL,
`prod_promo_edate` date DEFAULT NULL,
`prod_promo_price` decimal(6,2) NOT NULL,
`prod_discountable` tinyint(1) unsigned NOT NULL,
`prod_on_hold` tinyint(1) unsigned NOT NULL,
`prod_note` varchar(150) DEFAULT NULL,
`prod_alter` varchar(150) DEFAULT NULL,
`prod_extdesc` text,
`prod_img` varchar(5) NOT NULL,
`prod_min_qty` smallint(6) unsigned NOT NULL,
`prod_recent` tinyint(1) unsigned NOT NULL,
`prod_name_url` varchar(150) NOT NULL,
`upc_code` varchar(50) DEFAULT NULL,
PRIMARY KEY (`product_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
When I run this statement in database1, it successfully inserts:
insert into t_product (product_id) values ('jlaihello');
When I run this exact statement in database2, I get the error:
ERROR 1364 (HY000): Field 'price' doesn't have a default value
Why is this error happening only in database2? As far as I can tell, the difference between database1 and database2 are:
database1 uses mysql Ver 14.14 Distrib 5.5.53, for debian-linux-gnu (i686) using readline 6.3
and
database2 uses mysql Ver 14.14 Distrib 5.7.16, for Linux (x86_64) using EditLine wrapper
How do I make database2 behave like database1?
EDIT
There are hundreds of tables affected by this. Basically we're moving a database over to a new server. And I did a mysqldump from db1, and imported into db2. t_product is just ONE of the tables affected by this. I'd like to avoid manually modifying the schema for the hundreds of tables. I prefer a "simple switch" that will make db2 behave like db1.
ERROR 1364 (HY000): Field 'price' doesn't have a default value
price decimal(6,2) NOT NULL,
Set price to null or assign a default value
EDIT:
This is caused by the STRICT_TRANS_TABLES SQL mode.
Open phpmyadmin and goto More Tab and select Variables submenu. Scroll down to find sql mode. Edit sql mode and remove STRICT_TRANS_TABLES Save it.
OR
You can run an SQL query within your database management tool, such as phpMyAdmin:
-- verify that the mode was previously set:
SELECT ##GLOBAL.sql_mode;
-- update mode:
SET ##GLOBAL.sql_mode= 'YOUR_VALUE';
OR
Find the line that looks like so in the mysql conf file:
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
Comment above line out and restart mysql server
Most probably, the default for column price is missing in the second database. To check this you should output your table structure:
describe database2.t_product;
OR
show create table database2.t_product;
and check if the default is defined.
You can alter your table and add the missing default constraint like this:
ALTER TABLE database2.t_product MODIFY COLUMN decimal(6,2) NOT NULL DEFAULT 0
EDIT
Based on comments and specification (data type default values), I think there is a difference in sql_mode of the MySQL:
For data entry into a NOT NULL column that has no explicit DEFAULT
clause, if an INSERT or REPLACE statement includes no value for the
column, or an UPDATE statement sets the column to NULL, MySQL handles
the column according to the SQL mode in effect at the time:
If strict SQL mode is enabled, an error occurs for transactional
tables and the statement is rolled back. For nontransactional tables,
an error occurs, but if this happens for the second or subsequent row
of a multiple-row statement, the preceding rows will have been
inserted.
If strict mode is not enabled, MySQL sets the column to the implicit
default value for the column data type.
So, if strict mode is not enabled for the first database, INSERT/UPDATE is allowed and storing the default value of that type (a 0 decimal)

MYSQL allowing me to insert nulls in PK and FK

I have been testing a database i am doing right now and i am noticing that it is letting me insert null values into fields that are part of a primary key, despite stating in the script that the value of the field should be NOT NULL. I am using MAC's MySQL Workbench, and I have been googling around and can't figure out why this is happening. (Maybe I am too brain-fried right now... I am even starting to doubt myself)
Part of the script of the database creation (these are the tables I have tested..):
DROP DATABASE IF EXISTS solytierra ;
CREATE DATABASE IF NOT EXISTS solytierra DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci ;
USE solytierra ;
DROP TABLE IF EXISTS solytierra.Cliente ;
CREATE TABLE IF NOT EXISTS solytierra.Cliente (
CIF VARCHAR(25) NOT NULL,
Nombre VARCHAR(100) NULL,
EmailGeneral VARCHAR(45) NULL,
Web VARCHAR(45) NULL,
Notas VARCHAR(150) NULL,
insertado Timestamp,
CONSTRAINT pk_Cliente PRIMARY KEY (CIF)
) ENGINE=InnoDB;
DROP TABLE IF EXISTS solytierra.PersonaContacto ;
CREATE TABLE IF NOT EXISTS solytierra.PersonaContacto (
Cliente_CIF VARCHAR(25) NOT NULL,
Nombre VARCHAR(50) NOT NULL,
Apellidos VARCHAR(100) NOT NULL,
Notas VARCHAR(150) NULL,
CONSTRAINT pk_PersonaContacto PRIMARY KEY (Cliente_CIF , Nombre , Apellidos),
CONSTRAINT fk_PersonaContacto_Cliente FOREIGN KEY (Cliente_CIF)
REFERENCES solytierra.Cliente (CIF)
ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB;
...
It will let me create Clients without CIF, "PersonaContacto" without Cliente_CIF or without "Nombre"....
I have also tested other databases that i already had that used to work and it is happening the same in an all them.
Got it!!
I don't know what sql mode i was running on by default, but with this:
SET sql_mode = TRADITIONAL;
It is now running perfectly! I didn't know that there were different sql modes! Thanks a lot to everyone for your time and efforts! It really helped me to see that the problem was in my workbench, not the code and look for the answer accordingly! I hope this thread will be useful for future beginners like me!
If the value being stored in the column CIF is actually a NULL, then the expression LENGTH(CIF) should also return NULL. (If it's a zero length string, then LENGTH(CIF) will return 0.
To verify:
SELECT c.CIF, LENGTH(c.CIF) FROM solytierra.Cliente c ;
SELECT c.CIF FROM solytierra.Cliente c WHERE c.CIF IS NULL;
If you are running an INSERT statement, I can't explain the behavior you are observing, either MySQL allowing a NULL value to be stored or MySQL providing an implicit default value.)
If it's a zero length string being stored, that's the behavior we would expect if the columns were not explicitly declared to be NOT NULL but were later declared to part of the primary key. It's also the behavior we'd expect if the column were defined NOT NULL DEFAULT ''.
When the NOT NULL is omitted from the column declaration and the column is later declared to be part of the PRIMARY KEY, MySQL will use an an implicit default value based on the datatype of the column (zero length string for VARCHAR, zero for an integer, etc.)
But I'm not able to reproduce the problem you report, with the table definitions you've posted.
I recommend you check the table definition by getting the output from:
SHOW CREATE TABLE solytierra.Cliente;

Errors while migrating from MS SQL 2000 to MySQL 5.1

I am using migration toolkit for the migration but i am getting these errors in the process of migration
Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause
Incorrect string value: '\xEF\xBF\xBDs d...' for column 'MESSAGE' at row 5
0 row(s) transferred.
For the fixing the first error i got something here http://terrencemiao.com/Webmail/msg00949.html
but i am not getting the second error what it is and why is it there how to fix it also suggest me some better ideas for fixing the first one if there any apart from what mentioned in the link
USE `MyDB`
Creating tables ...
Creating table MyTable...
DROP TABLE IF EXISTS `MyTable`
Creating table MyTable ...
SET NAMES UTF8;
CREATE TABLE `MyTable` (
`PrimaryKey` INT(10) NOT NULL AUTO_INCREMENT,
`FK_QUESTION_ID` INT(10) NOT NULL,
`ANSWER` LONGTEXT CHARACTER SET utf8 COLLATE utf8_general_ci NULL
PRIMARY KEY (`PK_ID`)
)
ENGINE = INNODB
i am getting error for answer column
*Incorrect table definition; there can be only one TIMESTAMP column with CURRENT_TIMESTAMP in DEFAULT or ON UPDATE clause*
This is right, you should not create more then one such fields.
Incorrect string value: '\xEF\xBF\xBDs d...' for column 'MESSAGE' at row 5 0 row(s) transferred.
Possible encoding error, try to run 'SET NAMES UTF8;' before inserting data
Try this statement,
CREATE TABLE `MyTable` (
PK_ID INT(11) NOT NULL AUTO_INCREMENT,
FK_QUESTION_ID INT(11) NOT NULL,
ANSWER LONGTEXT CHARACTER SET utf8 COLLATE utf8_general_ci NULL,
PRIMARY KEY (`PK_ID`)
)
ENGINE = INNODB;
You missed a comma and it was wrong field name. Be careful with migration toolkit. Check generated field types, for example if you do not need 4GB text values, you could use simple VARCHAR instead of LONGTEXT.