Error when importing virtual column from MySQL to MariaDB - mysql

I'm moving a database from MySQL to MariaDB, and testing export/import. One issue that's come up consistent is when a table has virtual columns. SHOW CREATE TABLE in MySQL returns this:
CREATE TABLE `table1` (
`colA` varchar(50) NOT NULL,
`colB' varchar(50) NOT NULL,
'vir1` GENERATED ALWAYS AS (concat_ws(' ', `colA`, `colB`)) VIRTUAL NOT NULL,
`colC` varchar(50) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
MariaDB then reports an error when importing it:
#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'NOT NULL,
`colC` varchar(50) DEFAULT NULL
The problem appears to be in the "VIRTUAL NOT NULL" part of the virtual column definition. If I edit the import sql file by hand, to this:
CREATE TABLE `table1` (
`colA` varchar(50) NOT NULL,
`colB' varchar(50) NOT NULL,
'vir1` GENERATED ALWAYS AS (concat_ws(' ', `colA`, `colB`)) VIRTUAL,
`colC` varchar(50) DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
then it imports just fine, so I guess MariaDB doesn't like being told whether a virtual field is allowed to be null or not (which seems logical, since it wouldn't know whether the inputs are null or not), but the exported MySQL file always has either VIRTUAL NULL or VIRTUAL NOT NULL as part of the exported table definitions.
Is there a way to avoid this? I could grep through the exported file to s&r those definitions, but that seems kludgey and at risk of running into other issues later if it's a compatibility issue with a known solution.

The syntax you show works in MySQL, I just tested with MySQL 5.7 and it does not cause an error.
The MariaDB syntax is not compatible. This has been reported as a bug: https://jira.mariadb.org/browse/MDEV-10964
You could vote for that bug, or even contribute a patch to resolve it.
The bottom line is that MariaDB forked from MySQL in 2010, and the two products have been growing further and further apart ever since then. They should no longer be considered compatible.
Just like if you were to migrate from a MySQL database to PostgreSQL or Microsoft SQL Server, there will be some edits needed to make MySQL syntax work on different brands of RDBMS.

The problem is that null / not null is a part of mysql's definition of generated columns, therefore mysqldump exports these properties as part of dumping the table structures. This is the right thing to do as mysqldump is designed to work with mysql and not with mariadb.
You should use a proper ETL tool for migrating data between different database products, even if those pruducts are as closely related to each other as mysql and mariadb are.

Related

Mysqldump fails on dumping virtual column

I have a largish (4GB) database, that I would like to dump, but when using the mysqldump tool (the MariaDB version, Ver 10.19 Distrib 10.4.21-MariaDB, for Linux (x86_64)), my dumping process has always failed at the same table, with the not so helpful error message:
mysqldump: Couldn't execute 'SHOW CREATE TABLE `AffiliateProgramsCampaigns`': Lost connection to MySQL server during query (2013)
I've tried to debug this error, but none of the obvious solutions worked for me, so I did a little experimenting, and found the culprit of my problem. The table in question, contains a VIRTUAL column, which strangely, if I remove, the dump finishes succesfully. I've digged a little more, but found no such error anywhere else relating to dumping MariaDB databases with virtual columns. Adding the --verbose option to the dump, is not helping either, as it gives me no other significant information.
As the query fails at the SHOW CREATE TABLE part, I've figured it has something to do with the structure of the CREATE TABLE query, but when I only try to dump the structure of this database, everything works like a charm. So I am stuck at the moment, trying to solve this issue. I could give up on the virtual column in this specific table, but if there would be any alternative, even a different dump tool, I would more likely go with that solution. Any advice, on how to fix this, or at least how to debug the problem more throughly would be appreciated!
Here are some other debug informations, that could be helpful:
This is the end of the --verbose dump output:
-- Retrieving view structure for table ActionLogReferences...
-- It's base table, skipped
-- Retrieving view structure for table ActionLogs...
-- It's base table, skipped
-- Retrieving view structure for table AffiliatePrograms...
-- It's base table, skipped
-- Retrieving view structure for table AffiliateProgramsCampaigns...
mysqldump: Couldn't execute 'SHOW CREATE TABLE `AffiliateProgramsCampaigns`': Lost connection to MySQL server during query (2013)
And here is the CREATE TABLE syntax for the table in question:
CREATE TABLE `AffiliateProgramsCampaigns` (
`AffiliateProgramsCampaignId` bigint(20) NOT NULL AUTO_INCREMENT,
`Name` varchar(255) NOT NULL,
`Description` tinytext NOT NULL,
`StartDate` datetime NOT NULL,
`EndDate` datetime NOT NULL,
`IsActivated` tinyint(1) NOT NULL DEFAULT 0 COMMENT 'This column shows if this campaign was manually activated.',
`Status` tinyint(4) GENERATED ALWAYS AS (if(`IsActivated`,if(curdate() between `StartDate` and `EndDate`,1,0),0)) VIRTUAL COMMENT 'The final, computed status of the campaign. When querying, you should use this to check the status.',
`affiliatePrograms_AffiliateProgramId` mediumint(9) NOT NULL,
`images_ImageId_BaseImage` bigint(20) DEFAULT NULL COMMENT 'The id of the base image.',
`images_ImageId_CoverImage` bigint(20) DEFAULT NULL COMMENT 'The id of the cover image.',
PRIMARY KEY (`AffiliateProgramsCampaignId`)
) ENGINE=InnoDB AUTO_INCREMENT=2 DEFAULT CHARSET=latin1
The query that is reported by mysqldump btw runs every single time I try it, both from phpymadmin and from the command line mysql interface. I also tried dumping with different users, even with the root user, but I always get the same error, at the same spot.
The problem was with the CURDATE() function that was used in the virtual column. By changing the function, to CURRENT_TIMESTAMP(), the issue is solved.
Also posted a bug report on the official boards: https://jira.mariadb.org/browse/MDEV-26619

Importing a database script into my database throws an #1064 error

could you please help me? I bought a domain just for learning databases etc. and I created my model of a database in MySQL Workbench. I generated a script and tried importing it into my database using phpMyAdmin. This is the script:
CREATE TABLE IF NOT EXISTS `knight` (
`idKnight` INT NOT NULL AUTO_INCREMENT,
`strength` INT NOT NULL DEFAULT 1,
`agility` INT NOT NULL DEFAULT 1,
`vitality` INT NOT NULL DEFAULT 1,
`attack` INT GENERATED ALWAYS AS (agility*strength) STORED,
`defense` INT GENERATED ALWAYS AS (vitality*strength) STORED,
`idUser` INT NOT NULL,
`idTavern` INT NULL,
PRIMARY KEY (`idKnight`),
INDEX `fk_user_idx` (`idUser` ASC),
INDEX `fk_tavern_idx` (`idTavern` ASC),
CONSTRAINT `fk_user` FOREIGN KEY (`idUser`)
REFERENCES `user` (`idUser`)
ON DELETE NO ACTION
ON UPDATE NO ACTION,
CONSTRAINT `fk_tavern` FOREIGN KEY (`idTavern`)
REFERENCES `tavern` (`idTavern`)
ON DELETE NO ACTION
ON UPDATE NO ACTION)
ENGINE = InnoDB
And this is the error:
#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'GENERATED ALWAYS AS (agility*strength) STORED,
defense INT GENERATED ALWAYS ' at line 6
Now the question is how do I synchronize the MySQL Workbench version of a database for which the script is generated, and the database itself. The database is Inno DB.
Thanks for your help
EDIT: MySQL version of my server is: 5.6.28
This type of problem shows the importance of using the same version of database in development as you will eventually use when you deploy to your production server. So you don't get surprised by incompatibilities.
You can run that script against your MySQL 5.6 server only if you avoid SQL features introduced in more recent versions of MySQL. This includes generated columns, which were first introduced in MySQL 5.7.
So you need to remove these columns, or else change them to plain INT columns, without the generated option.
`attack` INT GENERATED ALWAYS AS (agility*strength) STORED,
`defense` INT GENERATED ALWAYS AS (vitality*strength) STORED,
If you need those columns in query result sets, you have a few alternative solutions:
Add them as expressions in the select-list of a SELECT query:
SELECT (agility*strength) AS `attack`, (vitality*strength) AS `defense`
FROM `knight` ...
Or you could create a VIEW to encode a query with those expressions.
Or you could add those columns as plain integers, and write TRIGGERs on INSERT and UPDATE to keep them in sync with the other columns.
MySQL 5.6.28 was released in December 2015, and the whole 5.6 branch is past its end-of-support date. That means if any security bugs are discovered from now on, they won't be fixed. Besides, you're already using an outdated release of 5.6, with many bugs. The last 5.6 release was 5.6.51 in January 2021.

MySQL autoincrement column have diffrent behaviour with different databases

I have one working project in php and mysql.
In which I am using one column syntax for all my auto increment columns like below -
CREATE TABLE `mytable` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`sometext` int(11) NOT NULL
)
And for inserting records in this table in my whole proect I am using below syntax -
INSERT INTO mytable(ID,sometext)
VALUES(0,'Sometext')
And this is working fine.
But when I copied same DB and project and this code stopped working
So I changed my insert with below
INSERT INTO mytable( sometext)
VALUES( 'Sometext')
But this is very weird... In previous project old syntax is working fine but for new I have to make code change in 100 of places.
Can somebody tell me whats wrong with new MYSQL DB that it stopped supporting old syntax.
The difference is probably that your new database servers has the configuration option sql_mode=NO_AUTO_VALUE_ON_ZERO. Therefore only a NULL will cause an auto-increment to be generated.
Read https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html for full explanation of how sql modes affect your database server.
To avoid having to make code changes, you can change the server option.

mysqldump output is not a valid Oracle "create table" format

I'm trying to move data from a MySQL database (5.6.32-78) to an Oracle Database (11g). Using mysqldump, the output causes a "missing right parenthesis" error when creating a table in oracle. ie...
mysqldump output:
CREATE TABLE "table1" (
"ID" int(11) NOT NULL,
"column1" int(11) NOT NULL DEFAULT '0',
"column2" varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY ("ID")
);
Oracle is expecting the following (the order of NOT NULL and DEFAULT switched):
CREATE TABLE "table1" (
"ID" int(11) NOT NULL,
"column1" int(11) DEFAULT '0' NOT NULL,
"column2" varchar(255) DEFAULT '' NOT NULL,
PRIMARY KEY ("ID")
);
Is there an option I'm missing to correct this? I have a couple hundred tables to move and do not want to "reinvent the wheel" by writing procedure to get the correct output.
(--compatible=oracle does not make any difference).
Thanks.
Doug
I'll try to make the question more specific.
I am trying to migrate a MySQL to Oracle database and tried using mysqldump and various options, but it does not generate an Oracle usable output. I can't use Oracle's SQL Developer because it requires connecting to both the MySQL database (internet) and Oracle database (inside of a "no internet access" firewall) at the same time. Is anyone aware of a way to create an Oracle friendly export of a MySQL database?
The order of NOT NULL and DEFAULT is only one problem you'll face. There are many differences between MySQL and Oracle that cannot be fixed with mysqldump options.
http://dev.mysql.com/doc/refman/5.7/en/mysqldump.html#option_mysqldump_compatible says:
This option does not guarantee compatibility with other servers. It only enables those SQL mode values that are currently available for making dump output more compatible. For example, --compatible=oracle does not map data types to Oracle types or use Oracle comment syntax.
So you would have to do some hand-editing of your dump file before importing it to Oracle.
You're better off using Oracle SQL Developer to migrate your MySQL database to Oracle. You can find a step-by-step guide with videos here: http://www.oracle.com/technetwork/database/migration/index.html
Note the steps for MySQL can be found by clicking the link "and others..." below the list of other commercial RDBMS products. Here's a direct link: http://www.oracle.com/technetwork/database/migration/mysql-093223.html

MySQL BIT Data Type confusion

I have developed a database system utilizing MySQL to house some testing data.
CREATE TABLE testtable (
TEST_IDX int(11) NOT NULL AUTO_INCREMENT,
PASS_FLAG bit(1) NOT NULL,
RESULT_STRING varchar(500) NOT NULL,
TEST_DATE timestamp NULL DEFAULT NULL,
LAST_MODDATE timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
TESTED_BY varchar(45) NOT NULL,
PRIMARY KEY (TEST_IDX) )
ENGINE=InnoDB AUTO_INCREMENT=31 DEFAULT CHARSET=latin1;
One of the fields used is a flag to indicate the Pass/Fail Status of a test set. On my development machine I used the BIT data type, developed the database interaction code and tested the system successfully. I have a second development laptop that I used to perform bug fixes and such when deployment time came where the system also worked properly.
When I went to deploy the system on a production machine I set up MySQL and imported the database from a dump made off of the laptop. When the program, which had run successfully on both of my development machines, attempted to execute the error "data too long for column" was generated causing my inserts to fail. This doesn't make sense to me unless mysql has a setting that makes the bit/tinyint/int(1) behave in odd ways from install to install. I was able to make this function properly by simply setting the field to an INT (INT(11) it think) but I should not have had to do this and I would like to know why this happened. Perhaps someone could clarify how BIT data types work in MYSQL.