MYSQL (Percona) Failed to read auto-increment value from storage engine - mysql

I am getting a "Failed to read auto-increment value from storage engine" error from MySQL (Percona), after trying to:
LOAD DATA LOCAL INFILE
I have been trying to find an answer on stackoverflow and google, but the answers did not solve the problem/error for my case.
The following shows Auto_increment as 1:
show table status like 'my_table_name';
Setting the auto increment value to the next value or to 1 did not solve it:
ALTER TABLE `my_table_name` AUTO_INCREMENT = 1;
Dropping the auto incrementing id column and recreating it did not solve it.
Dropping the table and recreating it did not solve it.
Dropping the database and recreating it did not solve it.
Destroying all data and rebuilding my docker development environment did not solve it.
Check table extended gave "OK":
CHECK TABLE distributor_users EXTENDED;
And it is the only auto incrementing value in the table.
This is what I use to create the table:
CREATE TABLE `my_table_name` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`x0` varchar(191) DEFAULT NULL,
`x1` varchar(4) DEFAULT NULL,
`x2` varchar(191) DEFAULT NULL,
`x3` varchar(191) DEFAULT NULL,
`x4` varchar(191) DEFAULT NULL,
`x5` varchar(191) DEFAULT NULL,
`x6` DATE DEFAULT NULL,
`x7` int(10) DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `x0` (`x0`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
What could it be? Any lead to start looking at is highly welcomed!

I solved it myself by adding to my query:
SET id = NULL
The new query now looks like this:
LOAD DATA LOCAL INFILE '$csv_file_location' INTO TABLE $destination_table
FIELDS TERMINATED BY ','
ENCLOSED BY '\"'
IGNORE 1 LINES
(x0,x1,x2,x3,x4,x5,x6,x7)
SET id = NULL;
Although my query worked previously without it.
MySQL was trying to place the first column in the csv file into the first column of the database. Setting the column that has the auto_increment to NULL solved it.

Related

how to make mysql database schema to be compatible with h2 database

I am currently using mysql as my database and use flyway to manage database schema. All my unit tests are running against mysql and they are running really slow with adding more unit tests. Now I want to change the database from mysql to h2 memory database in unit tests. Below is my setting for h2 db connection:
#Datasource
spring.datasource.url=jdbc:h2:mem:testDb;MVCC=TRUE;LOCK_TIMEOUT=10000;DB_CLOSE_ON_EXIT=FALSE;DATABASE_TO_UPPER=true
spring.datasource.username=
spring.datasource.password=
spring.datasource.driver-class-name=org.h2.Driver
spring.datasource.default-transaction-isolation-level=1
When I run flywayMigrate, I got some sql errors. Below is one example, this sql is used to create a table on mysql but failed to run on h2.
CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8;
below is the error I got from h2. I don't know what wrong with my sql. Is there a way for h2 to accept mysql database schema?
Execution failed for task ':dbschema:flywayMigrate'.
> Error occurred while executing flywayMigrate
Migration V2016_02_26_12_59__create_file_storage.sql failed
-----------------------------------------------------------
SQL State : 42000
Error Code : 42000
Message : Syntax error in SQL statement "CREATE TABLE ""FILE_STORAGE"" (
""ID"" BIGINT(64) NOT NULL AUTO_INCREMENT,
""FILE_NAME"" VARCHAR(45) NULL,
PRIMARY KEY (""ID""))
DEFAULT CHARACTER[*] SET = UTF8 "; SQL statement:
CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8 [42000-190]
Location : db/migration/V2016_02_26_12_59__create_file_storage.sql (/Users/yzzhao/dev/cooltoo/cooltoo_backend/dbschema/build/resources/main/db/migration/V2016_02_26_12_59__create_file_storage.sql)
Line : 1
Statement : CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8
Syntax error in SQL statement "CREATE TABLE ""FILE_STORAGE"" (
""ID"" BIGINT(64) NOT NULL AUTO_INCREMENT,
""FILE_NAME"" VARCHAR(45) NULL,
PRIMARY KEY (""ID""))
DEFAULT CHARACTER[*] SET = UTF8 "; SQL statement:
CREATE TABLE `file_storage` (
`id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`))
DEFAULT CHARACTER SET = utf8 [42000-190]
EDIT
I have hundreds of sql scripts which is running fine in mysql. So I don't want to change anything in these scripts. Is there a way to allow h2 accepts mysql script?
According to this description, you may try to use your H2 database in MySQL Compatibility Mode, by setting it in the connection string as MODE=MySQL. Here is exactly what is said about it:
To use the MySQL mode, use the database URL jdbc:h2:~/test;MODE=MySQL or the SQL statement SET MODE MySQL.
When inserting data, if a column is defined to be NOT NULL and NULL is inserted, then a 0 (or empty string, or the current timestamp for timestamp columns) value is used. Usually, this operation is not allowed and an exception is thrown.
Creating indexes in the CREATE TABLE statement is allowed using INDEX(..) or KEY(..). Example: create table test(id int primary key, name varchar(255), key idx_name(name));
Meta data calls return identifiers in lower case.
When converting a floating point number to an integer, the fractional digits are not truncated, but the value is rounded.
Concatenating NULL with another value results in the other value.
Text comparison in MySQL is case insensitive by default, while in H2 it is case sensitive (as in most other databases). H2 does support case insensitive text comparison, but it needs to be set separately, using SET IGNORECASE TRUE. This affects comparison using =, LIKE, REGEXP.
Your issue can be seen with your example
CREATE TABLE `file_storage`
(
'id` BIGINT(64) NOT NULL AUTO_INCREMENT,
`file_name` VARCHAR(45) NULL,
PRIMARY KEY (`id`)
)
DEFAULT CHARACTER SET = utf8;
The last line "DEFAULT CHARACTER SET = utf8" is setting a mySQL table option. H2 does not have such an option at either the table or schema level as it operates using Unicode at all times.
If you have a lot of SQL DDL statements that have been written over the years for MySQL you are likely to see a lot of such issues.

Issue Converting varchar to nvarchar mysql

Sorry if this is an easy question, I am coming to MySQL from SQL Server.
When I execute my create statement it contains nvarchar but commits to the database as varchar. Even in my alter statement afterwards the column does not change at all. Does the collation or DB engine make a difference?
During execution I am not encountering any issues in results, other than the fact the column changes datatype. I attached a screencast of my activity http://screencast.com/t/wc94oei2
I have not been able to find anyone with similar issues through my Google searches
Did you mean, this..
CREATE TABLE stars (
idstars int(11) NOT NULL AUTO_INCREMENT,
Name nvarchar(200) DEFAULT NULL,
PRIMARY KEY (idstars),
UNIQUE KEY Name_UNIQUE (Name)
)
----turns to---
CREATE TABLE stars (
idstars int(11) NOT NULL AUTO_INCREMENT,
Name varchar(200) DEFAULT NULL,
PRIMARY KEY (idstars),
UNIQUE KEY Name_UNIQUE (Name)
)

Best way to add column to a big table with longblob

I have the following table:
CREATE TABLE IF NOT EXISTS PDF_STORAGE (
ID_PDF_STORAGE bigint(20) NOT NULL AUTO_INCREMENT,
DESC_FILE varchar(255) DEFAULT NULL,
PDF_FILE longblob,
LINK_FILE varchar(255) DEFAULT NULL,
VERSION int(11) NOT NULL DEFAULT '0',
PRIMARY KEY (ID_PDF_STORAGE)
) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;
Where PDF_FILE is a file of 10 MB on average. Today it has about 50.000 rows. I need to add a new column to this table but it is taking a long time, more than 10 min, some times giving a 401 error in PhpMyAdmin, so I'd like to know what is the proper way to achieve this...
I already tried:
ALTER TABLE PDF_STORAGE ADD VERSION INT NOT NULL DEFAULT '0' AFTER LINK_FILE ;
and
SET FOREIGN_KEY_CHECKS=0;
CREATE TABLE PDF_STORAGE_new LIKE PDF_STORAGE;
ALTER TABLE PDF_STORAGE_new ADD VERSION INT NOT NULL DEFAULT '0' AFTER LINK_FILE ;
INSERT INTO PDF_STORAGE_new (PDF_STORAGE, DESC_FILE, ID_PDF_STORAGE, LINK_FILE) SELECT * FROM PDF_STORAGE;
RENAME TABLE PDF_STORAGE TO PDF_STORAGE_old, PDF_STORAGE_new TO PDF_STORAGE;
DROP TABLE PDF_STORAGE_old;
SET FOREIGN_KEY_CHECKS = 1;
but they are also slow.. is there a better way?
Thanks
What you are doing now ALTER TABLE is the best approach to my knowledge. You can try making this change when there is not much transaction (or) DB operation going on. I mean say, do the changes in idle time.
ALTER TABLE PDF_STORAGE ADD VERSION INT NOT NULL DEFAULT '0' AFTER LINK_FILE ;
You can as well create a new table same as this table schema along with the new column.
Insert all the records from this table to the newly created table.
Rename the new table to the old table name.
delete the old table.

MYSQL allowing me to insert nulls in PK and FK

I have been testing a database i am doing right now and i am noticing that it is letting me insert null values into fields that are part of a primary key, despite stating in the script that the value of the field should be NOT NULL. I am using MAC's MySQL Workbench, and I have been googling around and can't figure out why this is happening. (Maybe I am too brain-fried right now... I am even starting to doubt myself)
Part of the script of the database creation (these are the tables I have tested..):
DROP DATABASE IF EXISTS solytierra ;
CREATE DATABASE IF NOT EXISTS solytierra DEFAULT CHARACTER SET latin1 COLLATE latin1_swedish_ci ;
USE solytierra ;
DROP TABLE IF EXISTS solytierra.Cliente ;
CREATE TABLE IF NOT EXISTS solytierra.Cliente (
CIF VARCHAR(25) NOT NULL,
Nombre VARCHAR(100) NULL,
EmailGeneral VARCHAR(45) NULL,
Web VARCHAR(45) NULL,
Notas VARCHAR(150) NULL,
insertado Timestamp,
CONSTRAINT pk_Cliente PRIMARY KEY (CIF)
) ENGINE=InnoDB;
DROP TABLE IF EXISTS solytierra.PersonaContacto ;
CREATE TABLE IF NOT EXISTS solytierra.PersonaContacto (
Cliente_CIF VARCHAR(25) NOT NULL,
Nombre VARCHAR(50) NOT NULL,
Apellidos VARCHAR(100) NOT NULL,
Notas VARCHAR(150) NULL,
CONSTRAINT pk_PersonaContacto PRIMARY KEY (Cliente_CIF , Nombre , Apellidos),
CONSTRAINT fk_PersonaContacto_Cliente FOREIGN KEY (Cliente_CIF)
REFERENCES solytierra.Cliente (CIF)
ON DELETE NO ACTION ON UPDATE NO ACTION
) ENGINE=InnoDB;
...
It will let me create Clients without CIF, "PersonaContacto" without Cliente_CIF or without "Nombre"....
I have also tested other databases that i already had that used to work and it is happening the same in an all them.
Got it!!
I don't know what sql mode i was running on by default, but with this:
SET sql_mode = TRADITIONAL;
It is now running perfectly! I didn't know that there were different sql modes! Thanks a lot to everyone for your time and efforts! It really helped me to see that the problem was in my workbench, not the code and look for the answer accordingly! I hope this thread will be useful for future beginners like me!
If the value being stored in the column CIF is actually a NULL, then the expression LENGTH(CIF) should also return NULL. (If it's a zero length string, then LENGTH(CIF) will return 0.
To verify:
SELECT c.CIF, LENGTH(c.CIF) FROM solytierra.Cliente c ;
SELECT c.CIF FROM solytierra.Cliente c WHERE c.CIF IS NULL;
If you are running an INSERT statement, I can't explain the behavior you are observing, either MySQL allowing a NULL value to be stored or MySQL providing an implicit default value.)
If it's a zero length string being stored, that's the behavior we would expect if the columns were not explicitly declared to be NOT NULL but were later declared to part of the primary key. It's also the behavior we'd expect if the column were defined NOT NULL DEFAULT ''.
When the NOT NULL is omitted from the column declaration and the column is later declared to be part of the PRIMARY KEY, MySQL will use an an implicit default value based on the datatype of the column (zero length string for VARCHAR, zero for an integer, etc.)
But I'm not able to reproduce the problem you report, with the table definitions you've posted.
I recommend you check the table definition by getting the output from:
SHOW CREATE TABLE solytierra.Cliente;

import csv file with data to mysql

I'm trying to import csv file to MYSQL, and I have the following schema.
CREATE TABLE `monitor` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`time` time DEFAULT NULL,
`domain_name` text,
`cpu_ns` int(11) DEFAULT NULL,
`cpu_percentage` int(11) DEFAULT NULL,
`mem_bytes` int(11) DEFAULT NULL,
`mem_percentage` int(11) DEFAULT NULL,
`block_rdby` int(11) DEFAULT NULL,
`block_wrby` int(11) DEFAULT NULL,
`net_rxby` int(11) DEFAULT NULL,
`net_wrby` int(11) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
I'm having issues importing a file with data presented as follows.
16:48:27,2,s-1-VM,220000000.,0.448204684384,262144,0,0,0,60,0
16:48:30,2,s-1-VM,260000000.,0.528932926209,262144,0,0,16384,300,0
16:48:33,2,s-1-VM,300000000.,0.609786677944,262144,0,0,0,180,0
16:48:37,2,s-1-VM,290000000.,0.59000206364,262144,0,0,16384,120,0
16:48:40,2,s-1-VM,270000000.,0.54985661784,262144,0,0,0,649,425
16:48:43,2,s-1-VM,310000000.,0.631207212346,262144,0,0,0,180,0
16:48:46,2,s-1-VM,220000000.,0.44728232907,262144,0,0,20480,60,0
16:48:49,2,s-1-VM,200000000.,0.407008216196,262144,0,0,0,300,0
16:48:52,2,s-1-VM,250000000.,0.508946559213,262144,0,0,0,240,0
16:48:55,2,s-1-VM,240000000.,0.488674160215,262144,0,0,0,120,0
How can import this to my database?
I have tried the following and I get lots of warnings.
LOAD DATA LOCAL INFILE '/tmp/domain2.csv' INTO TABLE vtop FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n';
your help is highly appreciated.
Thank you
If I understand http://dev.mysql.com/doc/refman/5.1/de/load-data.html correctly, you should set ID to null (that makes it auto_increment) via
"SET id=NULL "
at the end of the statement. Otherwise column counts and column orders have to match perfectly.
But your columns don't match at all (what is the "2" at position 2?). So create a temp-Table with the structure of your CSV and then assign via insert into ... select the matching columns.
MySQL isn't an AI and can't figure out that 16:48:27 should go into the the time field - it'll be trying to stuff that into id instead.
You need to explictly map the columns in your CSV file to the fields they should go into in the table:
LOAD DATA .... (time, domain_name, x, y, z, foo, bar)