I have a new wampserver installation on a windows 10 machine and I am trying to import a previously existing database containing around 10 different tables into a new database via phpmyadmin.
When I run a 'create table' command for a specific table in the sql query editor I receive the following error:
#1071 - Specified key was too long; max key length is 1000 bytes
This is the command that I am running:
DROP TABLE IF EXISTS `subscription_items`;
CREATE TABLE IF NOT EXISTS `subscription_items` (
`id` bigint(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`subscription_id` bigint(20) UNSIGNED NOT NULL,
`stripe_id` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`stripe_plan` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`quantity` int(11) DEFAULT NULL,
`created_at` timestamp NULL DEFAULT NULL,
`updated_at` timestamp NULL DEFAULT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `subscription_items_subscription_id_stripe_plan_unique` (`subscription_id`,`stripe_plan`) USING HASH,
KEY `subscription_items_stripe_id_index` (`stripe_id`(250))
) ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
Any ideas as to which setting could be changed to enable this error to disappear? I have tried looking in the my.ini file but cannot spot the setting to alter. Also this database worked in a previous older installation of wamp.
No setting can change this limit of MyISAM tables.
https://dev.mysql.com/doc/refman/8.0/en/myisam-storage-engine.html says:
The maximum key length is 1000 bytes. This can also be changed by changing the source and recompiling.
Your indexes columns are 8 bytes for the bigint, plus 255 * 4 bytes per character for the string. For indexing purposes, it has to assume you might use the full length of the utf8mb4 string in some row, even if you haven't done so yet. That totals 1028 bytes.
You could fix this by using varchar(248) or shorter.
You can also fix this by using InnoDB instead of MyISAM. InnoDB supports up to 3072 bytes for an index length in recent versions of MySQL.
I recommend using InnoDB instead of MyISAM regardless.
Executed: alter table device_msg convert to character set 'utf8' COLLATE 'utf8_unicode_ci';"
As my expect，table data size change to smaller.
But at the same time, table index size change to bigger ?
What happen and why ?
ps: table data size and index size are calculated by information_schema.TABLES
DbEngine: InnoDB
Table Before:
CREATE TABLE `device_msg` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`sn` varchar(30) COLLATE utf8_unicode_ci NOT NULL,
`time` datetime(3) NOT NULL,
`msg` json NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `device_UNIQUE` (`sn`,`time`)
) ENGINE=InnoDB AUTO_INCREMENT=62077733 DEFAULT CHARSET=utf8mb4;
Table After:
CREATE TABLE `device_msg` (
`id` bigint(20) NOT NULL AUTO_INCREMENT,
`sn` varchar(30) COLLATE utf8_unicode_ci NOT NULL,
`time` datetime(3) NOT NULL,
`msg` json NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `device_UNIQUE` (`sn`,`time`)
) ENGINE=InnoDB AUTO_INCREMENT=62077733 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
Before:
totalSize: 2.14 GB
indexSize: 282.98 MB
dataSize: 1.86 GB
avg_row_len: 297B
After
totalSize: 1.93 GB
indexSize: 413.97 MB
dataSize: 1.52 GB
avg_row_len: 260B
If data of information_schema.TABLES is not accurate,
How to make it right ?
The space taken by utf8mb4, then utf8 (assuming there were no 4-byte characters beforehand) is the same, in spite of the numbers you show.
This ALTER required rebuilding the table and the indexes.
InnoDB structures the data and each secondary index in a BTrees.
Depending on the order by which you insert elements into a BTree, more or fewer "block splits" will occur.
So, You can't really say whether it is the character set change or the rebuild that lead to the index getting bigger and the data getting smaller.
I say it was not the charset change.
Just in my opinion
As I read on MySQL document about the limitation.
https://dev.mysql.com/doc/refman/5.6/en/innodb-restrictions.html
By default, the index key prefix length limit is 767 bytes
if the index column exceeds this size, it will be truncated.
I assume your indexed column value has 255 characters.
in the case of utf8mb4, 1 character = 4 bytes, the limit is around 191 characters.
So 191 characters will be added to index, other (255-191=64) characters will be truncated from the index.
When you change encoding to utf8 (at that time 1 character = 3 bytes), the indexed limit will become around 255 characters.
It means your column value, all 255 characters, will be added to index without truncating.
The characters that are added to the index increased from 191 characters to 255 characters, so the index size was also increased.
I know questions with this title have been answered before, but please do read on. I've read thoroughly all the other questions/answers on this error before posting.
I am getting the above error for the following query:
CREATE TABLE IF NOT EXISTS `pds_core_menu_items` (
`menu_id` varchar(32) NOT NULL,
`parent_menu_id` int(32) unsigned DEFAULT NULL,
`menu_name` varchar(255) DEFAULT NULL,
`menu_link` varchar(255) DEFAULT NULL,
`plugin` varchar(255) DEFAULT NULL,
`menu_type` int(1) DEFAULT NULL,
`extend` varchar(255) DEFAULT NULL,
`new_window` int(1) DEFAULT NULL,
`rank` int(100) DEFAULT NULL,
`hide` int(1) DEFAULT NULL,
`template_id` int(32) unsigned DEFAULT NULL,
`alias` varchar(255) DEFAULT NULL,
`layout` varchar(255) DEFAULT NULL,
PRIMARY KEY (`menu_id`),
KEY `index` (`parent_menu_id`,`menu_link`,`plugin`,`alias`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
Does anyone have idea why and how to fix it?
The catch is - this same query works perfectly on my local machine, and worked as well on my previous host. Btw.it's from a mature project - phpdevshell - so I'd guess these guys know what they are doing, although you never know.
Any clue appreciated.
I'm using phpMyAdmin.
As #Devart says, the total length of your index is too long.
The short answer is that you shouldn't be indexing such long VARCHAR columns anyway, because the index will be very bulky and inefficient.
The best practice is to use prefix indexes so you're only indexing a left substring of the data. Most of your data will be a lot shorter than 255 characters anyway.
You can declare a prefix length per column as you define the index. For example:
...
KEY `index` (`parent_menu_id`,`menu_link`(50),`plugin`(50),`alias`(50))
...
But what's the best prefix length for a given column? Here's a method to find out:
SELECT
ROUND(SUM(LENGTH(`menu_link`)<10)*100/COUNT(`menu_link`),2) AS pct_length_10,
ROUND(SUM(LENGTH(`menu_link`)<20)*100/COUNT(`menu_link`),2) AS pct_length_20,
ROUND(SUM(LENGTH(`menu_link`)<50)*100/COUNT(`menu_link`),2) AS pct_length_50,
ROUND(SUM(LENGTH(`menu_link`)<100)*100/COUNT(`menu_link`),2) AS pct_length_100
FROM `pds_core_menu_items`;
It tells you the proportion of rows that have no more than a given string length in the menu_link column. You might see output like this:
+---------------+---------------+---------------+----------------+
| pct_length_10 | pct_length_20 | pct_length_50 | pct_length_100 |
+---------------+---------------+---------------+----------------+
| 21.78 | 80.20 | 100.00 | 100.00 |
+---------------+---------------+---------------+----------------+
This tells you that 80% of your strings are less than 20 characters, and all of your strings are less than 50 characters. So there's no need to index more than a prefix length of 50, and certainly no need to index the full length of 255 characters.
PS: The INT(1) and INT(32) data types indicates another misunderstanding about MySQL. The numeric argument has no effect related to storage or the range of values allowed for the column. INT is always 4 bytes, and it always allows values from -2147483648 to 2147483647. The numeric argument is about padding values during display, which has no effect unless you use the ZEROFILL option.
This error means that length of index index is more then 1000 bytes. MySQL and storage engines may have this restriction. I have got similar error on MySQL 5.5 - 'Specified key was too long; max key length is 3072 bytes' when ran this script:
CREATE TABLE IF NOT EXISTS test_table1 (
column1 varchar(500) NOT NULL,
column2 varchar(500) NOT NULL,
column3 varchar(500) NOT NULL,
column4 varchar(500) NOT NULL,
column5 varchar(500) NOT NULL,
column6 varchar(500) NOT NULL,
KEY `index` (column1, column2, column3, column4, column5, column6)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
UTF8 is multi-bytes, and key length is calculated in this way - 500 * 3 * 6 = 9000 bytes.
But note, next query works!
CREATE TABLE IF NOT EXISTS test_table1 (
column1 varchar(500) NOT NULL,
column2 varchar(500) NOT NULL,
column3 varchar(500) NOT NULL,
column4 varchar(500) NOT NULL,
column5 varchar(500) NOT NULL,
column6 varchar(500) NOT NULL,
KEY `index` (column1, column2, column3, column4, column5, column6)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
...because I used CHARSET=latin1, in this case key length is 500 * 6 = 3000 bytes.
I had this issue, and solved by following:
Cause
There is a known bug with MySQL related to MyISAM, the UTF8 character
set and indexes that you can check here.
Resolution
Make sure MySQL is configured with the InnoDB storage engine.
Change the storage engine used by default so that new tables will always be created appropriately:
set GLOBAL storage_engine='InnoDb';
For MySQL 5.6 and later, use the following:
SET GLOBAL default_storage_engine = 'InnoDB';
And finally make sure that you're following the instructions provided in Migrating to MySQL.
Reference
run this query before creating or altering table.
SET ##global.innodb_large_prefix = 1;
this will set max key length to 3072 bytes
I was facing the same issue; used the below query to resolve it.
While creating the DB you can use utf-8 encoding.
eg. create database my_db character set utf8 collate utf8mb4;
EDIT:
(Considering suggestions from comments)
Changed utf8_bin to utf8mb4
This index size limit seems to be larger on 64 bit builds of MySQL.
I was hitting this limitation trying to dump our dev database and load it on a local VMWare virt. Finally I realized that the remote dev server was 64 bit and I had created a 32 bit virt. I just created a 64 bit virt and I was able to load the database locally.
I have just made bypass this error by just changing the values of the "length" in the original database to the total of around "1000" by changing its structure, and then exporting the same, to the server. :)
if you are using Laravel 7 or Laravel 8,
goto to config/database.php
'engine' => 'innoDb',
that should work especially using Wamp or Xampp.
Well , I just changed from MyISAM to InnoDB like this
Before changing
ENGINE=MyISAM DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
After changing
ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
I had this error and I changed my tables column length smaller for the indexed foreign key columns so I changed it like this:
VARCHAR(1024)
To:
VARCHAR(512)
And run the query again.
If the sql is not created yourself, maybe just check and set the length of column to usual length.
I have gone through so many tricks, but none worked for me.
Then I found the best and simplest one:
If you are working with mysql-workbench, while altering index select (Engine:)-innoDb.
Getting the same exception while running back-end application in this case we can set out engine "InnoDb"
set GLOBAL storage_engine='InnoDb';
if the above configuration is not working for you then change your key length for example if it is by default selected 255 char you can change it below of 100
Example - create table role ( role_name varchar(99) not null, role_description varchar(255), primary key (role_name));
it is works for me
Whenever i fire this query from one of the mysql client (emma):
CREATE TABLE `tbl_mappings` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`private_id` int(11) unsigned NOT NULL,
`name` tinytext NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`private_id`,`name`(255)),
KEY `FK_tbl__private_integrations_mappings_tbl__private_integrations` (`private_id`),
CONSTRAINT `FK_tbl__private_integrations_mappings_tbl__private_integrations` FOREIGN KEY (`private_id`) REFERENCES `tbl__private_integrations` (`id`) ON DELETE CASCADE ON UPDATE CASCADE
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
i get error : specified key was too long max key length is 255 bytes
i am using mysql server 5.7,ubuntu 16.04
And i have tried adding configuration in my.cnf under [mysqld] :
innodb_file_format=barracuda
innodb_file_per_table=1
innodb_large_prefix=1
init_connect='SET collation_connection = utf8mb4_unicode_ci'
init_connect='SET NAMES utf8mb4'
character-set-server=utf8mb4
collation-server=utf8mb4_unicode_ci
default-storage-engine=InnoDB
And then restarted mysql service .still it wont work.
Any help is appreciated.
Thank you.
EDIT
Issue appears to be related to the TINYTEXT datatype. (I can replicate the observed behavior with MySQL version 5.7.17-0ubuntu0.16.04.1-log, using either InnoDB or MyISAM.)
The short answer (as a workaround, how to resolve the 1071 warning) is to use datatype VARCHAR(255) in place of TINYTEXT.
I ran several test cases with various character sets (utf8, utf8mb4, latin1) and using InnoDB and MyISAM storage engines. The 1071 warning appears to be related to the prefix length specified in the index on the TINYTEXT column... appears to be a MySQL limit on the prefix length (not specifically related to InnoDB, since I can replicate the behavior with MyISAM.) I did not test with any other TEXT types other than TINYTEXT.
PREVIOUS ANSWER
Index key length limit for InnoDB tables is 767 bytes.
The name(255) in the key definition is specifying the first 255 characters of name. With the MySQL utf8 characterset, a character can take from one to three bytes. And 255 times three is 765. Add in the four bytes for the int private_id, and that's 769, which exceeds the maximum.
That's why you are getting the error.
Several approaches to resolving that.
Easiest would be to reduce the number of characters of name that are included in the index, e.g.
UNIQUE KEY `name` (`private_id`,`name`(254))
If that doesn't satisfy your use case, then you might need to consider using the deprecated innodb_large_prefix setting. You would need to use DYNAMIC or COMPRESSED row format. See the discussions here:
https://dev.mysql.com/doc/refman/5.7/en/innodb-restrictions.html
https://dev.mysql.com/doc/refman/5.7/en/innodb-row-format-specification.html
There are 5 solutions here .
If you are hitting the limit because of trying to use CHARACTER SET utf8mb4. Then do one of the following (each has a drawback) to avoid the error:
⚈ Upgrade to 5.7.7 for 3072 byte limit -- your cloud may not provide this;
⚈ Change 255 to 191 on the VARCHAR -- you lose any keys longer than 191 characters (unlikely?);
⚈ ALTER .. CONVERT TO utf8 -- you lose Emoji and some of Chinese;
⚈ Use a "prefix" index -- you lose some of the performance benefits.
⚈ Stay with 5.6/5.5/10.1 but perform 4 steps to raise the limit to 3072 bytes:
SET GLOBAL innodb_file_format=Barracuda;
SET GLOBAL innodb_file_per_table=1;
SET GLOBAL innodb_large_prefix=1;
logout & login (to get the global values);
ALTER TABLE tbl ROW_FORMAT=DYNAMIC; (or COMPRESSED)
On second glance, don't use TINYTEXT, change to VARCHAR(255) which does not need the prefixing!
On third glance, UNIQUE(x, y(255)) is very likely to be wrong. It says "the combination of x and part of y is unique". It does not say x and all of y is unique.
Fourth... Which version of 5.7? Works fine with 5.7.15:
mysql> CREATE TABLE `tbl_mappings` (
-> `id` int(11) unsigned NOT NULL AUTO_INCREMENT,
-> `private_id` int(11) unsigned NOT NULL,
-> `name` tinytext NOT NULL,
-> PRIMARY KEY (`id`),
-> UNIQUE KEY `name` (`private_id`,`name`(255)),
-> KEY `private_id` (`private_id`)
-> ) ENGINE=InnoDB DEFAULT CHARSET=utf8
-> ;
Query OK, 0 rows affected (0.03 sec)
mysql> select ##version;
+-----------+
| ##version |
+-----------+
| 5.7.15 |
+-----------+
1 row in set (0.00 sec)
mysql> SHOW CREATE TABLE tbl_mappings\G
*************************** 1. row ***************************
Table: tbl_mappings
Create Table: CREATE TABLE `tbl_mappings` (
`id` int(11) unsigned NOT NULL AUTO_INCREMENT,
`private_id` int(11) unsigned NOT NULL,
`name` tinytext NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`private_id`,`name`(255)),
KEY `private_id` (`private_id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8
1 row in set (0.01 sec)
CREATE TABLE wp_locations (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`city` VARCHAR(255) NOT NULL,
`name` VARCHAR(255) NOT NULL,
CONSTRAINT `city_name` UNIQUE (`city`, `name`)
) DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
I got an sql error '#1071 - Specified key was too long; max key length is 767 bytes'
What am I doing wrong?
MySQL always reserves the max amount for a UTF8 field which is 4 bytes so with 255 + 255 with your DEFAULT CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci; you are over the 767 max key length limit.
You can only reduce the single varchar length or don't use a composite key.
I had the same exact issue. I've added these lines to a new conifg file in /etc/my.conf.d directory named umb4-support.cnf
[mysqld]
innodb_large_prefix=true
innodb_file_format=barracuda
innodb_file_per_table=true
After restarting the maria db service, my import scripts ran without this issue.
Godspeed!