I am trying to create an index on a particular text string using Hibernate annotations. What I am currently doing is:
#Index(name="guid_index")
#Column(length=1400)
private String myGUID;
However, since the length of the column is 1400, this maps to SQL using the longtext type. hbm2ddl is able to create the table without mishap, but when it tries to create the index, I get:
Unsuccessful: create index guid_index on table_name (myguid);
BLOB/TEXT column 'myguid' used in key specification without a key length
How can I create an index on this column using Hibernate?
I initially thought that HHH-4389 was what you're looking for but your case seems to be a different issue: Hibernate is simply not generating the expected DDL statement. I would open a new Jira issue.
You have to specify a length of index.
Maximum length depends on storage engine, but usually it's not more than 1000 bytes,
if charset is utf-8, you have to divide maximum lengh by 3.
create index guid_index on table_name (myguid (1000) );
It's good enough for WHERE field LIKE 'prefix%' and WHERE field = 'text' conditions.
Related
I've unsuccessfully been through the AWS forum and Stack Overflow trying to find a solution to the following error:
Index column size too large. The maximum column size is 767 bytes
I am running a WordPress website with 1.5M records in the postmeta table. I recently added an index to the postmeta table, and all was testing ok. However I had an incident with my server today (botnet scan dwindled my RDS credits), and restarted both my Lightsail instance and my RDS MySQL instance. After the restart I noticed that the site wasn't working properly and upon further investigation found the postmeta table was returning the error Index column size too large. The maximum column size is 767 bytes.
I'm running on MySQL 8.0.20
The table is:
Engine = InnoDB
Charset = utf8mb4
Collation = utf8mb4_0900_ai_ci
Row Format = Compact
Many existing "solutions" talk about recreating the table, however I need the data that's currently in the table.
Unfortunately this issue is present in my oldest AWS RDS Snapshot, so back ups don't appear to be an option.
Every time I try run an ALTER or SELECT statement, I get the same error, Index column size too large. The maximum column size is 767 bytes.
I've tried:
Changing the ROWFORMAT=DYNAMIC
Converting the charset and records to utf8
Changing the meta_value column from 255 to 191
Removing the custom index
Dumping the table
I can see that the default ROWFORMAT is now "DYNAMIC", however this table is still "COMPACT" from when it was running on MySQL 5.7
I've also tried updating the AWS RDS MySQL from 8.0.20 to 8.0.23, however the update fails cause it reports the table is corrupt in PrePatchCompatibility.log.
Ref: https://dba.stackexchange.com/questions/234822/mysql-error-seems-unfixable-index-column-size-too-large#answer-283266
There are some other suggestions about modifying the environment and file system, and running "innodb_force_recovery".
https://dba.stackexchange.com/questions/116730/corrupted-innodb-table-mysqlcheck-and-mysqldump-crash-server
However being an RDS instance, I don't have access to this lower level of the instance.
I suspect this issue is the column length and utf8mb4, however my main priority is getting the data from the currently in the table.
I also understand that changing the ROWFORMAT to DYNAMIC should fix this issue - however getting the same error.
Ref: http://mysql.rjweb.org/doc.php/limits#767_limit_in_innodb_indexes
I have also tried the "RDS Export to S3" option with no luck.
Please help, I'm lost as to what else to try.
I had, and solved, the same problem. Here's the situation.
In legacy MySQL table formats, the maximum size of an index on a VARCHAR or blob column is 767 bytes (not characters). These wp_somethingmeta WordPress tables have key columns (like meta_key) with the VARCHAR(255) datatype. When utf8mb4 is the character set each character can take up to four of those 767 bytes. that means indexes have to be defined as prefix. meta_key(191).
What makes a MySQL table into a legacy table?
MyISAM access method, or
An old version (5.5, early 5.6) of MySQL which only supports the older InnoDB Antelope ondisk table format and not the newer Barracuda file format, or
InnoDB and the ROW_FORMAT is COMPACT (or REDUNDANT).
So, to get away from prefix indexes on the varchar(255) columns, the table needs to be InnoDB and use the DYNAMIC (or COMPRESSED) ROW_FORMAT.
There's no need to rebuild a legacy table from scratch. You can convert it by saying
ALTER TABLE whatever ENGINE=InnoDB, ROW_FORMAT=DYNAMIC;
Then you stop having the prefix-key (191) issue.
Back up your database before you do this kind of thing. You knew that.
And, upgrade to a recent version of MySQL or MariaDB. Seriously. MySQL 5.6 long-term support ended on 1-February-2021, and the newer versions are better. (GoDaddy! I'm looking at you.)
WordPress' wp_postmeta table normally has an index on its meta_key column, which is varchar(255). That's too long.
First, drop the index that is too large.
SHOW CREATE TABLE wp_postmeta; -- to verify the name of the index
ALTER TABLE wp_postmeta DROP KEY meta_key;
I'm assuming the name of the index will be meta_key, which is the default name for an index on that column. But double-check the index name to be sure.
Then, add the index back, but make it a prefix index such that it's not larger than 767 bytes.
Since you're using utf8mb4, which allows multibyte characters up to 4 bytes per character, you can define the index with a prefix length of floor(767/4), or 191.
ALTER TABLE wp_postmeta ADD KEY (meta_key(191));
That index length will be permitted by the COMPACT row format, and it should be more than long enough to make the index just as useful as it was before. There's virtually no chance that you have a lot of meta key values that have the same leading characters and differ only after the 191th character.
Another alternative is to create a new table:
CREATE TABLE wp_postmeta_new (
meta_id BIGINT UNSIGNED,
post_id BIGINT UNSIGNED,
meta_key VARCHAR(255),
meta_value LONGTEXT,
KEY (post_id),
KEY (meta_key)
) ROW_FORMAT=DYNAMIC;
Double-check that it created this table with DYNAMIC rowformat.
Copy all the old data into it:
INSERT INTO wp_postmeta_new SELECT * from wp_postmeta;
Then swap the tables:
RENAME TABLE wp_postmeta TO wp_postmeta_old,
wp_postmeta_new TO wp_postmeta;
I'm assuming there are no new posts being created while you're doing this operation. You'd have to ensure no one is adding content to this WP instance so you don't miss any data in the process.
I am converting one of the "Varchar" column in mysql table to a text field. I am simply using single alter command to convert the type. When i read about text and varchar came to know that it had difference in storing mechanisms.
Am I supposed to write any migration script to change the type of column and move data or a single type altering command is enough ?
ALTER TABLE table_name MODIFY column_name TEXT NOT NULL;
There is no a big problem to change varchar to text because text supports more data length than varchar, but if the field has a index it must be drop and create new index with prefix col_name(length) (see CREATE INDEX syntax).
According to your data content maybe would be a good idea use fulltext indexes but that implies change your search expressions on that field.
If you are in production environment, the table will be locked meanwhile the migration is in progress to prevent data loss.
I recently noticed that there are several bigint type fields in one of my databases that could be replaced with type "int". The problem is that the database is already online and in use and also that there are foreign keys in action, so when I try changing the data type in the offline DB, mysql won't let me and produces the error message: "#1025 - Error on rename of (...)".
So how can I change these fields while keeping the foreign keys happy? (and not wiping out the existing DB!)
You can alter a column from BIGINT to INT as long as there is not any data in the column that would not fit into the smaller INT datatype.
ALTER TABLE mytable
MODIFY COLUMN mycolumn INT(11);
You could try:
Add new columns with proper types.
Copy values from old columns to new columns.
Rename columns.
Update constrains (maybe in two steps -- DROP and CREATE).
Drop old columns.
You can't reduce the size of field when it holds data. You can only expand them.
What you can do is export all the data, disable the keys, empty the tables, change the field sizes & import the data again.
I recently migrated a database from mysql to pgsql 9.
But now when I try to create a new object (in django admin) it tells me that the id I'm trying to use (started at one and has increased each time I tried) is already used.
I'm guessing that there is a pointer or index which needs to be set to the last used id. Am I correct?
When you define your table, the PostgreSQL equivalent to 'auto_increment' is:
CREATE TABLE foo (
id SERIAL,
...
);
If your table is already created (as I suspect it is), you can add this manually:
CREATE SEQUENCE foo_id_seq;
ALTER TABLE foo ALTER COLUMN id SET DEFAULT nextval('foo_id_seq');
Note that if you want to stick with the default name that Pg would have given you use the following format for your sequence name:
<table name>_<column name>_seq
Thus in my example, foo_id_seq.
If the table was migrated and it uses serial to replace the mysql auto increment column, your data was probably migrated without incrementing the serial sequence. Look up the postgresql setval function to set your sequence to a value above the highest existing key in your table.
When we add a new table, we made a mistake : a field was a varchar(255), but we have to write join queries.
So we alter table to make a char(6).
My question is : what does MySQL in this case ?
It trims right ?
Rather than worry about what MySQL does or doesn't do why not just convert the data yourself before the change.
e.g.
UPDATE YourTable
SET YourField = LEFT(YourField,6);
ALTER TABLE YourTable MODIFY COLUMN YourField Char(6)
You should note that if your column data is too long it won't let you do the alter assuming enable strict SQL mode see Shef's complete answer
It will try to convert the format. In your case, the column is a string type, so it will truncate the value, if the strict SQL mode is not enabled.
When you change a data type using CHANGE or MODIFY, MySQL tries to
convert existing column values to the new type as well as possible.
Warning This conversion may result in alteration of data. For example,
if you shorten a string column, values may be truncated. To prevent
the operation from succeeding if conversions to the new data type
would result in loss of data, enable strict SQL mode before using
ALTER TABLE (see Section 5.1.6, “Server SQL Modes”).
Read the fine warning down two fifth of the page.
MySQL won't even let you alter the table if there's any case you'll lose data (e.g. a row has an entry where the field's text is longer than six characters), so you're better off converting the data how you see fit.