My query is not working, it always show related 0. Even against data do exist.
SELECT number, MATCH(number)
AGAINST('02' '01' '03' WITH QUERY EXPANSION)
as related FROM lottery_entries;
you can see the result below.
i don't know what is the reason.
It is because MySQL has a server parameter - The minimum length of the word to be included in a FULLTEXT index. Default value for this parameter is 4 so your words like 'XX' are not included in this index. You should change this system parameter, restart server and then rebuild all FULLTEXT indexes.
REPAIR TABLE lottery_entries QUICK;
Change the full text index minimum word length with MySQL
Related
I've unsuccessfully been through the AWS forum and Stack Overflow trying to find a solution to the following error:
Index column size too large. The maximum column size is 767 bytes
I am running a WordPress website with 1.5M records in the postmeta table. I recently added an index to the postmeta table, and all was testing ok. However I had an incident with my server today (botnet scan dwindled my RDS credits), and restarted both my Lightsail instance and my RDS MySQL instance. After the restart I noticed that the site wasn't working properly and upon further investigation found the postmeta table was returning the error Index column size too large. The maximum column size is 767 bytes.
I'm running on MySQL 8.0.20
The table is:
Engine = InnoDB
Charset = utf8mb4
Collation = utf8mb4_0900_ai_ci
Row Format = Compact
Many existing "solutions" talk about recreating the table, however I need the data that's currently in the table.
Unfortunately this issue is present in my oldest AWS RDS Snapshot, so back ups don't appear to be an option.
Every time I try run an ALTER or SELECT statement, I get the same error, Index column size too large. The maximum column size is 767 bytes.
I've tried:
Changing the ROWFORMAT=DYNAMIC
Converting the charset and records to utf8
Changing the meta_value column from 255 to 191
Removing the custom index
Dumping the table
I can see that the default ROWFORMAT is now "DYNAMIC", however this table is still "COMPACT" from when it was running on MySQL 5.7
I've also tried updating the AWS RDS MySQL from 8.0.20 to 8.0.23, however the update fails cause it reports the table is corrupt in PrePatchCompatibility.log.
Ref: https://dba.stackexchange.com/questions/234822/mysql-error-seems-unfixable-index-column-size-too-large#answer-283266
There are some other suggestions about modifying the environment and file system, and running "innodb_force_recovery".
https://dba.stackexchange.com/questions/116730/corrupted-innodb-table-mysqlcheck-and-mysqldump-crash-server
However being an RDS instance, I don't have access to this lower level of the instance.
I suspect this issue is the column length and utf8mb4, however my main priority is getting the data from the currently in the table.
I also understand that changing the ROWFORMAT to DYNAMIC should fix this issue - however getting the same error.
Ref: http://mysql.rjweb.org/doc.php/limits#767_limit_in_innodb_indexes
I have also tried the "RDS Export to S3" option with no luck.
Please help, I'm lost as to what else to try.
I had, and solved, the same problem. Here's the situation.
In legacy MySQL table formats, the maximum size of an index on a VARCHAR or blob column is 767 bytes (not characters). These wp_somethingmeta WordPress tables have key columns (like meta_key) with the VARCHAR(255) datatype. When utf8mb4 is the character set each character can take up to four of those 767 bytes. that means indexes have to be defined as prefix. meta_key(191).
What makes a MySQL table into a legacy table?
MyISAM access method, or
An old version (5.5, early 5.6) of MySQL which only supports the older InnoDB Antelope ondisk table format and not the newer Barracuda file format, or
InnoDB and the ROW_FORMAT is COMPACT (or REDUNDANT).
So, to get away from prefix indexes on the varchar(255) columns, the table needs to be InnoDB and use the DYNAMIC (or COMPRESSED) ROW_FORMAT.
There's no need to rebuild a legacy table from scratch. You can convert it by saying
ALTER TABLE whatever ENGINE=InnoDB, ROW_FORMAT=DYNAMIC;
Then you stop having the prefix-key (191) issue.
Back up your database before you do this kind of thing. You knew that.
And, upgrade to a recent version of MySQL or MariaDB. Seriously. MySQL 5.6 long-term support ended on 1-February-2021, and the newer versions are better. (GoDaddy! I'm looking at you.)
WordPress' wp_postmeta table normally has an index on its meta_key column, which is varchar(255). That's too long.
First, drop the index that is too large.
SHOW CREATE TABLE wp_postmeta; -- to verify the name of the index
ALTER TABLE wp_postmeta DROP KEY meta_key;
I'm assuming the name of the index will be meta_key, which is the default name for an index on that column. But double-check the index name to be sure.
Then, add the index back, but make it a prefix index such that it's not larger than 767 bytes.
Since you're using utf8mb4, which allows multibyte characters up to 4 bytes per character, you can define the index with a prefix length of floor(767/4), or 191.
ALTER TABLE wp_postmeta ADD KEY (meta_key(191));
That index length will be permitted by the COMPACT row format, and it should be more than long enough to make the index just as useful as it was before. There's virtually no chance that you have a lot of meta key values that have the same leading characters and differ only after the 191th character.
Another alternative is to create a new table:
CREATE TABLE wp_postmeta_new (
meta_id BIGINT UNSIGNED,
post_id BIGINT UNSIGNED,
meta_key VARCHAR(255),
meta_value LONGTEXT,
KEY (post_id),
KEY (meta_key)
) ROW_FORMAT=DYNAMIC;
Double-check that it created this table with DYNAMIC rowformat.
Copy all the old data into it:
INSERT INTO wp_postmeta_new SELECT * from wp_postmeta;
Then swap the tables:
RENAME TABLE wp_postmeta TO wp_postmeta_old,
wp_postmeta_new TO wp_postmeta;
I'm assuming there are no new posts being created while you're doing this operation. You'd have to ensure no one is adding content to this WP instance so you don't miss any data in the process.
I'm using MySQL database to store student projects (every single project has its own unique ID, as the first screenshot shows).
The 'ID' column is set as auto_increment and PRIMARY KEY.
All projects must be ordered (only) by their ID ascending. But every time I insert a new project into my database and set lower value in the 'Year' field (lower value than I entered last time, when I was inserting my previous project), my projects become automatically ordered by this 'Year' field (as shown on the second screenshot).
Is there any way, how to set my table for automatic ordering all newly added projects only by the 'ID' column? Yeah, I know that I can change the ordering with ALTER TABLE tablename ORDER BY columnname ASC;after I place every new record, but can it be done automatically?
Thx to everyone who helps.
Q: Is there any way, how to set my table for automatic ordering all newly added projects only by the 'ID' column?
A: There is no "automatic ordering" in a MySQL table. (Some storage engines, such as InnoDB, are "index organized" structures, and do store rows in order by the cluster key.) But this organization does not define or specify the order of rows returned by a SELECT statement.
Without an ORDER BY clause on a SELECT statement, then the MySQL server can return rows in any order it chooses to. There is no guarantee of any "automatic" or "default" ordering.
When we run a SQL SELECT statement without an ORDER BY clause, we do observe that rows tend to be returned in a consistent order. This behavior isn't guaranteed, and it isn't "automatic".
This consistent "ordering" behavior we observe is due to the MySQL server performing a consistent set of operations, on a consistent set of data.
Performing an ALTER TABLE statement to rebuild the entire table is not the solution to the "automatic ordering" issue.
If you want the MySQL server to return rows in a specific order, then the solution is to add an ORDER BY clause to the SELECT statement.
The client that executes the SELECT statement is free to do whatever it wants with the rows it retrieves. The client can perform operations such as filtering, ordering, or whatever, to come up with what gets returned in the user interface.
Some clients (like the mysql command line) don't implement any functions for "filtering" or "ordering" rows, so the only way for the client to specify an order that rows should be returned in is the ORDER BY clause on statement itself. (The MySQL command line client returns rows in the user interface in the same order that they are retrieved.)
I expect that phpMyAdmin does the same thing, it displays the rows in the same order that they are returned from the MySQL server.
Order in query results should be determined by the ORDER clause. Don't rely on default order applied by phpmyadmin (or some other tool).
You have to distinct retrieving from inserting/updating.
About automatic order INTO phpmyadmin, maybe using bookmarks on queries : How can I set a default sort for tables in PHPMyAdmin (i.e. always "Primary key - Descending")
I had to run this command to alter a column in one of my table.
ALTER table XYZ modify value VARCHAR(1024);
When I run it, I get this error.
ERROR 1071 (42000) at line 1: Specified key was too long; max key length is 767 bytes
The original column size was 250. Here is the strange part. I have a 2nd MySQL server with identical configurations and it is doing 2 way replication with the 1st server. When I run this "alter table" command on my 2nd MySQL server, I didn't get this error. In fact, after the alter command finished running, this command was replicated to my primary mysql server without any error.
My two mysql servers are setup as active-active, but only one of them is being actively use. So to clarify, I got this "max key length" error on the actively used server. But it ran fine on the 2nd server that was not being use.
Does anyone have any idea why the command ran find on my 2nd server?
Thanks../Ken
You might need to adjust the index on this table.
DROP INDEX index_name ON tbl_name;
CREATE INDEX index_name ON tbl_name (col1,col2(100));
Where col2 is the value column from your example and 100 is the max characters to include in the index.
If you are unsure of the indexes on the table, try the following (replacing tbl_name with your table name).
SHOW INDEXES IN tbl_name;
I have also faced same issue when the encoding of the database is set to UTF8.
If you use latin1 then the largest column you can index is varchar(767), but if you use utf8 then the limit is varchar(255).
Try changing the value length to 255.
It might work.
Following link may guide you to the same.
http://mechanics.flite.com/blog/2014/07/29/using-innodb-large-prefix-to-avoid-error-1071/
I'm building a search for a site, which utilizes a fulltext search. The search itself works great, that's not my problem. I string together user provided keywords (MATCH... AGAINST...) with AND's so that multiple words further narrow the results. Now, I know that certain stop words aren't indexed, and that's fine with me I don't really want to use them as selection criteria. But, if a stopword is provided in the keyword set (by the user), it kills all the results (as expected) even if the word actually is in a certain text block.
My question: is there any way to check to see if a certain word is a stop word at the time of the query? My preferred solution would just be to exclude the relevant word from the search criteria (I don't care if a user can narrow results by the word 'neither', I just don't want MySQL to return an empty result set because the user provided it, even though neither does exist in the results). Or, am I just going to have to empty the stopword list? Thanks very much for any help.
edit ----
I'm sorry, but there's really no code snippets to provide for this one. The code works fine, actually exactly as expected. It's more of a logical problem I'm dealing with. But as an example, in the way of explanation:
lets say there are three records, which include the words (but are not limited to)
1: apple, orange, mango, banana
2: grape, orange, pineapple, mango
3: potato, mango, melon, keira knightly
If the search word entered by the user is mango, all results are returned correctly. If the words are orange AND mango, results 1 and 2 are returned (correctly). Now, let's say banana is a stop word (it's not... but let's assume it is), if the search is for orange, mango, AND banana, no results are returned (because banana isn't in the fulltext index).
What I'm looking for is if anyone else has encountered this problem, and has a way to work around it. Sort of an:
if 'banana' NOT STOP WORD match 'banana' against `words`. (OBVIOUSLY not real code).
Or... am I just going to have to drop the stopword list...
You can verify the keywords by comparing all stopwords. Here is the list of stopwords
I've found out a solution to disable stopwords from fulltext.
You just need to locate .cnf file and add this,
ft_stopword_file = ""
restart mysql engine and rebuild indexes;
Hope this work
How to disable fulltext stopwords in MySQL:
In my.ini text file (MySQL) :
ft_stopword_file = "" or link an empty file "empty_stopwords.txt"
ft_min_word_len = 2
// set your minimum length, but be aware that shorter words (3,2) will increase the query time dramatically, especially if the fulltext indexed column fields are large.
Save the file, restart the server.
The next step should be to repair the indexes with this query:
REPAIR TABLE tbl_name QUICK.
However, this will not work if you table is using InnoDB storage engine. You will have to change it to MyISAM :
ALTER TABLE t1 ENGINE = MyISAM;
So, once again:
1. Edit my.ini file and save
2. Restart your server (this cannot be done dynamically)
3. Change the table engine (if needed) ALTER TABLE tbl_name ENGINE = MyISAM;
4. Perform repair REPAIR TABLE tbl_name QUICK.
Be aware that InnoDB and MyISAM have their speed differences. One read faster, other writes faster ( read more about that on the internet )
disable stopword for fulltext search in mysql using this steps
1: open my.ini file in mysql
2: place below two line after [mysqld] line in my.ini (search [mysqld] in file)
ft_min_word_len=1
ft_stopword_file=""
3: restart your server
4: repair your table using below command
> repair table tablename;
5: now your search is working....
For the INNODB case, it is possible to disable stop_words when you create the index.
SET ##SESSION.innodb_ft_enable_stopword = 'OFF';
create table foo
....
fulltext (search_col)
This will cause the full text index to be created with the stopwords disabled. You can verify by using the following queries.
SET GLOBAL innodb_ft_aux_table = 'schema/foo';
select * from information_schema.innodb_ft_config;
Your results will look like this:
Notice that use_stopword is set to 0.
Search for use_stopwords on this mysql documentation page.
and Checkout innodb_ft_enable_stopword here
setting
ft_stopword_file = ""
didn't work for me, I'm using INNODB tables and MySQL 5.6 (stop words still not indexed in full text indexes after optimizing associated table)
this solution works (even if you are not super user) :
CREATE TABLE mydb.stopwordslist(value VARCHAR(20)) ENGINE = INNODB;
INSERT INTO mydb.stopwordslist(value) VALUES ('skipthisword');
for all users but you still need super user rights :
SET GLOBAL innodb_ft_server_stopword_table = 'mydb/stopwordslist';
for just the user (assuming it the one who recreating indexes and updating columns)
SET SESSION innodb_ft_user_stopword_table = 'mydb/stopwordslist';
as it is a session variable, it won't last when your session is closed so please make sure you set it at each session or before you optimize or insert into tables having fulltext index or when you update column indexed by fulltext index
try using MATCH…AGAINST…IN BOOLEAN MODE
Like this one:
WHERE MATCH(author,title)
AGAINST('"origin of"' IN BOOLEAN MODE);
I recently came across a wierd issue with MySQL Fulltext search. My statement is really simple:
SELECT * FROM `mytable` WHERE MATCH (`desc`) AGAINST ('+NOR +710' IN BOOLEAN MODE)
And this is what in the desc column: "The NOR 710 also has smoke seal ..."
For some reason it won't find that row. I added Fulltext index to that column, mysql version is 5.1.56 , database engine of that table is MyISAM. Is there anything else i need to check?
Thanks
By default, fulltext indexes will ignore words that are shorter than 4 charaters. Adjust your ft_min_word_len to also include the shorter words.