MySQL Fulltext MATCH/AGAINST showing not always results - mysql

I have the following table setup:
CREATE TABLE IF NOT EXISTS `search_table` (
`fulltext_id` int(10) unsigned NOT NULL AUTO_INCREMENT COMMENT 'Entity ID',
`data_index` longtext COMMENT 'Data index',
PRIMARY KEY (`fulltext_id`),
FULLTEXT KEY `FTI_CATALOGSEARCH_FULLTEXT_DATA_INDEX` (`data_index`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COMMENT='Search table'
AUTO_INCREMENT=1;
INSERT INTO `search_table` (`fulltext_id`, `data_index`)
VALUES (1, 'Test Hello abc');
Then I try a full text search on it with 3 different query texts:
SELECT `s`.`fulltext_id`, MATCH (s.data_index) AGAINST ('Test' IN BOOLEAN MODE) AS `relevance` FROM `search_table` AS `s`
WHERE (MATCH (s.data_index) AGAINST ('Test' IN BOOLEAN MODE));
SELECT `s`.`fulltext_id`, MATCH (s.data_index) AGAINST ('Hello' IN BOOLEAN MODE) AS `relevance` FROM `search_table` AS `s`
WHERE (MATCH (s.data_index) AGAINST ('Hello' IN BOOLEAN MODE));
SELECT `s`.`fulltext_id`, MATCH (s.data_index) AGAINST ('abc' IN BOOLEAN MODE) AS `relevance` FROM `search_table` AS `s`
WHERE (MATCH (s.data_index) AGAINST ('abc' IN BOOLEAN MODE));
Only the first query (the search for Test) gives a result back, the other two not. I don't understand why?

You should check the list of currently defined stopwords. You can do this like this:
mysql> SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_DEFAULT_STOPWORD;
More information about MySQL stopwords can be found here: https://dev.mysql.com/doc/refman/5.7/en/fulltext-stopwords.html
Hello for example is a known stopword therefore it is being ignored during FTS matching. If you check comments on the linked MySQL doc page you will find links from user to English language stopwords lists, like https://www.ranks.nl/stopwords/.
Note, MySQL as well as other DB engines allows you to specify your own custom list of stop words. So you should check both pre-defined system stopwords list and any existing custom stopwords lists.

Related

MYSql Full Text Search acting weird in boolean mode

I am trying to use ngram in MySQL FTS (Full Text Search), created this table and added some rows (I set ngram size to 3 using ngram_token_size=3 in my.ini) :
CREATE TABLE articles (id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
bookid INT,chapter INT,txt LONGTEXT ,FULLTEXT (txt) WITH PARSER ngram) ENGINE=InnoDB CHARACTER SET utf8mb4;
then started to search, my first query returns 0 items, which is correct :
SELECT * FROM articles WHERE MATCH (txt) AGAINST ('myterm' in boolean mode);
but when I enter this query it returns some rows that does not contain myterm!
SELECT * FROM articles WHERE MATCH (txt) AGAINST ('"myterm" #1' in boolean mode);
also this returns rows that does not contain myterm
SELECT * FROM articles WHERE MATCH (txt) AGAINST ('+myterm +anotherterm' in boolean mode);

MySQL full text search - no partial recognition

I'm trying to build a keyword search tool based on mysql and I can only get results for full words while I would like to get results for partial matches too.
My db structure looks like this:
My db content looks like this:
This query works:
select * from chromext_keyword where matches (keyword) against ('Redmi')
But this one doesn't work (no result):
select * from chromext_keyword where matches (keyword) against ('red')
I tried with % but it did not solve the problem. I tried the natural language option as well as boolean but it didn't help.
Update with create table query:
CREATE TABLE chromext_keywords (
id int(10) NOT NULL,
keyword text NOT NULL,
blacklist text NOT NULL,
category text NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
and insert:
INSERT INTO chromext_keywords (id, keyword, blacklist, category) VALUES
(1, 'Redmi Note 10', '9,8,pro', '2'),
(2, 'Realme GT', '6,7,8,narzo', '2');
and I added full text:
ALTER TABLE chromext_keywords
ADD UNIQUE KEY id (id);
ALTER TABLE chromext_keywords ADD FULLTEXT KEY keyword (keyword);
I have also tried innoDb and Myisam
Am I missing something?
Thanks
You should check for Minimum word lenght setting ..
in mysql the minimum length for full text search in limited by the param
ft_min_word_len
and the defualt value is for words > 3
take a look at the related docs
https://dev.mysql.com/doc/refman/8.0/en/fulltext-fine-tuning.html
I have finally found the answer.
The following query works:
SELECT * FROM chromext_keywords WHERE match(keyword) against('(re*)') IN BOOLEAN MODE)
With multiple keywords:
SELECT * FROM chromext_keywords WHERE match (keyword) against ('(+red*+not*)') IN BOOLEAN MODE)
I still need to figure out how to cover spelling mistakes. If anyone has an idea, let me know.

Mysql match against numeric keyword

I use mysql full-text index.
I found it can not matches a key numeric word as '1' in '1,2,3' or '1 2 3'.
I use this query "SELECT * FROM users u where match(u.leader_uids) against('1' IN BOOLEAN MODE);"
How to solve this issue?
Thanks a lot!
I am Providing the example hope it will works for you i think
MATCH (field) AGAINST ('+856049' IN BOOLEAN MODE)
It will work only with words of 4 or more digits. So you must concat some prefix in the leader_uid before saving it. Example:
CREATE TABLE mytable(
id INT NOT NULL KEY AUTO_INCREMENT,
myfield TEXT,
FULLTEXT KEY ix_mytable (myfield)
);
INSERT INTO mytable (myfield) VALUES
('id_1 id_2 id_3'),
('id_8'),
('id_4 id_1');
SELECT * FROM mytable
WHERE MATCH(myfield) AGAINST ('+id_1' IN BOOLEAN MODE);
-- will select rows 1 and 3
You can change the minimum amount of chars required for the words, in mysql config:
https://dev.mysql.com/doc/refman/8.0/en/innodb-parameters.html#sysvar_innodb_ft_min_token_size

MySql Full Index search not working

I set the example code here
http://sqlfiddle.com/#!2/6aa9ec/1
Below mentioned DB query used for searching the word Samsung but its not working
SELECT `idREQUEST`, `USER_NAME`, `idCATEGORY`, `TITLE`, `DESCRIPTION`, `IMAGE_URL1`, `IMAGE_URL2`, `IMAGE_URL3`, `POST_DATE`
FROM `requests`
WHERE MATCH (`TITLE`,`DESCRIPTION`) AGAINST('Samsung')
You can use Boolean Full-Text Searches
SELECT `idREQUEST`,
`USER_NAME`,
`idCATEGORY`,
`TITLE`,
`DESCRIPTION`,
`IMAGE_URL1`,
`IMAGE_URL2`,
`IMAGE_URL3`,
`POST_DATE`
FROM `requests`
WHERE MATCH (`TITLE`,`DESCRIPTION`) AGAINST('+Samsung' IN BOOLEAN MODE)
According to docs
MySQL can perform boolean full-text searches using
the IN BOOLEAN MODE modifier. With this modifier, certain characters
have special meaning at the beginning or end of words in the search
string. In the following query, the + operator indicate that a
word is required to be present, respectively, for a match to
occur.
DEMO
This query looks for tiles and descriptions which the value is merely 'Samsung' But there are no entries in your table which are only 'Samsung'. Hence I'd suggest you to use %
SELECT `idREQUEST`, `USER_NAME`, `idCATEGORY`, `TITLE`, `DESCRIPTION`,
`IMAGE_URL1`, `IMAGE_URL2`, `IMAGE_URL3`, `POST_DATE`
FROM `requests`
where `TITLE` like '%Samsung%' or `DESCRIPTION` like '%Samsung%'
This is the link

Full-text search on my MySQL table according to "field weight/order"

I would like to use full-text search on my MySQL table, but in addition to it, I would like also imply some kind of "weight order" for columns, in which the search is performed.
For example : I have three columns named - ChapterCol, TextCol, NoteCol.
When I search for the word "financial" it should return :
at the first place all rows which contains this word in ChapterCol (and/or TextCol, NoteCol too),
then all rows which has it in TextCol (and/or NoteCol)
and at last all rows which has it only in NoteCol.
I hope there is an answer...
thanx a lot
First, you need to use the MyISAM storage engine sicne it's the only one in MySQL that supports FULLTEXT indexes.
Create 3 separate fulltext indexes, one on each of those columns.
In your SELECT query, do a MATCH...AGAINST... for each of the 3 columns in both the WHERE clause and the ORDER BY clause so you can get all rows that have your search term in at least one of the 3 columns, and sort them based on which column(s) include(s) the search term.
Here's an example:
CREATE TABLE IF NOT EXISTS your_table (
ChapterCol TEXT,
TextCol TEXT,
NoteCol TEXT,
FULLTEXT INDEX (ChapterCol),
FULLTEXT INDEX (TextCol),
FULLTEXT INDEX (NoteCol)
) Engine = MyISAM;
-- insert test values
insert into your_table (ChapterCol,TextCol,NoteCol) values ('foo','foo','foo'),
('financial blah blah','foo','foo'),('foo','financial blah blah','foo'),
('foo','foo','financial blah blah'),('financial blah blah','financial blah blah',
'financial blah blah');
-- insert filler
insert into your_table (ChapterCol,TextCol,NoteCol)
values (md5(rand()),md5(rand()),md5(rand())),(md5(rand()),md5(rand()),md5(rand())),
(md5(rand()),md5(rand()),md5(rand())),(md5(rand()),md5(rand()),md5(rand())),
(md5(rand()),md5(rand()),md5(rand())),(md5(rand()),md5(rand()),md5(rand())),
(md5(rand()),md5(rand()),md5(rand())),(md5(rand()),md5(rand()),md5(rand())),
(md5(rand()),md5(rand()),md5(rand())),(md5(rand()),md5(rand()),md5(rand())),
(md5(rand()),md5(rand()),md5(rand())),(md5(rand()),md5(rand()),md5(rand()));
SELECT ChapterCol,TextCol,NoteCol
FROM your_table
WHERE MATCH (ChapterCol) AGAINST ('financial' IN BOOLEAN MODE)
OR MATCH (TextCol) AGAINST ('financial' IN BOOLEAN MODE)
OR MATCH (NoteCol) AGAINST ('financial' IN BOOLEAN MODE)
ORDER BY MATCH (ChapterCol) AGAINST ('financial' IN BOOLEAN MODE) DESC,
MATCH (TextCol) AGAINST ('financial' IN BOOLEAN MODE) DESC,
MATCH (NoteCol) AGAINST ('financial' IN BOOLEAN MODE) DESC;