I have a problem with MySQL's full text index, it treats underscore as part of a word (why? dunno).
This is the string I have in my table, VA_-_Some_Album
And this is the query for it:
SELECT
*
FROM
`mytable`
WHERE
MATCH (`name`) AGAINST ('+Some* +Album*' IN BOOLEAN MODE)
ORDER BY `sdate` DESC
LIMIT 3
MySQL returns an empty set for this query, unless I change it to +*Some* since the underscore is part of the word (_Some instead of Some). This is not good for me, since when adding the extra asterisk (*) the plus sign stops functioning and I don't get the "AND" done.
I tried to change the charset definition, and rebuild the full-text index but nothing.
Any ideas? changing the way the string is stored is not up to me.
Thank you!
I'm not very clear with your question but take a look here:
http://dev.mysql.com/doc/refman/5.0/en/fulltext-boolean.html
there they explain the differences between _, +, *, -, <>, etc In Boolean Mode
Related
I have a MySQL 5.7.31 InnoDB table with full text index enabled...
if I search for a longer word, I get results:
SELECT * FROM my_table WHERE match(my_title) against('landscape in' IN BOOLEAN MODE)
if I search full text for short word (e.g in), I get no results
SELECT * FROM my_table WHERE match(my_title) against('in' IN BOOLEAN MODE)
the data is there, I can find it with like %% query:
SELECT * FROM my_table WHERE my_title LIKE '%in%'
I set these two in /etc/my.cnf, I understand one is for InnoDB and one for MyIsam, I restarted MySQL, I still cannot run the above short full text query.
ft_min_word_len=1
innodb_ft_min_token_size=1
Edit:
If I have a value like landscape in Paris, then I get data for against('+landscape +Paris' IN BOOLEAN MODE) but NOT for against('+landscape +in +Paris' IN BOOLEAN MODE)
Is in a reserved word maybe ?
"in" is probably in the "stop list". Change the specification of the stoplist file.
After changing the min-len or the stoplist, you must rebuild the Fulltext index(es). (Restarting MySQL is not needed.)
An alternative I used on one situation: I added + to long words. For example, against('+landscape in +Paris' IN BOOLEAN MODE) would probably achieve your goal without changing either the min-len or the stopword list.
(Yes, there are several 'differences' between MyISAM and InnoDB. I have not found a definitive list in the docs. Here's my attempt at such a list: http://mysql.rjweb.org/doc.php/myisam2innodb#fulltext )
I have a query like below:
SELECT prd_id FROM products WHERE MATCH (prd_search_field)
AGAINST ('+gul* +yetistiren* +adam*' in boolean mode);
This doesn't return the rows including 'gul'.
http://dev.mysql.com/doc/refman/5.0/en/fulltext-boolean.html
The document says this.
Then a search for '+word +the*' will likely return fewer rows than a
search for '+word +the':
The former query remains as is and requires both word and the* (a word starting with the) to be present in the document.
The latter query is transformed to +word (requiring only word to be present). the is both too short and a stopword, and either condition is enough to cause it to be ignored.
So as I understood the too short word condition must not be applied in my situation since I use * after each word. What's wrong with this?
As a solution I use the below query but since it's slow, I need to find another solution. Any idea would be appreciated? Thanks in advance..
SELECT prd_id FROM products WHERE 1 AND MATCH (prd_search_field)
AGAINST ('+yetistiren* +adam*' in boolean mode) AND prd_search_field
LIKE '%gul%';
As a note ft_min_word_length=4 as default in all shared hosting environments, and I cannot change it.
I'm pretty new to MySQL full-text searches and I ran into this problem today:
My company table has a record with "e-magazine AG" in the name column. I have a full-text index on the name column.
When I execute this query the record is not found:
SELECT id, name FROM company WHERE MATCH(name) AGAINST('+"e-magazi"*' IN BOOLEAN MODE);
I need to work with quotes because of the dash and to use the wildcard because I implement a "search as you type" functionality.
When I search for the whole term "e-magazine AG", the record is found.
Any ideas what I'm doing wrong here? I read about adding the dash to the list of word characters (config update needed) but I'm searching for a way to do this programmatically.
This clause
MATCH(name) AGAINST('+"e-magazi"*' IN BOOLEAN MODE);
Will search for a AND "e" AND NOT "magazi"; i.e. the - inside "e-magazi" will be interpreted as a not even though it is inside quotation marks.
For this reason it will not work as expected.
A solution is to apply an extra having clause with a LIKE.
I know this having is slow, but it will only be applied to the results of the match, so not too many rows should be involved.
I suggest something like:
SELECT id, name
FROM company
WHERE MATCH(name) AGAINST('magazine' IN BOOLEAN MODE)
HAVING name LIKE '%e-magazi%';
MySQL fulltext treats the word e-magazine in a text as a phrase and not as a word. Because of that it results the two words e and magazine. And while it builds the search index it does not add the e to the index because of the ft_min_word_len (default is 4 chars).
The same length limitation is used for the search query. That is the reason why a search for e-magazine returns exactly the same results as a-magazine because a and - is fully ignored.
But now you want to find the exact phrase e-magazine. By that you use the quotes and that is the complete correct way to find phrases, but MySQL does not support operators for phrases, only for words:
https://dev.mysql.com/doc/refman/5.7/en/fulltext-boolean.html
With this modifier, certain characters have special meaning at the beginning or end of words in the search string
Some people would suggest to use the following query:
SELECT id, name
FROM company
WHERE MATCH(name) AGAINST('e-magazi*' IN BOOLEAN MODE)
HAVING name LIKE 'e-magazi%';
As I said MySQL ignores the e- and searches for the wildcard word magazi*. After those results are optained it uses HAVING to aditionally filter the results for e-magazi* including the e-. By that you will find the phrase e-magazine AG. Of course HAVING is only needed if the search phrase contains the wildcard operator and you should never use quotes. This operator is used by your user and not you!
Note: As long you do not surround the search phrase with % it will find only fields that start with that word. And you do not want to surround it, because it would find bee-magazine as well. So maybe you need an additional OR HAVING name LIKE ' %e-magazi%' OR HAVING NAME LIKE '\\n%e-magazi%' to make it usable inside of texts.
Trick
But finally I prefer a trick so HAVING isn't needed at all:
If you add texts to your database table, add them additionally to a separate fulltext indexed column and replace words like up-to-date with up-to-date uptodate.
If a user searches for up-to-date replace it in the query with uptodate.
By that you can still find specific in user-specific but up-to-date as well (and not only date).
Bonus
If a user searches for -well-known huge ports MySQL treats that as not include *well*, could include *known* and *huge*. Of course you could solve that with an other extra query variant as well, but with the trick above you remove the hyphen so the search query looks simply like that:
SELECT id
FROM texts
WHERE MATCH(text) AGAINST('-wellknown huge ports' IN BOOLEAN MODE)
I am trying to follow: http://dev.mysql.com/doc/refman/4.1/en/fulltext-natural-language.html
in an attempt to improve search queries, both in speed and the ability to order by score.
However when using this SQL ("skitt" is used as a search term just so I can try match Skittles).
SELECT
id,name,description,price,image,
MATCH (name,description)
AGAINST ('skitt')
AS score
FROM
products
WHERE
MATCH (name,description)
AGAINST ('skitt')
it returns 0 results. I am trying to find out why, I think I might have set my index's up wrong I'm not sure, this is the first time I've strayed away from LIKE!
Here is my table structure and data:
Thank you!
By default certain words are excluded from the search. These are called stopwords. "a" is an example of a stopword. You could test your query by using a word that is not a stopword, or you can disable stopwords:
How can I write full search index query which will not consider any stopwords?
If you want to also match prefixes use the truncation operator in boolean mode:
*
The asterisk serves as the truncation (or wildcard) operator. Unlike the other operators, it should be appended to the word to be affected. Words match if they begin with the word preceding the * operator.
I'm trying to use wildcards to pass a stem of a word as part of a full text search in MySql. I would prefer to use match...against for the performance benefit instead of a like query.
I found this post which makes it sound as though this can be done:
MySQL fulltext with stems
...but I can't get it to work for me.
My data looks like this:
table name: "rxnorm_brands"
step_medname bn_name
Amoxicillin Wymox
Amoxicillin-Clavulanate Augmentin
This query works but uses "like":
select `step_medname`, `bn_name`
from `rxnorm_brands`
where (`bn_name` like 'Amox%' or `step_medname` like 'Amox%');
I want to use this query, but it returns nothing:
select `step_medname`, `bn_name`
from `rxnorm_brands`
where MATCH (`bn_name`, `step_medname`) AGAINST ('Amox*');
I do have a fulltext index on bn_name and step_medname. What am I doing wrong? Or can this not be done?
This can be done using IN BOOLEAN MODE, see: http://dev.mysql.com/doc/refman/5.1/en/fulltext-boolean.html.
So your query would become:
select `step_medname`, `bn_name`
from `rxnorm_brands`
where MATCH (`bn_name`, `step_medname`) AGAINST ('Amox*' IN BOOLEAN MODE);
but note that with the BOOLEAN MODE matching, rows either match or they don't - the results can no longer be ordered by relevance like they can with normal FULLTEXT searches.
The RxNorm API now has a method that will do matching of text that only approximately matches the RxNorm concept. See http://rxnav.nlm.nih.gov/RxNormAPI.html#label:23