I am using the latest version of MySQL 5.5.
I have a fulltext index spanning multiple columns in a table generated specifically for fulltext search (other tables in the database uses innodb):
somedata_search
========
id
name
about
note
dislike
I have a fulltext index on all the columns except for ID. I am able to run fulltext searches using:
SELECT * FROM account_search WHERE MATCH(name, about, note, dislike) AGAINST('mykeyword*' IN BOOLEAN MODE);
This all works fine, but is there a way to deteremine which column the match originates from for each row? If there are matches across columns in a row, I am happy to have just the first column.
I don't think there is any "native" way of getting it but it is possible to do it anyway.
I'm not sure this is fast but it returns the correct data
select text_test.*,
match(name) against ('dude' in boolean mode) as name_match,
match(info) against ('dude' in boolean mode) as info_match
from text_test
where match(name, info) against ('dude' in boolean mode);
http://sqlfiddle.com/#!2/5159c/1
Related
I have table with quite many rows (~2M).
When i search it like
SELECT * FROM product WHERE
MATCH(name) AGAINST('+some' '+word' IN BOOLEAN MODE)
it works like charm and finds what i need in less than 0.5s.
But when i search for 2 sets of words like this
SELECT * FROM product WHERE
MATCH(name) AGAINST('+some' '-word' IN BOOLEAN MODE)
OR
MATCH(name) AGAINST('+something' '-other' IN BOOLEAN MODE)
search takes sometimes over minute.
I would expect it to work 2 times slower (it's 2 searches), maybe a bit more (you still have to compare results and remove duplicates, but if there are only few results it should not take long), but not so much longer. After adding OR it works slower, than LIKE "%...%" OR LIKE "%...%"
Anyone can tell me what am i doing wrong?
Unfortunately for you, fulltext indexes have some limitations, and not being able to properly merge the results of two independent fulltext searches is one them:
The Index Merge optimization algorithm has the following known limitations:
[...]
Index Merge is not applicable to full-text indexes.
Fortunately for you, fulltext searches can be more complex, so you can merge your searches yourself. Your second query can be written as a single search using:
SELECT * FROM product WHERE
MATCH(name) AGAINST('(+something -other) (+some -word)' IN BOOLEAN MODE)
This defines two search sets and is ok if either of the two (...) matches - which is an or.
Alternatively, you can use a union instead of an or, which allows MySQL to actually run two independent fulltext searches and then combine the two results, which is basically what you are thinking of:
SELECT * FROM product WHERE
MATCH(name) AGAINST('+some -word' IN BOOLEAN MODE)
UNION
SELECT * FROM product WHERE
MATCH(name) AGAINST('+something -other' IN BOOLEAN MODE)
This also works for more complicated situations, e.g. merging searches on different columns, but will not work that easy if you want to do something else than or.
I have a MySQL MyISAM table with a full text index on the keywords column and 20 millions rows. It works well when a search for rare words, for example:
SELECT count(*) FROM books WHERE MATCH(keywords) AGAINST ('+DUCK' IN BOOLEAN MODE)
(0.005s, 2k results)
But when I search for a more common terms it is much slowers:
SELECT count(*) FROM books WHERE MATCH(keywords) AGAINST ('+YES' IN BOOLEAN MODE)
(5s, 2millions results)
It makes sens because the last one returns much more rows, but then how can I pre-filter the rows before the text search? This doesn't work:
SELECT count(*) FROM books WHERE date > "2019-09-23" AND MATCH(keywords) AGAINST ('+YES' IN BOOLEAN MODE)
(5s, 0 result)
MyISAM's (and maybe InnoDB's) FULLTEXT will always do the MATCH first, then any other clauses. Hence, adding that extra filter does not help with speed.
Think of it this way... A FT index is constructed to test the entire table(s) for the MATCH clause. It is not ready to handle any filtering before it goes to work. So, you are stuck with FT first, then filter the results the other way but without benefit of any indexes.
I want to find all rows that match a full-text search for one pair of columns but also do not match the same text in another column.
Both of these seem to work
SELECT * FROM docs WHERE MATCH(title, descript) AGAINST ('energy' IN BOOLEAN MODE) AND NOT MATCH(categories) AGAINST ('energy' IN BOOLEAN MODE);
Or using a subquery:
SELECT * FROM docs WHERE MATCH(title, descript) AGAINST ('energy' IN BOOLEAN MODE) AND id NOT IN (SELECT id FROM docs where MATCH(categories) AGAINST ('energy' IN BOOLEAN MODE));
The docs field has the relevant full-text indexes set up.
Any reason to prefer one over the other?
On the (small) database I'm using they are both very fast, too fast to measure reliably.
Thanks for any suggestions.
My current query is a full text search, on a particular user's records. For this table, I have a FULLTEXT index over compColumn and a bTree over userID.
SELECT K.* FROM k_table AS K WHERE K.userID=2 AND (MATCH (K.compColumn) AGAINST ('+gatsby' IN BOOLEAN MODE));
From what I can tell, only one index gets used, and the WHOLE table is searched for the fulltext result, as opposed to just doing a fulltext search over user 2's records.
I was wondering how to set the above up having the user ID as a subquery, from which the fulltext search is then made, and if that would use the two indexes?
Thanks for your time and help.
To make sub queries you put a query where you would logically put a table:
SELECT
k2.*
FROM
(SELECT
K.*
FROM
k_table AS K
WHERE K.userID=2) k2
WHERE
MATCH (K2.compColumn) AGAINST ('+gatsby' IN BOOLEAN MODE);
I have a search query which performs a fulltext search on the DB.
$sql = "SELECT
*
FROM
`tbl_auction_listing` AS `al`
JOIN
`tbl_user` AS `u` ON `al`.`user_id` = `u`.`user_id`
LEFT JOIN
`tbl_gallery_details` AS `gd` ON `al`.`user_id` = `gd`.`user_id`
LEFT JOIN
`tbl_self_represented_details` AS `sr` ON `u`.`user_id` = `sr`.`user_id`
WHERE
`al`.`status` = '" . ACTIVE . "'
AND
`al`.`start_date` < NOW()
AND
`al`.`end_date` > NOW()
AND
MATCH(`al`.`listing_title`,
`al`.`description`,
`al`.`provenance`,
`al`.`title`,
`al`.`artist_full_name`,
`al`.`artist_first_name`,
`al`.`artist_last_name`,
`sr`.`artist_name`,
`gd`.`gallery_name`,
`u`.`username`) AGAINST('$search_query' IN BOOLEAN MODE)";
When I search for 'Cardozo, Horacio' or 'cardozo' or 'horacio' I get no results however I know there is an artist with 2 records in the db with artist_full_name = Cardozo, Horacio.
If I remove all MATCH fields and just have al.artist_full_name I get 2 results. If I add in al.description I get 1 result because 'Horacio Cardozo' exists in the description.
Is there a way to have the search return all records if any condition (any search query word) is met in any of the MATCH fields? I tried removing IN BOOLEAN MODE but that produced same results.
It appears that InnoDB tables do not allow searches over several fulltext indexes in the same MATCH() condition.
Here your fields do not all belong to the same table, therefore they are covered by different indexes. Notice the same limitation applies if you had a table like this:
CREATE TABLE t (
f1 VARCHAR(20),
f2 VARCHAR(20),
FULLTEXT(f1), FULLTEXT(f2)
) ENGINE=InnoDB;
SELECT * FROM t
WHERE MATCH(f1, f2) AGAINST ('something in f2'); -- likely to return no row
It looks like a fulltext search may only search on the first fulltext index it encounters but this is only something I deduct from this experience, please do not take this for granted.
The bottomline is that you should split your search so as to use one single fulltext index per MATCH() clause:
SELECT * FROM auction, user, gallery, ...
WHERE
MATCH(auction.field1, auction.field2) AGAINST ('search query' IN BOOLEAN MODE) OR
MATCH(auction.field3) AGAINST ('search query' IN BOOLEAN MODE) OR
MATCH(user.field1, user.field2, user.field3) AGAINST...
This is an illustration of a possible query if you had two distinct indexes on auction and one one on user. You need to adapt it to your actual structure (please post your tables' descriptions if you need more guidance).
Notice this only applies to InnoDB tables. Interestingly, MyISAM tables do not seem to show the same limitation.
Update: it turns out this was a bug in the InnoDB engine, fixed in 5.6.13/5.7.2. The above example now rightfully fails with "Can't find FULLTEXT index matching the column list". Indeed, there is no index on (f1, f2), but one on (f1) and another one on (f2). As the changelog advises:
Unlike MyISAM, InnoDB does not support boolean full-text searches on
nonindexed columns, but this restriction was not enforced, resulting
in queries that returned incorrect results.
It is noteworthy that while such queries return a correct result set with MyISAM, they run slower than one might expect, as they silently ignore existing fulltext indexes.