My current query is a full text search, on a particular user's records. For this table, I have a FULLTEXT index over compColumn and a bTree over userID.
SELECT K.* FROM k_table AS K WHERE K.userID=2 AND (MATCH (K.compColumn) AGAINST ('+gatsby' IN BOOLEAN MODE));
From what I can tell, only one index gets used, and the WHOLE table is searched for the fulltext result, as opposed to just doing a fulltext search over user 2's records.
I was wondering how to set the above up having the user ID as a subquery, from which the fulltext search is then made, and if that would use the two indexes?
Thanks for your time and help.
To make sub queries you put a query where you would logically put a table:
SELECT
k2.*
FROM
(SELECT
K.*
FROM
k_table AS K
WHERE K.userID=2) k2
WHERE
MATCH (K2.compColumn) AGAINST ('+gatsby' IN BOOLEAN MODE);
Related
Using MySQL 5.7, I have a table of urls containing url_title, url_description, url_keywords
Sometimes I just need to look in url_title, but sometimes look for something in all columns.
Is it better to just create one index containing all 3 columns or create a separate index for url_title alone and another index containing all 3 columns ?
e.g Will it search for url_title slower in the 3 columns index vs single column ?
Or can MySQL only search/read in given column even if index would contain 3 columns ?
Later edit: this is a sample query but I do have other less important variations:
SELECT *
FROM urls
WHERE match(url_title, url_description,
url_keywords, url_paragraphs)
against('red boots' IN BOOLEAN MODE)
LIMIT 500
Update: You didn't mention in your original post that you were talking about fulltext indexes, not conventional B-tree indexes.
Fulltext indexes are a different type. You must specify ALL the columns of the fulltext index in your MATCH() clause. No fewer, and no more, and they must be in the same order as they appear in the index definition.
If you want to do a fulltext search only on a single column sometimes, then you will have to create another fulltext with that single column.
Below is my original answer, that I wrote before you clarified that you were using a fulltext index. Perhaps it will help someone else.
MySQL can use the index if the column(s) you search are the leftmost column(s) of that index. It can use a subset of the columns of a multi-column index.
For example, given an index on (a, b, c), the following query uses all three columns:
SELECT ... WHERE a = ? AND b = ? AND c = ?
The following query uses the first column a of the index, because it's the leftmost column.
SELECT ... WHERE a = ?
The following query uses the first two columns of the index, because they're consecutive and the leftmost subset of columns.
SELECT ... WHERE a = ? AND b = ?
The following query uses only the first column a of the index, because the conditions don't match consecutive columns of the index. It will use the index to narrow down the search to rows matching the a condition, but then it will have to examine each of those rows to evaluate the c condition, even though c is part of the same index.
SELECT ... WHERE a = ? AND c = ?
MySQL has an optimization called index condition pushdown which does a short-cut for this. It delegates to the storage engine to evaluate the c condition, knowing that c is part of the index. So it still counts as examining the row, but it make the row read a little bit less costly.
The following query cannot use the index at all, because the conditions are not on leftmost columns of that index.
SELECT ... WHERE b = ? AND c = ?
The guidelines for FULLTEXT indexes and MATCH...AGAINST are different than for INDEX. For this:
SELECT *
FROM urls
WHERE match(url_title, url_description,
url_keywords, url_paragraphs)
against('red boots' IN BOOLEAN MODE)
LIMIT 500
(and assuming ENGINE=InnoDB), you need a FULLTEXT index with all 4 columns in it.
FULLTEXT(url_title, url_description,
url_keywords, url_paragraphs)
If you might also be searching, say, just url_title in another query, then you would also need FULLTEXT(url_title). (Etc)
See if either of these would be 'better' for your application:
against('+red +boots' IN BOOLEAN MODE)
against('red boots')
I have a query like this:
SELECT * FROM staffs
WHERE MATCH staff_name AGAINST ('johnny')
ORDER BY staff_city ASC
Just an example, I want to ask which Index should I use here. For the MATCH() and AGAINST() there is FULLTEXT index on column staff_name, that's okay. But in the query there is also ORDER BY on the staff_city column. The FULLTEXT works fast, but when it comes to ordering the matched results, the search is slower. What INDEX should need there?
MySQL can never (almost never) use two indexes in a single SELECT. The Optimizer picks from among the indexes you have, and it usually picks the best for the query.
For this query, only the FULLTEXT index you have will be used, regardless of the other indexes the table has.
The other index might be useful for some other query.
More: Assuming you care only about rows with the word 'johnny' in it, change:
AGAINST ('johnny')
-->
AGAINST ('+johnny' IN BOOLEAN MODE)
I have a MySQL MyISAM table with a full text index on the keywords column and 20 millions rows. It works well when a search for rare words, for example:
SELECT count(*) FROM books WHERE MATCH(keywords) AGAINST ('+DUCK' IN BOOLEAN MODE)
(0.005s, 2k results)
But when I search for a more common terms it is much slowers:
SELECT count(*) FROM books WHERE MATCH(keywords) AGAINST ('+YES' IN BOOLEAN MODE)
(5s, 2millions results)
It makes sens because the last one returns much more rows, but then how can I pre-filter the rows before the text search? This doesn't work:
SELECT count(*) FROM books WHERE date > "2019-09-23" AND MATCH(keywords) AGAINST ('+YES' IN BOOLEAN MODE)
(5s, 0 result)
MyISAM's (and maybe InnoDB's) FULLTEXT will always do the MATCH first, then any other clauses. Hence, adding that extra filter does not help with speed.
Think of it this way... A FT index is constructed to test the entire table(s) for the MATCH clause. It is not ready to handle any filtering before it goes to work. So, you are stuck with FT first, then filter the results the other way but without benefit of any indexes.
I was trying to make a very fast & efficient approach to fetch the records using keywords as search.
Our MYSQL table MASTER tablescontains 30,000 rows and has 4 fields.
ID
title (FULLTEXT)
short_descr (FULLTEXT)
long_descr (FULLTEXT)
Can any one suggest which is one more efficient?
LIKE %
MYSQL's AGAINST
It would be nice if some one can write a SQL query for the keywords
Weight Loss Secrets
SELECT id FROM MASTER
WHERE (title LIKE '%Weight Loss Secrets%' OR
short_descr LIKE '%Weight Loss Secrets%' OR
long_descr LIKE '%Weight Loss Secrets%')
Thanks in advance
The FULLTEXT index should be faster, maybe its a good idea to add all columns into 1 fulltext index.
ALTER TABLE MASTER
ADD FULLTEXT INDEX `FullTextSearch`
(`title` ASC, `short_descr` ASC, `long_descr` ASC);
Then execute using IN BOOLEAN MODE
SELECT id FROM MASTER WHERE
MATCH (title, short_descr, long_descr)
AGAINST ('+Weight +Loss +Secrets' IN BOOLEAN MODE);
This will find rows that contains all 3 keywords.
However, this wont give you exact match the keywords just need to be present in same row.
If you also want exact match you could do like this, but its a bit hacky and would only work if your table doesnt get to big.
SELECT id FROM
(
SELECT CONCAT(title,' ',short_descr,' ', long_descr) AS SearchField
FROM MASTER WHERE
MATCH (title, short_descr, long_descr)
AGAINST ('+Weight +Loss +Secrets' IN BOOLEAN MODE)
) result WHERE SearchField LIKE '%Weight Loss Secrets%'
I have a search query which performs a fulltext search on the DB.
$sql = "SELECT
*
FROM
`tbl_auction_listing` AS `al`
JOIN
`tbl_user` AS `u` ON `al`.`user_id` = `u`.`user_id`
LEFT JOIN
`tbl_gallery_details` AS `gd` ON `al`.`user_id` = `gd`.`user_id`
LEFT JOIN
`tbl_self_represented_details` AS `sr` ON `u`.`user_id` = `sr`.`user_id`
WHERE
`al`.`status` = '" . ACTIVE . "'
AND
`al`.`start_date` < NOW()
AND
`al`.`end_date` > NOW()
AND
MATCH(`al`.`listing_title`,
`al`.`description`,
`al`.`provenance`,
`al`.`title`,
`al`.`artist_full_name`,
`al`.`artist_first_name`,
`al`.`artist_last_name`,
`sr`.`artist_name`,
`gd`.`gallery_name`,
`u`.`username`) AGAINST('$search_query' IN BOOLEAN MODE)";
When I search for 'Cardozo, Horacio' or 'cardozo' or 'horacio' I get no results however I know there is an artist with 2 records in the db with artist_full_name = Cardozo, Horacio.
If I remove all MATCH fields and just have al.artist_full_name I get 2 results. If I add in al.description I get 1 result because 'Horacio Cardozo' exists in the description.
Is there a way to have the search return all records if any condition (any search query word) is met in any of the MATCH fields? I tried removing IN BOOLEAN MODE but that produced same results.
It appears that InnoDB tables do not allow searches over several fulltext indexes in the same MATCH() condition.
Here your fields do not all belong to the same table, therefore they are covered by different indexes. Notice the same limitation applies if you had a table like this:
CREATE TABLE t (
f1 VARCHAR(20),
f2 VARCHAR(20),
FULLTEXT(f1), FULLTEXT(f2)
) ENGINE=InnoDB;
SELECT * FROM t
WHERE MATCH(f1, f2) AGAINST ('something in f2'); -- likely to return no row
It looks like a fulltext search may only search on the first fulltext index it encounters but this is only something I deduct from this experience, please do not take this for granted.
The bottomline is that you should split your search so as to use one single fulltext index per MATCH() clause:
SELECT * FROM auction, user, gallery, ...
WHERE
MATCH(auction.field1, auction.field2) AGAINST ('search query' IN BOOLEAN MODE) OR
MATCH(auction.field3) AGAINST ('search query' IN BOOLEAN MODE) OR
MATCH(user.field1, user.field2, user.field3) AGAINST...
This is an illustration of a possible query if you had two distinct indexes on auction and one one on user. You need to adapt it to your actual structure (please post your tables' descriptions if you need more guidance).
Notice this only applies to InnoDB tables. Interestingly, MyISAM tables do not seem to show the same limitation.
Update: it turns out this was a bug in the InnoDB engine, fixed in 5.6.13/5.7.2. The above example now rightfully fails with "Can't find FULLTEXT index matching the column list". Indeed, there is no index on (f1, f2), but one on (f1) and another one on (f2). As the changelog advises:
Unlike MyISAM, InnoDB does not support boolean full-text searches on
nonindexed columns, but this restriction was not enforced, resulting
in queries that returned incorrect results.
It is noteworthy that while such queries return a correct result set with MyISAM, they run slower than one might expect, as they silently ignore existing fulltext indexes.