I have a search query which performs a fulltext search on the DB.
$sql = "SELECT
*
FROM
`tbl_auction_listing` AS `al`
JOIN
`tbl_user` AS `u` ON `al`.`user_id` = `u`.`user_id`
LEFT JOIN
`tbl_gallery_details` AS `gd` ON `al`.`user_id` = `gd`.`user_id`
LEFT JOIN
`tbl_self_represented_details` AS `sr` ON `u`.`user_id` = `sr`.`user_id`
WHERE
`al`.`status` = '" . ACTIVE . "'
AND
`al`.`start_date` < NOW()
AND
`al`.`end_date` > NOW()
AND
MATCH(`al`.`listing_title`,
`al`.`description`,
`al`.`provenance`,
`al`.`title`,
`al`.`artist_full_name`,
`al`.`artist_first_name`,
`al`.`artist_last_name`,
`sr`.`artist_name`,
`gd`.`gallery_name`,
`u`.`username`) AGAINST('$search_query' IN BOOLEAN MODE)";
When I search for 'Cardozo, Horacio' or 'cardozo' or 'horacio' I get no results however I know there is an artist with 2 records in the db with artist_full_name = Cardozo, Horacio.
If I remove all MATCH fields and just have al.artist_full_name I get 2 results. If I add in al.description I get 1 result because 'Horacio Cardozo' exists in the description.
Is there a way to have the search return all records if any condition (any search query word) is met in any of the MATCH fields? I tried removing IN BOOLEAN MODE but that produced same results.
It appears that InnoDB tables do not allow searches over several fulltext indexes in the same MATCH() condition.
Here your fields do not all belong to the same table, therefore they are covered by different indexes. Notice the same limitation applies if you had a table like this:
CREATE TABLE t (
f1 VARCHAR(20),
f2 VARCHAR(20),
FULLTEXT(f1), FULLTEXT(f2)
) ENGINE=InnoDB;
SELECT * FROM t
WHERE MATCH(f1, f2) AGAINST ('something in f2'); -- likely to return no row
It looks like a fulltext search may only search on the first fulltext index it encounters but this is only something I deduct from this experience, please do not take this for granted.
The bottomline is that you should split your search so as to use one single fulltext index per MATCH() clause:
SELECT * FROM auction, user, gallery, ...
WHERE
MATCH(auction.field1, auction.field2) AGAINST ('search query' IN BOOLEAN MODE) OR
MATCH(auction.field3) AGAINST ('search query' IN BOOLEAN MODE) OR
MATCH(user.field1, user.field2, user.field3) AGAINST...
This is an illustration of a possible query if you had two distinct indexes on auction and one one on user. You need to adapt it to your actual structure (please post your tables' descriptions if you need more guidance).
Notice this only applies to InnoDB tables. Interestingly, MyISAM tables do not seem to show the same limitation.
Update: it turns out this was a bug in the InnoDB engine, fixed in 5.6.13/5.7.2. The above example now rightfully fails with "Can't find FULLTEXT index matching the column list". Indeed, there is no index on (f1, f2), but one on (f1) and another one on (f2). As the changelog advises:
Unlike MyISAM, InnoDB does not support boolean full-text searches on
nonindexed columns, but this restriction was not enforced, resulting
in queries that returned incorrect results.
It is noteworthy that while such queries return a correct result set with MyISAM, they run slower than one might expect, as they silently ignore existing fulltext indexes.
Related
Using MySQL 5.7, I have a table of urls containing url_title, url_description, url_keywords
Sometimes I just need to look in url_title, but sometimes look for something in all columns.
Is it better to just create one index containing all 3 columns or create a separate index for url_title alone and another index containing all 3 columns ?
e.g Will it search for url_title slower in the 3 columns index vs single column ?
Or can MySQL only search/read in given column even if index would contain 3 columns ?
Later edit: this is a sample query but I do have other less important variations:
SELECT *
FROM urls
WHERE match(url_title, url_description,
url_keywords, url_paragraphs)
against('red boots' IN BOOLEAN MODE)
LIMIT 500
Update: You didn't mention in your original post that you were talking about fulltext indexes, not conventional B-tree indexes.
Fulltext indexes are a different type. You must specify ALL the columns of the fulltext index in your MATCH() clause. No fewer, and no more, and they must be in the same order as they appear in the index definition.
If you want to do a fulltext search only on a single column sometimes, then you will have to create another fulltext with that single column.
Below is my original answer, that I wrote before you clarified that you were using a fulltext index. Perhaps it will help someone else.
MySQL can use the index if the column(s) you search are the leftmost column(s) of that index. It can use a subset of the columns of a multi-column index.
For example, given an index on (a, b, c), the following query uses all three columns:
SELECT ... WHERE a = ? AND b = ? AND c = ?
The following query uses the first column a of the index, because it's the leftmost column.
SELECT ... WHERE a = ?
The following query uses the first two columns of the index, because they're consecutive and the leftmost subset of columns.
SELECT ... WHERE a = ? AND b = ?
The following query uses only the first column a of the index, because the conditions don't match consecutive columns of the index. It will use the index to narrow down the search to rows matching the a condition, but then it will have to examine each of those rows to evaluate the c condition, even though c is part of the same index.
SELECT ... WHERE a = ? AND c = ?
MySQL has an optimization called index condition pushdown which does a short-cut for this. It delegates to the storage engine to evaluate the c condition, knowing that c is part of the index. So it still counts as examining the row, but it make the row read a little bit less costly.
The following query cannot use the index at all, because the conditions are not on leftmost columns of that index.
SELECT ... WHERE b = ? AND c = ?
The guidelines for FULLTEXT indexes and MATCH...AGAINST are different than for INDEX. For this:
SELECT *
FROM urls
WHERE match(url_title, url_description,
url_keywords, url_paragraphs)
against('red boots' IN BOOLEAN MODE)
LIMIT 500
(and assuming ENGINE=InnoDB), you need a FULLTEXT index with all 4 columns in it.
FULLTEXT(url_title, url_description,
url_keywords, url_paragraphs)
If you might also be searching, say, just url_title in another query, then you would also need FULLTEXT(url_title). (Etc)
See if either of these would be 'better' for your application:
against('+red +boots' IN BOOLEAN MODE)
against('red boots')
This might seem to be a redundant question but i can't find the right answer to this issue.
I have a TableA with more than 50 columns.I am implementing a search functionality for searching a query in about 10 columns of this table. TableA contains more than a million rows
For this I have created a composite index on these 10 columns.
index (col1,col_2,col_3,col_4,col_5,col_6,col_7,col_8,col_9,col_10)
Now i am splitting user's query using space as regex. i.e. $search_words = $search_query.split(' '); and using individual words to match in my search query. Example :
SELECT something FROM tableA
WHERE ( MATCH ( col_1, col_2,col_3,col_4,col_5,col_6,col_7,col_8,col_9,col_10 )
AGAINST ( ' +word1* +word2* +word3* +word4* ' IN BOOLEAN MODE ) )
This query works fine for general searches but if users searches for individual alphabets in query like A E I O Co. it takes too much time. What is the best way to optimise the query or another way to perform search in this situation?
If you feed a too-short string to InnoDB's FULLTEXT, it returns zero results. So... Filter out any strings that are shorter than innodb_ft_min_token_size.
If necessary, test for them separately using REGEXP '[[:<:]]A[[:>:]] to look for a 1-letter word A.
Or throw them together. This would check for the only 1-letter English words: REGEXP '[[:<:]][AI][[:>:]]
I was trying to make a very fast & efficient approach to fetch the records using keywords as search.
Our MYSQL table MASTER tablescontains 30,000 rows and has 4 fields.
ID
title (FULLTEXT)
short_descr (FULLTEXT)
long_descr (FULLTEXT)
Can any one suggest which is one more efficient?
LIKE %
MYSQL's AGAINST
It would be nice if some one can write a SQL query for the keywords
Weight Loss Secrets
SELECT id FROM MASTER
WHERE (title LIKE '%Weight Loss Secrets%' OR
short_descr LIKE '%Weight Loss Secrets%' OR
long_descr LIKE '%Weight Loss Secrets%')
Thanks in advance
The FULLTEXT index should be faster, maybe its a good idea to add all columns into 1 fulltext index.
ALTER TABLE MASTER
ADD FULLTEXT INDEX `FullTextSearch`
(`title` ASC, `short_descr` ASC, `long_descr` ASC);
Then execute using IN BOOLEAN MODE
SELECT id FROM MASTER WHERE
MATCH (title, short_descr, long_descr)
AGAINST ('+Weight +Loss +Secrets' IN BOOLEAN MODE);
This will find rows that contains all 3 keywords.
However, this wont give you exact match the keywords just need to be present in same row.
If you also want exact match you could do like this, but its a bit hacky and would only work if your table doesnt get to big.
SELECT id FROM
(
SELECT CONCAT(title,' ',short_descr,' ', long_descr) AS SearchField
FROM MASTER WHERE
MATCH (title, short_descr, long_descr)
AGAINST ('+Weight +Loss +Secrets' IN BOOLEAN MODE)
) result WHERE SearchField LIKE '%Weight Loss Secrets%'
I am trying to create a SELECT query that combines 3 tables via LEFT JOIN and then MATCH specific keywords AGAINST the resulting table.
My testing so far reveals that the query is working fine and the output results are correct, however I have a doubt about FULLTEXT indexes when joining tables via LEFT JOIN.
The question is - should I add additional code to reflect the FULLTEXT index requirement for MATCH AGAINST when doing LEFT JOIN or only correct indexing of the relevant columns in the original tables is enough?
Reason for asking is this post. If I understand it correct my code should not work (since "You can't define fulltext indexes...across multiple tables in MySQL"). Therefore MATCH AGAINST for ma.name_url and v.model, would not work as they are not indexed (or this how I understand it, correct me if I am wrong :) ).
However, as the query output seems to be correct I need an expert advice if the below code is correct or I should add/modify something:
SELECT v.*, vt.color, ma.*
FROM tbl_items AS v
LEFT JOIN tbl_options
AS vt on v.id = vt.v_id
LEFT JOIN tbl_names
AS ma on v.make = ma.id
WHERE
(
(
MATCH (ma.name_url) AGAINST ('keyword1' IN BOOLEAN MODE)
OR MATCH (v.model) AGAINST ('keyword1' IN BOOLEAN MODE)
)
AND
(
MATCH (ma.name_url) AGAINST ('keyword2' IN BOOLEAN MODE)
OR MATCH (v.model) AGAINST ('keyword2' IN BOOLEAN MODE)
)
)
AND v.type = 4
I think you are misreading the post. A single full text index only refers to one table. Actually, this is true for all other indexes, so I'm not sure why there is confusion. The index can refer to multiple columns within the table, but only to one table.
As for your query, I'm not sure which is more important -- the full text indexes or the regular indexes.
As your query is written, you are returning ma rows that do not match the match condition. Perhaps this is fine, but it looks suspicious.
And, if performance is an issue, you might find that breaking the query into separate subqueries (connected by UNION or UNION ALL) gives better performance. Each subquery can be optimized more efficiently.
My current query is a full text search, on a particular user's records. For this table, I have a FULLTEXT index over compColumn and a bTree over userID.
SELECT K.* FROM k_table AS K WHERE K.userID=2 AND (MATCH (K.compColumn) AGAINST ('+gatsby' IN BOOLEAN MODE));
From what I can tell, only one index gets used, and the WHOLE table is searched for the fulltext result, as opposed to just doing a fulltext search over user 2's records.
I was wondering how to set the above up having the user ID as a subquery, from which the fulltext search is then made, and if that would use the two indexes?
Thanks for your time and help.
To make sub queries you put a query where you would logically put a table:
SELECT
k2.*
FROM
(SELECT
K.*
FROM
k_table AS K
WHERE K.userID=2) k2
WHERE
MATCH (K2.compColumn) AGAINST ('+gatsby' IN BOOLEAN MODE);