This might seem to be a redundant question but i can't find the right answer to this issue.
I have a TableA with more than 50 columns.I am implementing a search functionality for searching a query in about 10 columns of this table. TableA contains more than a million rows
For this I have created a composite index on these 10 columns.
index (col1,col_2,col_3,col_4,col_5,col_6,col_7,col_8,col_9,col_10)
Now i am splitting user's query using space as regex. i.e. $search_words = $search_query.split(' '); and using individual words to match in my search query. Example :
SELECT something FROM tableA
WHERE ( MATCH ( col_1, col_2,col_3,col_4,col_5,col_6,col_7,col_8,col_9,col_10 )
AGAINST ( ' +word1* +word2* +word3* +word4* ' IN BOOLEAN MODE ) )
This query works fine for general searches but if users searches for individual alphabets in query like A E I O Co. it takes too much time. What is the best way to optimise the query or another way to perform search in this situation?
If you feed a too-short string to InnoDB's FULLTEXT, it returns zero results. So... Filter out any strings that are shorter than innodb_ft_min_token_size.
If necessary, test for them separately using REGEXP '[[:<:]]A[[:>:]] to look for a 1-letter word A.
Or throw them together. This would check for the only 1-letter English words: REGEXP '[[:<:]][AI][[:>:]]
I am currently looking into using FULLTEXT indexes in MySQL for search functionality within a web site.
Basically, the user can go to an advanced search page, and select 1 or more columns to search against, e.g. they can search Title, Description and Comments or either only 1 column or a mixture of the three and when they perform the search these selected columns are searched for against the keywords.
I had created 1 index for the title, 1 index for the description and 1 index for the comments and then tried to run the following query:
SELECT * FROM support_calls WHERE MATCH(Title, Description) AGAINST('+these, +are, +some, +keywords')
I got an error from MySQL saying that the MATCH didn't match any fulltext indexes and I found that I need to create an index which included Title and Description together instead of having them in separate indexes.
This is going to add some complexity if this is the case as I am going to have to create an index for every single variation of what columns the user selects. Am I going about this the right away or is there a better solution?
first execute below query and then run your MATCH() query.
ALTER TABLE support_calls ADD FULLTEXT (
Title, Description
)
i'm not very very experimented with the indexes so that's why i'm asking this silly question. i searched like everywhere but i didn't get a clear answer.
I will have a table items with columns: id,name,category,price
Here will be 3 indexes:
id - Primary Index
name - FullText Index
category,price - Composite Index
I estimate my table in future will get like 700.000-1.000.00 rows.
I need to do a fulltext search for name and where category is a specified category and order by price.
So my query will be this:
SELECT * FROM items
WHERE MATCH(name) AGAINST(‘my search’) and category='my category' order by price
My question is:
How many index will be used to perform this search?
It will use 2 indexes?
[fulltext index] & [category,price] index - Will get results for words and then will use the next index to match my category and price order
It will use 1 index
[fulltext index] only - Will get results for words, but after will have to manually match my category and price order
I want my query to be fast, what are you opinions? I know the fulltext search is fast, but what happen if i apply clauses like: category and price order? will be same fast?
MySQL will only ever use one index in any search. The reason being that using two indexes will require two searches. This will make the query much more slower. You can force MySQL to use a specific index in a query but this is not a good idea.
In summary: MySQL will only ever use one index it cant use two indexes.
I have a database with 75,000+ rows with 500+ entries added per day.
Each row has a title and description.
I created an RSS feed which gives you the latest entries for a specific search term (ex. http://site.com/rss.rss?q=Pizza would output an RSS for the search term "Pizza").
I was wondering what would be the best way to write the SQL query for this. Right now I have:
SELECT *
FROM 'table'
WHERE (('title' LIKE %searcherm%) OR ('description' LIKE %searcherm%))
LIMIT 20;
But the problem is it takes between 2 to 10 seconds to execute the query.
Is there a better way to write the query, do I have to cache the results (and how would I do that?) or would changing something in the database structure speed up the query (indexes?)
A relatively simple solution for this would be incorporating a FULLTEXT index on these two fields and subsequently searching by using this index.
ALTER TABLE table ADD FULLTEXT(title, description);
Then would you need to perform a search, you'd do the following:
SELECT id FROM table
WHERE MATCH (title, description) AGAINST ('keyterm');
Fulltext indexed search is the automatic solution included in most SQL databases. It's much speedier comparing to doing LIKES. This is also optimized for your specific case because you are only interested in natural language search terms.
As well, fulltext index has some limiting algorithm for detecting relevancy. You can read more about it here
EDIT
In the alter statement, I missed the fulltext index name, it should be:
ALTER TABLE table ADD FULLTEXT ft_index_name(title, description);
Try:
SELECT * FROM table
WHERE MATCH (title,description) AGAINST (searchterm);
Make sure you add a full text index on title, description together.
Dont try to reinvent the wheel. MATCH and AGAINST are provided by mysql to do exactly that and to make your life easy. However, note full text search works on MyISAM tables. You can workaround for InnoDb too. You can simply add FT index by altering table like:
ALTER TABLE table ADD FULLTEXT(title,description);
If you're using a query with LIKE '%term%' the indexes can't be used. They can be used only if you use a query like 'term%'. Think about an address book with tabs, you can find really fast contacts starting with letter L, but to find contacts with a on somewhere in the word, you've to scan the whole addressbook.
The better alternative could be to use full text indexes:
CREATE FULLTEXT INDEX title_desc
ON table (title, description)
And then in the query:
SELECT title, description FROM table
WHERE MATCH (title, description) AGAINST ('+Pizza')
I would go with JohnB's or gtr32x's answer (Full Text Indexing). To complement their answer, there's a manual way to create a simple full text index that's simple and it's super fast...
Split title and description into keywords, and place them in a Keywords table, which has a foreign key to the original RSS article. Make sure the keyword column in Keywords is indexed. The you can do something like:
SELECT DISTINCT ra.*
FROM RssArticle ra
INNER JOIN Keywords k ON k.ArticleID = ra.ArticleID
WHERE k IN ( 'SearchTerm1', 'SearchTerm2', 'SearchTerm3')
LIMIT 20;
And it's fast!
Try either of the following four queries:
select * from myTable where concat_ws(' ',title,description) like '%pizza%';
select * from myTable where concat_ws(' ',title,description) regexp '.*pizza+.*';
select title,description from myTable where concat_ws(' ',title,description) like '%pizza%';
select title,description from myTable where concat_ws(' ',title,description) regexp '.*pizza+.*';
the point is to use concat before searching
A few pointers: Drop the * in your select statement and pull only the searched criteria, and make sure to add indexes to the columns that are getting searched.
SELECT `title`,`description`
FROM `table`
WHERE `title` LIKE '%$searchterm%' OR `description` LIKE '%$searchterm%' LIMIT 25;
Did you create an index for title and for description?
You should consider Sphinx for Full Text Search capabilities.
Thanks for the comment Tyler.
I restate my answer:
1) Create an index on title and description columns, but your query would be limited to the example below, and that's not ideal for finding all relevant rows:
SELECT *
FROM 'table'
WHERE title LIKE 'searcherm%' OR description LIKE 'searcherm%'
LIMIT 20;
2) As others have mentioned, use MySQL Full-Text Search, but you are limited to MyISAM table engine, as it isn't available for InnoDB. However, you can mix engines in MySQL, so you can make this table MyISAM, even if all your other tables are InnoDB.
3) Use an external Full-Text Search engine, such as Sphinx. This will give you more relevant search results (MySQL Full-Text Search leaves much to be desired), it will perform better, and it abstracts the burden of Full-Text Searching away from your database.
I am trying to create a simplified search box that will match multiple columns against a keyword or keywords the user inputs. The following code is my attempt at using MySQL's MATCH/AGAINST. However, I cannot get the query to execute properly when there are two columns specified, in this case 'topic, country'. It will execute when I run the code as either 'country' or 'topic', but not both.
Is there a secret to this?
SELECT * ,
MATCH (
topic, country
)
AGAINST (
'China'
) AS score
FROM reports2
WHERE MATCH (
topic, country
)
AGAINST (
'China'
)
ORDER BY score DESC
yields:
#1191 - Can't find FULLTEXT index matching the column list
Which I find to be an inaccurate description of the error, because there are FULLTEXT indices for both of these. I even copied the table, turned it into MyISAM from INNODB, in order to do so.
Any suggestions would be appreciated.
You don't need fulltext indices for each column, but a single fulltext index covering both columns
FULLTEXT_INDEX(topic, country)
Having a single index on each column will not work
FULLTEXT(topic); FULLTEXT(country); /* will not work as expected */
I also think that order is important, but I could be mistaken in that regard