I am working with MySQL full text search but find it lacking in situations where your string is part of a word within a field. If my field is "New York Times" and I search for "Time" I get no results. The hackish way to solve this is to set up two queries, one that does a full text search and the other that does:
SELECT * FROM ___ WHERE 'string' LIKE %searchterm%
Is there any way that I can set up my full text search to solve this issue so I don't have to run the extra query?
You can use wild cards in your full text search. More info here
SELECT * FROM _____ WHERE MATCH (title) AGAINST ('time*' IN BOOLEAN MODE);
I've basically given up on MySql's full text search in favor of Sphinx -- a dedicated full-text search engine which implements MySql's network protocol so you can "interpose" it between your clients and a MySql server, losing nothing and gaining rich, serious full-text capabilities. Maybe it's not suitable for your needs (which you don't fully express), but I think it's at least work checking out!
Related
Is there any way to search on specific text in MySQL without using the Full Text Search
I know LIKE is a solution but using wildcard at the beginning will disable using indexes , therefore not best performance for large data
Please specify more details of your use case.
Meanwhile, I have found this to be beneficial in some use cases. For example, suppose you wanted to search for a bracketed word:
WHERE MATCH(col) AGAINST('+word' IN BOOLEAN MODE)
AND col LIKE '%[word]%'
The MATCH would rapidly find the few rows with "word", then the LIKE would slowly check those few rows. It gives reasonably fast overall speed while checking for some types of non-words.
Here is the scenario:
I want to do a keyword query in MySQL with big data amount, which is in 10 million level.
The match is just to judge whether the keyword is a substring of the current specified field.
If there is a string: "A BC DEF", and the keyword is "BC", then it is matched. Just this simple, but I want it to be as quickly as possible. Because this is gonna applied to a website's search module （with relatively high concurrency）, I don't want the user to wait for a long time.
Could anyone give me an idea? Thanks a lot!
P.S. I've searched things about fulltext in MySQL, as well as some search engines like Lucene and Sphinx, which one is better and more appropriate to apply? My web project is based on Java EE. Thanks!
Consider using MySQL Full-Text Search Functions
http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html
Then you can use a SQL Query like this:
SELECT * FROM articles
WHERE MATCH (title, body) AGAINST ('BC');
I've looked for this question on stackoverflow, but didn't found a really good answer for it.
I have a MySQL database with a few tables with information about a specific product. When end users use the search function in my application, it should search for all the tables, in specific columns.
Because the joins and many where clauses where not performing really well, I created a stored procedure, which splits all the single words in these tables and columns up, and inserts them in the table. It's a combination of 'word' and 'productID'.
This table contains now over 3.3 million records.
At the moment, I can search pretty quick if I match on the whole word, or the beginning of the word (LIKE 'searchterm%'). This is obvious, because it uses an index right now.
However, my client want to search on partial words (LIKE '%searchterm%'). This isn't performing at all. Also FULLTEXT search isn't option, because it can only search for the beginning of a word, with a wildcard after it.
So what is the best practice for a search function like this?
While more work to set up, using a dedicated fulltext search package like Lucene or Solr may be what you are looking for.
MySQL is not well tailored for text search. Use other software to do that. For example use Sphinx to index data for text search. It will do a great job and is very simple to set up. If you user MySQL 5.1 you could use sphinx as an engine.
There are other servers for performing text search better than Spinx, but they are eather not free or require other software installed.
You can read more about: ElasticSearch, Sphinx, Lucene, Solr, Xapian. Which fits for which usage?
I'm building a small search with PHP. Here's an example query:
SELECT * FROM tools
WHERE name LIKE "%example%"
ORDER BY name;
A couple of names:
this example will be found
this example1 will not be found
It only finds the rows, if the string that is being searched is a separate word. Any advice? :)
Thanks,
Martti Laine
I ran the following script
CREATE TABLE tools (
name TEXT,
FULLTEXT (name)
) ENGINE=MyISAM;
INSERT tools VALUES('this example will be found');
INSERT tools VALUES('this example1 will not be found');
SELECT * FROM tools
WHERE name LIKE "%example%"
ORDER BY name;
Which gave the output
this example will be found
this example1 will not be found
I even got the same output when running the SELECT without the FULLTEXT index.
Can you post the DDL for the creation of your database and any errors you receive?
Also, does the SQL execute correctly in MySQL Workbench or have you only tried it from your PHP code?
(I realise this is not strictly an answer to your question but posting as an answer allows for better formatting of the text)
EDIT: After seeing the comment from ajreal I also tried
SELECT * FROM tools
WHERE MATCH(name) AGAINST ('example')
and this time it did not return any rows...
That's the way it's supposed to work. MySQL's Full Text Search using what's known as stemming to build the search index. So it'll take a word like shopping and store shop (since it's the same word). That way, when you search for one variant of the word, it can find all variants of the word.
However, in your case, that may not be the greatest situation. Well, for that, you have 2 possibilities:
Use LIKE with Full Text searching.
Well, LIKE performs a direct case-insensitive search. Full Text Search performs a stemmed word search. Both have significant flaws. But together they will let you hit a broader scope of results.
Use a 3pd search engine tool such as Sphinx or SOLR.
This will likely be the most efficient option, since these tools are both very good in how they search, and are very fast. Basically, the tools indexes the MySQL database from time to time (depending on the setup), and then you search against the tool instead of MySQL. This is the most robust option, and the most flexible.
I am trying to build a product search for a jewelry store. I know that if a term is in over 50% of the entries then it has a weight of zero. So right now if I do a search for "diamond" I get no results because over 50% contain diamond. Is there a way to change that?
Quoting the documentation of MySQL : 11.9.6. Fine-Tuning MySQL Full-Text Search
If you really need to search for such
common words, it would be better to
search using IN BOOLEAN MODE instead,
which does not observe the 50%
threshold.
See : 11.9.2. Boolean Full-Text Searches
The other solution seems to go with patching MySQL's source-code and recompiling -- which is probably not something you want to do...
Another approach, commented on MySQL website, is to use boolean mode only if the fulltext gives no results, but keep in mind that the second search won't sort the results in order of relevance.
11.9.1. Natural Language Full-Text Searches