I just use following code right now.
SELECT terms FROM searches WHERE MATCH(q) AGAINST('search term') LIMIT 20;
The table is MyISAM 90MB. terms has FULLTEXT INDEX and it is varchar(255). There are 1,000,000 rows on the table.
I wonder if there is any solution which is more resource usage friendly than fulltext search on MySQL? Especially in terms of memory.
By saying solution, I refer to any solution such as other types of databases, table structures etc.
and if the solution would be adaptable to a standart VPS or hosting in general, it would be extremely super duper perfect!
Thanks for your time!
I would check out Apache Solr. You can continue to use your MySQL database, have the Solr server index that column and use the Solr server to later do full-text searches on that column. There are even hosted solutions, see WebSolr.
Related
SELECT firstname, lastname,comments
FROMusers
WHERE MATCH(firstname,lastname,comments)
AGAINST('$searchterm')
I tried the above one as query for search engine ,but mysql says FULL-TEXT INDEXING is supported only on MYISAM ,Engine i am using is innoDB,Please tell me the best way of coloumn INDEX searching ON InnoDB Engine.
Based on my knowledge, MySQL FTS has been available for InnoDB since version 5.6 (http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html)
You should take a look at MySQL Fulltex search document here http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html
Basically, I think you should understand what is 'index' and how MySQL do indexing. This article is very useful for helping understand the mechanism behind MySQL fulltext search http://dev.mysql.com/doc/internals/en/full-text-search.html
There are several important concepts in full-text search:
Boolean mode
Natural language mode
I also recommend you read about stopwords list in MySQL FTS.
System variable likes ft_min_word_len is also important.
After understanding these things, I think you will know how to apply MySQL fulltext search properly.
I read that MySQL fulltext search can cause table locking. It means people can't insert or update the table when it's being searched on.
I read that there are many search servers (Lucence and Sphinx) can do it without table locking and even faster. It requires many configuration and hard to implement.
Is there any other way to use fulltext or some searching like that without using search service? I don't want to configure one more server other than MySQL.
Create an extra table which will be used only to perform FULLTEXT searches. In your code you have to ensure that all data and actions (create, update, delete) are properly replicated to this table. This solution is also handy if your data tables are running e.g. InnoDB engine.
Apache Lucene doesn't need many configuration and isn't hard to implement. Moreover, it's one of the most popular fulltext search engine, and allows the users to do very precise queries, like "to be or not to be", j?hn d?e, func*, etc.
I already did some database indexing with Lucene, so if you could be a bit more precise about which fields of which tables you wanna index, I can give you pieces of code which should do the trick.
I vote for Sphinxsearch anyway. It has one of APIs close to Mysql, easy to install and configure. Not so universal as Apache Lucene, but jet quick and very helpful in my projects.
I know, that InnoDB tables do not support fulltext searches, yet. So I thought of using a third party search engine like solr, xapian or whoosh. Do those third party tools work equivalently fine with InnoDB tables as they work with MyIsam tables? I need to find e.g. spelling suggestions, and similar strings...
You could use Solr/Lucene to do the fulltext-search over your DB data. Since my MySQL DB is to big for an fast fulltext-search, i decided to combine mysql and Solr/lucene.
It's important to know, that Solr/Lucene is not an MySQL Plugin. So you will not be able to search the fulltext-index by using typical MySQL SQL-Statements. An fulltext-search, initiated by the application, should be first send the request to the 3rd party fulltext-index (Solr), which returns the primary keys of the related documents. Second step is to run an SQL statement against your MySQL innoDB with an where clause with the corresponding primary keys from the Solr result set.
That solution works in my case very well and much, much faster (and better) than an typical MySQL Myisam fulltext-search.
As an alternative you could not only index the data in solr. You also could store the data in solr additionally. In that case, solr is able to return the full text. So you don't need get the data form the database, as in the example above.
Do those third party tools work equivalently fine with InnoDB tables as they work with MyIsam tables?
Absolutely. Solr has an DataImportHandler. Ther you define an SQL statement in order to get the data you like to index in solr, like: select * from MyTable;
But keep in mind: right now (as far as I know) ther is no MySQL solr plugin available. The cooperation of Solr and MySQL should be handled by the application.
Third-party fulltext search engines typically copy data returned by a MySQL query, and use it to populate their search index. There's no difference between MyISAM and InnoDB data sources in this respect.
I gave a presentation Practical Full-Text Search in MySQL a few years ago. You might find it interesting.
Sphinx supports its own index and just takes data from MySQL on a timely basis by issuing a query.
It is not even aware of the underlying table structure and as long as the query runs and returns the results, it's OK for Sphinx.
Other third party engines work in a similar way.
I have a question that , i use the index by the mysql database itself to index is better or using the index function in sphinx index.
it depends.
sphinx is a better index by all standards but 1. it's faster, has a more advanced syntax, is smaller, is more scalable, doesn't involve myisam.
mysql is more maintainable, simpler to install, does not involve another tier to your application, and if it's good enough you might as well use it.
Sphinx provides you full text search option. You can say its a mini search engine embedded in your app. And with no doubts its better Mysql Index.
If you just want to index auto incremented integer columns then there seems no point adding sphinx in your app. Still database size matters.
Checkout some the previously asked questions to get better idea what suits your needs.
https://stackoverflow.com/questions/tagged/sphinx?sort=votes
MySQL's default fulltext search sucks when the data volume is big.
So I'm counting on sphinx to get rid of the trouble.
Is there a benchmark data available that compares these two kinds of searches?
This claims 50-100x, up to 1000x, while this has even more drastic results. I've used both on dbs, and in my experience, there really isn't a comparison; you just can't use FULLTEXT for a large database, while sphinx is both fast and accurate.