I realize that MySQL 5.6 is still in beta, but does anyone have experience using the new InnoDB FTS engine? How does it compare to something like Sphinx?
Thanks
Jason
Never used Sphinx, but tried MySQL 5.6 FTS on an Innodb table with about 170k rows. Made an FTS index on the name column (contains all names of a person). To find a word in any position of the string MATCH(name) AGAINST("+word*") IN BOOLEAN MODE does work a lot faster (2-3 times in my case) than using name LIKE "word%" OR name LIKE "% word". However when making joins do check EXPLAIN to see if the FTS index is actually used. It seems MySQL optimizer is not that good at guessing when the FTS index should be used.
The FULLTEXT feature that formerly required downloading a special build from labs.mysql.com is now part of the mainline MySQL build in 5.6.5 and up (still in beta). The documentation for the FULLTEXT functions now includes the InnoDB-specific details: MySQL Full-Text Search Functions
Remember, that Sphinx search is developed for full text searching in mysql it's just a feature...
Here you have compare of sphinx and mysql FTS:
http://www.percona.com/files//presentations/opensql2008_sphinx.pdf
Here is performance test of InnoDB FTS compared to MyISAM:
http://blogs.innodb.com/wp/2011/07/innodb-fts-performance/
InnoDB its bit faster especially in indexing, but it's still far away from sphinx performance...
Related
In PostgreSQL, we can search table based on full text search like this -
SELECT title
FROM pgweb
WHERE to_tsvector('english', body) ## to_tsquery('english', 'friend');
Source - http://www.postgresql.org/docs/current/static/textsearch-tables.html
How can we do similar search in MySQL 5.5 which is quite easily done in PostgreSQL?
You probably want MySQL's full text search functionality. Essentially you create a FULLTEXT index then search against it using MATCH() ... AGAINST.
I'm not aware of a facility to set the search language per-query in MySQL, but that doesn't mean no such support exists. It wasn't clear if per-query language settings were a requirement for you.
The latest stable release of MySQL supports full text search on the modern transactional and crash-safe InnoDB table type as well as the unsafe MyISAM table type. If your MySQL only does FTS on MyISAM it's time to upgrade. 5.6 supports full text search on InnoDB.
Alternately, if you really can't upgrade, you can store your important data in InnoDB tables and run a periodic query to update a MyISAM table you use as a materialized view for fulltext search only:
Create a new MyISAM table
INSERT INTO ... SELECT the data from the InnoDB table into the new MyISAM table
CREATE the fulltext index on the new MyISAM table
DROP the old MyISAM table you were using for fulltext indexing; and
finally ALTER TABLE ... RENAME the new MyISAM table to have the name of the old one.
You'll have a very short window during which the fulltext index is unavailable between when you drop the old table and re-create the new one. Your data also gets out of date and stale between view refreshes, though it's possible you can work around that with triggers (I don't use MySQL enough to know). If you can't live with these limitations, upgrade to 5.6.
MySQL's full text search offers control of stopwords and other tuning. It's a solid offering that should do the job nicely.
SELECT firstname, lastname,comments
FROMusers
WHERE MATCH(firstname,lastname,comments)
AGAINST('$searchterm')
I tried the above one as query for search engine ,but mysql says FULL-TEXT INDEXING is supported only on MYISAM ,Engine i am using is innoDB,Please tell me the best way of coloumn INDEX searching ON InnoDB Engine.
Based on my knowledge, MySQL FTS has been available for InnoDB since version 5.6 (http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html)
You should take a look at MySQL Fulltex search document here http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html
Basically, I think you should understand what is 'index' and how MySQL do indexing. This article is very useful for helping understand the mechanism behind MySQL fulltext search http://dev.mysql.com/doc/internals/en/full-text-search.html
There are several important concepts in full-text search:
Boolean mode
Natural language mode
I also recommend you read about stopwords list in MySQL FTS.
System variable likes ft_min_word_len is also important.
After understanding these things, I think you will know how to apply MySQL fulltext search properly.
I'm currently doing research on the best ways to provide an advanced search for my php project.
I have narrowed it down to using FULLTEXT search rather than using LIKE to find matching strings in a table row. However, to do this it appears I need to sacrifice using the InnoDB engine which will make me lose the ACIDity of transactions and table relationships.
Is it really worth using the MYISAM mysql engine or are there better ways of providing search functionality.
Any pointers would be appreciated!
It really depends on the application... Using MyISAM for anything that needs referential integrity is an instant fail. At the same time, it's text search isn't all that efficient.
Basically, there are two ways to go. If you find you don't need true referential integrity, consider a NoSQL datastore. MongoDB is a great document store database.
If, on the other hand, you really need referential integrity, but also need fast, indexed full-text searching, you might do better to use Sphinx or Apache Solr to create an indexed cache for full-text search.
Either way, I consider MyISAM to be a legacy datastore. I wouldn't use it on a new project. YMMV.
MyISAM has several drawbacks - lack of transaction support, table-level locks which makes it very slow in heavy read+write load type. Another inconvenience of MyISAM tables - they are not crash safe so you can lost some data in case of unexpected shutdown or power loss on server. However MyISAM is very fast on some queries.
Regarding the FullText search I would suggest to use InnoDB + external search engine like Lucene or Sphinx so you could benefit from both safe and reliable storage engine and fast Full-text queries.
For quick start with InnoDB and Sphinx you can refer to http://astellar.com/2011/12/replacing-mysql-full-text-search-with-sphinx/
MySQL 5.6 supports FULLTEXT indexes with InnoDB (released Feb 2013). See:
http://dev.mysql.com/doc/refman/5.6/en/fulltext-search.html
I've done some searching into fulltext searches for MySQL InnoDB and found a few on stack overflow, but they either don't provide an exact solution or they are a bit dated and I think it's time to rediscuss.
All of my tables are InnoDB and I'd prefer not to lock the entire database with MyISAM. What are my options in regards to fulltext searching? Are there any simple solutions? I'd like to do the equivalant of MATCH (content) AGAINST ("my search query" IN BOOLEAN MODE)
There is no equivalant of MATCH AGAINST in Innodb. Fulltext searching is one of pros of using MyIsam. So you should use standalone seaching server like Sphinx or Solr
InnoDB full-text search (FTS) is finally available in MySQL 5.6.4 release.
These indexes are physically represented as entire InnoDB tables, which are acted upon by SQL keywords such as the FULLTEXT clause of the CREATE INDEX statement, the MATCH() ... AGAINST syntax in a SELECT statement, and the OPTIMIZE TABLE statement.
From FULLTEXT Indexes
I know, that InnoDB tables do not support fulltext searches, yet. So I thought of using a third party search engine like solr, xapian or whoosh. Do those third party tools work equivalently fine with InnoDB tables as they work with MyIsam tables? I need to find e.g. spelling suggestions, and similar strings...
You could use Solr/Lucene to do the fulltext-search over your DB data. Since my MySQL DB is to big for an fast fulltext-search, i decided to combine mysql and Solr/lucene.
It's important to know, that Solr/Lucene is not an MySQL Plugin. So you will not be able to search the fulltext-index by using typical MySQL SQL-Statements. An fulltext-search, initiated by the application, should be first send the request to the 3rd party fulltext-index (Solr), which returns the primary keys of the related documents. Second step is to run an SQL statement against your MySQL innoDB with an where clause with the corresponding primary keys from the Solr result set.
That solution works in my case very well and much, much faster (and better) than an typical MySQL Myisam fulltext-search.
As an alternative you could not only index the data in solr. You also could store the data in solr additionally. In that case, solr is able to return the full text. So you don't need get the data form the database, as in the example above.
Do those third party tools work equivalently fine with InnoDB tables as they work with MyIsam tables?
Absolutely. Solr has an DataImportHandler. Ther you define an SQL statement in order to get the data you like to index in solr, like: select * from MyTable;
But keep in mind: right now (as far as I know) ther is no MySQL solr plugin available. The cooperation of Solr and MySQL should be handled by the application.
Third-party fulltext search engines typically copy data returned by a MySQL query, and use it to populate their search index. There's no difference between MyISAM and InnoDB data sources in this respect.
I gave a presentation Practical Full-Text Search in MySQL a few years ago. You might find it interesting.
Sphinx supports its own index and just takes data from MySQL on a timely basis by issuing a query.
It is not even aware of the underlying table structure and as long as the query runs and returns the results, it's OK for Sphinx.
Other third party engines work in a similar way.