I have a problem. I have a managed VPS server running MySQL 5.1.x. I am currently building a new database where I want to store tweets (via search, stream, timeline etc). So I want to use InnoDB database engine because of the row locking! But unfortunately, MySQL 5.1 does not support Full-text search in InnoDB tables.
The problem is that I cannot update my server by myself. So I cannot install MySQL 5.6 (that should support Full-text search) and I cannot install lucene (or solr or whatever).
Are there other options to achieve Full-text search in MySQL or whatever. Or maybe in PostgreSQL (never used that before)
The only other option I have so far is going to an unmanaged VPS but I don't prefer that :)
You can take a look to MyISAM engine but it's not transactional.
The other other possible solution is create another table with engine MyISAM or ARIA and create a relationship between the new table and the "store tweets" table, so before insert into the 'new table' make sure that 'store tweets' it's not locked.
Related
In PostgreSQL, we can search table based on full text search like this -
SELECT title
FROM pgweb
WHERE to_tsvector('english', body) ## to_tsquery('english', 'friend');
Source - http://www.postgresql.org/docs/current/static/textsearch-tables.html
How can we do similar search in MySQL 5.5 which is quite easily done in PostgreSQL?
You probably want MySQL's full text search functionality. Essentially you create a FULLTEXT index then search against it using MATCH() ... AGAINST.
I'm not aware of a facility to set the search language per-query in MySQL, but that doesn't mean no such support exists. It wasn't clear if per-query language settings were a requirement for you.
The latest stable release of MySQL supports full text search on the modern transactional and crash-safe InnoDB table type as well as the unsafe MyISAM table type. If your MySQL only does FTS on MyISAM it's time to upgrade. 5.6 supports full text search on InnoDB.
Alternately, if you really can't upgrade, you can store your important data in InnoDB tables and run a periodic query to update a MyISAM table you use as a materialized view for fulltext search only:
Create a new MyISAM table
INSERT INTO ... SELECT the data from the InnoDB table into the new MyISAM table
CREATE the fulltext index on the new MyISAM table
DROP the old MyISAM table you were using for fulltext indexing; and
finally ALTER TABLE ... RENAME the new MyISAM table to have the name of the old one.
You'll have a very short window during which the fulltext index is unavailable between when you drop the old table and re-create the new one. Your data also gets out of date and stale between view refreshes, though it's possible you can work around that with triggers (I don't use MySQL enough to know). If you can't live with these limitations, upgrade to 5.6.
MySQL's full text search offers control of stopwords and other tuning. It's a solid offering that should do the job nicely.
I'm using MySQL 5.3 and have no possiblilty to update it to a newer version (5.6 supports full-text search for innodb).
So my question is, should I user Myisam instead so I can use full-text search for my forums table?
The users need to be able to search in the table or should I solve it someway else, (what way?).
EDIT: Or should I use LIKE in SQL?
SELECT firstname, lastname,comments
FROMusers
WHERE MATCH(firstname,lastname,comments)
AGAINST('$searchterm')
I tried the above one as query for search engine ,but mysql says FULL-TEXT INDEXING is supported only on MYISAM ,Engine i am using is innoDB,Please tell me the best way of coloumn INDEX searching ON InnoDB Engine.
Based on my knowledge, MySQL FTS has been available for InnoDB since version 5.6 (http://dev.mysql.com/tech-resources/articles/whats-new-in-mysql-5.6.html)
You should take a look at MySQL Fulltex search document here http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html
Basically, I think you should understand what is 'index' and how MySQL do indexing. This article is very useful for helping understand the mechanism behind MySQL fulltext search http://dev.mysql.com/doc/internals/en/full-text-search.html
There are several important concepts in full-text search:
Boolean mode
Natural language mode
I also recommend you read about stopwords list in MySQL FTS.
System variable likes ft_min_word_len is also important.
After understanding these things, I think you will know how to apply MySQL fulltext search properly.
I know, that InnoDB tables do not support fulltext searches, yet. So I thought of using a third party search engine like solr, xapian or whoosh. Do those third party tools work equivalently fine with InnoDB tables as they work with MyIsam tables? I need to find e.g. spelling suggestions, and similar strings...
You could use Solr/Lucene to do the fulltext-search over your DB data. Since my MySQL DB is to big for an fast fulltext-search, i decided to combine mysql and Solr/lucene.
It's important to know, that Solr/Lucene is not an MySQL Plugin. So you will not be able to search the fulltext-index by using typical MySQL SQL-Statements. An fulltext-search, initiated by the application, should be first send the request to the 3rd party fulltext-index (Solr), which returns the primary keys of the related documents. Second step is to run an SQL statement against your MySQL innoDB with an where clause with the corresponding primary keys from the Solr result set.
That solution works in my case very well and much, much faster (and better) than an typical MySQL Myisam fulltext-search.
As an alternative you could not only index the data in solr. You also could store the data in solr additionally. In that case, solr is able to return the full text. So you don't need get the data form the database, as in the example above.
Do those third party tools work equivalently fine with InnoDB tables as they work with MyIsam tables?
Absolutely. Solr has an DataImportHandler. Ther you define an SQL statement in order to get the data you like to index in solr, like: select * from MyTable;
But keep in mind: right now (as far as I know) ther is no MySQL solr plugin available. The cooperation of Solr and MySQL should be handled by the application.
Third-party fulltext search engines typically copy data returned by a MySQL query, and use it to populate their search index. There's no difference between MyISAM and InnoDB data sources in this respect.
I gave a presentation Practical Full-Text Search in MySQL a few years ago. You might find it interesting.
Sphinx supports its own index and just takes data from MySQL on a timely basis by issuing a query.
It is not even aware of the underlying table structure and as long as the query runs and returns the results, it's OK for Sphinx.
Other third party engines work in a similar way.
I'm running a django system over mysql in amazon's cloud, and the database default is innodb. But now I want to put a fulltext index on a couple of tables for searching, which evidently requires myisam.
The obvious solution is to just tell mysql to ALTER TABLE to myisam, but are there going to be any issues with that?
One that comes to mind is that I'll have to remember to do that any time I build a new version of the database, which should theoretically be rare, but there doesn't seem to be a way to tell django to please set the storage engine at the table level. I guess I could write a migration (we use south).
Any other things I might be missing? What could possibly go wrong?
Will the application notice? Probably not.
Will it cause problems? Only when things go wrong. MyISAM is not a transactional storage engine. If you change the data in a MyISAM table while inside of a transaction, then have to roll back changes, the changes in that table won't be rolled back. It's been a while since I tried to break it horribly, but I'm willing to wager that MySQL won't even issue a warning when this happens. This will lead to data consistency issues.
You should seriously consider external search software instead of a fulltext index, like ElasticSearch (integrates at the application level), or Sphinx (integrates at the MySQL level, though if you're using RDS instead of MySQL directly, I don't think you'll be able to use it).
the following may be of help:
use a myisam fulltext table to index back into your innodb tables for example:
Build your system using innodb:...
Any way to achieve fulltext-like search on InnoDB