I've a web app developed by java. Currently I'm in a part of my app that I need to use MySql like in order to search for a string in mysql table contain 100000+ rows. When I had my research I found that MySql like doesn't use indexes but if you have the wildcard at the end of your string example: hello% but I need %hello% which like doesn't use index in these kinds of wildcards. And I also read on the internet that there are other technologies such as postgresql which can give you the ability of using indexes for searching string.
My question is Just because of like do I need to change MySql DB with all it's other features to postgresql DB, Do we have any alternative way on MySql To search for a string that uses indexes?, Do I Install them both and use each for it's own use ( If there is no other way );
All replies are much appreciated.
Do we have any alternative way on MySql To search for a string
Have you looked into MySQL Full-Text Search which uses fulltext index; provided you are using either InnoDB or MyISAM engine
Related
I have my own non English dictionary with words in CSV format where every line represents a word. A single line starts with the word's base form followed by all its inflected forms.
I would like to use that file to create my own dictionary that sould be used by the FULLTEXT search.
I prefer to use MySQL with that FULLTEXT search but if it is not possible to use MySQL with a custom dictionary then I can switch my project to PostgreSQL or MariaDB.
How to make MySQL or other RDBMS use custom dictionary for FULLTEXT search?
It would be nice to have a solution that works also with SQLite so it could be deployed to mobile apps as well.
So far I found related links that unlucky don't say how to do that:
MySQL Stemming for full-text Status: Un-Assigned
PostgreSQL Dictionaries
If using a Lemma dictionary for FULLTEXT search is not possible with any of these RDBMS I think It should be possible to create at least a Stemming dictionary for the PostgreSQL as its extension but I haven't done any extension for PostgreSQL and don't know where to start especially in the context of creating your own dictionary for the FULLTEXT search.
If you curently use MySQL then you could use the ngram plugin as specified in ngram Full-Text Parser (you may also want to see this article also).
Regarding the PostgreSQL link that you provided I don't think that you actually have any questions on that, it is straight forward how to configure it.
This is a follow-up from this question. Although I can write a non-binary LIKE query such as - SELECT COUNT(*) FROM TABLE WHERE MID LIKE 'TEXT%' in raw SQL, I would like to know if it's possible through the Django ORM.
Both startswith and contains seem to be using a binary pattern search.
Try istartswith and icontains, which in MySQL resolve to LIKE rather than LIKE BINARY.
Note that with MySQL, the case-sensitivity of the comparison depends on the collation set in the database (meaning that i lookups may still be case-sensitive!).
I created a fuzzy search in C# for a PostgreSQL database using the similarity() function from the pg_trgm module. Now I want to port this search to a MySQL database, but MySQL has no similar trigram functionality.
Is there a way to import the pg-trgm module from PostgreSQL in MySQL or is there a similar implementation of Trigrams for MySQL?
Unfortunately I was not able to find any satisfying implementation yet.
I am reluctant to use a external search engine like Solr due to the effort of installation, maintenance and becoming acquainted with the syntax and configuration.
I know this question is old, but I got here Google searching for this and there is a bit of new information I also found.
As of Mysql 5.7.6, there is built in support for using nGram in full text searches.
Mysql team article regarding ngram searches
I am not familiar with PostgreSQL but if you are using Innodb storage engine then make the column in which you are tring to search "Full-Text" search. then you can use the following search syntax to search your contant
SELECT subject, MATCH (subject) AGAINST ("Find This String") AS relevance_notes
FROM yourTable
http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html#function_match
We're using Doctrine 2 and have some fairly simple search requirements in which we'd normally set fulltext on a field. I've had a good google around and it seems there is no way to do this with Doctrine.
We're using Zend Framework 2 for this project and I wondered if anyone had any ideas of a good workaround?
I don't think using lots of LIKEs in the query would yeild fast enough search results, but I think at the same time using something like Solr or Elastic search would be way too overkill for searching just one field in a simple manner.
Any suggestions? I get the feeling we're going to have to hack something together. At the moment we're creating the database by running the orm:schema:create tool from the command line.
Any suggestions?
Simply said, there is no solution if you stick with Doctrine2 and you don't want to use LIKE queries and you don't want to introduce a search engine.
Doctrine2 relies on InnoDB and InnoDB (currently) does not support fulltext. So focusing on fulltext or LIKE queries are no option I'd say. However, there is a much simpler way than using Solr or ElasticSearch, as they are both using Lucene as engine. You can create a Lucene index on your file system (within your project dir) and use ZendSearch for indexing and querying.
Require zendframework/zendsearch via composer and do this for your search:
use ZendSearch\Lucene\Lucene;
use ZendSearch\Lucene\Document;
use ZendSearch\Lucene\Document\Field;
// this is relative to your project say /var/www/site/data/search
$dir = 'data/search';
// Create index
Lucene::create($dir);
// Insert a new document
$index = Lucene::open($dir);
$doc = new Document;
$doc->addField(Field::keyword('foo', 'bar'));
$index->addDocument($doc);
// Search the index
$index = Lucene::open($dir);
$result = $index->query('foo:bar');
echo count($result);
There is no need to install a binary on your server (like Solr and ElasticSearch) to support search. It's even faster than fulltext search, but you have to keep your index up2date of course to support proper search.
I'm trying to push a brand new Ruby on Rails app to Heroku. Currently, it sits on MySQL. It looks like Heroku doesn't really support MySQL and so we are considering using PostgreSQL, which they DO support.
How difficult should I expect this to be? What do I need to do to make this happen?
Again, please note that my DB as of right now (both development & production) are completely empty.
Common issues:
GROUP BY behavior. PostgreSQL has a rather strict GROUP BY. If you use a GROUP BY clause, then every column in your SELECT must either appear in your GROUP BY or be used in an aggregate function.
Data truncation. MySQL will quietly truncate a long string to fit inside a char(n) column unless your server is in strict mode, PostgreSQL will complain and make you truncate your string yourself.
Quoting is different, MySQL uses backticks for quoting identifiers whereas PostgreSQL uses double quotes.
LIKE is case insensitive in MySQL but not in PostgreSQL. This leads many MySQL users to use LIKE as a case insensitive string equality operator.
(1) will be an issue if you use AR's group method in any of your queries or GROUP BY in any raw SQL. Do some searching for column "X" must appear in the GROUP BY clause or be used in an aggregate function and you'll see some examples and common solutions.
(2) will be an issue if you use string columns anywhere in your application and your models aren't properly validating the length of all incoming string values. Note that creating a string column in Rails without specifying a limit actually creates a varchar(255) column so there actually is an implicit :limit => 255 even though you didn't specify one. An alternative is to use t.text for your strings instead of t.string; this will let you work with arbitrarily large strings without penalty (for PostgreSQL at least). As Erwin notes below (and every other chance he gets), varchar(n) is a bit of an anachronism in the PostgreSQL world.
(3) shouldn't be a problem unless you have raw SQL in your code.
(4) will be an issue if you're using LIKE anywhere in your application. You can fix this one by changing a like b to lower(a) like lower(b) (or upper(a) like upper(b) if you like to shout) or a ilike b but be aware that PostgreSQL's ILIKE is non-standard.
There are other differences that can cause trouble but those seem like the most common issues.
You'll have to review a few things to feel safe:
group calls.
Raw SQL (including any snippets in where calls).
String length validations in your models.
All uses of LIKE.
If you have no data to migrate, it should be as simple as telling your Gemfile to use the pg gem instead, running bundle install, and updating your database.yml file to point to your PostgreSQL databases. Then just run your migrations (rake db:migrate) and everything should work great.
Don't feel you have to migrate to Postgres - there are several MySQL Addon providers available on Heroku - http://addons.heroku.com/cleardb is the one I've had the most success with.
It should be simplicity itself: port the DDL from MySQL to PostgreSQL.
Does Heroku have any schema creation scripts? I'd depend on those if they were available.
MySQL and PostgreSQL are different (e.g. identity type for MySQL, sequences for PostgreSQL). But the port shouldn't be too hard. How many tables? Tens are doable.