We're using Doctrine 2 and have some fairly simple search requirements in which we'd normally set fulltext on a field. I've had a good google around and it seems there is no way to do this with Doctrine.
We're using Zend Framework 2 for this project and I wondered if anyone had any ideas of a good workaround?
I don't think using lots of LIKEs in the query would yeild fast enough search results, but I think at the same time using something like Solr or Elastic search would be way too overkill for searching just one field in a simple manner.
Any suggestions? I get the feeling we're going to have to hack something together. At the moment we're creating the database by running the orm:schema:create tool from the command line.
Any suggestions?
Simply said, there is no solution if you stick with Doctrine2 and you don't want to use LIKE queries and you don't want to introduce a search engine.
Doctrine2 relies on InnoDB and InnoDB (currently) does not support fulltext. So focusing on fulltext or LIKE queries are no option I'd say. However, there is a much simpler way than using Solr or ElasticSearch, as they are both using Lucene as engine. You can create a Lucene index on your file system (within your project dir) and use ZendSearch for indexing and querying.
Require zendframework/zendsearch via composer and do this for your search:
use ZendSearch\Lucene\Lucene;
use ZendSearch\Lucene\Document;
use ZendSearch\Lucene\Document\Field;
// this is relative to your project say /var/www/site/data/search
$dir = 'data/search';
// Create index
Lucene::create($dir);
// Insert a new document
$index = Lucene::open($dir);
$doc = new Document;
$doc->addField(Field::keyword('foo', 'bar'));
$index->addDocument($doc);
// Search the index
$index = Lucene::open($dir);
$result = $index->query('foo:bar');
echo count($result);
There is no need to install a binary on your server (like Solr and ElasticSearch) to support search. It's even faster than fulltext search, but you have to keep your index up2date of course to support proper search.
Related
I have my own non English dictionary with words in CSV format where every line represents a word. A single line starts with the word's base form followed by all its inflected forms.
I would like to use that file to create my own dictionary that sould be used by the FULLTEXT search.
I prefer to use MySQL with that FULLTEXT search but if it is not possible to use MySQL with a custom dictionary then I can switch my project to PostgreSQL or MariaDB.
How to make MySQL or other RDBMS use custom dictionary for FULLTEXT search?
It would be nice to have a solution that works also with SQLite so it could be deployed to mobile apps as well.
So far I found related links that unlucky don't say how to do that:
MySQL Stemming for full-text Status: Un-Assigned
PostgreSQL Dictionaries
If using a Lemma dictionary for FULLTEXT search is not possible with any of these RDBMS I think It should be possible to create at least a Stemming dictionary for the PostgreSQL as its extension but I haven't done any extension for PostgreSQL and don't know where to start especially in the context of creating your own dictionary for the FULLTEXT search.
If you curently use MySQL then you could use the ngram plugin as specified in ngram Full-Text Parser (you may also want to see this article also).
Regarding the PostgreSQL link that you provided I don't think that you actually have any questions on that, it is straight forward how to configure it.
I've a web app developed by java. Currently I'm in a part of my app that I need to use MySql like in order to search for a string in mysql table contain 100000+ rows. When I had my research I found that MySql like doesn't use indexes but if you have the wildcard at the end of your string example: hello% but I need %hello% which like doesn't use index in these kinds of wildcards. And I also read on the internet that there are other technologies such as postgresql which can give you the ability of using indexes for searching string.
My question is Just because of like do I need to change MySql DB with all it's other features to postgresql DB, Do we have any alternative way on MySql To search for a string that uses indexes?, Do I Install them both and use each for it's own use ( If there is no other way );
All replies are much appreciated.
Do we have any alternative way on MySql To search for a string
Have you looked into MySQL Full-Text Search which uses fulltext index; provided you are using either InnoDB or MyISAM engine
I have a text field in my database and a index on it for the first 10 characters. How do I specify that in my Doctrine Entity?
I can't find any information about database specific options for indexes anywhere :/
This is my "partial" MySQL create statement:
KEY `sourceaddr_index` (`sourceaddr`(10)),
And this is my #Index in doctrine:
#ORM\Index(name="sourceaddr_index", columns={"sourceaddr"}, options={}),
This dosnt interfere with the regular use, but I noticed the problem when deploying development to a new laptop, and creating the database based on my entities...
Any help would be appreciated :)
Possible since Doctrine 2.9, see: https://github.com/doctrine/dbal/pull/2412
#Index(name="slug", columns={"slug"}, options={"lengths": {191}})
Unfortunately, Doctrine seem to be very picky with whitespace location, so e.g. update --dump-sql yields:
DROP INDEX slug ON wp_terms;
CREATE INDEX slug ON wp_terms (slug(191));
and even if you execute those, they messages will stay there (tested with MariaDB 10.3.14).
I've had very good luck naming the index in Doctrine, after manually creating it in MySQL. It's not pretty or elegant, and it's prone to cause errors moving from dev to production if you forget to recreate the index. But, Doctrine seems to understand it respect it.
In my entity, I have the following definition. Doctrine ignores the length option - it's wishful thinking on my part.
/**
* Field
*
* #ORM\Table(name="field", indexes={
* #ORM\Index(name="field_value_bt", columns={"value"}, options={"length": 100})
* })
And in MySQL, I execute
CREATE INDEX field_value_bt ON field (value(100))
As far as I've seen, Doctrine just leaves the index alone so long as it's named the same.
In short: you can't set this within Doctrine. Doctrine's ORM is specifically focused on cross vendor compatability and the type of index you're describing, though supported in many modern RDBMS, is somewhat outside the scope of Doctrine to handle.
Unfortunately there isn't an easy way around this if you use Doctrine's schema updater (in Symfony that would be php app/console doctrine:schema:update --force) as if you manually update the database, Doctrine will sometimes, regress that change to keep things in sync.
In instances where I've needed something like this I've just set up a fixture that sends the relevant ALTER TABLE statement via SQL. If you're going to be distributing your code (i.e. it may run on other/older databases) you can wrap the statement with a platform check to make sure.
It's not ideal but once your app/software stabilises, issues like this shouldn't happen all that often.
I created a fuzzy search in C# for a PostgreSQL database using the similarity() function from the pg_trgm module. Now I want to port this search to a MySQL database, but MySQL has no similar trigram functionality.
Is there a way to import the pg-trgm module from PostgreSQL in MySQL or is there a similar implementation of Trigrams for MySQL?
Unfortunately I was not able to find any satisfying implementation yet.
I am reluctant to use a external search engine like Solr due to the effort of installation, maintenance and becoming acquainted with the syntax and configuration.
I know this question is old, but I got here Google searching for this and there is a bit of new information I also found.
As of Mysql 5.7.6, there is built in support for using nGram in full text searches.
Mysql team article regarding ngram searches
I am not familiar with PostgreSQL but if you are using Innodb storage engine then make the column in which you are tring to search "Full-Text" search. then you can use the following search syntax to search your contant
SELECT subject, MATCH (subject) AGAINST ("Find This String") AS relevance_notes
FROM yourTable
http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html#function_match
I have to implement a keyword search from a CLOB column in my MySQL / JPA project.
I know that I can use a JPA query something like SELECT something FROM something WHERE..., but are there any other 'Enterprise' way of doing this?
(I am asking this question keeping Hibernate search in mind, but there seems to be no equivalent for Hibernate search in JPA)
What do you mean under "enterprise"-way? Hibernate is pretty enterprisish as most of Fortune-500 companies actually use that in one or another way. Though latest Hibernate Search version is still in beta, that's probably not what most of the enterprises will accept. But there is a stable release you can probably use.
And you can still use Apache Lucene to index your CLOBs and search in the index instead of DB. That's what basically Hibernate Search is also doing under the hood. And that's the aproach used by many companies.
UPDATE: I never used Hibernate Search as a separate product and what they say in their documentation is that Hibernate Core is a requirement. But you can still try plain Lucene instead.