I have a question about integrating Sphinx with MariaDB 10.0. The db comes packaged with the SphinxSE storage engine, but I'm having trouble finding detailed information about how to set this up. Consider the following scenario:
I have a book-selling app using a database with the following tables: "Book", "Listing", and "School". A "listing" contains a relationship to a single book and a single school. A "book" contains several columns, most importantly "title" and "author". I would like to perform full-text searches on the "Book" table using Sphinx, and a typical query might go something like this (in pseudo-SQL): "Find all books whose title contains the string 'Harry P' for which a current listing exists at Harvard University".
According to MariaDB's article: "you need to create a special ENGINE=SPHINX 'search table', and then SELECT from it with full text query..." - so does this mean that Sphinx cannot access/index my existing MariaDB tables at all? Must I recreate my "Book" table as a SphinxSE table? If this is the case, how do I specify relationships between this table and my other non-Sphinx tables (Listing, School, etc)? The above article gives a very simple JOIN example, but I'm still confused. If all "books" are stored in a Sphinx table, can MariaDB access this table for different types of queries (where a full-text search is not needed)?
As a related question, what would be a good way to enable Sphinx to index both the Title and the Author of the books? The app will always specify which field is being queried.
Any help would be greatly appreciated, because hours of searching and reading has still left me mostly in the dark on this topic.
The SphinxSE engine is NOT a real engine. All it does is serve a 'proxy' to a underlying Sphinx daemon. It runs the query against sphinx engine, then presents the 'results' as a FAKE table, so that mysql can then join it back with the original table(s).
Basically a convenience method to get sphinx results and the apprirate data in one query (with mysql doing the hard work of doing the 'JOIN')
... so before "worrying" about sphinxSE. You need to figure out a normal Sphinx index, that you can run queries against. ie create an index to be able to run that query. Normally the sphinx query will just give you a list of ids.
So play with getting sphinx setup independantly (without SphinxSE or MariaDB) - create an index, and run some queries against it. Get that working first.
Only once you have the index working (and you understand the concepts) - come back and work out how to set it up with SphinxSE.
There are plenty of articles on getting sphinx working
https://www.google.com/search?q=sphinx+search+getting+started
Related
Is it possible to search for something that is in two databases? For example, I want to do a "starts with" search on a column in Postgres as well as a column in MySQL where one is "name" and one is "email"
Copying over data is not reliable as new data will be created in both databases constantly.
Yes, it is possible. For the "starts with" part, you should be able to use the standard Postgres string functions, of which starts_with is one, and indexing on the desired columns.
Getting the data from MySQL is the more complicated part.
You would most likely want to use a foreign data wrapper (e.g. FDW) from Postgres to access the MySQL data, and then handle the unioning of it (or other desired processing) with the Postgres data for returning the combined data set.
You could write your own FDW if you have particularly specific requirements, or you could try an open source one, such as this one from EnterpriseDB. EnterpriseDB is a Postgres consultancy and offers their own Postgres version, but the doc on the Github page for this says it is compatible with base Postgres as well as their own version.
I am currently trying to figure out how to set up a search database on localhost which is set up with sphinx which I have no experience with, nor did I create this crazy database and scripting I have to work with. I successfully have install sphinx on the computer and it connects well. Now my question is do I need to set up the same index fields as it was set up in phpMyAdmin?
Thanks
No.
You setup sphinx to index the column(s) you need to be able to search. They dont need to match mysql indexes at all.
But if you recreating a search that used to use the mysql full-text indexes, then you would create the sphinx index on the same column(s) plus any additional columns you need to be able to filter, group or sort by.
We are looking at ways of improving "search" functionality in our large business application which currently uses SQL Like syntax to do it. So we started evaluating Solr server and were able to index few of our database tables and search. But I am newbie and wanted to know if
1) We have large number of tables in our application. Is there any utility that generates schema xml in solr using the database tables?
2) Our current search lists the database row that meets the search criteria (this was written using SQL 'like' and takes lot of time to generate the search results). We want to simulate the exact functionality using solr. Is that possible?
For importing a database into SOLR, you might want to look into DataImportHandler.
There will be a fair amount of configuration required for it, defining what tables and columns to import, what should be stored, and how it should be indexed.
I'm using MySQL workbench 5.2.35 CE to reverse a Mysql Database and show the diagram of tables with their relationships.
I am able to get every table and its fields but the relations between them is not shown. Why is that? Is there any way it can be fixed?
Most often, the relations between tables are not stored in the DB. How to combine two tables is entirely up to the SQL query and because there are near endless possibilities to name your columns this is not easy to guess. Try to find some SELECT statements or use brute force to find out wich columns match.
If you are using MyISAM as the database engine, your database is not fully relational. Make your database fully relational by using InnoDB engine. Then you will be able to see the relations when you do reverse engineering. But please note that each has pros and cons of its own. Google MyISAM vs InnoDB to find out more about the differences.
I have tested mysql-workbench for both 5.2 and 6.0 latest version. My database got foreign key constraints defined clearly. My database is using mysql-native SQL. But the relationships are shown as line between tables. I have created another test database with only two tables. In this case the InoDb was used as engine. The foreign key constrain clearly defined. Still it only generate two lonely table object in the ER diagram after going through the "reverse engineering" module.
While using another commercial software, I was able to generate the 'lines' between the Entities.
My question is, is this a problem of workbench itself, or some error on my part. My database server is the latest stable version 5.6.12. If I can get an answer, this will be good for everyone. You don't have to waste time trying to get an ER-diagram with this piece of software.
I consider using Sphinx search in one of my projects so I have a few questions related to it.
When using SphinxSE and RT index, every UPDATE or INSERT in the SphinxSE table will update the index, right? No need to call indexer or anything?
Can I search on both tags (user entered keywords for a document) and the content and give more relevance to the tag matches? And if it's possible how do I implement the tag search (now I have them in separate tables like an inverted index)
For the fillter attributes is it better to stick duplicates of them in the SphinxSE table or fillter using mysql from the regular documents table I have?
Thanks in advance!
OK, I finally understand how things work with the sphinx thing.
You cannot INSERT or UPDATE directly the SphinxSE table. Instead you use INSERT/REPLACE while connected to SphinxQL (directly to sphinx daemon).
With 1.10 you can add multiple FullText searchable fields. I added title, tags and content. And the query to give more weight to the title, then tags and then content looks like this:
SELECT SQL_NO_CACHE * FROM sphinx_docs WHERE query = 'a lot of keywords;weights=3,2,1;';
I use the SQL_NO_CACHE to tell mysql not to cache the result of this, because on next calls I can't get the number of rows returned from sphinx (SHOW STATUS LIKE 'sphinx_total_found')
It's better to let sphinx do all the sorting, filltering and use mysql only to JOIN the table you need more info from.
In addition I have to say that I tried many times to add the sphinxse plugin to mysql without success (endless make waiting hours) so I switched to MariaDB 5.2.4 which includes the SphinxSE storage engine.