I have a table with different fields, in particular name (varchar 255), surname (varchar 255), but actually I used them as name (varchar 90) and surname (varchar 70), considering the maxlength limit.
This table has ~620,000 rows and I'm doing a lot of queries like the following one:
SELECT * FROM table WHERE name LIKE "%word1%" AND surname LIKE "%word2%"
I've added FULLTEXT indexes on both field (tried also one index contains both fields), but the performance is slower than queries without any index.
What I missing?
Thanks in advance
Edit:
Following your replies, I show you some results.
Case A.
Table without any index
Query:
SELECT *
FROM `table`
WHERE `name` LIKE '%word1%'
AND `surname` LIKE '%word2%'
executed two times was solved in 0.8870 and 0.8952 sec
Case B.
I've added a FULLTEXT index on name and a FULLTEXT index on surname.
Query
SELECT *
FROM table
WHERE match(name) AGAINST ('+word1*' in boolean mode)
AND match(surname) AGAINST ('+word2*' in boolean mode)
executed two times was solved in 0.9647 and 1.0380 sec
Note: table has InnoDB engine and unfortunatly MySQL is older than 5.6; I cannot convert easily from InnoDB to MyISAM because of foreign keys.
Other ideas?
Thanks again
It will be better if you use Mysql (InnoDB)'s Full text search in boolean mode.
More Details here
Fulltext search is designed to be efficient in cases like yours and is considerably faster than full scan.
A fulltext search query for your case might look like this :
SELECT * FROM table WHERE match(name) against ('+word1*' in boolean mode) AND match(surname) against ('+word2*' in boolean mode);
"cannot convert easily" -- Sure you can. Make a MyISAM table with nothing but the text column(s) and the id. Then do the MATCH..AGAINST using that table and JOIN (without FKs) to the InnoDB table. The id is already the PRIMARY KEY of the other table, correct? The JOIN just needs an index; the PK qualifies as such; no FK is required.
You do have a 2-column FULLTEXT index, correct? FULLTEXT(name, surname)
Related
I have a column with a index of type Hash with has value from 1..128
Like this
CREATE INDEX indexName ON myTable (studentType) USING HASH;
Sometimes we have queries like this.
Select * from myTable where studentType =1;
Select * from myTable where studentType =2;
Here the index is used very good but I was reading the docs and some stuff really caught my attention, I think maybe I didn't get it well.
Only whole keys can be used to search for a row. (With a B-tree index, any leftmost prefix of the key can be used to find rows.)
Does this mean that if I have a query like this:
Select * from myTable where studentType =2 and status=121;
Here MySQL is ignoring the hash index 'studentType'? because the where clause uses other column besides the column index? Or what it says in a example.
InnoDB has no HASH index. Your declaration was turned into BTree.
Either of these BTree indexes works just fine for your last SELECT:
INDEX(studentType, status)
INDEX(status, studentType)
Background:
Hey I'm using MySQL 5.6.17 InnoDB, I've read on mysql website that FULLTEXT is now avaiable for InnoDB in 5.6+ version. so i don't have to change from InnoDB to MyISAM Here is the link I altered TWO of my table columns for FULLTEXT search by using the following query
ALTER TABLE `es_officers` ADD FULLTEXT Index_officer_name (es_officer_name)
ALTER TABLE `es_officers` ADD FULLTEXT Index_officer_fname (es_officer_fname)
Altered Table Registered in Information Schema:
Then i checked in my information schema if the altered table columns are registered or not by running the following query
SELECT TABLE_SCHEMA, TABLE_NAME, COLUMN_NAME
FROM statistics
WHERE index_type LIKE 'FULLTEXT%'
It showed me exacttly two results which i was expecting
Problem:
when i write a query to MATCH a sting AGAINST two columns it gives me an error
SELECT * FROM `es_officers` WHERE MATCH (es_officer_name, es_officer_fname) AGAINST ('abc')
#1191 - Can't find FULLTEXT index matching the column list
BUT:
when i try to query the columns separately like below it works absolutely fine
SELECT * FROM `es_officers` WHERE MATCH (es_officer_name) AGAINST ('abc')
SELECT * FROM `es_officers` WHERE MATCH (es_officer_fname) AGAINST ('abc')
I don't know what is it I'm doing wrong, help is highly appreciated.
Multiples issues:
You have two separate fulltext indexes, each covering a single field. You have WHERE MATCH (es_officer_name, es_officer_fname) as your query, which requires a SINGLE index covering both fields. MySQL will not use two separate indexes for this query - it can't. that's not how fulltext indexing works. You need an alter ... fulltext (es_officer_name, es_officer_fname) instead added.
And even then, abc will fall under the default minimum word length and won't get indexed.
Besides, you can try something like:
SELECT * FROM es_officers
WHERE MATCH es_officer_name AGAINST ('abc') + MATCH es_officer_fname AGAINST ('abc')
Though, as Marc B said, 'abc' value is too short.
I am having following simple query, but its taking too much time (.5 seconds/half of a second), I am unable to find the reason. I want to improve the performance of Search.
Title
VARCHAR (50), Collation = latin1_general_ci, NULL allowed, MySQL Engine: MyISAM
Number of record in Product table: 163129
CatID
Foreign Key (int) From Category table
I already have tried this Optimize statement
OPTIMIZE table Product
Here is the simple query.
SELECT COUNT(*)
FROM
Product
WHERE
CatID=123
AND Title = 'abc xyz'
Please help. Thanks in advance.
Taking more time meaning the table is not properly indexed and probably doing full table scan.
For this particular query a covering index would be useful
alter table Product add index cat_title_idx(CatID,Title);
Make sure to take a backup of the table before applying the index.
Having this table:
CREATE TABLE `example` (
`id` int(11) unsigned NOT NULL auto_increment,
`keywords` varchar(200) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB;
We would like to optimize the following query:
SELECT id FROM example WHERE keywords LIKE '%whatever%'
The table is InnoDB, (so no FULLTEXT for now) which would be the best index to use in order to optimize such query?
We've tried a simple :
ALTER TABLE `example` ADD INDEX `idxSearch` (`keywords`);
But an explain query shows that need to scan the whole table
if our queries where LIKE 'whatever%' instead, this index performs well, but otherwise has no value.
Is there anyway to optimize this for innoDB ?
Thanks!
Indexes are built from the start of the string towards the end. When you use LIKE 'whatever%' type clause, MySQL can use those start-based indexes to look for whatever very quickly.
But switching to LIKE '%whatever%' removes that anchor at the start of the string. Now the start-based indexes can't be used, because your search term is no longer anchored at the start of the string - it's "floating" somewhere in the middle and the entire field has to be search. Any LIKE '%... query can never use indexes.
That's why you use fulltext indexes if all you're doing are 'floating' searches, because they're designed for that type of usage.
Of major note: InnoDB now supports fulltext indexes as of version 5.6.4. So unless you can't upgrade to at least 5.6.4, there's nothing holding you back from using InnoDB *AND fulltext searches.
I would like to comment that surprisingly, creating an index also helped speed up queries for like '%abc%' queries in my case.
Running MySQL 5.5.50 on Ubuntu (leaving everything on default), I have created a table with a lot of columns and inserted 100,000 dummy entries. In one column, I inserted completely random strings with 32 characters (i.e. they are all unique).
I ran some queries and then added an index on this column.
A simple
select id, searchcolumn from table_x where searchcolumn like '%ABC%'
returns a result in ~2 seconds without the index and in 0.05 seconds with the index.
This does not fit the explanations above (and in many other posts). What could be the reason for that?
EDIT
I have checked the EXPLAIN output. The output says rows is 100,000, but Extra info is "Using where; Using index". So somehow, the DBMS has to search all rows, but still is able to utilise the index?
I have a table with a nullable datetime field.
I'll execute queries like this:
select * from TABLE where FIELD is not null
select * from TABLE where FIELD is null
Should I index this field or is not necessary? I will NOT search for some datetime value in that field.
It's probably not necessary.
The only possible edge case when index can be used (and be of help) is if the ratio of null / not-null rows is rather big (e.g. you have 100 NULL datetimes in the table with 100,000 rows). In that case select * from TABLE where FIELD is null would use the index and be considerably faster for it.
In short: yes.
Slightly longer: yeeees. ;-)
(From http://dev.mysql.com/doc/refman/5.0/en/mysql-indexes.html) - "A search using col_name IS NULL employs indexes if col_name is indexed."
It would depend on the number of unique values and the number of records in the table. If your just searching on whether or not a column is null or not, you'll probably have one query use it and one not depending on the amount of nulls in the table overall.
For example: If you have a table with 99% of the records have the querying column as null and you put/have an index on the column and then execute:
SELECT columnIndexed FROM blah WHERE columnIndexed is null;
The optimizer most likely won't use the index. It won't because it will cost more to read the index and then read the associated data for the records, than to just access the table directly. Index usage is based on the statistical analysis of a table, and one major player in that is cardinality of the values. In general, indexes work best and give the best performance when they select a small subset of the rows in the table. So if you change the above query to select where columnIndexed is not null, your bound to use the index.
For more details check out the following: http://dev.mysql.com/doc/refman/5.1/en/myisam-index-statistics.html