I have a pretty simple LIKE search for MySQL that i'd like to transform into a fulltext. The problem is i need to be able to implement it so that it starts with X. Like the example below:
SELECT column FROM table WHERE column LIKE "startswith%"
as you can see that query returns all results that begins with "startswith". I need to do this with a fulltext.
Is this possible to do?
No, that isn't how fulltext works (it's actually just a list with loose words underneath, no information about location relative to the string) but there's no reason why you can't have that LIKE ... as an extra WHERE clause. FULLTEXT can still help to get a smaller subset of results if you haven't got another key on column. If you do have a key on column, using FULLTEXT for this is useless.
You can set a key on just the start of a column with ADD INDEX (column(123)); (which would only index the first 123 characters). THis also works for text/blob columns (in the latter case it's the binary length you give).
I am not sure about MySQL, but in SQL Server you can convert the column into a varchar and perform a LIKE on the result, such as below:
SELECT column FROM table WHERE CONVERT(varchar(255), column) LIKE 'startswidth%'
Since this is ANSI standard, I presume MySQL will handle this as well.
Related
I have a table and one of the column is TEXT type. I need to search the table for the rows with the text similar to the given string.
As the string can be pretty long (let's say 10000 bytes) I decided that it will be enough to compare only first 20 bytes of the string. To do this search faster I created a key:
KEY `description` (`description`(20))
So what I want to do now is one of the following query:
SELECT * FROM `table` WHERE STRCMP(SUBSTRING(`description`,0,20),'string_to_compare') = 0
or
SELECT * FROM `table` WHERE `description` LIKE 'string_to_compare%')
Note that I put only one percentage sign at the end of string_to_compare for saying to DB that I want to compare only first bytes.
I hope that MySQL brains will do the best to use key and not to do any extra moves.
Questions:
Is there any difference which query is better? I'm personally prefer the second as it looks clearer and hopefully will be better understand by the DB engine (MyISAM).
Is that correct the MySQL MyISAM will make an efficient code for these queries?
How do I put '%' in the PDO's prepare statement? SELECT * FROM table
WHERE description LIKE ":text%"?
Yes, there's a difference. When the WHERE condition calls a function on the column value, indexes can't be used. I don't think it will realize that your SUBSTRING() call happens to match the indexed part of the text and use that. On the other hand, LIKE is specifically coded to recognize the cases where it can use an index. Also, if you want to compare two strings for equality, you should use =, not STRCMP(), e.g.
WHERE SUBSTRING(`description`,1,20) = 'string_to_compare'
I believe it will make an efficient query for the LIKE version.
The placeholder can't be in quotes for it to work. Use CONCAT() to combine it: WHERE description LIKE CONCAT(:text, '%'). Or you can put the % at the end of the PHP variable that you bind to the placeholder, and use WHERE description LIKE :text.
I have strings like the following in my VARCHAR InnoDB table column:
"This is a {{aaaa->bbb->cccc}} and that is a {{dddd}}!"
Now, I'd like to search for e.g. {{xxx->yyy->zzz}}. Brackets are part of the string. Sometimes searched together with another colum, but which only contains an ordinary id and hence don't need to be considered (I guess).
I know I can use LIKE or REGEXP. But these (already tried) ways are too slow. Can I introduce a fulltext index? Or should I add another helping table? Should I replace the special characters {, }, -, > to get words for the fulltext search? Or what else could I do?
The search works with some ten-thousand rows and I assume that I often get about one hundred hits.
This link should give you all the info you need regarding FULLTEXT indexes in MySQL.
MySQL dev site
The section that you will want to pay particular attention to is:
"Full-text searching is performed using MATCH() ... AGAINST syntax. MATCH() takes a comma-separated list that names the columns to be searched. AGAINST takes a string to search for, and an optional modifier that indicates what type of search to perform. The search string must be a string value that is constant during query evaluation. This rules out, for example, a table column because that can differ for each row."
So in short, to answer your question you should see an improvement in query execution times by implementing a full text index on wide VARCHAR columns. Providing you are using a compatible storage engine ( InnoDB or MyISAM)
Also here is an example of how you can query the full text index and also an additional ID field as hinted in your question:
SELECT *
FROM table
WHERE MATCH (fieldlist) AGAINST ('search text here')
AND ( field2= '1234');
I'm curious how this is handled. I have a varchar(255) column named title that has the first 10 characters indexed.
If I create a MySQL query that orders by title, does it take advantage of that index solely, or does it ignore it, or do I have to issue a command to use that index?
For example let's say I have two title items named:
This is the same thing
This is the same thing only slightly different
The first 10 chars are the same, so how does MySQL handle that? Does it use the prefix index up to the first 10 chars and then row by row index afterwards?
Well, it's said that:
The index can also be used even if the ORDER BY does not match the
index exactly, as long as all of the unused portions of the index and
all the extra ORDER BY columns are constants in the WHERE clause.
But I strongly suggest checking this query with EXPLAIN, to see which path the optimizer will choose.
I have this "log-queries-not-using-indexes" enabled in MySQL. I have one query which is being logged by MySQL as such i.e. query that is not using indexes. The thing is this query uses LIKE for e.g.
category like '%fashion%'
and if I remove LIKE or change it to
category = 'fashion'
then it says it is using indexes.
So when we are using LIKE in our query, MySQL will log it as not using indexes no matter what?
Thanks
Using a clause like %fashion% will never use a regular index. You need a full-text index if you want to do that kind of search.
Remember that a varchar indexes on first part of the string. So, if you are searching for an ocurrence of fashion on any part of the string, then index will offer no help to improve performance since you will need to search every single string.
However, if you only search for the first part like this:
select * from table where field like 'fashion%'
Then index could be used and could be helpful.
If I store an HTML TEXTAREA in my ODBC database each time the user submits a form, what's the SELECT statement to retrieve 1) all rows which contain a given sub-string 2) all rows which don't (and is the search case sensitive?)
Edit: if LIKE "%SUBSTRING%" is going to be slow, would it be better to get everything & sort it out in PHP?
Well, you can always try WHERE textcolumn LIKE "%SUBSTRING%" - but this is guaranteed to be pretty slow, as your query can't do an index match because you are looking for characters on the left side.
It depends on the field type - a textarea usually won't be saved as VARCHAR, but rather as (a kind of) TEXT field, so you can use the MATCH AGAINST operator.
To get the columns that don't match, simply put a NOT in front of the like: WHERE textcolumn NOT LIKE "%SUBSTRING%".
Whether the search is case-sensitive or not depends on how you stock the data, especially what COLLATION you use. By default, the search will be case-insensitive.
Updated answer to reflect question update:
I say that doing a WHERE field LIKE "%value%" is slower than WHERE field LIKE "value%" if the column field has an index, but this is still considerably faster than getting all values and having your application filter. Both scenario's:
1/ If you do SELECT field FROM table WHERE field LIKE "%value%", MySQL will scan the entire table, and only send the fields containing "value".
2/ If you do SELECT field FROM table and then have your application (in your case PHP) filter only the rows with "value" in it, MySQL will also scan the entire table, but send all the fields to PHP, which then has to do additional work. This is much slower than case #1.
Solution: Please do use the WHERE clause, and use EXPLAIN to see the performance.
Info on MySQL's full text search. This is restricted to MyISAM tables, so may not be suitable if you wantto use a different table type.
http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
Even if WHERE textcolumn LIKE "%SUBSTRING%" is going to be slow, I think it is probably better to let the Database handle it rather than have PHP handle it. If it is possible to restrict searches by some other criteria (date range, user, etc) then you may find the substring search is OK (ish).
If you are searching for whole words, you could pull out all the individual words into a separate table and use that to restrict the substring search. (So when searching for "my search string" you look for the the longest word "search" only do the substring search on records containing the word "search")
I simply use SELECT ColumnName1, ColumnName2,.....WHERE LOCATE(subtr, ColumnNameX)<>0
To get rows with ColumnNameX having the substring.
Replace <> with = to get rows NOT having the substring.