I am trying to optimize the search engine of my wensite. My data base contains phrases. #word is defined as varchar and contains a similar phrase to the one which exist in the DB.
I want if the given search phrase matches exactly to the entry in DB the entry will be chosen, if nothing could be found, search with like '%phrase%' and if nothing will be find with this method, then the MATCH columnx against (phrase) method should be used. This is the code I tried with:
select distinct columnx
from tabley
where
( CASE when columnx LIKE #word is not null
then columnx LIKE #word
when columnx like concat('%',#word,'%') is not null
then columnx like concat('%',#word,'%')
else MATCH (columnx) AGAINST (#word) END
);
To make sure if the cases on a standalone select query works fine I used them separately in where clause. I get result for these queries:
select distinct columnx from view_materialisiert where MATCH (columnx) AGAINST (#word);
and
select distinct columnx from view_materialisiert where columnx like concat('%',#word,'%');
And as expected no result for:
select distinct columnx from view_materialisiert where columnx like #word;
The question is why i dont get any result when I use the case condition at all?
If you want values that match on any of the three condition, you can use boolean logic:
select columnx
from tabley
where columnx = #word
or columnx like concat('%', #word, '%')
or match(columnx) against (#word)
Or you can extend the case logic:
where case
when columnx = #word then 1
when columnx like concat('%', #word, '%') then 1
when match(columnx) against (#word) then 1
end
However I am unsure that's really what you expect. It seems like you want to iteratively check the conditions, trying the next one only when the preceding had no match on the entire table. This type of logic would be typically implemented with union all and not exists:
select columnx
from tabley
where columnx = #word
union all
select columnx
from tabley
where columnx like concat('%', #word, '%')
and not exists (select 1 from tabley where columnx = #word)
union all
select columnx
from tabley
where match(columnx) against (#word)
and not exists (select 1 from tabley where columnx = #word)
and not exists (select 1 from tabley where columnx like concat('%', #word, '%'))
How the database will optimize such query is highly dependent on your data and other factors. In the best case scenario, the following members will be "skipped" as soon as one member returns anything (because the not exists subquery is quite explicit about that), but there is no guarantee. You would need to assess the performance on your actual dataset.
Do only
MATCH (columnx) AGAINST ("+word" IN BOOLEAN MODE)
It is very fast. That expression eliminates the need for the exact match and some of the other cases.
The OR approaches mentioned are quite inefficient. They will check every row, usually with every test.
Caveats:
"Short" words cannot be used.
"Stop" words cannot be used.
If you aren't careful about those limitations, you will get nothing or everything. So do some preprocessing to decide when MATCH will work.
Related
(MySQL)
I have a query to check if 'phone_number' or 'fax_number' startsWith a value from a given array,
lets say const possibleValues = [123,432,645,234]
currently my query runs with the 'or' condition, to check if -
'phone_number' or 'fax_number' that starts with 123
or
'phone_number' or 'fax_number' that starts with 432
or
'phone_number' or 'fax_number' that starts with 645
or
'phone_number' or 'fax_number' that starts with 234
it runs extremely slow on a big database, and I wish to make it faster,
is there a way to make it run faster?
I'm kinda new to sql queries,
any help would be highly appreciated!
You can try something like:
SELECT * FROM table_1
WHERE CONCAT(',', `phone_number`, ',') REGEXP ',(123|432|645|234),'
OR CONCAT(',', `fax_number`, ',') REGEXP ',(123|432|645|234),';
Demo
Try creating an in-line table and join with it.
WITH telnostart(telnostart) AS (
SELECT '123'
UNION ALL SELECT '432'
UNION ALL SELECT '645'
UNION ALL SELECT '234'
)
SELECT
*
FROM your_search_table
JOIN telnostart ON (
LEFT(tel_number,3) = telnostart
OR LEFT(fax_number,3) = telnostart
you can use a case statement to add a flag column
select *
,case when left(phone_number,3) in (123,432,645,234) or left(fax_number,3) in (123,432,645,234) then 1 else 0 end as contact_check_flag
from table_name
As per your requirement, you can filter it or use it elsewhere.
SELECT * FROM table_1
WHERE `phone_number` REGEXP '^(123|432|645|234)'
OR `fax_number` REGEXP '^(123|432|645|234)';
But it won't be fast. (And no regular INDEX will help.)
If there phone numbers are spelled out like in the US: "123-456-7890", then you could use a FULLTEXT(phone_number, fax_number) index and
SELECT * FROM table_1
WHERE MATCH(phone_number, fax_number)
AGAINST('123 432 645 234');
This is likely to be much faster, but not as "general".
I want to get all the rows that are the prefix of 'abcd' in mysql.
This query will get the job done, but it will not using the index of field1.
select field1
from table1
where 'abcd' LIKE CONCAT(field1, '%')
PS: this query will get field1='a' field1='ab' field1='abc'
Is there a way to get the same query result and use the index?
I assume you don't want field1='' to match?
Here's a way to get the index to be somewhat useful:
WHERE field1 BETWEEN 'a' AND 'abcd'
AND 'abcd' LIKE 'field1%'
To be slightly more maintainable, given that #x is 'abcd':
WHERE field1 BETWEEN LEFT(#x, 1) AND #x
AND #x LIKE CONCAT(field1, '%')
It may be that #variables won't work; in that case, conspire to use a JOIN:
SELECT a.field1
FROM ( SELECT 'abcd' AS x ) init
JOIN table1
WHERE field1 BETWEEN LEFT(init.x, 1) AND init.x
AND init.x LIKE CONCAT(field1, '%')
(and that may not be good enough)
A third approach would be to use a Stored Procedure to construct my WHERE suggestion with literals, then execute it.
I have created a subquery that searches for a particular string from one table, using the SQL LIKE condition. I would like to use this subquery's result as the string to search for in my main SQL query also using the LIKE condition. I tried the below code but I get syntax errors, although it seems to be the way it should be done...sadly I am not an SQL expert and just trying to feel this out.
SELECT * FROM `allcesseries`
WHERE series_id LIKE '%'+(SELECT industry_code FROM `ceindustry` WHERE industry_name LIKE '%Technical and trade schools%')+'%'
SELECT * FROM `allcesseries`
WHERE series_id LIKE concat('%',
(SELECT industry_code FROM `ceindustry`
WHERE industry_name LIKE '%Technical and trade schools%'),
'%')
I would suggest that you use exists in this case:
SELECT *
FROM `allcesseries` a
WHERE EXISTS (SELECT 1
FROM `ceindustry` c
WHERE c.industry_name LIKE '%Technical and trade schools%' AND
a.series_id LIKE CONCAT('%', c.industry_code, '%')
);
If you have multiple matches, then this will work as expected.
You can also phrase this directly as a join, if you want:
SELECT a.*
FROM `allcesseries` a JOIN
ceindustry c
ON c.industry_name LIKE '%Technical and trade schools%' AND
a.series_id LIKE CONCAT('%', c.industry_code, '%')
But if there are multiple rows that satisfy the conditions in ceindustry, you will get duplicates.
I am creating a "simple" search query. I'd like to select a column that should have the value 1 if a specified column LIKE('test'), it's kind of hard to explain.
What i want to do is like this
SELECT *,(CASE mycol LIKE('%test%') as match THEN 1 END) FROM mytable
So if mycol matches the condition, then match=1, if else 0. How would i do that? Sorry if hard to understand.
You are nearly there, but you have made four errors and used one bad practice:
Add ELSE 0 in your CASE expression.
CASE expressions need one or more WHEN expressions.
Put the alias in the correct place - after the end of the expression.
Don't use the reserved word match as an alias.
Also, don't use SELECT *.
Try this:
SELECT
col1,
col2,
...,
coln,
CASE WHEN mycol LIKE '%test%' THEN 1 ELSE 0 END AS zipmatches
FROM mytable
However there's no need for the CASE expression in MySQL because TRUE and 1 are equivalent:
SELECT
col1,
col2,
...,
coln,
mycol LIKE '%test%' AS zipmatches
FROM mytable
Here is another expression
SELECT *, IF(mycol LIKE('%test%'),1,0) is_a_match FROM mytable;
I have the following problem with a MySQL query in C#:
Given a list of strings, I want to query the database for any rows that match said strings. The strings are unique in that each string matches no more than one row. Today, my query looks something like this:
SELECT Id FROM SomeTable
WHERE SomeColumn IN("foo", "bar", "baz")
Now, ideally I would like to be able to map the result from the query directly to the list of strings I supplied in the IN clause:
String Returned ID
------------------------------------------
foo 123
bar NULL <-- Missing row filled with NULL
baz 42
This works fine as long as all strings I pass to the query match a row. When one is missing, however, I would like to fill in the blank with a NULL as in the example above.
Is there any way to accomplish this?
Edit: I should probably have pointed out that the solution must scale to a lot of strings. The way I do it right now is that I pass 100 at a time through the IN clause.
You could do this:
SELECT
helper.SomeColumn,
SomeTable.Id
FROM
(
SELECT 'foo' AS SomeColumn
UNION SELECT 'bar'
UNION SELECT 'baz'
) AS helper
LEFT JOIN SomeTable ON SomeTable.SomeColumn = helper.SomeColumn
Of course you can create the helper table (as a temp table) beforehand instead of inline.
Anyway, maybe it is smarter and more efficient to just do the query you have (WHERE SomeColumn IN (...)) and simply figure out the missing rows in your application. You will loop over them anyway, so you will notice.
What you could do is SELECT the set of strings as a result set and then LEFT JOIN on SomeTable.SomeColumn.
Try this:
SELECT Id
FROM (
SELECT "foo" SomeColumn
UNION ALL
SELECT "bar" AS SomeColumn
UNION ALL
SELECT "baz" AS SomeColumn
) b
LEFT JOIN
SomeTable a
ON a.SomeColumn = b.SomeColumn