I have a dictionary database in which every entery looks like this
column
word; synonym; synonym | example of usage; example of usage
I want to make a select function that will only get the row if it appears in the first part of the data (words and synonyms) and not in the examples of usage (as there are more words there)
I have been trying to do it with REGEXP
SELECT * FROM dictionary WHERE column REGEXP '[^\|]*word.*\|.*'
But for some reason this matches everything in the tables - even where the word doesn't appear at all.
What am I doing wrong?
You need to use double slashes to escape special characters in MySQL regex. Thus, your \| is treated as a | and you match every empty space before each "character" in each string.
I suggest just checking if word appears before | with
'^[^|]*word'
or - if you need a whole word check:
'^[^|]*[[:<:]]word[[:>:]]'
The regex matches...
^ - start of string
[^|]* - 0 or more characters other than |
[[:<:]] - leading word boundary
word - literal sequence of letters
[[:>:]] - trailing word boundary.
Also, this regex is case-insensitive by default. To make it case-sensitive, use BINARY keyword.
SELECT * FROM dictionary WHERE column REGEXP BINARY '^[^|]*[[:<:]]word[[:>:]]'
What am I doing wrong?
You don't have a database. You have 3 types of information (word, synonym, usage) jammed into one column in one row.
You need a table of word - synonym pairs
You need a table of word - usage pairs
You probably need a word - definition pair table
Do all the parsing before inserting the data into the tables.
Related
I need 2 regex case insensitive patterns. One of them are equivalent of SQL's %. So %word%. My attempt at this was '^[a-zA-Z]*word[a-zA-Z]*$'.
Question 1: This seems to work, but I am wondering if this is the equivalent of %word%.
Finally the last pattern being similar to %, but requires 3 or more characters either before and after the word. So for example if the target word was word:
words = matched because it doesn't have 3 or more characters either before or after it.
swordfish = not matched because it has 3 or more characters after word
theword = not matched because it has 3 or more characters before it
mywordly = matched because it doesn't contain 3 or more characters before or after word.
miswordeds = not matched because it has 3 characters before it. (it also has 3 words after it too, but it met the criteria already).
Question 2: For the second regex, I am not very sure how to start this. I will be using the regex in a MySQL query using the REGEXP function for example:
SELECT 1
WHERE 'SWORDFISH' REGEXP '^[a-zA-Z]*word[a-zA-Z]*$'
First Question:
According to https://dev.mysql.com/doc/refman/8.0/en/string-comparison-functions.html#operator_like
With LIKE you can use the following two wildcard characters in the pattern:
% matches any number of characters, even zero characters.
_ matches exactly one character.
It means the REGEX ^[a-zA-Z]*word[a-zA-Z]*$' is not equivalent to %word%
Second Question:
Change * to {0,2} to indicate you want to match at maximum 2 characters either before or after it:
SELECT 1
WHERE 'SWORDFISH' REGEXP '^[a-zA-Z]{0,2}word[a-zA-Z]{0,2}$'
And to make case insensitive:
SELECT 1 WHERE LOWER('SWORDFISH') REGEXP '^[a-z]{0,2}word[a-z]{0,2}$'
Assuming
The test string (or column) has only letters. (Hence, I can use . instead of [a-z]).
Case folding and accents are not an issue (presumably handled by a suitable COLLATION).
Either way:
WHERE x LIKE '%word%' -- found the word
AND x NOT LIKE '%___word%' -- but fail if too many leading chars
AND x NOT LIKE '%word___%' -- or trailing
WHERE x RLIKE '^.{0,2}word.{0,2}$'
I vote for RLIKE being slightly faster than LIKE -- only because there are fewer ANDs.
(MySQL 8.0 introduced incompatible regexp syntax; I think the syntax above works in all versions of MySQL/MariaDB. Note that I avoided word boundaries and character class shortcuts like \\w.)
First let me apologize I have not been successful in finding anything online with this specific scenario.
I have been using MySQL for quite some time, but I am hoping to get some clarification on a certain situation I have come across, which honestly bothers me quite a bit.
I'm trying to match a string in a MySQL column that contains both \ and % literal characters using the LIKE operator.
Inside the table I have two records:
id value
-----------------------
1 100\\%A
2 100\%A
They both contain literal special characters.
If I do a SELECT, in an attempt to only match the first record (id=1), I would expect to write the query as such:
SELECT * FROM table_name WHERE value LIKE '%0\\\\\%A'
(\\\\ to match two literal backslashes, plus a backslash before % to match a literal %)
However, It only matches the row (id=2), which makes no sense to me.
If I change the query a little to be:
SELECT * FROM table_name WHERE value LIKE '%0\\\\%A'
I would expect to match the id=1 row only, (\\\\ to match 2 literal backslashes, and the % is not literal and should represent a wildcard). But instead, it matches both rows?
row (id=2) only has a single backslash but still matches.
Is row id=2 matching because the first 2 backslashes are matching the \, the 3rd backslash is ignored for some reason, and the 4th backslash is allowing a literal match on the %?
If I do a:
SELECT * FROM table_name WHERE value LIKE '%0\\\\\\\%A'
I for some reason get row (id=1), when I would expect to get no matches whatsoever.
I'm trying to find a solution in which I can do partial matches on any series of characters accurately, including those with consecutive special characters such as the scenario above. However, I'm having an impossible time trying to plan for situations such as these.
Any input would be greatly appreciated.
Maybe this help you understand the usage of escape chars in mySQL
https://stackoverflow.com/a/27061961/634698
So I'm trying to find what "special characters" have been used in my customer names. I'm going through updating this query to find them all one-by-one, but it's still showing all customers with a - despite me trying to exlude that in the query.
Here's the query I'm using:
SELECT * FROM customer WHERE name REGEXP "[^\da-zA-Z\ \.\&\-\(\)\,]+";
This customer (and many others with a dash) are still showing in the query results:
Test-able Software Ltd
What am I missing? Based on that regexp, shouldn't that one be excluded from the query results?
Testing it on https://regex101.com/r/AMOwaj/1 shows there is no match.
Edit - So I want to FIND any which have characters other than the ones in the regex character set. Not exclude any which do have these characters.
Your code checks if the string contains any character that does not belong to the character class, while you want to ensure that none does belong to it.
You can use ^ and $ to check the while string at once:
SELECT * FROM customer WHERE name REGEXP '^[^\da-zA-Z .&\-(),]+$';
This would probably be simpler expressed with NOT, and without negating the character class:
SELECT * FROM customer WHERE name NOT REGEXP '[\da-zA-Z .&\-(),]';
Note that you don't need to escape all the characters within the character class, except probably for -.
Use [0-9] or [[:digit:]] to match digits irrespective of MySQL version.
Use the hyphen where it can't make part of a range construction.
Fix the expression as
SELECT * FROM customer WHERE name REGEXP "[^0-9a-zA-Z .&(),-]+";
If the entire text should match this pattern, enclose with ^ / $:
SELECT * FROM customer WHERE name REGEXP "^[^0-9a-zA-Z .&(),-]+$";
- implies a range except if it is first. (Well, after the "not" (^).)
So use
"[^-0-9a-zA-Z .&(),]"
I removed the + at the end because you don't really care how many; this way it will stop after finding one.
I have a column containing values as strings. I need to keep only those that contain one of the following substrings: |MB1, |MB2, |MB3, |MB4, |MB5 and |MB6.
My starting point is:
select * from table
where column like '%|MB_%';
However, this would keep any other row with values such as |MBa or others. How do I get rid of them?
P.S. I am using MySQL
You can use MySQL's regular expression pattern matching;
WHERE `column` REGEXP '\\|MB[1-6]'
The pattern '\\|[1-6]' can be analysed as follows:
\\ is the string-encoding of a literal backslash, which is the regular expression escape character, so that
| is given no special meaning (if not escaped by backslash it would signify alternation, which would lead to an invalid pattern in this case)
MB are literal characters
[1-6] represents a single character within the range 1 through to 6
I have keywords like "some-or-other" where the hyphens matter in the search through my mysql database. I'm currently using the fulltext function.
Is there a way to escape the hyphen character?
I know that one option is to comment out #define HYPHEN_IS_DELIM in the myisam/ftdefs.h file, but unfortunately my host does not allow this. Is there another option out there?
Here's the code I have right now:
$search_input = $_GET['search_input'];
$keyword_safe = mysql_real_escape_string($search_input);
$keyword_safe_fix = "*'\"" . $keyword_safe . "\"'*";
$sql = "
SELECT *,
MATCH(coln1, coln2, coln3) AGAINST('$keyword_safe_fix') AS score
FROM table_name
WHERE MATCH(coln1, coln2, coln3) AGAINST('$keyword_safe_fix')
ORDER BY score DESC
";
From here http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
One solution to find a word with a dashes or hyphens in is to use FULL TEXT SEARCH IN BOOLEAN MODE, and to enclose the word with the hyphen / dash in double quotes.
Or from here http://bugs.mysql.com/bug.php?id=2095
There is another workaround. It was recently added to the manual:
"
Modify a character set file: This requires no recompilation. The true_word_char() macro
uses a “character type” table to distinguish letters and numbers from other
characters. . You can edit the contents in one of the character set XML
files to specify that '-' is a “letter.” Then use the given character set for your
FULLTEXT indexes.
"
Have not tried it on my own.
Edit: Here is some more additional info from here http://dev.mysql.com/doc/refman/5.0/en/fulltext-boolean.html
A phrase that is enclosed within double quote (“"”) characters matches only rows that contain the phrase literally, as it was typed. The full-text engine splits the phrase into words and performs a search in the FULLTEXT index for the words. Prior to MySQL 5.0.3, the engine then performed a substring search for the phrase in the records that were found, so the match must include nonword characters in the phrase. As of MySQL 5.0.3, nonword characters need not be matched exactly: Phrase searching requires only that matches contain exactly the same words as the phrase and in the same order. For example, "test phrase" matches "test, phrase" in MySQL 5.0.3, but not before.
If the phrase contains no words that are in the index, the result is empty. For example, if all words are either stopwords or shorter than the minimum length of indexed words, the result is empty.
Some people would suggest to use the following query:
SELECT id
FROM texts
WHERE MATCH(text) AGAINST('well-known' IN BOOLEAN MODE)
HAVING text LIKE '%well-known%';
But by that you need many variants depending on the used fulltext operators. Task: Realize a query like +well-known +(>35-hour <39-hour) working week*. Too complex!
And do not forget the default len of ft_min_word_len so a search for up-to-date returns only date in your results.
Trick
Because of that I prefer a trick so constructions with HAVING etc aren't needed at all:
Instead of adding the following text to your database table: "The Up-to-Date Sorcerer" is a well-known science fiction short story. copy the hyphen words without hypens to the end of the text inside a comment: "The Up-to-Date Sorcerer" is a well-known science fiction short story.<!-- UptoDate wellknown -->
If the users searches for up-to-date remove the hyphen in the sql query:
MATCH(text) AGAINST('uptodate ' IN BOOLEAN MODE)
By that you're user can find up-to-date as one word instead of getting all results that contain only date (because ft_min_word_len kills up and to).
Of course before you echo the texts you should remove the <!-- ... --> comments.
Advantages
the query is simpler
the user is able to use all fulltext operators as usual
the query is faster.
If a user searches for -well-known +science MySQL treats that as not include *well*, could include *known* and must include *science*. This isn't what the user expected. The trick solves that, too (as the sql query searches for -wellknown +science)
Maybe simpler to use the Binary operator.
SELECT *
FROM your_table_name
WHERE BINARY your_column = BINARY "Foo-Bar%AFK+LOL"
http://dev.mysql.com/doc/refman/5.0/en/cast-functions.html#operator_binary
The BINARY operator casts the string following it to a binary string. This is an easy way to force a column comparison to be done byte by byte rather than character by character. This causes the comparison to be case sensitive even if the column is not defined as BINARY or BLOB. BINARY also causes trailing spaces to be significant.
My preferred solution to this is to remove the hyphen from the search term and from the data being searched. I keep two columns in my full-text table - search and return. search contains sanitised data with various characters removed, and is what the users' search terms are compared to, after my code has sanitised those as well.
Then I display the return column.
It does mean I have two copies of the data in my database, but for me that trade-off is well worth it. My FT table is only ~500k rows, so it's not a big deal in my use case.