I have a field called myfield that contains this string:
{'Content-Language': 'en', 'X-Frame-Options': 'SAMEORIGIN', 'X-Generator': 'Drupal 7 (http://drupal.org)', 'Link': '<https://01.org/node>; rel="shortlink"', 'Some-Header-Key': 'max-age=31; addSomething', 'Content-Encoding': 'gzip'}
I want to capture the 'Some-Header-Key': 'max-age=31; addSomething' where:
1) 'Some-Header-Key', max-age are fixed values that should always be present.
2) The addSomething is a optional.
3) There may be one or more spaces between the double colon and the equal sign
4) The general formal is 'key': 'value', with either single or double quotes.
5) The ([^""|'])* to say: zero or more characters that are not single or double quotes. This to capture addSomething.
I wrote this query:
select myfield
from mytable
where mycol regexp "('|"")Some-Header-Key('|"")\s*:\s*('|"")([^""|'])*max-age\s*=\s*[0-9]+([^""|'])*('|"")";
But it does not return anything!! although myfield contains the above example string.
When I copied the field value into an external text file and run the regexp in grep, the regexp captured the string correctly.
What is wrong in MySQL? I use MySQL workbench 8.0 in Ubuntu 18.04.
Your problem is with the \s in your regex expression. Versions of MySQL prior to 8 do not support this notation, you need to use the character class [:blank:] instead i.e.
where mycol regexp "('|"")Some-Header-Key('|"")[[:blank:]]*:[[:blank:]]*('|"")([^""|'])*max-age[[:blank:]]*=[[:blank:]]*[0-9]+([^""|'])*('|"")"
In MySQL 8, you can use \s but you need to escape the backslash as MySQL uses C-style escape syntax in strings, thus \s just translates to s. So change the \s to \\s and it should work:
where mycol regexp "('|"")Some-Header-Key('|"")\\s*:\\s*('|"")([^""|'])*max-age\\s*=\\s*[0-9]+([^""|'])*('|"")"
Demo on dbfiddle
Not single or double quotes: [^'"]
Zero or more of such: [^'"]
Either a single quote or two double quotes: ('|"")
Either a double quote or two single quotes: ("|'')
One of either type of quote: ['"] or ('|")
A single-quoted string: '[^']*'
A double-quoted string: "[^"]*"
Either of the above: ('[^']*'|"[^"]*")
Next problem: How to quote a regexp string: If it contains ' or ", escape that with a backslash:
my_json REGEXP "('[^']*'|\"[^\"]*\")"
If you use something that does "binding" for you, you don't need to do the escaping. PHP has mysqli_real_escape_string and add_slashes.
But... I you are going to use JSON, you should upgrade to MySQL 5.7 or MariaDB 10.2 so you can use JSON functions instead of REGEXP.
Related
I want to use a regualr expression in a mysql search that macth a string and NOT the same string followed by a dot . character.
As an example i want to match the product code 3.4.5.1 but not 3.4.5.1.4 I have a workaround using a double like :
SELECT * from mytable where myclolumn like '%3.4.5.1%' AND mycolumn NOT like '%3.4.5.1.%'....
Not a very elegant solution.
So I was trying to use REGEXP_LIKE(mycolumn,'(3.4.5.1)[^\.]')
In fact I have tested that pattern and works nicely in php and even online : https://regexr.com/66olc
but seems that regex works in their own way in MYSQL because I obtain an empty search result.
What I'm doing wrong?
You can use
([^0-9]|^)3\.4\.5\.1([^.0-9]|$)
See the regex demo. Details:
([^0-9]|^) - a non-digit char ([^0-9]) or (|) start of string (^)
3\.4\.5\.1 - 3.4.5.1 literal string (dots in regex must be escaped with a literal backslash to match a . char, else, . matches any single char)
([^.0-9]|$) - any char other than a . and digit ([^.0-9]), or (|) end of string ($).
"Groups of digits separated by dots":
([0-9]+[.])+[0-9]+
Explanations:
[0-9]+ -- 1 or more digit; you could change "+" to "{1,2}" to say "1 or 2"
[.] -- a single dot; this would work, too: \.
(...)+ -- 1 or more of what's between the parens
I need to use special character like ( \ ) character in mysql string function and unfortunately it doesn't work properly!for example couldn't search this character alone (locate-instr-substring_index-concat and even in set variable value are function that i need and test )
like thses
SELECT LOCATE("\", "Schools.co\m", 1) AS MatchPosition;
select SUBSTRING_INDEX("footba\l","\",1)
I will appreciate if anybody could help me
Backslash needs to be escaped. To fix your SUBSTRING_INDEX example, consider the following:
SELECT SUBSTRING_INDEX("footba\\l","\\",1) FROM dual
Here, backslash has to be escaped both in the string literal and in the text to match.
To escape a literal backslash inside a LIKE expression, use four backslashes, e.g.
SELECT 'match' FROM dual WHERE "footba\\l" LIKE '%\\\\%';
Demo
I tried running this query:
SELECT column FROM table WHERE column REGEXP '[^A-Za-z\-\']'
but this returns
#1139 - Got error 'invalid character range' from regexp
which seems to me like the - in the character class is not being escaped, and instead read as an invalid range. Is there some other way that it's suppose to be escaped for mysql to be the literal -?
This regex works as expected outside of mysql, https://regex101.com/r/wE8vY5/1.
I came up with an alternative to that regex which is
SELECT column FROM table WHERE column NOT REGEXP '([:alpha:]|-|\')'
so the question isn't how do I get this to work. The question is why doesn't the first regex work?
Here's a SQL fiddle of the issue, http://sqlfiddle.com/#!9/f8a006/1.
Also, there is no language being used here, query is being run at DB level.
Regex in PHP: http://sandbox.onlinephpfunctions.com/code/10f5fe2939bdbbbebcc986c171a97c0d63d06e55
Regex in JS: https://jsfiddle.net/6ay4zmrb/
Just change the order.
SELECT column FROM table WHERE column REGEXP '[^-A-Za-z\']'
#Avinash Raj is correct the - must be first (or last). The \ is not an escape character in POSIX, which is what mysql uses, https://dev.mysql.com/doc/refman/5.1/en/regexp.html.
One key syntactic difference is that the backslash is NOT a metacharacter in a POSIX bracket expression.
-http://www.regular-expressions.info/posixbrackets.html
What special characters must be escaped in regular expressions?
Inside character classes, the backslash is a literal character in POSIX regular expressions. You cannot use it to escape anything. You have to use "clever placement" if you want to include character class metacharacters as literals. Put the ^ anywhere except at the start, the ] at the start, and the - at the start or the end of the character class to match these literally
I have a set of queries with randoms data that i want to insert in database. Randoms data may have any special characters.
for example:
INSERT INTO tablename VALUES('!^'"twco\dq');
Are there any scope that escape all special characters?
please help.
No, there is no "scope" in MySQL to automatically escape all special characters.
If you have a text file containing statements that were created with potentially unsafe "random values" like this:
INSERT INTO tablename VALUES('!^'"twco\dq');
^^^^^^^^^^^
You're basically screwed. MySQL can't unscramble a scrambled egg. There's no "mode" that makes MySQL work with a statement like that.
Fortunately, that particular statement will throw an error. More tragic would be some nefariously random data,
x'); DROP TABLE students; --
if that random string got incorporated into your SQL text without being escaped, the result would be:
INSERT INTO tablename VALUES('x'); DROP TABLE students; --');
The escaping of special characters has to be done before the values are incorporated into SQL text.
You'd need to take your random string value:
!^'"twco\dq
And run it through a function that performs the necessary escaping to make that value safe for including that as part of the the SQL statement.
MySQL provides the real_escape_string_function as part of their C library. Reference https://dev.mysql.com/doc/refman/5.5/en/mysql-real-escape-string.html. This same functionality is exposed through the MySQL Connectors for several languages.
An even better pattern that "escaping" is to use prepared statements with bind placeholders, so your statement would be a static literal, like this:
INSERT INTO tablename VALUES ( ? )
You can use \ character to escape special characters like below. See this DEMO if in doubt.
INSERT INTO tablename VALUES('\!\^\'\"twco\\dq');
Per MySQL documentation, below are the defined escape sequences
Table 9.1 Special Character Escape Sequences
\0 An ASCII NUL (0x00) character.
\' A single quote (“'”) character.
\" A double quote (“"”) character.
\b A backspace character.
\n A newline (linefeed) character.
\r A carriage return character.
\t A tab character.
\Z ASCII 26 (Control+Z). See note following the table.
\\ A backslash (“\”) character.
\% A “%” character. See note following the table.
\_ A “_” character. See note following the table.
I have a table where a column allows special characters like '/' (forward slash) and '' (back slash).
Now when I try to search such records from table, I am unable to get those.
For example: abc\def or abc/def
I am generating a search query like:
select * from table1_1 where column10 like '%abc\def%'
It is returning 0 rows, but actually there is 1 record existing that should be returned. How do I write the query in this case?
The trick is to double escape ONLY the backslash; for string escapes only a single escape is needed.
For example
The single quote ' only needs escaping once LIKE '%\'%'
But to query backslash \ you need to double escape to LIKE '%\\\\%'
If you wanted to query backslash+singlequote \' then LIKE '%\\\\\'%' (with 5 backslashes)
Explanation Source
excerpt:
Because MySQL uses C escape syntax in strings (for example, “\n” to
represent a newline character), you must double any “\” that you use
in LIKE strings. For example, to search for “\n”, specify it as “\n”.
To search for “\”, specify it as “\\”; this is because the
backslashes are stripped once by the parser and again when the pattern
match is made, leaving a single backslash to be matched against.
In MySQL, this works:
select * from Table1
where column10 like '%abc\\\\def%'
FIDDLE
Backslash is an escape prefix for both strings and LIKE patterns. So you need to double it once for LIKE, and again for string literal syntax.
You have to escape the \ with another \
select * from table1_1 where column10 like '%abc\\def%'
Escaping the LIKE value didn't work when I tried it on MySQL v5.5. After a few tries, what did work was a regex (and I had to escape there too, plus hide it in a character class). I finally got it to work like this:
select * from table1_1 where column10 rlike 'abc[\\]def'
These didn't work:
... column10 like '%abc\\def%'
... column10 like concat('%abc', char(92), 'def%')
... column10 rlike 'abc\\def'
Note you also have this tagged as PL/SQL, which is Oracle. The query you posted works as-is on Oracle. Is the PL/SQL tag a mistake?
Use escaping.
https://dev.mysql.com/doc/refman/5.0/en/string-literals.html
Table 9.1. Special Character Escape Sequences
Escape Sequence Character Represented by Sequence
\0 An ASCII NUL (0x00) character.
\' A single quote (“'”) character.
\" A double quote (“"”) character.
\b A backspace character.
\n A newline (linefeed) character.
\r A carriage return character.
\t A tab character.
\Z ASCII 26 (Control+Z). See note following the table.
\ A backslash (“\”) character.
\% A “%” character. See note following the table.
_ A “_” character. See note following the table.
If you use PHP sprintf() to put together your query where you want:
AND column10 LIKE '%/%'
This worked for me inside sprintf():
AND column10 LIKE '%%/%%'
(MySSQL 8 - WAMP)