For example, I am having a column storing data like this.
Apple
12.5.126.40
Smite
Abby
127.0.0.1
56.5.4.8
9876543210
Notes
How to select out only the rows with data in IP format?
I have tried with '^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$'
but I have no idea why it also matches 9876543210
You're going to need to use REGEXP to match the IP address dotted quad pattern.
SELECT *
FROM yourtable
WHERE
thecolumn REGEXP '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$'
Technically, this will match values that are not valid IP addresses, like 999.999.999.999, but that may not be important. What is important, is fixing your data such that IP addresses are stored in their own column separate from whatever other data you have in here. It is almost always a bad idea to mix data types in one column.
mysql> SELECT '9876543210' REGEXP '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$';
+---------------------------------------------------------------------------+
| '9876543210' REGEXP '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$' |
+---------------------------------------------------------------------------+
| 0 |
+---------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT '987.654.321.0' REGEXP '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$';
+------------------------------------------------------------------------------+
| '987.654.321.0' REGEXP '^[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}\\.[0-9]{1,3}$' |
+------------------------------------------------------------------------------+
| 1 |
+------------------------------------------------------------------------------+
Another method is to attempt to convert the IP address to a long integer via MySQL's INET_ATON() function. An invalid address will return NULL.
This method is likely to be more efficient than the regular expression.
You may embed it in a WHERE condition like: WHERE INET_ATON(thecolumn) IS NOT NULL
SELECT INET_ATON('127.0.0.1');
+------------------------+
| INET_ATON('127.0.0.1') |
+------------------------+
| 2130706433 |
+------------------------+
SELECT INET_ATON('notes');
+--------------------+
| INET_ATON('notes') |
+--------------------+
| NULL |
+--------------------+
SELECT INET_ATON('56.99.9999.44');
+----------------------------+
| INET_ATON('56.99.9999.44') |
+----------------------------+
| NULL |
+----------------------------+
IS_IPV4() is a native mysql function that lets you check whether a value is a valid IP Version 4.
SELECT *
FROM ip_containing_table
WHERE IS_IPV4(ip_containing_column);
I don't have data, but I reckon that this must be the most solid and efficient way to do this.
There are also similar native functions that check for IP Version 6 etc.
This may not be the most efficient way, and it's not technically regex, but it should work:
SELECT col1 FROM t1 WHERE col1 LIKE '%.%.%.%';
you could also use the useful function inet_aton()
SELECT *
FROM yourtable
WHERE inet_aton(thecolumn) is not null
Lengthy but works fine:
mysql> SELECT '1.0.0.127' regexp '^([0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\\.([0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\\.([0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])\\.([0-1]?[0-9]{1,2}|2[0-4][0-9]|25[0-5])$';
Related
I have a MySQL data table which stores metadata for client transactions. I am writing a query to extract a number out of the metadata column, which is essentially a JSON stored as a string.
I am trying to find 'clients' and extract the first number after clients. The data can be stored in several different ways; see the examples below:
..."type\":\"temp\",\"typeOther\":\"\",\"clients\":\"2\",\"hours\":\"5\",\...
..."id\":31457,\"clients\":2,\"cancel\":false\...
I've tried the following Regexp:
(?<=clients.....)[0-9]+
(?<=clients...)[0-9]*(?=[[^:digit:]])
And I've tried the following json_extract, but it returned a null value:
json_extract(rd.meta, '$.clients')
The regexp functions do work, but the first one only works on the first example, while the second only works on the second example.
What regexp should I use such that it's dynamic and will pull the number nested between two non-word char sets after 'clients'?
clients.*?([0-9]+)
^^^^^^^ -- exact match
^^^ -- non-greedy string of anything
^ ^ -- capture
^^^^^^ -- string of 1 or more digits (greedy)
I did this test on MySQL 8.0.29, but it should work on MySQL 5.x too:
mysql> set #s1 = '..."type\":\"temp\",\"typeOther\":\"\",\"clients\":\"2\",\"hours\":\"5\",\...';
mysql> set #s2 = '..."id\\":31457,\\"clients\\":2,\\"cancel\\":false\\...';
mysql> select trim(leading '\\"' from substring_index(#s1, '\\"clients\\":', -1)) as clients;
+--------------------------+
| clients |
+--------------------------+
| 2\",\"hours\":\"5\",\... |
+--------------------------+
mysql> select trim(leading '\\"' from substring_index(#s2, '\\"clients\\":', -1)) as clients;
+------------------------+
| clients |
+------------------------+
| 2,\"cancel\":false\... |
+------------------------+
Then cast the result as an integer to get rid of the non-numeric part following the number.
mysql> select cast(trim(leading '\\"' from substring_index(#s1, '\\"clients\\":', -1)) as unsigned) as clients;
+---------+
| clients |
+---------+
| 2 |
+---------+
mysql> select cast(trim(leading '\\"' from substring_index(#s2, '\\"clients\\":', -1)) as unsigned) as clients;
+---------+
| clients |
+---------+
| 2 |
+---------+
Based on https://stackoverflow.com/a/59666211/4250302 I created the stored function get_enum_item for future processing the lists of possible values in the ENUM() type fields.
It works fine enough, but... but I can't determine what to do if the delimiter itself is the part of a string being split. For example:
(square brackets are for readability)
mysql> set #q=",v1,',v2'" --empty string, "v1", "comma-v2";
mysql> select concat('[',get_enum_item(#q,',',0),']') as item;
+------+
| item |
+------+
| [] |
+------+
it is OK
mysql> select concat('[',get_enum_item(#q,',',1),']') as item;
+------+
| item |
+------+
| [v1] |
+------+
it is also OK
mysql> select concat('[',get_enum_item(#q,',',2),']') as item;
+------+
| item |
+------+
| ['] |
+------+
It is not OK
the #q contains 3 commas, the first two of these are real delimiters, while the last one is the part of the third possible value: "comma-v-two". And I have no idea how to avoid confusion of splitting function. MySQL WorkBench in the "form editor" mode solves this trouble somehow, but how can I solve this with MySQL's code?
Well, I can rely on the fact that the show_columns-like queries show the enums in "hardcoded" manner:
select column_name,column_type
from information_schema.columns
where data_type='enum' and table_name='assemblies';
+--------------+------------------------------------------------------------------+
| COLUMN_NAME | COLUMN_TYPE |
+--------------+------------------------------------------------------------------+
| AssetTagType | enum('','И/Н','Н/Н',',fgg') |
| PCTagType | enum('','И/Н','Н/Н') |
| MonTagType | enum('','И/Н','Н/Н') |
| UPSTagType | enum('','И/Н','Н/Н') |
| OtherTagType | enum('','И/Н','Н/Н') |
| state | enum('в работе','на списание','списано') |
+--------------+------------------------------------------------------------------+
Thus I can try to use ',' as a delimiter, but this will not save me from the case if the "comma-apostrophe" combination is the part of possible value... :-(
The only thing I can imagine is to count apostrophes and if the delimiting comma is after the even number of ''s, then it is the delimiter, while if it follows an odd number of ''s, it is the part of the value.
And I can't invent anything except for dumb scanning the input string inside the loop. But maybe there are some other suggestions to get the values split correctly?
Please, don't suggest use PHP, Python, AWK, and so on. The query will be executed from the Pascal (Lazarus, CodeTyphoon) application, and calling external processors is highly unsafe.
As a last resort, I can process the column_type with Pascal's code, but at first, I must make myself sure that the task is not solvable by MySQL's features.
edit:
select column_type from information_schema.columns
where column_name='assettagtype' and table_name='assemblies';
+------------------------------------------+
| COLUMN_TYPE |
+------------------------------------------+
| enum('','И/Н','Н/Н',''''',fgg','''') |
+------------------------------------------+
1 row in set (0.00 sec)
Fourth field: '',fgg, fifth field: '
set #q="'в работе','на списание','списано'";
WITH RECURSIVE cte as (
select 1 as a union all
select a+1 from cte where a<35
)
select distinct regexp_substr(#q,'''[^,]*''',a) as E from cte;
Too high values for 35 raise an error ERROR 3686 (HY000): Index out of bounds in regular expression search.. (I created a bug for this)
The null value should be filtered out... 😉
output:
E
'в работе'
'на списание'
'списано'
null
EDIT: With some effort, this also works for a more complex example (not for every "staged" example!)
set #q="'в работе','на списание','списано',''',fgg'";
select #q;
WITH RECURSIVE cte as (
select 1 as a union all
select a+1 from cte where a<35
)
select distinct regexp_substr(#q,'(''([^,]|[^''][^''])*'')',a) E from cte;
output:
E
'в работе'
'на списание'
'списано'
''',fgg'
I'm simply testing MySQL AES_ENCRYPT() and AES_DECRYPT() before I start using it in my app. So I write a simple query to test it like:
SELECT AES_DECRYPT(AES_ENCRYPT('SERV92','TESTTTTTTT'),'TESTTTTTTT') AS `TEST`
I get an error because there are to few parameters in AES_ENCRYPT()
I do some research and find that my version(5.6) of MySQL does indeed take an extra parameter so I rewrite the query
SELECT AES_DECRYPT(AES_ENCRYPT('SERV92','TESTTTTTTT',RANDOM_BYTES(16)),'TESTTTTTTT',RANDOM_BYTES(16)) AS `TEST`
Result:
+-----------+
| TEST |
|-----------|
| NULL |
+-----------+
Important MySQL Variables:
block encryption mode=aes-256-cbc
I'm trying to use AES 256
You apparently need to use the same init_vector 3rd argument as this works:
> set #a=RANDOM_BYTES(16);
> SELECT AES_DECRYPT(AES_ENCRYPT('SERV92','TESTTTTTTT',#a),'TESTTTTTTT',#a) AS `TEST`;
+--------+
| TEST |
+--------+
| SERV92 |
+--------+
In your case you used RANDOM_BYTES(16) twice so that different values are used in encrypt and decrypt.
Okay I found the problem, AES_DECRYPT() returns data as a blob. Basically I just needed to tell it that it was utf8 text, as show below.
SET #a=RANDOM_BYTES(16); #Thanks Hartmut Holzgraefe
SELECT CONVERT(AES_DECRYPT(AES_ENCRYPT('SERV92','TESTTTTTTT',#a),'TESTTTTTTT',#a) USING utf8) AS `TEST`
+----------+
| TEST |
+----------+
| SERV92 |
+----------+
We have a varchar(255) column and some values are integers, how do I isolate only the rows which contain integer values
This should work.
select *
from table
where column
regexp '^\-?[1-9][0-9]*$'
EDIT: thanks Alma Do for pointing out that my solution did not consider signed integers and leading zeroes! Also his solution is much more performant than using regular expressions.
You can do this with CAST():
SELECT * FROM t WHERE CAST(col AS SIGNED)=col
You can use REGEXP() for your issue, but I will not recommend that: for large tables CAST() will be extremely faster. Compare:
mysql> select benchmark(1E7, '17453454.6655744' REGEXP '^[0-9]+$');
+------------------------------------------------------+
| benchmark(1E7, '17453454.6655744' REGEXP '^[0-9]+$') |
+------------------------------------------------------+
| 0 |
+------------------------------------------------------+
1 row in set (17.59 sec)
With:
mysql> select benchmark(1E7, CAST('17453454.6655744' AS SIGNED)='17453454.6655744');
+-----------------------------------------------------------------------+
| benchmark(1E7, CAST('17453454.6655744' AS SIGNED)='17453454.6655744') |
+-----------------------------------------------------------------------+
| 0 |
+-----------------------------------------------------------------------+
1 row in set, 1 warning (0.36 sec)
-and see the difference.
Here col means, will have to provide column name which has to be checked that having intergers or not ..?
SELECT * FROM t WHERE CAST(col AS SIGNED)=col
My table filed's value is "<script type="text/javascript"src="http://localhost:8080/db/widget/10217EN/F"></script>",
I want to analyse this string and fetch the id 10217,how to do use mysql regex?
I know python regex group function can return the id 10217,but i'm not familiar with mysql regex.
Please help me,Thank you very much.
MySQL regular expressions do not support subpattern extraction. You will probably have better luck iterating over all of the rows in your database and storing the results in a new column.
As far as I know, you can't use MySQL's REGEXP for substring retrieval; it is designed for use in WHERE clauses and is limited to returning 0 or 1 to indicate failure or success at a match.
Since your pattern is pretty well defined, you can probably retrieve the id with a query that uses SUBSTR and LOCATE. It will be a bit of a mess since SUBSTR wants the start index and the length of the substring (it would be easier if it took the end index). Perhaps you could use TRIM to chop off the unwanted trailing part.
This query get the Id from the field
SELECT substring_index(SUBSTRING_INDEX(testvar,'/',-3),'EN',1) from testtab;
where as testtab - is table name , testvar - is field name
inner substring get string starts with last 3 / which is
mysql> SELECT SUBSTRING_INDEX(testvar,'/',-3) from testtab;
+----------------------------+
| SUBSTRING_INDEX(testvar,'/',-3) |
+----------------------------+
| 10217EN/F"> |
| 10222EN/F"> |
+----------------------------+
2 rows in set (0.00 sec)
outer substring get
mysql> SELECT substring_index(SUBSTRING_INDEX(testvar,'/',-3),'EN',1) from testtab;
+----------------------------------------------------+
| substring_index(SUBSTRING_INDEX(testvar,'/',-3),'EN',1) |
+----------------------------------------------------+
| 10217 |
| 10222 |
+----------------------------------------------------+
2 rows in set (0.00 sec)