Looking for a function to using IF ELSE condition in MYSQL - mysql

I have a table data with combination of Integer and strings. Like this
CREATE TABLE EMPLOYEE (
empId INTEGER,
name TEXT NOT NULL,
email TEXT NOT NULL,
phone TEXT NOT NULL
);
INSERT INTO EMPLOYEE VALUES (12345, 'Clark Duff', 'Sales#yahoo.com',9001234567);
INSERT INTO EMPLOYEE VALUES (22245, 'Dave Johnson', 'Accounting#gmail.com',9000123456);
INSERT INTO EMPLOYEE VALUES (55456, 'Ava evelene', 'Sales_Marketing#gmail.com',9000012345);
But I'm looking for a common function in MYSQL to check the condition whether it is a special character or integer or string. Below syntax will mask the data:
If it is an integer then this condition needs to execute:
concat(SUBSTRING(phone,1,3) , '*****' , SUBSTRING(phone,7,4)) phone
If it is a special character (#) then this condition needs to execute:
CONCAT(LEFT(UUID(), 8), '#', SUBSTRING_INDEX(`Mail`, '#', -1)) as Mail
ELSE string then other script

seem you are looking for case statement applied to like condition for # char or cast to usingned for chekc a valid integer number
select case
when email like '%#%' then CONCAT(LEFT(UUID(), 8), '#', SUBSTRING_INDEX(`Mail`, '#', -1))
when cast(phone AS UNSIGNED) != 0 THEN concat(SUBSTRING(phone,1,3) , '*****' , SUBSTRING(phone,7,4))
else 'not managed'
end

One UDF to mask them.
CREATE FUNCTION fnMaskUserInfo (
input TEXT
)
RETURNS TEXT
DETERMINISTIC
BEGIN
IF input LIKE '%_#_%._%'
THEN
RETURN CONCAT(RIGHT(SHA1(input),8),'#',SUBSTRING_INDEX(input,'#',-1));
END IF;
IF input REGEXP '^[0-9]{7,10}$'
THEN
RETURN CONCAT(SUBSTRING(input,1,3),'*****',SUBSTRING(input,7,4));
END IF;
RETURN input;
END
SELECT empId
, fnMaskUserInfo(Name) AS Name
, fnMaskUserInfo(email) AS email
, fnMaskUserInfo(phone) AS phone
FROM EMPLOYEE
empId
Name
email
phone
12345
Clark Duff
1beb980f#yahoo.com
900*****4567
22245
Dave Johnson
1dadfff9#gmail.com
900*****3456
55456
Ava evelene
3a0768e7#gmail.com
900*****2345
Demo on db<>fiddle here

Here is an example stored function. Is this what you mean?
CREATE FUNCTION `someFunc`(
input VARCHAR(255)
)
RETURNS VARCHAR(255)
NOT DETERMINISTIC
BEGIN
IF input LIKE '%#%' THEN
RETURN CONCAT(LEFT(UUID(), 8), '#', SUBSTRING_INDEX(input, '#', -1));
ELSEIF input REGEXP '^[0-9 ]{7,10}$' THEN
RETURN CONCAT(SUBSTRING(input, 1, 3) , '*****' , SUBSTRING(input, 7, 4));
ELSE
RETURN input;
END IF;
END
You may want to use LEFT(MD5(input), 8) instead of LEFT(UUID(), 8) so that the function can be declared as deterministic.
CREATE FUNCTION `someFunc`(
input VARCHAR(255)
)
RETURNS VARCHAR(255)
DETERMINISTIC
BEGIN
IF input LIKE '%#%' THEN
RETURN CONCAT(LEFT(MD5(input), 8), '#', SUBSTRING_INDEX(input, '#', -1));
ELSEIF input REGEXP '^[0-9 ]{7,10}$' THEN
RETURN CONCAT(SUBSTRING(input, 1, 3) , '*****' , SUBSTRING(input, 7, 4));
ELSE
RETURN input;
END IF;
END
WITH `EMPLOYEE` (`empId`, `name`, `email`, `phone`) AS (
SELECT 12345, 'Clark Duff', 'Sales#yahoo.com',9001234567 UNION
SELECT 22245, 'Dave Johnson', 'Accounting#gmail.com',9000123456 UNION
SELECT 55456, 'Ava evelene', 'Sales_Marketing#gmail.com',9000012345
)
SELECT
empId,
someFunc(`name`) AS `name`,
someFunc(`email`) AS `email`,
someFunc(`phone`) AS `phone`
FROM `EMPLOYEE`;
# empId
name
email
phone
12345
Clark Duff
b3282336#yahoo.com
900*****4567
22245
Dave Johnson
47c44ab0#gmail.com
900*****3456
55456
Ava evelene
ab3fee5c#gmail.com
900*****2345

Related

MYSQL: ERROR 1241 (21000): Operand should contain 1 column(s)

this should be working but I get an error somehow. I want to get the credits column from the table and multiply it by 100. The problem is to get the number of credits for a given student id and year and get the total payment. Assuming each credit is $100.
delimiter //
create function fTest (stuYear varchar(4), stuID varchar(4))
returns varchar(200)
begin
declare msg varchar(200) default '';
if (stuYear = '' or stuYear is null) then
select 'Please input a valid year' into msg;
elseif (stuID = '' or stuID is null) then
select 'Please input a student id' into msg;
else
begin
if (msg = '' or msg is null) then
select ('No result found for student ID: ', stuID, ' at year: ', stuYear) into msg;
select (credits * 100) into msg from Students_Courses natural join Courses where sid=stuID and year=stuYear group by credits;
return msg ;
end if;
end ;
end if;
end ;
//
delimiter ;
This is incorrect:
select ('No result found for student ID: ', stuID, ' at year: ', stuYear)
A select statement can contain multiple columns, but you shouldn't enclose them in parentheses. Also, that would return multiple values, which you can't select into a single message.
So, I guess you wanted to concat the values into a single value. You can do that using the concat function, like so:
select concat('No result found for student ID: ', stuID, ' at year: ', stuYear)
Btw, a normal assignment using the concat function should also work in a trigger:
SET msg = concat('No result found for student ID: ', stuID, ' at year: ', stuYear);
See MySQL concat function
PS: In the next statement, you also got parentheses: (credits * 100)
In this case it accidentally will work, because it's a single expression. They don't have any functional value, though, and might as well be removed.

MySQL: SUM() of trailing number in VARCHAR [duplicate]

I'm looking to find records in a table that match a specific number that the user enters. So, the user may enter 12345, but this could be 123zz4-5 in the database.
I imagine something like this would work, if PHP functions worked in MySQL.
SELECT * FROM foo WHERE preg_replace("/[^0-9]/","",bar) = '12345'
What's the equivalent function or way to do this with just MySQL?
Speed is not important.
I realise that this is an ancient topic but upon googling this problem I couldn't find a simple solution (I saw the venerable agents but think this is a simpler solution) so here's a function I wrote, seems to work quite well.
DROP FUNCTION IF EXISTS STRIP_NON_DIGIT;
DELIMITER $$
CREATE FUNCTION STRIP_NON_DIGIT(input VARCHAR(255))
RETURNS VARCHAR(255)
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE iterator INT DEFAULT 1;
WHILE iterator < (LENGTH(input) + 1) DO
IF SUBSTRING(input, iterator, 1) IN ( '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' ) THEN
SET output = CONCAT(output, SUBSTRING(input, iterator, 1));
END IF;
SET iterator = iterator + 1;
END WHILE;
RETURN output;
END
$$
You can easily do what you want with REGEXP_REPLACE (compatible with MySQL 8+ and MariaDB 10.0.5+)
REGEXP_REPLACE(expr, pat, repl[, pos[, occurrence[, match_type]]])
Replaces occurrences in the string expr that match the regular expression specified by the pattern pat with the replacement string repl, and returns the resulting string. If expr, pat, or repl is NULL, the return value is NULL.
Go to REGEXP_REPLACE doc: MySQL or MariaDB
Try it:
SELECT REGEXP_REPLACE('123asd12333', '[a-zA-Z]+', '');
Output:
12312333
Updated 2022: as per Marlom's answer, you can now use REGEX_REPLACE - which will perform even better than my historical answer here.
Most upvoted answer above isn't the fastest.
Full kudos to them for giving a working proposal to bounce off!
This is an improved version:
DELIMITER ;;
DROP FUNCTION IF EXISTS `STRIP_NON_DIGIT`;;
CREATE DEFINER=`root`#`localhost` FUNCTION `STRIP_NON_DIGIT`(input VARCHAR(255)) RETURNS VARCHAR(255) CHARSET utf8
READS SQL DATA
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE iterator INT DEFAULT 1;
DECLARE lastDigit INT DEFAULT 1;
DECLARE len INT;
SET len = LENGTH(input) + 1;
WHILE iterator < len DO
-- skip past all digits
SET lastDigit = iterator;
WHILE ORD(SUBSTRING(input, iterator, 1)) BETWEEN 48 AND 57 AND iterator < len DO
SET iterator = iterator + 1;
END WHILE;
IF iterator != lastDigit THEN
SET output = CONCAT(output, SUBSTRING(input, lastDigit, iterator - lastDigit));
END IF;
WHILE ORD(SUBSTRING(input, iterator, 1)) NOT BETWEEN 48 AND 57 AND iterator < len DO
SET iterator = iterator + 1;
END WHILE;
END WHILE;
RETURN output;
END;;
Testing 5000 times on a test server:
-- original
Execution Time : 7.389 sec
Execution Time : 7.257 sec
Execution Time : 7.506 sec
-- ORD between not string IN
Execution Time : 4.031 sec
-- With less substrings
Execution Time : 3.243 sec
Execution Time : 3.415 sec
Execution Time : 2.848 sec
While it's not pretty and it shows results that don't match, this helps:
SELECT * FROM foo WHERE bar LIKE = '%1%2%3%4%5%'
I would still like to find a better solution similar to the item in the original question.
There's no regexp replace, only a plain string REPLACE().
MySQL has the REGEXP operator, but it's only a match tester not a replacer, so you would have to turn the logic inside-out:
SELECT * FROM foo WHERE bar REGEXP '[^0-9]*1[^0-9]*2[^0-9]*3[^0-9]*4[^0-9]*5[^0-9]*';
This is like your version with LIKE but matches more accurately. Both will perform equally badly, needing a full table scan without indexes.
On MySQL 8.0+ there is a new native function called REGEXP_REPLACE. A clean solution to this question would be:
SELECT * FROM foo WHERE REGEXP_REPLACE(bar,'[^0-9]+',"") = '12345'
The simplest way I can think to do it is to use the MySQL REGEXP operator a la:
WHERE foo LIKE '1\D*2\D*3\D*4\D*5'
It's not especially pretty but MySQL doesn't have a preg_replace function so I think it's the best you're going to get.
Personally, if this only-numeric data is so important, I'd keep a separate field just to contain the stripped data. It'll make your lookups a lot faster than with the regular expression search.
This blog post details how to strip non-numeric characters from a string via a MySQL function:
SELECT NumericOnly("asdf11asf");
returns 11
http://venerableagents.wordpress.com/2011/01/29/mysql-numeric-functions/
There's no regex replace as far as I'm concerned, but I found this solution;
--Create a table with numbers
DROP TABLE IF EXISTS ints;
CREATE TABLE ints (i INT UNSIGNED NOT NULL PRIMARY KEY);
INSERT INTO ints (i) VALUES
( 1), ( 2), ( 3), ( 4), ( 5), ( 6), ( 7), ( 8), ( 9), (10),
(11), (12), (13), (14), (15), (16), (17), (18), (19), (20);
--Then extract the numbers from the specified column
SELECT
bar,
GROUP_CONCAT(SUBSTRING(bar, i, 1) ORDER BY i SEPARATOR '')
FROM foo
JOIN ints ON i BETWEEN 1 AND LENGTH(bar)
WHERE
SUBSTRING(bar, i, 1) IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9')
GROUP BY bar;
It works for me and I use MySQL 5.0
Also I found this place that could help.
I have a similar situation, matching products to barcodes where the barcode doesn't store none alpha numerics sometimes, so 102.2234 in the DB needs to be found when searching for 1022234.
In the end I just added a new field, reference_number to the products tables, and have php strip out the none alpha numerics in the product_number to populate reference_number whenever a new products is added.
You'd need to do a one time scan of the table to create all the reference_number fields for existing products.
You can then setup your index, even if speed is not a factor for this operation, it is still a good idea to keep the database running well so this query doesn't bog it down and slow down other queries.
I came across this solution. The top answer by user1467716 will work in phpMyAdmin with a small change: add a second delimiter tag to the end of the code.
phpMyAdmin version is 4.1.14; MySQL version 5.6.20
I also added a length limiter using
DECLARE count INT DEFAULT 0; in the declarations
AND count < 5 in the WHILE statement
SET COUNT=COUNT+1; in the IF statement
Final form:
DROP FUNCTION IF EXISTS STRIP_NON_DIGIT;
DELIMITER $$
CREATE FUNCTION STRIP_NON_DIGIT(input VARCHAR(255))
RETURNS VARCHAR(255)
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE iterator INT DEFAULT 1;
DECLARE count INT DEFAULT 0;
WHILE iterator < (LENGTH(input) + 1) AND count < 5 DO --limits to 5 chars
IF SUBSTRING(input, iterator, 1) IN ( '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' ) THEN
SET output = CONCAT(output, SUBSTRING(input, iterator, 1));
SET COUNT=COUNT+1;
END IF;
SET iterator = iterator + 1;
END WHILE;
RETURN output;
END
$$
DELIMITER $$ --added this
How big is table with foo? If it is small, and speed really doesn't matter, you might pull the row ID and foo, loop over it using the PHP replace functions to compare, and then pull the info you want by row number.
Of course, if the table is too big, this won't work well.
try this example. this is used for phone numbers, however you can modify it for your needs.
-- function removes non numberic characters from input
-- returne only the numbers in the string
CREATE DEFINER =`root`#`localhost` FUNCTION `remove_alpha`(inputPhoneNumber VARCHAR(50))
RETURNS VARCHAR(50)
CHARSET latin1
DETERMINISTIC
BEGIN
DECLARE inputLenght INT DEFAULT 0;
-- var for our iteration
DECLARE counter INT DEFAULT 1;
-- if null is passed, we still return an tempty string
DECLARE sanitizedText VARCHAR(50) DEFAULT '';
-- holder of each character during the iteration
DECLARE oneChar VARCHAR(1) DEFAULT '';
-- we'll process only if it is not null.
IF NOT ISNULL(inputPhoneNumber)
THEN
SET inputLenght = LENGTH(inputPhoneNumber);
WHILE counter <= inputLenght DO
SET oneChar = SUBSTRING(inputPhoneNumber, counter, 1);
IF (oneChar REGEXP ('^[0-9]+$'))
THEN
SET sanitizedText = Concat(sanitizedText, oneChar);
END IF;
SET counter = counter + 1;
END WHILE;
END IF;
RETURN sanitizedText;
END
to use this user defined function (UDF).
let's say you have a column of phone numbers:
col1
(513)983-3983
1-838-338-9898
phone983-889-8383
select remove_alpha(col1) from mytable
The result would be;
5139833983
18383389898
9838898383
thought I would share this since I built it off the function from here. I rearranged just so I can read it easier (I'm just server side).
You call it by passing in a table name and column name to have it strip all existing non-numeric characters from that column. I inherited a lot of bad table structures that put a ton of int fields as varchar so I needed a way to clean these up quickly before I can modify the column to an integer.
drop procedure if exists strip_non_numeric_characters;
DELIMITER ;;
CREATE PROCEDURE `strip_non_numeric_characters`(
tablename varchar(100)
,columnname varchar(100)
)
BEGIN
-- =============================================
-- Author: <Author,,David Melton>
-- Create date: <Create Date,,2/26/2019>
-- Description: <Description,,loops through data and strips out the bad characters in whatever table and column you pass it>
-- =============================================
#this idea was generated from the idea STRIP_NON_DIGIT function
#https://stackoverflow.com/questions/287105/mysql-strip-non-numeric-characters-to-compare
declare input,output varchar(255);
declare iterator,lastDigit,len,counter int;
declare date_updated varchar(100);
select column_name
into date_updated
from information_schema.columns
where table_schema = database()
and extra rlike 'on update CURRENT_TIMESTAMP'
and table_name = tablename
limit 1;
#only goes up to 255 so people don't run this for a longtext field
#just to be careful, i've excluded columns that are part of keys, that could potentially mess something else up
set #find_column_length =
concat("select character_maximum_length
into #len
from information_schema.columns
where table_schema = '",database(),"'
and column_name = '",columnname,"'
and table_name = '",tablename,"'
and length(ifnull(character_maximum_length,100)) < 255
and data_type in ('char','varchar')
and column_key = '';");
prepare stmt from #find_column_length;
execute stmt;
deallocate prepare stmt;
set counter = 1;
set len = #len;
while counter <= ifnull(len,1) DO
#this just removes it by putting all the characters before and after the character i'm looking at
#you have to start at the end of the field otherwise the lengths don't stay in order and you have to run it multiple times
set #update_query =
concat("update `",tablename,"`
set `",columnname,"` = concat(substring(`",columnname,"`,1,",len - counter,"),SUBSTRING(`",columnname,"`,",len - counter,",",counter - 1,"))
",if(date_updated is not null,concat(",`",date_updated,"` = `",date_updated,"`
"),''),
"where SUBSTRING(`",columnname,"`,",len - counter,", 1) not REGEXP '^[0-9]+$';");
prepare stmt from #update_query;
execute stmt;
deallocate prepare stmt;
set counter = counter + 1;
end while;
END ;;
DELIMITER ;
To search for numbers that match a particular numeric pattern in a string, first remove all the alphabets and special characters in a similar manner as below then convert the value to an integer and then search
SELECT *
FROM foo
WHERE Convert(Regexp_replace(bar, '[a-zA-Z]+', ''), signed) = 12345
I think you don't need complicated functions for that.
I've found a REGEXP_REPLACE solution using built-in mysql character class names. You can read about them in a table in the docs. Basically, they are mysql-specific names for commonly matched groups of characters like [:alnum:] for alpha-numeric characters, [:alpha:] for only alphabetic characters and so on.
So my version of the REGEXP_REPLACE:
REGEXP_REPLACE('My number is: +59 (29) 889-23-56', '[[:alpha:][:blank:][:punct:][:cntrl:]]', '')
will yield 59298892356 as per requirements.
Being someone that has version 5.7, doesn't have the privilege to create functions, and it's not practical to bring my data down into my code, I found Nelson Miranda's Answer amazing. I wanted to share it in a subquery version which I found more useful.
DROP TABLE IF EXISTS ints;
CREATE TABLE ints (i INT UNSIGNED NOT NULL PRIMARY KEY);
INSERT INTO ints (i) VALUES
( 1), ( 2), ( 3), ( 4), ( 5), ( 6), ( 7), ( 8), ( 9), (10),
(11), (12), (13), (14), (15), (16), (17), (18), (19), (20);
SELECT * FROM foo f
WHERE (SELECT GROUP_CONCAT(SUBSTRING(f.bar, i, 1) ORDER BY i SEPARATOR '')
FROM ints
WHERE i BETWEEN 1 AND LENGTH(f.bar)
AND SUBSTRING(f.bar, i, 1) IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9')) = '12345'
Note: The ints table can be a temporary table.
If you are on MySQL 5.7 or below, and you just need something quick and dirty without having to define a new function, and you have a small number of known non-numerics to filter out, something like this can work well...
Select replace(replace(replace(replace(replace(phone, '+1', ''), '(', ''), ')', ''), '-', ''), ' ', '') from customers;

Query to Search All possible words

I need to write a Delphi 7 and mysql database query which would return the records including ALL words in the submitted name. So query will return records which has all those name words but can have different order.
For example, if search string is John Michael Smith, query should be able to return records with names such as John Smith Michael, Michael Smith John, Smith John Michael or other combination with all those words there.
As can be seen return only records which still has all words in name field but can have different order.
I can't figure out how to write a query for such requirement that I have. Please help.
procedure Tfrm_Query.Button1Click(Sender: TObject);
var
mask : string;
begin
mask:='''%'+StringReplace(Edit1.text,' ','%',[rfReplaceAll, rfIgnoreCase])+'%''';
if Edit1.Text > '' then
begin
Adosorgulama.Close;
Adosorgulama.SQL.Clear;
Adosorgulama.SQL.Add('SELECT * FROM stok.product');
Adosorgulama.SQL.ADD('Where (P_Name like '+mask+') limit 50');
Adosorgulama.Open;
end;
end;
as a result;
edit1.text:='Jo Mich'; // Result Ok!
edit1.text:='Smi Jo Mic'; //No result
edit1.text:='Mich Sm'; // No result
Instead of replacing spaces with %, you could replace them with % AND P_Name LIKE %:
mask:='''WHERE (P_Name LIKE %'+StringReplace(Edit1.text,' ','% AND P_Name LIKE %',[rfReplaceAll, rfIgnoreCase])+'%)''';
Apologies if there is some problem with the syntax (I don't know Delphi), but if Edit1.text:= 'John Michael Smith' this should generate the following WHERE clause:
WHERE (P_Name LIKE %John% AND P_Name LIKE %Michael% AND P_Name LIKE %Smith%)
Which should find all records where P_Name contains the strings 'John', 'Michael' and 'Smith'.
Then, of course, instead of
Adosorgulama.SQL.ADD('Where (P_Name like '+mask+') limit 50');
you'd do something like
Adosorgulama.SQL.ADD(mask + ' limit 50');
If the input can contain extraneous spaces, you will need to remove those first, otherwise this won't work.
Forming SQL queries with string concatenation could make your application vulnerable to SQL injection, just so you know. I don't know how to do prepared statements with Delphi, so I can't help you there.
Write a function that can split the name according to space. Use the following code inside a loop that is looping split results.
Declare #sqlq as nvarchar(max);
-- loop start
#sqlq = sqlq + 'Select * from mytable where names';
#sqlq = sqlq + 'like ''%' + loopvalue + '%''';
--loop end
Exec #sqlq
You can build table of words dynamically. To find yours match do query that join both tables in possible match, and by grouping results test it - is name have all of words, try this:
WITH
words AS (SELECT 'John' AS word FROM dual union
SELECT 'Michael' FROM dual union
SELECT 'Smith' FROM dual ) , --build your table of words (this is example on oracle DB engine)
names AS (SELECT 'John Michael Smith' AS name FROM dual UNION
SELECT 'John SmithMichael' FROM dual union
SELECT 'Smith Michael' FROM dual union
SELECT 'Smith Michael John' FROM dual union
SELECT 'John' FROM dual union
SELECT 'John John' FROM dual union
SELECT 'John John John' FROM dual union
SELECT 'xyz abc' FROM dual ) --this is simulation of yours table of names
SELECT name, Count(DISTINCT word)
FROM names, words
WHERE ' ' || name || ' ' LIKE '% ' || word || ' %'
GROUP BY name
HAVING Count(DISTINCT word) = (SELECT Count(1) FROM words)
;
Problem is solved.
Delphi + MySQL connection with word order with the following code, regardless of calls can be made. Thank you for inspiration. Respects.
Database model;
CREATE TABLE IF NOT EXISTS `TableName` (
`ID` int(11) NOT NULL AUTO_INCREMENT,
`PNUMBER` varchar(20) DEFAULT NULL,
`PNAME` varchar(255) DEFAULT NULL,
`PBARCODE` varchar(30) DEFAULT NULL,
`PSearch` mediumtext,
PRIMARY KEY (`ID`),
FULLTEXT KEY `PSearch` (`PSearch`)
) ENGINE=MyISAM DEFAULT CHARSET=latin5 ;
Psearch = PNUMBER + PNAME + PBARCODE ...; (Type in all areas PSearch)
Delphi7 Code;
procedure TForm1.Button1Click(Sender: TObject);
var
mask : string;
begin
mask:='+'+StringReplace(Edit1.text,' ','* +',[rfReplaceAll, rfIgnoreCase])+'*';
if Edit1.Text > '' then
begin
Query1.Close;
Query1.SQL.Clear;
Query1.SQL.Add('SELECT MATCH(PSearch) AGAINST("'+mask+'" IN BOOLEAN MODE), tablename.* FROM database.tablename');
Query1.SQL.Add('WHERE MATCH(PSearch) AGAINST("'+mask+'" IN BOOLEAN MODE) limit 300;');
Query1.Open;
end; end;

Forming text based on values from table

I am currently working on a import and need to build a comma delimited string based on the values a table row. Iam able to do it this way but want to know if i can optimize this.
If((Select planValue from #tempTable) = 'Yes')
Begin
SET #ltValue='1'
END
If((Select planValue1 from #tempTable) = 'Yes')
Begin
IF(LEN(#ltValue) > 0)
BEGIN
SET #ltValue=#ltValue+',2'
END
ELSE
BEGIN
SET #ltValue='2'
END
END
If((Select planValue2 from #tempTable) = 'Yes')
Begin
IF(LEN(#ltValue) > 0)
BEGIN
SET #ltValue=#ltValue+',3'
END
ELSE
BEGIN
SET #ltValue='3'
END
END
An issue with your provided solution is if there are more than one row in the temp table. Here is an option that creates the string within a select.
DECLARE #tmpTable TABLE (
planValue varchar(5),
planValue1 varchar(5),
planValue2 varchar(5)
)
insert #tmpTable (planValue, planValue1, PlanValue2) VALUES ('Yes', 'Yes', 'No')
insert #tmpTable (planValue, planValue1, PlanValue2) VALUES ('Yes', 'No', 'No')
INSERT #tmpTable (planValue, planValue1, PlanValue2) VALUES ('No', 'Yes', 'Yes')
INSERT #tmpTable (planValue, planValue1, PlanValue2) VALUES ('No', 'No', 'Yes')
INSERT #tmpTable (planValue, planValue1, PlanValue2) VALUES ('Yes', 'Yes', 'Yes')
select REPLACE(REPLACE(CASE WHEN planValue = 'Yes' THEN '1|' ELSE '' END +
CASE WHEN planValue1 = 'Yes' THEN '|2|' ELSE '' END +
CASE WHEN planValue2 = 'Yes' THEN '|3|' ELSE '' END, '||', ','), '|', '') [ResultData]
FROM #tmpTable
OUTPUT
ResultData
1,2
1
2,3
3
1,2,3
The case statements add in the necessary values with a unique delimiter. The inner replace finds two consecutive delimiters and changes it to a comma. The last outer replace finds a single delimiter and removes it.

MySQL strip non-numeric characters to compare

I'm looking to find records in a table that match a specific number that the user enters. So, the user may enter 12345, but this could be 123zz4-5 in the database.
I imagine something like this would work, if PHP functions worked in MySQL.
SELECT * FROM foo WHERE preg_replace("/[^0-9]/","",bar) = '12345'
What's the equivalent function or way to do this with just MySQL?
Speed is not important.
I realise that this is an ancient topic but upon googling this problem I couldn't find a simple solution (I saw the venerable agents but think this is a simpler solution) so here's a function I wrote, seems to work quite well.
DROP FUNCTION IF EXISTS STRIP_NON_DIGIT;
DELIMITER $$
CREATE FUNCTION STRIP_NON_DIGIT(input VARCHAR(255))
RETURNS VARCHAR(255)
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE iterator INT DEFAULT 1;
WHILE iterator < (LENGTH(input) + 1) DO
IF SUBSTRING(input, iterator, 1) IN ( '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' ) THEN
SET output = CONCAT(output, SUBSTRING(input, iterator, 1));
END IF;
SET iterator = iterator + 1;
END WHILE;
RETURN output;
END
$$
You can easily do what you want with REGEXP_REPLACE (compatible with MySQL 8+ and MariaDB 10.0.5+)
REGEXP_REPLACE(expr, pat, repl[, pos[, occurrence[, match_type]]])
Replaces occurrences in the string expr that match the regular expression specified by the pattern pat with the replacement string repl, and returns the resulting string. If expr, pat, or repl is NULL, the return value is NULL.
Go to REGEXP_REPLACE doc: MySQL or MariaDB
Try it:
SELECT REGEXP_REPLACE('123asd12333', '[a-zA-Z]+', '');
Output:
12312333
Updated 2022: as per Marlom's answer, you can now use REGEX_REPLACE - which will perform even better than my historical answer here.
Most upvoted answer above isn't the fastest.
Full kudos to them for giving a working proposal to bounce off!
This is an improved version:
DELIMITER ;;
DROP FUNCTION IF EXISTS `STRIP_NON_DIGIT`;;
CREATE DEFINER=`root`#`localhost` FUNCTION `STRIP_NON_DIGIT`(input VARCHAR(255)) RETURNS VARCHAR(255) CHARSET utf8
READS SQL DATA
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE iterator INT DEFAULT 1;
DECLARE lastDigit INT DEFAULT 1;
DECLARE len INT;
SET len = LENGTH(input) + 1;
WHILE iterator < len DO
-- skip past all digits
SET lastDigit = iterator;
WHILE ORD(SUBSTRING(input, iterator, 1)) BETWEEN 48 AND 57 AND iterator < len DO
SET iterator = iterator + 1;
END WHILE;
IF iterator != lastDigit THEN
SET output = CONCAT(output, SUBSTRING(input, lastDigit, iterator - lastDigit));
END IF;
WHILE ORD(SUBSTRING(input, iterator, 1)) NOT BETWEEN 48 AND 57 AND iterator < len DO
SET iterator = iterator + 1;
END WHILE;
END WHILE;
RETURN output;
END;;
Testing 5000 times on a test server:
-- original
Execution Time : 7.389 sec
Execution Time : 7.257 sec
Execution Time : 7.506 sec
-- ORD between not string IN
Execution Time : 4.031 sec
-- With less substrings
Execution Time : 3.243 sec
Execution Time : 3.415 sec
Execution Time : 2.848 sec
While it's not pretty and it shows results that don't match, this helps:
SELECT * FROM foo WHERE bar LIKE = '%1%2%3%4%5%'
I would still like to find a better solution similar to the item in the original question.
There's no regexp replace, only a plain string REPLACE().
MySQL has the REGEXP operator, but it's only a match tester not a replacer, so you would have to turn the logic inside-out:
SELECT * FROM foo WHERE bar REGEXP '[^0-9]*1[^0-9]*2[^0-9]*3[^0-9]*4[^0-9]*5[^0-9]*';
This is like your version with LIKE but matches more accurately. Both will perform equally badly, needing a full table scan without indexes.
On MySQL 8.0+ there is a new native function called REGEXP_REPLACE. A clean solution to this question would be:
SELECT * FROM foo WHERE REGEXP_REPLACE(bar,'[^0-9]+',"") = '12345'
The simplest way I can think to do it is to use the MySQL REGEXP operator a la:
WHERE foo LIKE '1\D*2\D*3\D*4\D*5'
It's not especially pretty but MySQL doesn't have a preg_replace function so I think it's the best you're going to get.
Personally, if this only-numeric data is so important, I'd keep a separate field just to contain the stripped data. It'll make your lookups a lot faster than with the regular expression search.
This blog post details how to strip non-numeric characters from a string via a MySQL function:
SELECT NumericOnly("asdf11asf");
returns 11
http://venerableagents.wordpress.com/2011/01/29/mysql-numeric-functions/
There's no regex replace as far as I'm concerned, but I found this solution;
--Create a table with numbers
DROP TABLE IF EXISTS ints;
CREATE TABLE ints (i INT UNSIGNED NOT NULL PRIMARY KEY);
INSERT INTO ints (i) VALUES
( 1), ( 2), ( 3), ( 4), ( 5), ( 6), ( 7), ( 8), ( 9), (10),
(11), (12), (13), (14), (15), (16), (17), (18), (19), (20);
--Then extract the numbers from the specified column
SELECT
bar,
GROUP_CONCAT(SUBSTRING(bar, i, 1) ORDER BY i SEPARATOR '')
FROM foo
JOIN ints ON i BETWEEN 1 AND LENGTH(bar)
WHERE
SUBSTRING(bar, i, 1) IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9')
GROUP BY bar;
It works for me and I use MySQL 5.0
Also I found this place that could help.
I have a similar situation, matching products to barcodes where the barcode doesn't store none alpha numerics sometimes, so 102.2234 in the DB needs to be found when searching for 1022234.
In the end I just added a new field, reference_number to the products tables, and have php strip out the none alpha numerics in the product_number to populate reference_number whenever a new products is added.
You'd need to do a one time scan of the table to create all the reference_number fields for existing products.
You can then setup your index, even if speed is not a factor for this operation, it is still a good idea to keep the database running well so this query doesn't bog it down and slow down other queries.
I came across this solution. The top answer by user1467716 will work in phpMyAdmin with a small change: add a second delimiter tag to the end of the code.
phpMyAdmin version is 4.1.14; MySQL version 5.6.20
I also added a length limiter using
DECLARE count INT DEFAULT 0; in the declarations
AND count < 5 in the WHILE statement
SET COUNT=COUNT+1; in the IF statement
Final form:
DROP FUNCTION IF EXISTS STRIP_NON_DIGIT;
DELIMITER $$
CREATE FUNCTION STRIP_NON_DIGIT(input VARCHAR(255))
RETURNS VARCHAR(255)
BEGIN
DECLARE output VARCHAR(255) DEFAULT '';
DECLARE iterator INT DEFAULT 1;
DECLARE count INT DEFAULT 0;
WHILE iterator < (LENGTH(input) + 1) AND count < 5 DO --limits to 5 chars
IF SUBSTRING(input, iterator, 1) IN ( '0', '1', '2', '3', '4', '5', '6', '7', '8', '9' ) THEN
SET output = CONCAT(output, SUBSTRING(input, iterator, 1));
SET COUNT=COUNT+1;
END IF;
SET iterator = iterator + 1;
END WHILE;
RETURN output;
END
$$
DELIMITER $$ --added this
How big is table with foo? If it is small, and speed really doesn't matter, you might pull the row ID and foo, loop over it using the PHP replace functions to compare, and then pull the info you want by row number.
Of course, if the table is too big, this won't work well.
try this example. this is used for phone numbers, however you can modify it for your needs.
-- function removes non numberic characters from input
-- returne only the numbers in the string
CREATE DEFINER =`root`#`localhost` FUNCTION `remove_alpha`(inputPhoneNumber VARCHAR(50))
RETURNS VARCHAR(50)
CHARSET latin1
DETERMINISTIC
BEGIN
DECLARE inputLenght INT DEFAULT 0;
-- var for our iteration
DECLARE counter INT DEFAULT 1;
-- if null is passed, we still return an tempty string
DECLARE sanitizedText VARCHAR(50) DEFAULT '';
-- holder of each character during the iteration
DECLARE oneChar VARCHAR(1) DEFAULT '';
-- we'll process only if it is not null.
IF NOT ISNULL(inputPhoneNumber)
THEN
SET inputLenght = LENGTH(inputPhoneNumber);
WHILE counter <= inputLenght DO
SET oneChar = SUBSTRING(inputPhoneNumber, counter, 1);
IF (oneChar REGEXP ('^[0-9]+$'))
THEN
SET sanitizedText = Concat(sanitizedText, oneChar);
END IF;
SET counter = counter + 1;
END WHILE;
END IF;
RETURN sanitizedText;
END
to use this user defined function (UDF).
let's say you have a column of phone numbers:
col1
(513)983-3983
1-838-338-9898
phone983-889-8383
select remove_alpha(col1) from mytable
The result would be;
5139833983
18383389898
9838898383
thought I would share this since I built it off the function from here. I rearranged just so I can read it easier (I'm just server side).
You call it by passing in a table name and column name to have it strip all existing non-numeric characters from that column. I inherited a lot of bad table structures that put a ton of int fields as varchar so I needed a way to clean these up quickly before I can modify the column to an integer.
drop procedure if exists strip_non_numeric_characters;
DELIMITER ;;
CREATE PROCEDURE `strip_non_numeric_characters`(
tablename varchar(100)
,columnname varchar(100)
)
BEGIN
-- =============================================
-- Author: <Author,,David Melton>
-- Create date: <Create Date,,2/26/2019>
-- Description: <Description,,loops through data and strips out the bad characters in whatever table and column you pass it>
-- =============================================
#this idea was generated from the idea STRIP_NON_DIGIT function
#https://stackoverflow.com/questions/287105/mysql-strip-non-numeric-characters-to-compare
declare input,output varchar(255);
declare iterator,lastDigit,len,counter int;
declare date_updated varchar(100);
select column_name
into date_updated
from information_schema.columns
where table_schema = database()
and extra rlike 'on update CURRENT_TIMESTAMP'
and table_name = tablename
limit 1;
#only goes up to 255 so people don't run this for a longtext field
#just to be careful, i've excluded columns that are part of keys, that could potentially mess something else up
set #find_column_length =
concat("select character_maximum_length
into #len
from information_schema.columns
where table_schema = '",database(),"'
and column_name = '",columnname,"'
and table_name = '",tablename,"'
and length(ifnull(character_maximum_length,100)) < 255
and data_type in ('char','varchar')
and column_key = '';");
prepare stmt from #find_column_length;
execute stmt;
deallocate prepare stmt;
set counter = 1;
set len = #len;
while counter <= ifnull(len,1) DO
#this just removes it by putting all the characters before and after the character i'm looking at
#you have to start at the end of the field otherwise the lengths don't stay in order and you have to run it multiple times
set #update_query =
concat("update `",tablename,"`
set `",columnname,"` = concat(substring(`",columnname,"`,1,",len - counter,"),SUBSTRING(`",columnname,"`,",len - counter,",",counter - 1,"))
",if(date_updated is not null,concat(",`",date_updated,"` = `",date_updated,"`
"),''),
"where SUBSTRING(`",columnname,"`,",len - counter,", 1) not REGEXP '^[0-9]+$';");
prepare stmt from #update_query;
execute stmt;
deallocate prepare stmt;
set counter = counter + 1;
end while;
END ;;
DELIMITER ;
To search for numbers that match a particular numeric pattern in a string, first remove all the alphabets and special characters in a similar manner as below then convert the value to an integer and then search
SELECT *
FROM foo
WHERE Convert(Regexp_replace(bar, '[a-zA-Z]+', ''), signed) = 12345
I think you don't need complicated functions for that.
I've found a REGEXP_REPLACE solution using built-in mysql character class names. You can read about them in a table in the docs. Basically, they are mysql-specific names for commonly matched groups of characters like [:alnum:] for alpha-numeric characters, [:alpha:] for only alphabetic characters and so on.
So my version of the REGEXP_REPLACE:
REGEXP_REPLACE('My number is: +59 (29) 889-23-56', '[[:alpha:][:blank:][:punct:][:cntrl:]]', '')
will yield 59298892356 as per requirements.
Being someone that has version 5.7, doesn't have the privilege to create functions, and it's not practical to bring my data down into my code, I found Nelson Miranda's Answer amazing. I wanted to share it in a subquery version which I found more useful.
DROP TABLE IF EXISTS ints;
CREATE TABLE ints (i INT UNSIGNED NOT NULL PRIMARY KEY);
INSERT INTO ints (i) VALUES
( 1), ( 2), ( 3), ( 4), ( 5), ( 6), ( 7), ( 8), ( 9), (10),
(11), (12), (13), (14), (15), (16), (17), (18), (19), (20);
SELECT * FROM foo f
WHERE (SELECT GROUP_CONCAT(SUBSTRING(f.bar, i, 1) ORDER BY i SEPARATOR '')
FROM ints
WHERE i BETWEEN 1 AND LENGTH(f.bar)
AND SUBSTRING(f.bar, i, 1) IN ('0', '1', '2', '3', '4', '5', '6', '7', '8', '9')) = '12345'
Note: The ints table can be a temporary table.
If you are on MySQL 5.7 or below, and you just need something quick and dirty without having to define a new function, and you have a small number of known non-numerics to filter out, something like this can work well...
Select replace(replace(replace(replace(replace(phone, '+1', ''), '(', ''), ')', ''), '-', ''), ' ', '') from customers;