Related
I have made a stored procedure. I want it to filter the data by different parameters. If I pass one parameter, it should be filtered by one; if I pass two, it should be filtered by two, and so on, but it is not working.
Can anyone help me please?
DROP PROCEDURE IF EXISTS medatabase.SP_rptProvince2;
CREATE PROCEDURE medatabase.`SP_rptProvince2`(
IN e_Region VARCHAR(45)
)
BEGIN
DECLARE strQuery VARCHAR(1024);
DECLARE stmtp VARCHAR(1024);
SET #strQuery = CONCAT('SELECT * FROM alldata where 1=1');
IF e_region IS NOT NULL THEN
SET #strQuery = CONCAT(#strQuery, ' AND (regionName)'=e_Region);
END IF;
PREPARE stmtp FROM #strQuery;
EXECUTE stmtp;
END;
AFAIK, you can't have a variable argument list like that. You can do one of a couple of things:
Take a fixed maximum number of parameters, and check them for null-ness before concatenating:
CREATE PROCEDURE SP_rptProvince2(a1 VARCHAR(45), a2 VARCHAR(45), ...)
...
IF a1 IS NOT NULL THEN
SET #strQuery = CONCAT(#strQuery, ' AND ', a2);
END IF;
If you need predetermined fields to which the criteria in the argument apply (like the e_Region parameter in your existing code), then you modify the CONCAT operation appropriately.
Possible invocation:
CALL SP_rptProvince2('''North''', 'column3 = ''South''')
Take a single parameter that is much bigger than just 45 characters, and simply append it to the query (assuming it is not null).
Clearly, this places the onus on the user to provide the correct SQL code.
Possible invocation:
CALL SP_rptProvince2('RegionName = ''North'' AND column3 = ''South''')
There's not a lot to choose between the two. Either can be made to work; neither is entirely satisfactory.
You might note that there was a need to protect the strings in the arguments with extra quotes; that is the sort of thing that makes this problematic.
I found a JSON-based approach which works with the latest MySQL/MariaDB systems. Check the link below (Original Author is Federico Razzoli): https://federico-razzoli.com/variable-number-of-parameters-and-optional-parameters-in-mysql-mariadb-procedures
Basically, you take a BLOB parameter which is actually a JSON object and then do JSON_UNQUOTE(JSON_EXTRACT(json object, key)) as appropriate.
Lifted an extract here:
CREATE FUNCTION table_exists(params BLOB)
RETURNS BOOL
NOT DETERMINISTIC
READS SQL DATA
COMMENT '
Return whether a table exists.
Parameters must be passed in a JSON document:
* schema (optional). : Schema that could contain the table.
By default, the schema containing this procedure.
* table : Name of the table to check.
'
BEGIN
DECLARE v_table VARCHAR(64)
DEFAULT JSON_UNQUOTE(JSON_EXTRACT(params, '$.table'));
DECLARE v_schema VARCHAR(64)
DEFAULT JSON_UNQUOTE(JSON_EXTRACT(params, '$.schema'));
IF v_schema IS NULL THEN
RETURN EXISTS (
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE
TABLE_SCHEMA = SCHEMA()
AND TABLE_NAME = v_table
);
ELSE
RETURN EXISTS (
SELECT TABLE_NAME
FROM information_schema.TABLES
WHERE
TABLE_SCHEMA = v_schema
AND TABLE_NAME = v_table
);
END IF;
END;
I have a MySQL database and I have a query as:
SELECT `id`, `originaltext` FROM `source` WHERE `originaltext` regexp '[0-9][0-9]'
This detects all originaltexts which have numbers with 2 digits in it.
I need MySQL to return those numbers as a field, so i can manipulate them further.
Ideally, if I can add additional criteria that is should be > 20 would be great, but i can do that separately as well.
If you want more regular expression power in your database, you can consider using LIB_MYSQLUDF_PREG. This is an open source library of MySQL user functions that imports the PCRE library. LIB_MYSQLUDF_PREG is delivered in source code form only. To use it, you'll need to be able to compile it and install it into your MySQL server. Installing this library does not change MySQL's built-in regex support in any way. It merely makes the following additional functions available:
PREG_CAPTURE extracts a regex match from a string. PREG_POSITION returns the position at which a regular expression matches a string. PREG_REPLACE performs a search-and-replace on a string. PREG_RLIKE tests whether a regex matches a string.
All these functions take a regular expression as their first parameter. This regular expression must be formatted like a Perl regular expression operator. E.g. to test if regex matches the subject case insensitively, you'd use the MySQL code PREG_RLIKE('/regex/i', subject). This is similar to PHP's preg functions, which also require the extra // delimiters for regular expressions inside the PHP string.
If you want something more simpler, you could alter this function to suit better your needs.
CREATE FUNCTION REGEXP_EXTRACT(string TEXT, exp TEXT)
-- Extract the first longest string that matches the regular expression
-- If the string is 'ABCD', check all strings and see what matches: 'ABCD', 'ABC', 'AB', 'A', 'BCD', 'BC', 'B', 'CD', 'C', 'D'
-- It's not smart enough to handle things like (A)|(BCD) correctly in that it will return the whole string, not just the matching token.
RETURNS TEXT
DETERMINISTIC
BEGIN
DECLARE s INT DEFAULT 1;
DECLARE e INT;
DECLARE adjustStart TINYINT DEFAULT 1;
DECLARE adjustEnd TINYINT DEFAULT 1;
-- Because REGEXP matches anywhere in the string, and we only want the part that matches, adjust the expression to add '^' and '$'
-- Of course, if those are already there, don't add them, but change the method of extraction accordingly.
IF LEFT(exp, 1) = '^' THEN
SET adjustStart = 0;
ELSE
SET exp = CONCAT('^', exp);
END IF;
IF RIGHT(exp, 1) = '$' THEN
SET adjustEnd = 0;
ELSE
SET exp = CONCAT(exp, '$');
END IF;
-- Loop through the string, moving the end pointer back towards the start pointer, then advance the start pointer and repeat
-- Bail out of the loops early if the original expression started with '^' or ended with '$', since that means the pointers can't move
WHILE (s <= LENGTH(string)) DO
SET e = LENGTH(string);
WHILE (e >= s) DO
IF SUBSTRING(string, s, e) REGEXP exp THEN
RETURN SUBSTRING(string, s, e);
END IF;
IF adjustEnd THEN
SET e = e - 1;
ELSE
SET e = s - 1; -- ugh, such a hack to end it early
END IF;
END WHILE;
IF adjustStart THEN
SET s = s + 1;
ELSE
SET s = LENGTH(string) + 1; -- ugh, such a hack to end it early
END IF;
END WHILE;
RETURN NULL;
END
There isn't any syntax in MySQL for extracting text using regular expressions. You can use the REGEXP to identify the rows containing two consecutive digits, but to extract them you have to use the ordinary string manipulation functions which is very difficult in this case.
Alternatives:
Select the entire value from the database then use a regular expression on the client.
Use a different database that has better support for the SQL standard (may not be an option, I know). Then you can use this: SUBSTRING(originaltext from '%#[0-9]{2}#%' for '#').
I think the cleaner way is using REGEXP_SUBSTR():
This extracts exactly two any digits:
SELECT REGEXP_SUBSTR(`originalText`,'[0-9]{2}') AS `twoDigits` FROM `source`;
This extracts exactly two digits, but from 20-99 (example: 1112 return null; 1521 returns 52):
SELECT REGEXP_SUBSTR(`originalText`,'[2-9][0-9]') AS `twoDigits` FROM `source`;
I test both in v8.0 and they work. That's all, good luck!
I'm having the same issue, and this is the solution I found (but it won't work in all cases) :
use LOCATE() to find the beginning and the end of the string you wan't to match
use MID() to extract the substring in between...
keep the regexp to match only the rows where you are sure to find a match.
I used my code as a Stored Procedure (Function), shall work to extract any number built from digits in a single block. This is a part of my wider library.
DELIMITER $$
-- 2013.04 michal#glebowski.pl
-- FindNumberInText("ab 234 95 cd", TRUE) => 234
-- FindNumberInText("ab 234 95 cd", FALSE) => 95
DROP FUNCTION IF EXISTS FindNumberInText$$
CREATE FUNCTION FindNumberInText(_input VARCHAR(64), _fromLeft BOOLEAN) RETURNS VARCHAR(32)
BEGIN
DECLARE _r VARCHAR(32) DEFAULT '';
DECLARE _i INTEGER DEFAULT 1;
DECLARE _start INTEGER DEFAULT 0;
DECLARE _IsCharNumeric BOOLEAN;
IF NOT _fromLeft THEN SET _input = REVERSE(_input); END IF;
_loop: REPEAT
SET _IsCharNumeric = LOCATE(MID(_input, _i, 1), "0123456789") > 0;
IF _IsCharNumeric THEN
IF _start = 0 THEN SET _start = _i; END IF;
ELSE
IF _start > 0 THEN LEAVE _loop; END IF;
END IF;
SET _i = _i + 1;
UNTIL _i > length(_input) END REPEAT;
IF _start > 0 THEN
SET _r = MID(_input, _start, _i - _start);
IF NOT _fromLeft THEN SET _r = REVERSE(_r); END IF;
END IF;
RETURN _r;
END$$
If you want to return a part of a string :
SELECT id , substring(columnName,(locate('partOfString',columnName)),10) from tableName;
Locate() will return the starting postion of the matching string which becomes starting position of Function Substring()
I know it's been quite a while since this question was asked but came across it and thought it would be a good challenge for my custom regex replacer - see this blog post.
...And the good news is it can, although it needs to be called quite a few times. See this online rextester demo, which shows the workings that got to the SQL below.
SELECT reg_replace(
reg_replace(
reg_replace(
reg_replace(
reg_replace(
reg_replace(
reg_replace(txt,
'[^0-9]+',
',',
TRUE,
1, -- Min match length
0 -- No max match length
),
'([0-9]{3,}|,[0-9],)',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
'^[0-9],',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
',[0-9]$',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
',{2,}',
',',
TRUE,
1, -- Min match length
0 -- No max match length
),
'^,',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
',$',
'',
TRUE,
1, -- Min match length
0 -- No max match length
) AS `csv`
FROM tbl;
I'm working on a stored procedure that will accept a string and return a new string of text. Input parameters are #OrderId and #OrderText which is a string with dollar sign enclosed variables like so... $Order Name$ sent to $Customer$
The valid variables are in a Variables table (values such as Order Name, Customer, a total of 25 of them which should remain fairly static). Variables can only be used once in the string.
The stored procedure needs to return the string but with the variables replaced with their respective values.
Example1
Input: 123, $Order Name$ sent to $Customer$
Returns: Toolkit sent to StackCustomer Inc.
Example2
Input: 456, $Customer$ requests $delivery method$ for $order type$
Returns: ABC Inc requests fast shipping for pallet orders.
Each of the variables can be retrieved using a function.
DECLARE #OrderId int = 123
DECLARE #InputText VARCHAR(500) = '$Order Name$ sent to $Customer$'
select
#InputText = case when #InputText like '%$order name$%'
then replace(#InputText, '$Order Name$', getOrderName(id) else '' end,
#InputText = case when #InputText like '%$customer$'
then replace(#InputText, '$Customer$', getCustomer(id) else '' end
-- repeat 25 times
Is there a better way? My main concern is maintainability - if a variable is added, renamed, or removed, this stored proc will need to be changed (although I'm told it would only happen a couple times a year, if that). Would dynamic sql be able to help in this case?
Personally, I would create a keywords table to maintain it. something like this
CREATE TABLE [keywords] (
key_value VARCHAR(100) NOT NULL,
function_value VARCHAR(100) NOT NULL
)
INSERT INTO [keywords]
VALUES
('$Customer$','getCustomer(id)'),
('$Order Name$' ,'getOrderName(id)'),
('$order type$','getOrderType(id)')
Then use dynamic sql create REPLACE SQL
DECLARE #OrderId int = 123
DECLARE #InputText VARCHAR(500) = '$Order Name$ sent to $Customer$'
DECLARE #sql VARCHAR(8000) = 'SELECT '
SELECT
#sql = #sql +
' #InputText = replace(#InputText, ''' + key_value + ''', ' + function_value + ')'
+ ' ,'
FROM keywords
WHERE #InputText LIKE '%' + key_value + '%'
SELECT #sql = LEFT(#sql, LEN(#sql) -1)
PRINT #sql
EXEC(#sql)
SQLFiddle
I am not really getting why you need to do tokenization if you are going to use input variables for the procedure.
There are token templates already available for SQL Management Studio in the form of:
<(tokenName),(datatype),(defaultvalue)>
You can get their data filled directly with SQL Managment Studio with CTRL + SHIFT + M, or with ALT > Q > S in 2012, or by text finding their values in an environment.
If you are trying to put in an input to be accessed by an outside developing platform that changes the strings like ADO.NET or the Entity Framework in C#/VB.NET I still am not getting why you would not just make more input variables.
You could do this quite easily:
Declare #OrderPlace varchar(128), #Customer varchar(64), #StringCombine varchar(512);
Select
#OrderPlace = 'Place I am at'
, #Customer = 'Mr Customer';
Select #StringCombine = #Customer + ' order at ' + #OrderPlace
Then if a different language accessed your SQL server procedure it would just have to put in two parameters #Customer and #OrderPlace. You could even set #StringCombine to be an output variable. This is much more preferable then text replacing characters and then running a string. This could be able for SQL injection attacks potentially so santizing your inputs is a big part of returning data, especially if you are altering something for SQL to run before it is ran.
You mentioned maintainability and this is much more robust because if I change the logic to the proc but DO NOT change the variables names, I did not have to change anything else. If I have to change an ADO.NET or other library for references and then SQL code, that is lot more work. Generally when working with tokens I strive for reuse of code where one part of it can go down and only hurt that part of it, not take the whole thing down.
I have a table with following Data in a table
abcd
abcd , pgw
ABcd , Pgw
I want output as
Abcd
Abcd , Pgw
Abcd , Pgw
the First letter in capitals and letter after ',' in capital.
to make first letter upper-case use this
select CONCAT(UPPER(LEFT(`field`, 1)), SUBSTRING(`field`, 2)) from mytable
to do it to also to word after comma declare a function like :
DELIMITER ||
CREATE FUNCTION `ucwords`( str VARCHAR(128) ) RETURNS VARCHAR(128) CHARSET latin1
BEGIN
DECLARE c CHAR(1);
DECLARE s VARCHAR(128);
DECLARE i INT DEFAULT 1;
DECLARE BOOL INT DEFAULT 1;
SET s = LCASE( str );
WHILE i < LENGTH( str ) DO
BEGIN
SET c = SUBSTRING( s, i, 1 );
IF c = ',' THEN
SET BOOL = 1;
ELSEIF BOOL=1 THEN
BEGIN
IF c >= 'a' AND c <= 'z' THEN
BEGIN
SET s = CONCAT(LEFT(s,i-1),UCASE(c),SUBSTRING(s,i+1));
SET BOOL = 0;
END;
END IF;
END;
END IF;
SET i = i+1;
END;
END WHILE;
RETURN s;
END ||
DELIMITER ;
and use
select ucwords(`field`) from mytable
Please don't do this if you ever expect your database to scale well. If you're only ever going to be using this for your personal phone book application or equally tiny data sets, you can get away with doing this at select time, as you suggest.
But, if you want a robust scalable database, you should sacrifice space (disk storage) for time (speed). Per-row functions on select statement rarely scale well.
One time-honored and tested way of doing this is to add another column of identical specifications to the one you already have and use an insert/update trigger to populate it with the data in the format you desire.
Then the cost of the transformation is incurred only when you have to (when the data is updated), not every single time you read the data. This amortises the cost across the reads which, in the vast majority of databases, outweigh writes considerably.
This answer shows an SQL formula which seems close to what you want but I do strongly suggest that you use it in a trigger rather than inefficiently running it every select.
take a look at this solution mysql-update-script-problem-all-txt-is-lower-case-just-first-letter-is-upper-cas , it should give you enough information to adapt to what you're trying to do. Change update to SELECT, etc. etc.
I have a MySQL database and I have a query as:
SELECT `id`, `originaltext` FROM `source` WHERE `originaltext` regexp '[0-9][0-9]'
This detects all originaltexts which have numbers with 2 digits in it.
I need MySQL to return those numbers as a field, so i can manipulate them further.
Ideally, if I can add additional criteria that is should be > 20 would be great, but i can do that separately as well.
If you want more regular expression power in your database, you can consider using LIB_MYSQLUDF_PREG. This is an open source library of MySQL user functions that imports the PCRE library. LIB_MYSQLUDF_PREG is delivered in source code form only. To use it, you'll need to be able to compile it and install it into your MySQL server. Installing this library does not change MySQL's built-in regex support in any way. It merely makes the following additional functions available:
PREG_CAPTURE extracts a regex match from a string. PREG_POSITION returns the position at which a regular expression matches a string. PREG_REPLACE performs a search-and-replace on a string. PREG_RLIKE tests whether a regex matches a string.
All these functions take a regular expression as their first parameter. This regular expression must be formatted like a Perl regular expression operator. E.g. to test if regex matches the subject case insensitively, you'd use the MySQL code PREG_RLIKE('/regex/i', subject). This is similar to PHP's preg functions, which also require the extra // delimiters for regular expressions inside the PHP string.
If you want something more simpler, you could alter this function to suit better your needs.
CREATE FUNCTION REGEXP_EXTRACT(string TEXT, exp TEXT)
-- Extract the first longest string that matches the regular expression
-- If the string is 'ABCD', check all strings and see what matches: 'ABCD', 'ABC', 'AB', 'A', 'BCD', 'BC', 'B', 'CD', 'C', 'D'
-- It's not smart enough to handle things like (A)|(BCD) correctly in that it will return the whole string, not just the matching token.
RETURNS TEXT
DETERMINISTIC
BEGIN
DECLARE s INT DEFAULT 1;
DECLARE e INT;
DECLARE adjustStart TINYINT DEFAULT 1;
DECLARE adjustEnd TINYINT DEFAULT 1;
-- Because REGEXP matches anywhere in the string, and we only want the part that matches, adjust the expression to add '^' and '$'
-- Of course, if those are already there, don't add them, but change the method of extraction accordingly.
IF LEFT(exp, 1) = '^' THEN
SET adjustStart = 0;
ELSE
SET exp = CONCAT('^', exp);
END IF;
IF RIGHT(exp, 1) = '$' THEN
SET adjustEnd = 0;
ELSE
SET exp = CONCAT(exp, '$');
END IF;
-- Loop through the string, moving the end pointer back towards the start pointer, then advance the start pointer and repeat
-- Bail out of the loops early if the original expression started with '^' or ended with '$', since that means the pointers can't move
WHILE (s <= LENGTH(string)) DO
SET e = LENGTH(string);
WHILE (e >= s) DO
IF SUBSTRING(string, s, e) REGEXP exp THEN
RETURN SUBSTRING(string, s, e);
END IF;
IF adjustEnd THEN
SET e = e - 1;
ELSE
SET e = s - 1; -- ugh, such a hack to end it early
END IF;
END WHILE;
IF adjustStart THEN
SET s = s + 1;
ELSE
SET s = LENGTH(string) + 1; -- ugh, such a hack to end it early
END IF;
END WHILE;
RETURN NULL;
END
There isn't any syntax in MySQL for extracting text using regular expressions. You can use the REGEXP to identify the rows containing two consecutive digits, but to extract them you have to use the ordinary string manipulation functions which is very difficult in this case.
Alternatives:
Select the entire value from the database then use a regular expression on the client.
Use a different database that has better support for the SQL standard (may not be an option, I know). Then you can use this: SUBSTRING(originaltext from '%#[0-9]{2}#%' for '#').
I think the cleaner way is using REGEXP_SUBSTR():
This extracts exactly two any digits:
SELECT REGEXP_SUBSTR(`originalText`,'[0-9]{2}') AS `twoDigits` FROM `source`;
This extracts exactly two digits, but from 20-99 (example: 1112 return null; 1521 returns 52):
SELECT REGEXP_SUBSTR(`originalText`,'[2-9][0-9]') AS `twoDigits` FROM `source`;
I test both in v8.0 and they work. That's all, good luck!
I'm having the same issue, and this is the solution I found (but it won't work in all cases) :
use LOCATE() to find the beginning and the end of the string you wan't to match
use MID() to extract the substring in between...
keep the regexp to match only the rows where you are sure to find a match.
I used my code as a Stored Procedure (Function), shall work to extract any number built from digits in a single block. This is a part of my wider library.
DELIMITER $$
-- 2013.04 michal#glebowski.pl
-- FindNumberInText("ab 234 95 cd", TRUE) => 234
-- FindNumberInText("ab 234 95 cd", FALSE) => 95
DROP FUNCTION IF EXISTS FindNumberInText$$
CREATE FUNCTION FindNumberInText(_input VARCHAR(64), _fromLeft BOOLEAN) RETURNS VARCHAR(32)
BEGIN
DECLARE _r VARCHAR(32) DEFAULT '';
DECLARE _i INTEGER DEFAULT 1;
DECLARE _start INTEGER DEFAULT 0;
DECLARE _IsCharNumeric BOOLEAN;
IF NOT _fromLeft THEN SET _input = REVERSE(_input); END IF;
_loop: REPEAT
SET _IsCharNumeric = LOCATE(MID(_input, _i, 1), "0123456789") > 0;
IF _IsCharNumeric THEN
IF _start = 0 THEN SET _start = _i; END IF;
ELSE
IF _start > 0 THEN LEAVE _loop; END IF;
END IF;
SET _i = _i + 1;
UNTIL _i > length(_input) END REPEAT;
IF _start > 0 THEN
SET _r = MID(_input, _start, _i - _start);
IF NOT _fromLeft THEN SET _r = REVERSE(_r); END IF;
END IF;
RETURN _r;
END$$
If you want to return a part of a string :
SELECT id , substring(columnName,(locate('partOfString',columnName)),10) from tableName;
Locate() will return the starting postion of the matching string which becomes starting position of Function Substring()
I know it's been quite a while since this question was asked but came across it and thought it would be a good challenge for my custom regex replacer - see this blog post.
...And the good news is it can, although it needs to be called quite a few times. See this online rextester demo, which shows the workings that got to the SQL below.
SELECT reg_replace(
reg_replace(
reg_replace(
reg_replace(
reg_replace(
reg_replace(
reg_replace(txt,
'[^0-9]+',
',',
TRUE,
1, -- Min match length
0 -- No max match length
),
'([0-9]{3,}|,[0-9],)',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
'^[0-9],',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
',[0-9]$',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
',{2,}',
',',
TRUE,
1, -- Min match length
0 -- No max match length
),
'^,',
'',
TRUE,
1, -- Min match length
0 -- No max match length
),
',$',
'',
TRUE,
1, -- Min match length
0 -- No max match length
) AS `csv`
FROM tbl;