removing adjacent newline characters sql - mysql

Im trying to remove newline characters when the next character is also a newline
using the Function I wrote below, and im running into a problem. When I input this string :
"Line1
Line2
Line3
Line4"
It removes the newlines after line 1 and 2 , but the lines between 3 and 4 remain ? Any ideas as to why the function doesn't work then ?
DELIMITER $$
DROP FUNCTION IF EXISTS removeLines;
CREATE FUNCTION test_dev.removeLines(address varchar(255))
RETURNS varchar(255)
DETERMINISTIC
BEGIN
DECLARE i int;
SET i = 0;
WHILE i < length(address)-1 DO
IF(((SUBSTR(address,i,1) in (CHAR(10) ,CHAR(13))) AND (SUBSTR(address,i+1,1) in (CHAR(10) ,CHAR(13))) ) )
THEN set address = INSERT(address,i,1,'');
SET i= i+1;
ELSE set i= i+1;
END IF;
END WHILE;
RETURN address;
END$$
DELIMITER ;

An alternate way to write your function so you don't have to manually keep track of character indexing.
while instr(address, '\r\n\r\n') do
address = replace(address, '\r\n\r\n', '\r\n');
end while;
I didn't have a chance to test this in a database so the syntax may not be 100% correct.

The problem is that you increment i after you replace a newline. Consider the following simple string:
1\n\n\n2
01 2 3 4 -- indexes
When i is 1, you see that there are newlines at indexes i=1 and i+1=2, so you remove the first one with INSERT(). Now the string is:
1\n\n2
01 2 3
and you do SET i = i + 1. Now i is 2, but indexes i=2 and i+1=3 don't have newlines, so you don't remove the next newline.
Change it so that you only increment i when you don't find a pair of newlines.
IF(((SUBSTR(address,i,1) in (CHAR(10) ,CHAR(13))) AND (SUBSTR(address,i+1,1) in (CHAR(10) ,CHAR(13))) ) )
THEN set address = INSERT(address,i,1,'');
ELSE set i= i+1;
END IF;

Related

MYSQL - Parse comma separated string of numbers and compare each to set number

I have a requirement in MySQL to take a comma separated string of numbers ("2000, 2001, 2002, 2003, 2004") passed into a stored procedure and compare each one to a another number e.g. 2005. If any of the numbers are greater than or equal to my target 2005, I need to execute code, else leave the loop. I cannot figure out how to iterate over this comma delimited string, turn each into a number using CAST(), and compare it to my target number.
Assuming the argument passed into the procedure is named arg_list ...
If the requirement is to determine if just some (any number other than zero) entries in the list is greater than 2005,
Then we can do something like this in the procedure:
a loop iterator and some work areas:
DECLARE i_ INT DEFAULT 0;
DECLARE ls_number VARCHAR(255) DEFAULT '';
DECLARE ls_greatest VARCHAR(255) DEFAULT '';
initialize, get first number in list, first number in list is greatest so far:
SET i_ := 1;
SET ls_number := TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(CONCAT( arg_list ,','),',',i),',',-1));
SET ls_greatest := ls_number;
WHILE ls_number > '' DO
-- evaluate as numeric and compare
IF ls_number+0 > ls_greatest+0 THEN
-- the one we just got is the biggest one so far, so keep it
SET ls_greatest := ls_number;
END IF;
-- get next number in list
SET i_ := i_ + 1;
SET ls_number := TRIM(SUBSTRING_INDEX(SUBSTRING_INDEX(CONCAT( arg_list ,','),',',i),',',-1));
END WHILE;
-- ls_greatest now contains the largest number from the list
IF ls_greatest+0 >= 2005 THEN
-- do some code
END IF;
Note: this assumes that the list in arg_list is well formed, and doesn't contain spurious commas, e.g. if the list was '1,2,3,,4,5' the loop would exit after processing the 3
Defined the procedure as follows.
DELIMITER //
CREATE PROCEDURE getValue(txt varchar(100), abc varchar(100))
BEGIN
SET #i = 1;
SET #txt = REPLACE(txt, '"', '');
SET #last_element = SUBSTRING_INDEX(#txt, ",", -1);
WHILE #txt != '' DO
SET #element = SUBSTRING_INDEX(#txt, ",", #i);
IF CAST(abc AS UNSIGNED) >= CAST(#element AS UNSIGNED) THEN
# execute your code
SET #txt = '';
ELSE
IF !LOCATE(',',#txt) && #element = #last_element THEN
# #element = #last_element comparison is not necessary
SET #txt = '';
ELSE
SET #txt = REPLACE(#txt, CONCAT(#element,","), '');
END IF;
END IF;
END WHILE;
END //
DELIMITER ;
Tried the following two procedure calls.
call getValue("200,400,100","100");
call getValue("200,400,600","100");

It is possible to sort particular cell value in mySql?

I need to sort below cell values using mysql
Example:
cell contain red,blue,green
But I want that in alphabetic order.
Steps to do this,
1.First you need to make a procedure call for sorting values
2.Call your procedure then
Here is the code to create mysql procedure
-- sort comma separated substrings with unoptimized bubble sort
DROP FUNCTION IF EXISTS sortString;
DELIMITER |
CREATE FUNCTION sortString(inString TEXT) RETURNS TEXT
BEGIN
DECLARE delim CHAR(1) DEFAULT ','; -- delimiter
DECLARE strings INT DEFAULT 0; -- number of substrings
DECLARE forward INT DEFAULT 1; -- index for traverse forward thru substrings
DECLARE backward INT; -- index for traverse backward thru substrings, position in calc. substrings
DECLARE remain TEXT; -- work area for calc. no of substrings
-- swap areas TEXT for string compare, INT for numeric compare
DECLARE swap1 TEXT; -- left substring to swap
DECLARE swap2 TEXT; -- right substring to swap
SET remain = inString;
SET backward = LOCATE(delim, remain);
WHILE backward != 0 DO
SET strings = strings + 1;
SET backward = LOCATE(delim, remain);
SET remain = SUBSTRING(remain, backward+1);
END WHILE;
IF strings < 2 THEN RETURN inString; END IF;
REPEAT
SET backward = strings;
REPEAT
SET swap1 = SUBSTRING_INDEX(SUBSTRING_INDEX(inString,delim,backward-1),delim,-1);
SET swap2 = SUBSTRING_INDEX(SUBSTRING_INDEX(inString,delim,backward),delim,-1);
IF swap1 > swap2 THEN
SET inString = TRIM(BOTH delim FROM CONCAT_WS(delim
,SUBSTRING_INDEX(inString,delim,backward-2)
,swap2,swap1
,SUBSTRING_INDEX(inString,delim,(backward-strings))));
END IF;
SET backward = backward - 1;
UNTIL backward < 2 END REPEAT;
SET forward = forward +1;
UNTIL forward + 1 > strings
END REPEAT;
RETURN inString;
END |
DELIMITER ;
To make procedure call just you have to use,
-- example call:
SET #Xstr = "red,blue,green"; // for query purpose only you need to write within (SQL Query here for that row)
SELECT sortString(#Xstr) AS s1
Please see the documentation guide map
Click here to read
Also there is an alternative way to do if you are interested to study is that about FIND_IN_SET, please you can find some idea from one of the question from stackoverflow. Click here to read
You can create a function which sorts the items in the column:
create function f_comma_list_order ( t text )
returns text
begin
declare v_c int;
drop temporary table if exists tmp;
create temporary table tmp ( v text );
set v_c = 1;
while( v_c > 0 ) do
select locate(',', t) into v_c;
if (v_c>0) then
insert into tmp select substr(t, 1, v_c-1);
set t = substr(t, v_c+1);
else
insert into tmp values (t);
end if;
end while;
select group_concat(v order by v) into t
from tmp;
return t;
end
and then call the function:
select f_comma_list_order('red,green,blue')

phpMyAdmin - mariaDB roman numerals function

can anybody help me with my sorting function - seriously I don't know how can I make it work as supposed to. :( Database is in MariaDB in Xampp. I use phpMyAdmin to execute the query.
DELIMITER $$
DROP FUNCTION IF EXISTS convRomanNumeral$$
CREATE FUNCTION convRomanNumeral (numeral CHAR(4))
RETURNS INT
BEGIN
DECLARE intnum INT;
CASE numeral
WHEN "I" THEN intnum = 1;
WHEN "II" THEN intnum = 2;
END CASE;
RETURN intnum;
END;
$$
SET #iteration = -1;
UPDATE `st0gk_docman_documents`
SET created_on = DATE('2016-06-14') + INTERVAL(#iteration := #iteration + 1) SECOND
WHERE `docman_category_id` = 141 ORDER BY convRomanNumeral(SUBSTRING(SUBSTRING_INDEX(title,'/',1),' ',-2) ASC, SUBSTRING_INDEX(title,'/',-2)+0 ASC;
So what I want to achieve is to sort documents by title. Example titles are:
Document Nr I/36/2006
Document Nr II/36/2006
Document Nr I/32/2006
Document Nr II/19/2006
After sorting them by first Roman number and then by second Arabic number I want to update the date. Code below for updating by only second Arabic number works properly:
SET #iteration = -1;
UPDATE `st0gk_docman_documents`
SET created_on = DATE('2016-06-14') + INTERVAL(#iteration := #iteration + 1) SECOND
WHERE `docman_category_id` = 141 ORDER BY SUBSTRING_INDEX(title,'/',-2)+0 ASC;
I would like to use CASE to return proper variable for Roman values. I know it's not perfect but I can't even make the CASE and FUNCTION work. What I am doing wrong? All suggestions are welcome.
The best way to do this is to add another column that has a sortable equivalent of that string. And use non-SQL code to do the parsing and building of that column before inserting into the table.
First mistake that I was making it was trying to execute the whole query at once... After taking the first lodge out of the way the debugging seemed way simpler. :D
So I created my case function to convert Roman numerals:
DELIMITER $$
DROP FUNCTION IF EXISTS convRomanNumeralSubFunction$$
CREATE FUNCTION convRomanNumeralSubFunction (numeral CHAR(1))
RETURNS INT
BEGIN
DECLARE intnum INT;
CASE numeral
WHEN "I" THEN SELECT 1 INTO intnum;
WHEN "X" THEN SELECT 10 INTO intnum;
WHEN "C" THEN SELECT 100 INTO intnum;
WHEN "M" THEN SELECT 1000 INTO intnum;
WHEN "V" THEN SELECT 5 INTO intnum;
WHEN "L" THEN SELECT 50 INTO intnum;
WHEN "D" THEN SELECT 500 INTO intnum;
END CASE;
RETURN intnum;
END;
$$
After that I declared the second function needed for conversion. I don't know if You can declare function inside function... and I didn't want to waste more time on this. For sure You can declare Function inside Procedure. Anyhow. WARNING: This function is not proof of BAD numerals like IIX. Numerals like that or will be badly counted. Also AXI will not count.
DELIMITER $$
DROP FUNCTION IF EXISTS convRomanNumeral$$
CREATE FUNCTION convRomanNumeral (numeral CHAR(10))
RETURNS INT
BEGIN
DECLARE currentintnum, previntnum, intnum, counter, numerallength INT;
SET numerallength = LENGTH(numeral);
SET counter = numerallength;
SET intnum = 0;
SET previntnum = 0;
WHILE counter > 0 DO
SET currentintnum = CAST(convRomanNumeralSubFunction(SUBSTRING(numeral,counter, 1)) as integer);
IF currentintnum < previntnum THEN
SET intnum = intnum - currentintnum;
ELSE
SET intnum = intnum + currentintnum;
END IF;
SET previntnum = currentintnum;
SET counter = counter - 1;
END WHILE;
RETURN intnum;
END;
$$
So that's it. Now You can convert all kind of Roman numerals and sort them up.
Use this to test the conversion:
SELECT convRomanNumeral("XIX");
This is example sorting code that I in the end used:
SET #iteration = -1;
UPDATE `st0gk_docman_documents`
SET created_on = DATE('2016-06-07') + INTERVAL(#iteration := #iteration + 1) SECOND
WHERE `docman_category_id` = 67 ORDER BY convRomanNumeralBreak(SUBSTRING_INDEX(SUBSTRING_INDEX(title,'/',1),' ',-1)) ASC, SUBSTRING_INDEX(title,'/',-2)+0 ASC;
Also one more thing - if You'll try to excecute this on mySQL then You have to fix this line:
SET currentintnum = CAST(convRomanNumeralSubFunction(SUBSTRING(numeral,counter, 1)) as integer);
into this:
SET currentintnum = CAST(convRomanNumeralSubFunction(SUBSTRING(numeral,counter, 1)) as SIGNED);
This code could be improved but as the #Rick James stated this should be done differently - not in as db update but in different table structure and sorting mechanism.

Generate a random string in MySQL

I'm trying to get a random string in phpmyadmin using a function.
I have the following code:
CREATE FUNCTION randomPassword()
RETURNS varchar(128)
BEGIN
SET #chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
SET #charLen = length(#chars);
SET #randomPassword = '';
WHILE length(#randomPassword) < 12
SET #randomPassword = concat(#randomPassword, substring(#chars,CEILING(RAND() * #charLen),1));
END WHILE;
RETURN #randomPassword ;
END;
Now I get the error:
1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 5
Does anyone know how I can fix this?
This is faster than concat + substring routine.
select substring(MD5(RAND()),1,20);
As I've tested inserting 1M random data, md5 routine consumes only 1/4 (even less) time of concat + substring routine;
The problem is a md5 string contains only 32 chars so if you need a longer one you'd have to manually generate more md5 strings and substring it yourself.
Try this more simple solution:
SELECT CONV(FLOOR(RAND() * 99999999999999), 10, 36)
SELECT SUBSTRING(REPLACE(REPLACE(REPLACE( TO_BASE64(MD5(RAND())), '=',''),'+',''),'/',''), 2, 40)
This solution to generate a fixed length random string that contains all lower- and upper-case chars and digits.
SELECT SUBSTRING(REPLACE(REPLACE(REPLACE( TO_BASE64(MD5(RAND())), '=',''),'+',''),'/',''), 2, FLOOR(10+RAND()*31))
If you need a random length string (from 10 to 40 symbols in this example)
It's solved by using the Delimiter, i don't know for sure how, but it works
Thanks
DELIMITER $$
CREATE FUNCTION randomPassword()
RETURNS varchar(128)
BEGIN
SET #chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
SET #charLen = length(#chars);
SET #randomPassword = '';
WHILE length(#randomPassword) < 12
DO
SET #randomPassword = concat(#randomPassword, substring(#chars,CEILING(RAND() * #charLen),1));
END WHILE;
RETURN #randomPassword ;
END$$
DELIMITER ;
CREATE FUNCTION randomPassword()
RETURNS varchar(128)
AS
BEGIN
declare #chars nvarchar(25);
declare #charlen int;
declare #randomPassword nvarchar(128);
SET #chars = 'ABCDEFGHIJKLMNOPQRSTUVWXYZ';
SET #charLen = len(#chars);
SET #randomPassword = '';
WHILE(LEN(#randomPassword) < 12)
BEGIN
SET #randomPassword = concat(#randomPassword, substring(#chars,CEILING(RAND() * #charLen),1));
END
RETURN #randomPassword
END

Convert all characters except a set of characters to whitespace in MySQL

I have a set of characters that are defined as valid characters. Let me define the valid string by the following regexp:
^[a-zA-Z0-9\ .-_]+$
(alphanumeric, space, dot, dash and underscore)
The question is that given a column containing a lot of invalid characters, how I can run an update to convert each invalid character to one space? And then possibly convert consequent spaces to one space?
I cannot run several replace commands because there are a lot of possible invalid characters. So I expect a regexp solution.
Currently, I am doing the task in Java (after exporting the table to tsv format). But I want a MySQL approach.
If your MySQL version supports it, create a function:
DELIMITER $$
CREATE FUNCTION my_func_1 (str TEXT)
RETURNS TEXT
BEGIN
DECLARE ret TEXT DEFAULT '';
DECLARE chr TEXT DEFAULT '';
DECLARE i INT DEFAULT 1;
WHILE i < (LENGTH(str) + 1) DO
SET chr = SUBSTRING(str, i, 1);
IF chr REGEXP '[-a-zA-Z0-9\\_.]'
THEN SET ret = CONCAT(ret, chr);
ELSE
SET ret = CONCAT(ret, ' ');
END IF;
SET i = i + 1;
END WHILE;
WHILE ret LIKE '% %' DO
SET ret = REPLACE(ret, ' ', ' ');
END WHILE;
RETURN TRIM(ret);
END;
$$
DELIMITER ;
Test it a bit:
SELECT my_func_1('$a-B\\?!=01._%'); > a-B\ 01._
and update with SET col = my_func_1(col)
If not needed anymore:
DROP FUNCTION IF EXISTS my_func_1;
Also I changed your regex a bit as - indicates a range, if between characters in a class or is .-_ intended? Then modify the pattern.