I have a string like so;
"field1=lance,field2=peter,field3=john"
The actual string has 20+fields, and I want to pull out specific values by name.
For instance, I want to pull out the value for "field2" and return the value "peter",
Can someone give me an elegant way of doing this in MySQL?
I should mention that this is a standard field format coming out of an eCommerce system. I have no control over the format. It would be possible to extract the data cleanly through the API, but that would be significant extra work, especially as I have the data already in this format.
interesting question. There is a lot below so let's break it down. We essentially build a query and execute the stmt
DELIMITER $$
DROP PROCEDURE IF EXISTS proc_loop_test$$
CREATE PROCEDURE proc_loop_test()
#create empty query string
set #sqlstring = '';
#set your string of fields, not sure where this comes from
set #mystring = 'field1=lance,field2=peter,field3=john';
#create number of times we will loop through the string
set #num = (select length(#mystring)
- length(replace('field1=lance,field2=peter,field3=john',',','')) +1);
#init loop
loop LOOP
#create a short string just taking the "last" field/value pair
set #shtstring = (select SUBSTRING_INDEX(#mystring,',',-1));
#recreate your query string removing the substring we created
set #mystring = (select left(replace(#mystring,#shtstring,''),length(replace(#mystring,#shtstring,''))-1));
#add to your query string, we will build this for each
set #sqlstring = concat(#sqlstring ,(select
concat('''',SUBSTRING_INDEX(#shtstring,'=',-1),''''
,' as ',
left(#shtstring,(position('=' in #shtstring) -1))) ),',');
#reduce our count by one as we have removed the latest field
set #num = #num - 1;
#leave the loop when no fields left
if #num = 0 then leave LOOP;
end if;
end loop LOOP;
END$$
DELIMITER ;
#create a query statement to execute
set #query = (select concat('select ',left(#sqlstring, length(#sqlstring)-1)));
#execute the query!
PREPARE stmt FROM #query;
EXECUTE stmt;
Result
field3 field2 field1
john peter lance
There is no array logic, this would be simple in presto SQL etc. Because you have an arbitrary number of fields being defined at any time we are going to need to loop, and unfortunately you cannot loop in mysql without creating a procedure
That is the first few lines. We also create our full string from your source and the number of iterations (number of fields in string).
Then basically we isolate the "last" field/value pair iterively, rearrange each one so field1=john turns into more sql friendly 'john' as field',
We reduce our counter and string each time we loop through until counter is 0. At which point we stop.
We then prepare a query with our value/field pairs and a 'select' string. Then execute and you get your values as fields
Credit
Dynamic Strings prepare/exec
Looping and stored procs
Simulating Split function
This answer from #Akina's comment works perfectly.
SUBSTRING_INDEX(SUBSTRING_INDEX(column, 'field2=', -1), ',', 1)
And WHERE accordingly.
Related
I have been trying to create a simple loop of SELECT statements in MySQL to reduce code. I have started this using CONCAT() however this causes the procedure to stop/fail. For example (where k is a loop counter):
CONCAT('SELECT (Child_', k, ' INTO #Age_Child_', k, ' FROM lookup_childage WHERE ModYear = ModYear_var LIMIT 1)');
To diagnose the issue, I simply tried to place the SELECT statement (without concatenated loop variables) inside a string to then be executed. While I could get this to work for simple statements it would not work for the following:
SET #queryString = CONCAT('SELECT Child_1 INTO #Age_Child_1 FROM lookup_childage WHERE ModYear = ModYear_var LIMIT 1');
PREPARE stmt FROM #queryString;
EXECUTE stmt;
Does anyone know why the #queryString containing the CONCAT() statement will not be executed/cause the procedure to fail?
tl;dr The statement you're trying to write has the form SELECT(rest of statement) LIMIT 1. It should have the form SELECT rest of statement LIMIT 1.
It looks like you want to create variable column names, ummm, because your lookup_childage table is denormalized. I guess that table has these columns.
Child_1 INT
Child_2 INT
Child_3 INT
Child_4 INT
It looks like you hope to get a #queryString value containing this sort of thing:
SELECT Child_4 INTO #Age_Child_4 FROM lookup_childage WHERE ModYear = ModYear_var LIMIT 1
Only the 4s are variable.
So to get that string you want
SELECT CONCAT('SELECT Child_', k,
' INTO #Age_Child_', k,
' FROM lookup_childage WHERE ModYear = ModYear_var LIMIT 1'
)
INTO #queryString;
I have the following code so far:
delimiter //
CREATE FUNCTION Iteration(InputArray CHAR)
RETURNS DECIMAL
BEGIN
RETURN
SELECT Max(IF(Stock = InputArray, ValueFrom, NULL)) AS Stock FROM DatabaseName.TableName LIMIT 1;
END
delimiter ;
SET #Stocks = (SELECT ColumnName FROM DatabaseName.TableName LIMIT 1);
SELECT Iteration(#Stocks)FROM DatabaseName.TableName;
What I'm interested in doing is passing an array to the Iteration function, which would then be operated on by looking up the corresponding value in the DatabaseName database, spitting back out a corresponding decimal for each value in the input array-- in other words, one array in, another array out.
I think I instantiated everything correctly, but I keep getting the following error:
Error Code: 1305. FUNCTION applications.Iteration does not exist
For example, I have the variables AAA, BBB, CCC, etc, which are stock inventory codes. I want to pass an array of these variables to the procedure/function, and then go onto get back an array as well. In this case, it would be 1.7, 1.3, and 1.8.
There's a few questions I have:
Does this make sense-- that is, can I pass an array to a function like this?
If 1. is yes, am I passing the array to the function properly?
Why am I getting this error?
First, we need to nail down the logic you actually want for the SQL query. You want to take the MAX value of the ValueFrom column, but only for those records where the Stock is contained in the input array. This can be accomplished by doing a simple SELECT, along with a WHERE clause which filters out the non matching records, e.g.
SELECT ValueFrom
FROM DatabaseName.TableName
WHERE Stock IN ('stock1', 'stock2', 'stock3')
Of course, we will need to replace the list of stocks with the input array. I would use a stored procedure here:
CREATE PROCEDURE Iteration(IN inputArray VARCHAR(255))
BEGIN
SET #sql = CONCAT('SELECT ValueFrom FROM DatabaseName.TableName WHERE Stock IN (', inputArray, ')');
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END
$$
DELIMITER ;
Usage:
SET #inputArray = '\'stock1\',\'stock2\',\'stock3\'';
CALL Iteration(#inputArray);
I have the following problem, this is going to be long, I want to tell exactly all what I know about my problem in my question.
I have a table, field_body_value, with two fields, body_value and body_summary, containing strings of the form "/webfm_send/#" where # is a number.
I have another table called webfm_file where I have two fields with information for the string substitution: the first one is called fid, and it is the number # that I mentioned before, and the second is called fpatch, and gives me a string holding a path (for instance /data/html/files/file1.pdf) which has to substitute /webfm_send/# in the first table. The numbers # go up over the records of webfm_file but there are jumps, that is they increase but there are missing # so the final # is not equal to the number of records in webfm_file
So I thought the strategy was to set up a procedure which loops over the second table, and at each step of the sequence retrieves the pair fid/fpath, searches for "/webfm_send/fid" in the first table, and substitutes this by fpath in the first table.
So this is as far a I could arrive with my coding:
BEGIN
DECLARE v1 INT DEFAULT 0;
SELECT COUNT(*) INTO #numrec FROM `webfm_file`;
WHILE v1 < #numrec DO
SELECT fpath,fid INTO #path,#file FROM `webfm_file` LIMIT v1,1;
SET #webfm = concat('/webfm_send/',#file);
SET #cpath = concat('/',#path);
UPDATE `field_data_body`
SET body_value = replace(body_value, #webfm, #cpath),
body_summary = replace(body_summary, #webfm, #cpath)
WHERE body_value LIKE concat('%',#webfm,'%') OR
body_summary LIKE concat('%',#webfm,'%');
SET v1 = v1 + 1;
END WHILE;
END
Let me explain what I think I'm doing with the code above:
1) I retrieve the number of records in webfm_file for the loop.
2) The first SELECT gets a pair in fpath/fid from webfm_file, with LIMIT v1,1 I just check one record at a time, I checked an it works, the while loops over each record of webfm_file and the records are retrieved correctly.
3) The two next "set" fix the pair of strings #file/#path to create #webfm whith is the way its written in body_value at field_body_value, and to put a slash in front of #cpath which is the way I need this string to finally appear.
4) Then comes the UPDATE which will actually substitute the string if it finds it in either body_value or body_summary of field_body_data.
Expected: each instance of /webfm_send/# is substituted by the corresponding fpath pair of # (fid) in webfm_file
What I actually get: All appearances of /webfm_send/# no matter the value of # are substituted by the value of fpath in record 1 of webfm_file.
Things I have tried:
1) Take out the "WHERE" clause in the UPDATE sentence, which I believe is not strictly necessary since the replace function already takes care of finding a match but could speed up things. Same result
2) Resctrict the loop to loop just over a single record of webfm_file. Here it works in substituting the corresponding single retrieved pair fid/fpath, in the two instances of body_value and body_summary in field_body_data where fid=# appears in the string webfm_send/#
Thanks for following my explanation until here and thanks in advance for any hint.
You could use a cursor to iterate over the replacement strings. (There are faster ways using group_concat and it would be easier to do this in a general-purpose language rather than in a stored procedure). The general cursor approach would be:
drop procedure if exists proc;
delimiter //
create procedure proc()
begin
declare done boolean default 0;
declare path varchar(255);
declare id int;
declare cur cursor for select fpath, fid from webfm_file order by fid desc;
declare continue handler for sqlstate '02000' set done = 1;
open cur;
block: loop
fetch cur into path, id;
if done then
leave block;
end if;
set #from = concat('/webfm_send/', id);
update field_data_body set
body_value = replace(body_value, #from, path),
body_summary = replace(body_summary, #from, path);
end loop;
close cur;
end//
delimiter ;
call proc();
Some stored procedures I work with need to interpolate WHERE criteria based on if procedure input parameters have been supplied. To avoid potential injection points, I'd like to utilize parameter binding for the values that are to be part of the interpolated criteria.
Since the criteria added to the prepared statement and thus the number of parameters to be bound may differ depending on the user input, I devised the method below to determine which variables will be passed to the EXECUTE statement. This works, but it seems inelegant.
CREATE PROCEDURE foo (IN mandatory INT, IN optional INT, IN optional2 VARCHAR(20))
BEGIN
SELECT
0, '', '', mandatory, optional, optional2
INTO
#params, #sql, #where, #m, #o1, #o2;
IF (#o1 > '' AND #o1 IS NOT NULL) THEN
SET #where = CONCAT(#where, ' AND field = ?');
SET #params = #params + 1;
END IF;
IF (#o2 > '' AND #o2 IS NOT NULL) THEN
SET #where = CONCAT(#where, ' AND field2 = ?');
SET #params = #params + 3;
END IF;
SET #sql = CONCAT('
SELECT id, bar FROM table
WHERE
baz = ?
', #where
);
PREPARE STMT FROM #sql;
CASE #params
WHEN 0 THEN EXECUTE STMT USING #m;
WHEN 1 THEN EXECUTE STMT USING #m, #o1;
WHEN 3 THEN EXECUTE STMT USING #m, #o2;
WHEN 4 THEN EXECUTE STMT USING #m, #o1, #o2;
END CASE;
DEALLOCATE PREPARE STMT;
END$$
I'm aware of alternatives:
The binaries that would call these stored procedures have a function that attempts to identify potential SQL injection by passing the user supplied strings through a regular expression.
A user-defined function could be used to dynamically construct the EXECUTE statement given a dynamic number of inputs.
However, I was wondering if anyone else has ran into this desire to handle dynamic construction of an EXECUTE statement purely with SQL.
However, I was wondering if anyone else has ran into this desire to handle dynamic construction of an EXECUTE statement purely with SQL.
Yes, me too.
Here's a PHP solution to generate the list of question marks for a prepared statement based on an array of unknown length:
/* My target query is this:
SELECT fun FROM fun_stuff WHERE fun_key IN ( ...unknown number of values... )
*/
/* For this example let's set our array to this: */
$val_arr = array(1,2,3,4,5,6,7,8,9);
$val_arr_cnt = count($val_arr); /* and count it */
/* Now make prepared statement q-mark string from values array */
$sql_prep = str_pad('?', ($val_arr_cnt * 2) - 1, ',?', STR_PAD_RIGHT);
/* add it to query */
$sql = "SELECT fun FROM fun_stuff WHERE fun_key IN ($sql_prep)";
/* And the result:
SELECT fun FROM fun_stuff WHERE fun_key IN (?,?,?,?,?,?,?,?,?)
*/
I've no idea how efficient this is. But I too every now and then want to implement the security and efficiency of MySQL prepared statements but have variable length input arrays
Not sure if it's possible to dynamically build the parameter list (changing the number of parameters on the fly, etc...). But since you can dynamically build your where clause, one very simple workaround is do something like this. Assuming your validations permit it, the else clause basically has the same effect as ignoring the parameter you may or may not be filtering on.
if p_cust_id is not null && p_cust_id > 0 then
set v_where_clause = concat(v_where_clause, ' c.cust_id = ? ');
set #v_cust_id := p_cust_id;
else
set v_where_clause = concat(v_where_clause, ' c.cust_id > ? ');
set #v_cust_id := 0;
end if;
then plug all the user variables above into your execute statement
execute str1 using #v_cust_id, #v_etc....;*
I'm doing a SELECT INTO OUTFILE and it's showing a "\N" for every NULL value. Is there any way for me to make it just be blank instead?
I'm on MySQL.
You can use the COALESCE function something like this:
COALESCE(yourfield, '')
Good question, and as relevant today as it was in 2011. I use a stored procedure for this, wich takes the name of a table as argument. Subsequently, it converts all null-values to empty strings (''), using a prepaired statement and no cursors, for cursors are so non-SQL. I've tested it with a table with some 200 columns and a resulting prepaired statement that's about 15,000 characters long:
CREATE DEFINER=`root`#`localhost` PROCEDURE `ExportFixNull`(
in tblname tinytext)
BEGIN
set #string=concat(
"Update ",#tblname," set ",
(
select group_concat(column_name,"=ifnull(",column_name,",'')")
from information_schema.columns
where table_name=#tblname
)
,";"
);
prepare s1 from #string;
execute s1;
drop prepare s1;
In the main SQL-file, there is a statement
SET ##group_concat_max_len = 60000;
that might be crucial.
More details (in Dutch): http://wiki.devliegendebrigade.nl/SELECT_INTO_OUTFILE_%28MySQL%29
Regards,
Jeroen Strompf