MySQL Prepared Statements Appeding Where Condition Not Working - mysql

Here is the actual query
select taken_date, DATE_FORMAT(taken_date, '%Y') taken_date_year, count(id) num_of_orders, sum(total_order_days) total_work_days from
(
select id, taken_date, getNumOfWorkDaysForOrder(order.order_num) total_order_days from order
where order.is_active = 1 and order.deleted_at is null and order.vendor_id = vendor_input and
order.company_id = company_input and order.contact_id = contact_input and order.candidate_id = candidate_input
order by taken_date
) as order_years group by YEAR(taken_date) order by taken_date desc;
I want to add where condition based on the input if it is not null, tried prepared statements and concatenation to add the where condition to the query but no luck.
DELIMITER $$
CREATE PROCEDURE
getAllActiveOrdersGroupByTakenDate(vendor_input INT, company_input INT, contact_input INT, candidate_input INT)
BEGIN
SET #prepareQuery = "select id, taken_date, getNumOfWorkDaysForOrder(order.order_num) total_order_days from order
where order.vendor_id = "+ vendor_input +" and order.is_active = 1 and order.deleted_at is null";
IF company_input IS NOT NULL THEN
SET #prepareQuery = CONCAT(#prepareQuery, ' ', "and order.company_id = "+company_input);
END IF;
IF contact_input IS NOT NULL THEN
SET #prepareQuery = CONCAT(#prepareQuery, ' ', "and order.contact_id = "+contact_input);
END IF;
IF candidate_input IS NOT NULL THEN
SET #prepareQuery = CONCAT(#prepareQuery, ' ', "and order.candidate_id = "+candidate_input);
END IF;
SET #finalQueryPart1 = CONCAT("select taken_date, DATE_FORMAT(taken_date, '%Y') taken_date_year, count(id) num_of_orders, sum(total_order_days) total_work_days from
(", #prepareQuery);
SET #finalQuery = CONCAT(#finalQueryPart1, ") as order_years group by YEAR(taken_date) order by taken_date desc");
PREPARE stmt FROM #finalQuery;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END $$
DELIMITER ;
Can someone help me to achieve this?

Update: you had an issue with your CONCAT() syntax before you edited your question.
When you want to append content, you must assign it back to the original string. CONCAT() is a function that returns the concatenated string. It does not have any side-effect of modifying the variable you use as an argument.
WRONG:
CONCAT(#prepareQuery, ' ', "and order.company_id=company_input");
RIGHT:
SET #prepareQuery = CONCAT(#prepareQuery, ' ', "and order.company_id=company_input");
Also, I'm not sure if you can reference the procedure input parameters in these expressions.
Frankly, I hardly ever use stored procedures. MySQL's implementation of stored procedures sucks. It's inefficient, doesn't save compiled procedures, there's no debugger, there are no packages, and so on.
Mostly I just execute dynamic SQL from my applications. There you have debugging, code reuse, familiar string manipulation in a familiar language.
I understand that stored procedures are the tradition in Oracle and Microsoft SQL Server communities, but it's really better to avoid stored procedures in MySQL.

Related

MySQL GROUP_CONCAT 'Invalid use of Group function' error that DOESN'T involve an aggregate function

I have a MySQL stored procedure (shown below) that's supposed to build a list of IDs from a table of hierarchically related records. I'm having to re-engineer an older stored procedure to switch from using a simple CONCAT function to GROUP_CONCAT because the former couldn't handle the sizes of the lists being generated (i.e., the lists are going well over the 1024 character limit of the CONCAT function).
DELIMITER $$
CREATE PROCEDURE SPTest (top_id INT)
BEGIN
DECLARE ids_all TEXT;
SET SESSION group_concat_max_len = 1000000;
SET ids_all = top_id;
SET #str = GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR \', \') ',
'FROM tbl WHERE nha_id = ', top_id, ' INTO #ids_tmp' SEPARATOR '');
PREPARE stmt FROM #str;
EXECUTE stmt;
WHILE #ids_tmp != "" DO
SET ids_all = GROUP_CONCAT(ids_all, #ids_tmp SEPARATOR ', ');
SET #str = GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR \', \') ',
'FROM tbl WHERE nha_id IN (', #ids_tmp, ') INTO #ids_tmp' SEPARATOR '');
PREPARE stmt FROM #str;
EXECUTE stmt;
END WHILE;
SELECT ids_all AS ids;
END $$
DELIMITER ;
The problem is this routine is generating the following error when I try to call it (and sometimes it returns this error when I try to create the Stored Procedure):
ERROR 1111 (HY000): Invalid use of group function
When I manually create the same kinds of queries this code would build and run them at the command-line, they work perfectly fine -- no errors of any kind, and I get the results I expect. I've seen posts (both here on Stack Exchange and elsewhere) which say that MySQL doesn't support nested aggregate functions, but the concatenation here is just being done to strings. So I thought that maybe somehow it was seeing the GROUP_CONCAT in the string and burping because of that, so I tried putting "XXXX" in place of "GROUP_CONCAT" in the string and then using the REPLACE function to switch it back, but that made no difference. I also tried it with WHERE and HAVING for the criteria clause, but neither one worked. I've also done an extensive web search and was unable to find anything that was in any way helpful.
I don't know what else to try, mainly because I can't see what's wrong with the syntax here. Any help would be appreciated.
Update 1:
I have since tried a modified version of the script where the GROUP_CONCAT merges data from a subquery like this:
SET #qrystr = GROUP_CONCAT('SELECT GROUP_CONCAT(c SEPARATOR ", ") ',
'FROM (SELECT id AS c FROM tbl WHERE nha_id IN (', #ids_tmp,
') AS d INTO #ids_tmp' SEPARATOR '');
but that made no difference either.
You can't use GROUP_CONCAT() as a scalar function; it must be used in the context of a set of rows. Similarly, you can't use any other aggregate function without a table reference:
SET #x = MAX(<expr>); -- makes no sense
Ideally you should upgrade to MySQL 8.0 if you haven't already, and use a recursive CTE query instead:
WITH RECURSIVE hierarchy AS (
SELECT top_id AS id
UNION
SELECT tbl.id FROM tbl JOIN hierarchy ON tbl.nha_id = hierarchy.id
)
SELECT GROUP_CONCAT(id SEPARATOR ', ') AS ids_all FROM hierarchy;
That would eliminate the need for using loops or prepare/execute or temp variables.
Thanks to Bill Karwin's answer regarding the need to use SELECT with GROUP_CONCAT, I was able to see how my code needed to be changed. Rather than using a SET statement to assign the values to the variables, I need to use a SELECT ... INTO ... construct, as shown here:
DELIMITER $$
CREATE PROCEDURE ASSEMBLY_LIST_TEST (top_id INT)
BEGIN
DECLARE ids_all TEXT DEFAULT '';
SET SESSION group_concat_max_len = 1000000;
SET ids_all = top_id;
# Find the 1st-level children of 'top_id' to start building the list
SELECT GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR ", ") FROM tbl ',
'WHERE nha_id = ', top_id, ' INTO #ids_tmp' SEPARATOR '') INTO #qrystr;
PREPARE stmt FROM #qrystr;
EXECUTE stmt;
# Recursively find the children of each level of children of the previous loop & add to the list
WHILE #ids_tmp != '' DO
SELECT GROUP_CONCAT(ids_all, ', ', #ids_tmp SEPARATOR '') INTO ids_all;
SELECT GROUP_CONCAT('SELECT GROUP_CONCAT(id SEPARATOR ", ") FROM tbl ',
'WHERE nha_id IN (', #ids_tmp, ') INTO #ids_tmp' SEPARATOR '') INTO #qrystr;
PREPARE stmt FROM #qrystr;
EXECUTE stmt;
END WHILE;
SELECT ids_all AS ids;
END $$
DELIMITER ;
This now produces exactly the result I was looking for. Of course, the recursive query approach is a much better solution for what I want this to do, but maybe my solution will help someone else.

Creating a MySQL stored procedure to update records

I'm converting all of my existing MSSQL databases and stored procedures am stuck on a new stored procedure where I need to update an existing record. The procedure gets called from a web form once a record has been inserted into the database and en email sent successfully (or at least passed off to the SMTP server)
I've had a working procedure in MSSQL for a long time but am trying to convert it to MySQL. I'm passing in 3 variables - a bit indicating the email got sent, a string indicating which SMTP server has been used to sent the email and a unique record id so I'll know what record to update. I'm also adding the date and time to another field to know when the procedure ran.
I've got the following but keep getting an error "#1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 7 - yet I don't see anything off at line 7 - at least to my eye.
The code I'm trying to use is:
CREATE PROCEDURE `sp_Test`(
`emailSent_In` BIGINT UNSIGNED,
`emailTransport_In` VARCHAR(100),
`formSecret_In` VARCHAR(32)
)
BEGIN
SET #`query` := CONCAT('UPDATE ',`tbl_JustSayThanks`,'
SET `emailSent` = `emailSent_In`,
`emailTransport` = ',`emailTransport_In`,',
`emailSentDate` = NOW()
WHERE `formSecret` = ', `formSecret_In`, '');
PREPARE `stmt` FROM #`query`;
EXECUTE `stmt`;
#`query` := NULL;
DEALLOCATE PREPARE `stmt`;
END//
DELIMITER ;
Just FYI, I'm using the CONCAT based on a previous answer I received from wchiquito and will be passing in the table name eventually. But, I wanted to get it to work on a simplified level before going there.
The following is wrong:
SET #`query` := CONCAT('UPDATE ',`tbl_JustSayThanks`,'
because you seem to be concatenating your SQL text with the value of tbl_JustSayThanks, but I think you mean to use the identifier itself. This should therefore be:
SET #`query` := CONCAT('UPDATE `tbl_JustSayThanks`',
The following is wrong:
`emailTransport` = ',`emailTransport_In`,',
because the variable is a VARCHAR but you don't quote it as a string literal in your SQL statement. It's easy to get mixed up with the multiple levels of quoting. It should be:
`emailTransport` = ''', `emailTransport_In`, ''',
The following is wrong for the same reason:
WHERE `formSecret` = ', `formSecret_In`, '');
it should be:
WHERE `formSecret` = ''', `formSecret_In`, '''');
This still suffers from SQL injection problems, unless you can guarantee that the input parameters are safe (which is not a good assumption). If you need to concatenate values into your SQL expressions, you should use the QUOTE() function to do escaping:
SET #query = CONCAT('
UPDATE tbl_JustSayThanks
SET emailSent = ', QUOTE(emailSent_In), '
emailTransport = ', QUOTE(emailTransport_In), '
emailSentDate = NOW()
WHERE formSecret = ', QUOTE(formSecret_In));
More comments:
You don't need to delimit every identifier with back-ticks, only those that conflict with SQL reserved words, or contain whitespace or punctuation or international characters. None of your identifiers you show require delimiting.
When you use prepared statements, you should use query parameters with the ? placeholders, intead of concatenating variables into the SQL string. You don't quote parameter placeholders in your SQL query. That way you won't run into hard-to-debug syntax errors like the ones you found.
Here's an example showing the fixes:
CREATE PROCEDURE sp_Test(
emailSent_In BIGINT UNSIGNED,
emailTransport_In VARCHAR(100),
formSecret_In VARCHAR(32)
)
BEGIN
SET #query = '
UPDATE tbl_JustSayThanks
SET emailSent = ?,
emailTransport = ?,
emailSentDate = NOW()
WHERE formSecret = ?';
SET #es = emailSent_In;
SET #et = emailTransport_In;
SET #fs = formSecret_In;
PREPARE stmt FROM #query;
EXECUTE stmt USING #es, #et, #fs;
DEALLOCATE PREPARE stmt;
END//
DELIMITER ;
Final comment:
Your example query has no dynamic syntax elements, only dynamic values. So you don't need to use a prepared statement at all.
This is how I'd really write the procedure:
CREATE PROCEDURE sp_Test(
emailSent_In BIGINT UNSIGNED,
emailTransport_In VARCHAR(100),
formSecret_In VARCHAR(32)
)
BEGIN
UPDATE tbl_JustSayThanks
SET emailSent = emailSent_In,
emailTransport = emailTransport_In,
emailSentDate = NOW()
WHERE formSecret = formSecret_In;
END//
DELIMITER ;
You should also be aware that MySQL stored procedures are greatly inferior to Microsoft SQL Server. MySQL doesn't keep compiled stored procedures, it doesn't support packages, it doesn't have a debugger... I recommend you do not use MySQL stored procedures. Use application code instead.

How to handle where clause within mysql procedure?

Here is my procedure.
DELIMITER //
drop procedure if exists GetID;
CREATE PROCEDURE GetID(IN tb VARCHAR(255), in name2 varchar(255))
BEGIN
set #sel = concat( 'select id FROM ', tb, ' WHERE ename = ', name2);
prepare stmt from #sel;
execute stmt;
deallocate prepare stmt;
END //
DELIMITER ;
When I tried to execute the stored procedure by using GetID('city', 'ny'). I got an error
unknown column 'ny' in where clause ...
Here 'city' is the table name. What is wrong?
Assuming that name2 is a string parameter which to be compared with ename column of the passed table
Put quotes around name2 in the SQL:
set #sel = concat('select id FROM ', tb, ' WHERE ename = ''', name2,'''');
It's usually recommended not to use string concatenation to build SQL queries. Since you are hardcoding the column name in the query, it makes little sense to provide the table name "dynamically". But, if you must, use QUOTE to properly escape and quote the passed string.
set #sel = concat('select id FROM ', tb, ' WHERE ename = ', quote(name2));
Never concatenate strings directly into queries. It's bad enough that you're passing a table name in, unsanitized. That needs to be fixed, too, but one correct solution to your immediate issue is this:
set #sel = concat( 'select id FROM ', tb, ' WHERE ename = ', QUOTE(name2));
The QUOTE() function correctly and safely quotes and escapes the argument, and also handles null values correctly... and prevents a SQL injection vulnerability here.
See https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_quote.

How to loop through all the tables on a database to update columns

I'm trying to update a column (in this case, a date) that is present on most of the tables on my database. Sadly, my database has more than 100 tables already created and full of information. Is there any way to loop through them and just use:
UPDATE SET date = '2016-04-20' WHERE name = 'Example'
on the loop?
One painless option would be to create a query which generates the UPDATE statements you want to run on all the tables:
SELECT CONCAT('UPDATE ', a.table_name, ' SET date = "2016-04-20" WHERE name = "Example";')
FROM information_schema.tables a
WHERE a.table_schema = 'YourDBNameHere'
You can copy the output from this query, paste it in the query editor, and run it.
Update:
As #PaulSpiegel pointed out, the above solution might be inconvenient if one be using an editor such as HeidiSQL, because it would require manually copying each record in the result set. Employing a trick using GROUP_CONCAT() would give a single string containing every desired UPDATE query in it:
SELECT GROUP_CONCAT(t.query SEPARATOR '; ')
FROM
(
SELECT CONCAT('UPDATE ', a.table_name,
' SET date = "2016-04-20" WHERE name = "Example";') AS query,
'1' AS id
FROM information_schema.tables a
WHERE a.table_schema = 'YourDBNameHere'
) t
GROUP BY t.id
You can use SHOW TABLES command to list all tables in database. Next you can check if column presented in table with SHOW COLUMNS command. It can be used this way:
SHOW COLUMNS FROM `table_name` LIKE `column_name`
If this query returns result, then column exists and you can perform UPDATE query on it.
Update
You can check this procedure on sqlfiddle.
CREATE PROCEDURE UpdateTables (IN WhereColumn VARCHAR(10),
IN WhereValue VARCHAR(10),
IN UpdateColumn VARCHAR(10),
IN UpdateValue VARCHAR(10))
BEGIN
DECLARE Finished BOOL DEFAULT FALSE;
DECLARE TableName VARCHAR(10);
DECLARE TablesCursor CURSOR FOR
SELECT c1.TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS c1
JOIN INFORMATION_SCHEMA.COLUMNS c2 ON (c1.TABLE_SCHEMA = c2.TABLE_SCHEMA AND c1.TABLE_NAME = c2.TABLE_NAME)
WHERE c1.TABLE_SCHEMA = DATABASE()
AND c1.COLUMN_NAME = WhereColumn
AND c2.COLUMN_NAME = UpdateColumn;
DECLARE CONTINUE HANDLER FOR NOT FOUND SET Finished = TRUE;
OPEN TablesCursor;
MainLoop: LOOP
FETCH TablesCursor INTO TableName;
IF Finished THEN
LEAVE MainLoop;
END IF;
SET #queryText = CONCAT('UPDATE ', TableName, ' SET ', UpdateColumn, '=', QUOTE(UpdateValue), ' WHERE ', WhereColumn, '=', QUOTE(WhereValue));
PREPARE updateQuery FROM #queryText;
EXECUTE updateQuery;
DEALLOCATE PREPARE updateQuery;
END LOOP;
CLOSE TablesCursor;
END
This is just an example how to iterate through all tables in database and perform some action with them. Procedure can be changed according to your needs.
Assuming you are using MySQL, You can use Stored Procedure.
This post is a very helpful.
Mysql-loop-through-tables

Prepared statement execution with variable number of parameters to be bound

Some stored procedures I work with need to interpolate WHERE criteria based on if procedure input parameters have been supplied. To avoid potential injection points, I'd like to utilize parameter binding for the values that are to be part of the interpolated criteria.
Since the criteria added to the prepared statement and thus the number of parameters to be bound may differ depending on the user input, I devised the method below to determine which variables will be passed to the EXECUTE statement. This works, but it seems inelegant.
CREATE PROCEDURE foo (IN mandatory INT, IN optional INT, IN optional2 VARCHAR(20))
BEGIN
SELECT
0, '', '', mandatory, optional, optional2
INTO
#params, #sql, #where, #m, #o1, #o2;
IF (#o1 > '' AND #o1 IS NOT NULL) THEN
SET #where = CONCAT(#where, ' AND field = ?');
SET #params = #params + 1;
END IF;
IF (#o2 > '' AND #o2 IS NOT NULL) THEN
SET #where = CONCAT(#where, ' AND field2 = ?');
SET #params = #params + 3;
END IF;
SET #sql = CONCAT('
SELECT id, bar FROM table
WHERE
baz = ?
', #where
);
PREPARE STMT FROM #sql;
CASE #params
WHEN 0 THEN EXECUTE STMT USING #m;
WHEN 1 THEN EXECUTE STMT USING #m, #o1;
WHEN 3 THEN EXECUTE STMT USING #m, #o2;
WHEN 4 THEN EXECUTE STMT USING #m, #o1, #o2;
END CASE;
DEALLOCATE PREPARE STMT;
END$$
I'm aware of alternatives:
The binaries that would call these stored procedures have a function that attempts to identify potential SQL injection by passing the user supplied strings through a regular expression.
A user-defined function could be used to dynamically construct the EXECUTE statement given a dynamic number of inputs.
However, I was wondering if anyone else has ran into this desire to handle dynamic construction of an EXECUTE statement purely with SQL.
However, I was wondering if anyone else has ran into this desire to handle dynamic construction of an EXECUTE statement purely with SQL.
Yes, me too.
Here's a PHP solution to generate the list of question marks for a prepared statement based on an array of unknown length:
/* My target query is this:
SELECT fun FROM fun_stuff WHERE fun_key IN ( ...unknown number of values... )
*/
/* For this example let's set our array to this: */
$val_arr = array(1,2,3,4,5,6,7,8,9);
$val_arr_cnt = count($val_arr); /* and count it */
/* Now make prepared statement q-mark string from values array */
$sql_prep = str_pad('?', ($val_arr_cnt * 2) - 1, ',?', STR_PAD_RIGHT);
/* add it to query */
$sql = "SELECT fun FROM fun_stuff WHERE fun_key IN ($sql_prep)";
/* And the result:
SELECT fun FROM fun_stuff WHERE fun_key IN (?,?,?,?,?,?,?,?,?)
*/
I've no idea how efficient this is. But I too every now and then want to implement the security and efficiency of MySQL prepared statements but have variable length input arrays
Not sure if it's possible to dynamically build the parameter list (changing the number of parameters on the fly, etc...). But since you can dynamically build your where clause, one very simple workaround is do something like this. Assuming your validations permit it, the else clause basically has the same effect as ignoring the parameter you may or may not be filtering on.
if p_cust_id is not null && p_cust_id > 0 then
set v_where_clause = concat(v_where_clause, ' c.cust_id = ? ');
set #v_cust_id := p_cust_id;
else
set v_where_clause = concat(v_where_clause, ' c.cust_id > ? ');
set #v_cust_id := 0;
end if;
then plug all the user variables above into your execute statement
execute str1 using #v_cust_id, #v_etc....;*