I am currently building a MySQL database. The database is storing data which is being supplied by another organisation and the format of the files we recieve is, shall we say, somewhat inconsistent.
I have writted a function which assess the file based on some simple rules and returns a message saying 'file ok' or else one of a list of hard coded error messages. The way I would like to use this function would be to say something like:
if check_10m_file() = 'file ok' then
load data infile '\\\\server\\filepath\file.csv'
fields terminated by ','
...(etc)
My problem is that the if..then..else control structure does not seem to be allowed outside of stored procedures and the load data infile command is not allowed inside stored procedures.
I have attempted to trick my way around this by building the load data infile statement as a prepared statement but then I get an error
"This command is not supported in the prepared statement protocol yet"
So my question is, does anyone know of a way in which I can run a load data statement only when conditions are met? I would ideally like to put this into a stored procedure but if I could just save the code as a script to run later, that would be acceptable.
One option is to use UDF, for example: lib_mysqludf_sys.
After installing the UDF, you can do something like:
Shell script (/server/loadpath/load.sh):
mysql -u [user] -p[pass] -e "LOAD DATA INFILE '$1' INTO TABLE $2;"
Stored Procedure:
DELIMITER $$
DROP PROCEDURE IF EXISTS load_data$$
CREATE PROCEDURE load_data(pfile VARCHAR(100), pdbtable VARCHAR(100))
BEGIN
DECLARE exec_str VARCHAR(500);
DECLARE ret_val INT;
IF (check_10m_file() = 'file ok') THEN
SET exec_str := CONCAT('sh /server/loadpath/load.sh ', pfile, ' ', pdbtable);
SET ret_val := sys_exec(exec_str);
IF ret_val = 0 THEN
SELECT 'OK' Result;
ELSE
SELECT 'ERROR' Result;
END IF;
END IF;
END$$
DELIMITER ;
CALL load_data('/server/filepath/file.csv', 'mydb.mytable');
IMPORTANT: Validate the input data to prevent any code injection.
Related
I am trying to write a mysql program, which has cursors in it. Due to an error in writing an DECLARE query, MySQL Workbench is always showing me the DECLARE is not valid at this position, expected EOF, ALTER, ANALYZE, BEGIN, BINLOG, CACHE, ...
Could you help me solve this problem?
Here is my code:
DELIMITER //
BEGIN
declare Naslov_knjige VARCHAR(24);
declare Cena_knjige DECIMAL(8,2);
DECLARE cursor_cene CURSOR
FOR SELECT
Naslov,
Cena
FROM
prvi_test_v2.knjige;
OPEN cursor_cene //
FETCH NEXT FROM cursor_cene INTO
#Naslov_knjige,
#Cena_knjige //
WHILE ##FETCH_STATUS = 0
BEGIN
PRINT #Naslov_knjige + CAST(#Cena_knjige AS VARCHAR) //
FETCH NEXT FROM cursor_cene INTO
#Naslov_knjige,
#Cena_knjige //
END //
CLOSE cursor_cene //
DEALLOCATE cursor_cene //
END //
DELIMITER ;
Thanks for your help!
I will assume you omitted a line for CREATE PROCEDURE, because in MySQL a BEGIN...END block must be part of a stored routine. See https://dev.mysql.com/doc/refman/8.0/en/begin-end.html
BEGIN ... END syntax is used for writing compound statements, which can appear within stored programs (stored procedures and functions, triggers, and events).
You changed the DELIMITER:
DELIMITER //
Using this delimiter terminates the whole CREATE PROCEDURE statement. You should not do this after the first statement in the body of the procedure. You need to use the normal ; terminators for each statement within the body of the procedure. The reason for changing the delimiter is so you can use ; for each statement in the procedure without ending the CREATE PROCEDURE.
See examples and documentation here: https://dev.mysql.com/doc/refman/8.0/en/stored-programs-defining.html
That's the reason for the error you got. You used // to terminate OPEN cursor_cene // which ended the CREATE PROCEDURE, but clearly there was more to that procedure.
There are other problems with your procedure. You seem to be using Microsoft SQL Server syntax, but MySQL is different.
Naslov_knjige is not the same variable as #Naslov_knjige in MySQL. Don't use the # sigil in front of local variables. If you use the # sigil, this refers to a user-defined variable.
The WHILE ##FETCH_STATUS = 0 syntax is specific to Microsoft SQL Server. MySQL has different syntax for running a cursor loop. See example in the documentation: https://dev.mysql.com/doc/refman/8.0/en/cursors.html
That's as far as I got. There may be more problems, but I am not going to look for them.
I've got a procedure that uses a loop with a SELECT statement, but the statement is actually just to set a variable. That means there's a lot of stuff being displayed that I don't need to see, and it's flooding my terminal.
Here's an example of what I mean, though this isn't actually what I'm running (because that's company information):
DROP PROCEDURE IF EXISTS test;
DELIMITER #
CREATE PROCEDURE test()
BEGIN
SET #key:=1;
testloop: REPEAT
SELECT
#dummyString := stringField
FROM
aTable;
SET #dummyStringAll :=CONCAT(#dummyStringAll,$dummyString);
SET #key := #key + 1;
UNTIL #key>10
END REPEAT testloop;
END #
DELIMITER ;
Is it possible to run SELECT (whether inside a procedure or not) and not show the results from a SELECT query? Maybe not the most important thing in the world, but it would be helpful.
Stored procedures will return a query resultset if it isn't stored in a variable.
How does it know that you are storing the result in a variable?
Not be using variables in the query but by using the SELECT value INTO <variable> syntax in the query. see: 13.2.9.1 SELECT ... INTO Syntax
From the FAQ:
1) Can MySQL 5.6 stored routines return result sets?
Stored procedures can, but stored functions cannot. If you perform an ordinary SELECT inside a stored procedure, the result set is returned directly to the client.
So, using the 'SELECT ... INTO ...' syntax will prevent the procedure returning the resultset from a query.
Problem
I have a stored procedure:
CREATE PROCEDURE `ProblematicProcedure` (IN dbName varchar(50), IN tableId INT)
MODIFIES SQL DATA
BEGIN
DROP VIEW IF EXISTS v1;
DROP VIEW IF EXISTS v2;
CALL ExecuteSql(CONCAT("CREATE VIEW v1 AS SELECT * FROM ",dbName,".my_table;"));
CALL ExecuteSql(CONCAT("CREATE VIEW v2 AS SELECT * FROM ",dbName,".table_",tableId,";"));
...
When called directly from command line or a client like Navicat or HeidiSql, it works well:
CALL ProblematicProcedure("my_schema",1);
But if called from a custom Apache module using the exactly same line above, it crashes on first ExecuteSql call. I have to make it work when called from the Apache module and couldn't find a reason to crash.
ExecuteSql definition
CREATE PROCEDURE ExecuteSql (IN sql_str TEXT)
BEGIN
SET #query = sql_str;
PREPARE stm FROM #query;
EXECUTE stm;
DEALLOCATE PREPARE stm;
END
What I tried?
Swapped two ExecuteSql calls.
Inlined ExecuteSql calls.
Removed ExecuteSql's and used direct SQL statements with hardcoded dbName and tableId values.
Created procedure without MODIFIES SQL DATA.
Granted CREATE VIEW privilege: GRANT ALL ON *.* TO 'myuser'#'%';
Note: I added simple insert statements between the lines to find where it is crashing. So, I am sure it crashes always on first ExecuteSql call.
Question
What can be reason to this crash?
Update: Finally, I managed to find error code:
ERROR 1312: Procedure can't return a result set in the given context
Solution
Use CLIENT_MULTI_STATEMENTS flag when connecting:
mysql_real_connect(conn, host, user, pass, db, 0, NULL, CLIENT_MULTI_STATEMENTS);
Why this is so?
Calling a stored procedure means executing multiple statements. So, I need to specify that I can execute multiple statements at once. Hence I am using MySql C API functions at client-side (in my Apache module), I need to specify CLIENT_MULTI_STATEMENTS flag when connecting:
mysql_real_connect(conn, host, user, pass, db, 0, NULL, CLIENT_MULTI_STATEMENTS);
Or set it later:
mysql_set_server_option(MYSQL_OPTION_MULTI_STATEMENTS_ON);
I learnt those from the C API Handling of Multiple Statement Execution page.
How I Debugged?
Debugging a stored procedure is not so easy. I used traditional log-table method, but performed a bit aggresively about finding the error code.
Firstly, defined two variables to keep the code and message about the error occurred:
DECLARE E INT DEFAULT 0; -- error code
DECLARE M TEXT DEFAULT NULL; -- error message
Then, defined possible error codes and messages both for client and server errors (full list here):
DECLARE CONTINUE HANDLER FOR 1000 SET E='1000', M="hashchk";
DECLARE CONTINUE HANDLER FOR 1001 SET E='1001', M="isamchk";
...
...
DECLARE CONTINUE HANDLER FOR 1312 SET E='1312', M="PROCEDURE %s can't return a result set in the given context";
...
...
DECLARE CONTINUE HANDLER FOR 1638 SET E='1638', M="Non-ASCII separator arguments are not fully supported";
DECLARE CONTINUE HANDLER FOR 1639 SET E='1639', M="debug sync point wait timed out";
DECLARE CONTINUE HANDLER FOR 1640 SET E='1640', M="debug sync point hit limit reached";
...
...
DECLARE CONTINUE HANDLER FOR 2057 SET E='2057', M="The number of columns in the result set differs from the number of bound buffers. You must reset the statement, rebind the result set columns, and execute the statement again";
And finally, put logs in critical points:
IF E> 0 THEN
CALL WriteLog(CONCAT("Error ", E, ": ", M));
END IF;
WriteLog is another procedure that only inserts into a log table. This method gave me the error code (1312) and then some Googling worked.
Is there a way to programmatically create stored procedures using MySQL? I can write a stored procedure to create databases and tables using prepared statements, but I get an error message saying it is not supported to create stored procedures with prepared statements.
I realize I can do this in PHP or Java, but so far I have been keeping all my schema management tools as SQL scripts, and I would like to avoid a second language dependency if I can.
One method you can try is to build the create procedure statement dynamically in SQL, then use select into outfile to dump the statement to local disk, and then source the file to load the procedure into the DB.
Here's a quick example:
set #proc_name = 'my_proc';
set #body1 = 'select ''hello''; ';
set #body2 = 'select ''world''; ';
set #delimiter = '$$';
set #create_proc_stmt = concat(
'create procedure ',
#proc_name,
'() begin ',
#body1,
#body2,
' end ',
#delimiter
);
select #create_proc_stmt into outfile '/tmp/create_proc_stmt.sql';
delimiter $$
\. /tmp/create_proc_stmt.sql
delimiter ;
call my_proc();
I think you can do it by inserting a record into INFORMATION_SCHEMA.ROUTINES table. I haven't tried it, but it should work (one day in the past I forgot to add --routines switch to mysqldump, and later I restored all procedures by dumping this one table).
Of course, I could go into mysql console and type the Function. But what if I want to store it for future use? What do I do?
Most projects have an SQL file to initialize the database from scratch. This way, one can set up the application database by simply running this SQL file. Your CREATE PROCEDURE/FUNCTION query would also go in this file.
There's a good tutorial here:
http://www.databasejournal.com/features/mysql/article.php/3525581/MySQL-Stored-Procedures-Part-1.htm
You need to use stored procedures. Once written, these are stored in the database along with your tables. They can be invoked using the CALL <procedure> statement.
Here's an example procedure that populates table1 with random values:
DELIMITER //
DROP PROCEDURE IF EXISTS autofill//
CREATE PROCEDURE autofill()
BEGIN
DECLARE i INT DEFAULT 0;
WHILE i < 20000 DO
INSERT INTO table1 (size) VALUES (FLOOR((RAND() * 1000)));
SET i = i + 1;
END WHILE;
END;
//
DELIMITER ;
Once the procedure has been written, it is called like this:
CALL autofill();