I am calling a stored procedure from JAVA, and it runs very slowly (about 120ms), even if it doesn't do much. For example:
BEGIN
SELECT '1';
END
If I have the stored procedure do literally nothing, or if I run the select commands outside of a stored procedure (ie execute the select * from.. directly in code), then it runs quickly (3ms or so).
BEGIN
END
I have 50 other SELECT stored procedures, and they all run equally slowly. If I run any if these commands outside of stored procedures (ie execute the SELECT * FROM... statement from JAVA without stored procedures), then they run very quickly.
I did look around and found others with similar issues, but couldn't find an answer that works in my case. I confirmed that collation and character set are the same between the database, the table, and the stored procedure.
INSERTS seem to run quickly. It really seems isolated to SELECT commands.
Environment: FreeBSD 10.3, DB - MariaDB 10.0.21
EDIT: Running the stored procedures from mysql commandline executes very quickly (0.00 sec).
EDIT: I enabled the general log and have the following output:
184381 Connect qwkdb#localhost as anonymous on productdb
184381 Query /* mysql-connector-java-6.0.2 ( Revision: c6da0fe501ad43d4ed6483b60ea796dc9fbe2d7b ) */SELECT ##session.auto_increment_increment AS auto_increment_increment, ##character_set_client AS character_set_client
, ##character_set_connection AS character_set_connection, ##character_set_results AS character_set_results, ##character_set_server AS character_set_server, ##init_connect AS init_connect, ##interactive_timeout AS interactive_timeout, ##l
icense AS license, ##lower_case_table_names AS lower_case_table_names, ##max_allowed_packet AS max_allowed_packet, ##net_buffer_length AS net_buffer_length, ##net_write_timeout AS net_write_timeout, ##query_cache_size AS query_cache_size
, ##query_cache_type AS query_cache_type, ##sql_mode AS sql_mode, ##system_time_zone AS system_time_zone, ##time_zone AS time_zone, ##tx_isolation AS tx_isolation, ##wait_timeout AS wait_timeout
184381 Query SET NAMES latin1
184381 Query SET character_set_results = NULL
184381 Query SET autocommit=1
184381 Query SET sql_mode = 'NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES'
184381 Query SELECT name, type, comment FROM mysql.proc WHERE name like 'GetAttributeValue' and db <=> 'productdb' ORDER BY name, type
184381 Query SHOW CREATE PROCEDURE productdb.GetAttributeValue
184381 Query CALL GetAttributeValue('hi')
I'm using MySQL 5.6. I have a specific query in my application that I don't want to be subject to the global innodb_lock_wait_timeout setting. Is there any way that I can specify that this specific query should wait a different amount of time for a lock before timing out?
You can only set this variable on global or session level, so you cannot set it just for a single query. Obviously, you can create a separate connection, set this variable on session level, execute your query and then close the connection.
You can find how to set a system variable on session level in mysql documentation on set command:
SET variable_assignment [, variable_assignment] ...
variable_assignment:
user_var_name = expr
| [GLOBAL | SESSION] system_var_name = expr
| [##global. | ##session. | ##]system_var_name = expr
I've reduced my issue down to this simple SP. The column names are getting cached in the SELECT * at the end. I have no idea why or how to stop it. I tried adding SQL_NO_CACHE but that makes no difference.
DROP TABLE IF EXISTS foo;
CREATE TABLE foo(
col1 int,
col2 int);
INSERT INTO foo VALUES(1,2),(3,4),(5,6);
DROP PROCEDURE IF EXISTS mysp;
DELIMITER ;;
CREATE DEFINER=root#localhost PROCEDURE mysp(c INT)
BEGIN
DROP TABLE IF EXISTS mydata;
SET #mycol='col1';
IF c > 0 THEN SET #mycol:='col2';
END IF;
SET #s=CONCAT('CREATE TEMPORARY TABLE mydata AS SELECT ', #mycol, ' FROM foo');
PREPARE stmt FROM #s;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
-- The following select call fails on 2nd and subsequent executions of the SP
SELECT SQL_NO_CACHE * FROM mydata;
SELECT "Please see new temp table mydata" as Result;
END ;;
DELIMITER ;
Version
mysql> SELECT VERSION();
+------------+
| VERSION() |
+------------+
| 5.5.15-log |
+------------+
1 row in set (0.00 sec)
First run works fine as expected
mysql> CALL mysp(0);
+------+
| col1 |
+------+
| 1 |
| 3 |
| 5 |
+------+
3 rows in set (0.17 sec)
+----------------------------------+
| Result |
+----------------------------------+
| Please see new temp table mydata |
+----------------------------------+
1 row in set (0.17 sec)
Query OK, 0 rows affected (0.17 sec)
Now if I try and run it again using the other column
mysql> CALL mysp(1);
ERROR 1054 (42S22): Unknown column 'qlgqp1.mydata.col1' in 'field list'
mysql> SELECT #mycol;
+--------+
| #mycol |
+--------+
| col2 |
+--------+
1 row in set (0.00 sec)
If I recreate the storedprocedure again its works
mysql> CALL mysp(1);
+------+
| col2 |
+------+
| 2 |
| 4 |
| 6 |
+------+
3 rows in set (0.18 sec)
+----------------------------------+
| Result |
+----------------------------------+
| Please see new temp table mydata |
+----------------------------------+
1 row in set (0.18 sec)
Query OK, 0 rows affected (0.18 sec)
But if I try switching back to the first column - even if I try dropping the temp table first - it still doesn't work
mysql> CALL mysp(0);
ERROR 1054 (42S22): Unknown column 'qlgqp1.mydata.col2' in 'field list'
mysql> DROP TABLE mydata;
Query OK, 0 rows affected (0.03 sec)
mysql> CALL mysp(0);
ERROR 1054 (42S22): Unknown column 'qlgqp1.mydata.col2' in 'field list'
mysql>
*Additional info asked for by eggyal. Also I tried this on another mysql version with same result. *
mysql> CALL mysp(1);
+------+
| col2 |
+------+
| 2 |
| 4 |
| 6 |
+------+
3 rows in set (0.20 sec)
+----------------------------------+
| Result |
+----------------------------------+
| Please see new temp table mydata |
+----------------------------------+
1 row in set (0.20 sec)
Query OK, 0 rows affected (0.20 sec)
mysql> describe mydata;
+-------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+---------+------+-----+---------+-------+
| col2 | int(11) | YES | | NULL | |
+-------+---------+------+-----+---------+-------+
1 row in set (0.00 sec)
mysql> CALL mysp(0);
ERROR 1054 (42S22): Unknown column 'test.mydata.col2' in 'field list'
mysql> describe mydata;
+-------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------+---------+------+-----+---------+-------+
| col1 | int(11) | YES | | NULL | |
+-------+---------+------+-----+---------+-------+
1 row in set (0.00 sec)
Interesting development of a fix - changing the last few lines to a prepared statement works - but using exactly the same query as before.
-- The following select call fails on 2nd and subsequent executions of the SP
PREPARE stmt FROM 'SELECT SQL_NO_CACHE * FROM mydata';
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
SELECT "Please see new temp table mydata" as Result;
MySQL is reusing the statement that was prepared on the previous execution. It's not really "caching" column names; what it's "caching" (if you will) is the prepared statement.
The easy workaround is to use dynamic SQL statement to gain control over the behavior, and avoid the reuse of the previously prepared statement:
SET #s=CONCAT('SELECT ',#mycol,' FROM mydata');
PREPARE stmt FROM #s;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
It's not matter of the column names being "cached", or the results of the query being cached. It's a performance optimization; it's a matter of that statement already being prepared, within your session.
By using dynamic SQL, you can control when the statement is prepared (i.e. parsing the SQL text for syntax (statement formation, keywords, etc.), checked for semantics (object names exist, column names exist, user has required privileges, etc.), and preparing an execution plan.
With the static SQL, all of that happens on the first execution, and then MySQL hangs on to the prepared statement.
For performance reasons, we wouldn't want the overhead of a "hard parse" every time a static statement is executed. This is especially true on a function that gets called multiple times, from a SQL statement.
(NOTE: Oracle does the same thing, BUT, Oracle does a good job of marking prepared statements as INVALID whenever a referenced object is altered or dropped.)
MySQL opted not to do that, probably because of the overhead of tracking all the dependencies. And, in the vast majority of cases, that overhead is not required.
I think the lesson here is if you are going to use dynamic SQL to create a table that is going to have DIFFERENT columns in it, you are going to have to use dynamic SQL to query that table.
My recommendation is that you avoid using SELECT *, unless your statement is in complete control of the columns being returned, for example, from an inline view. Otherwise, SQL statements that use SELECT * are fundamentally broken... they may work now, but a change to a table (adding a column for example) will break an application.
Q: Please explain how it is not a bug.
It is not a bug because the SELECT statement in your stored procedure is really just shorthand for what is actually happening.
On the first execution of your procedure, MySQL is doing the parse of your query text, and preparing and executing a statement. Basically, equivalent to:
PREPARE s1 FROM 'SELECT * FROM mydata';
EXECUTE s1;
On the second execution of the procedure, MySQL is simply executing the statement that has been previously prepared. Basically, equivalent to:
EXECUTE s1;
On that second execution, you seem to be expecting MySQL to run the equivalent of:
DEALLOCATE PREPARE s1;
PREPARE s1 FROM 'SELECT * FROM mydata';
EXECUTE s1;
You can make a case that this is what MySQL should be doing on the second execution. You could argue that statements prepared during a previous execution of a procedure should be discarded, and re-parsed and re-prepared on subsequent executions.
It would not be wrong for a DBMS to do this. But there would be, as always, the consideration about the impact on performance.
You could also make a case that MySQL should track all the database objects that a particular prepared statement is dependent on. You could argue that whenever one of those database objects is dropped or altered, MySQL should invalidate all the prepared statements (and all other objects) that depend on the altered or dropped object. Again, it would not be wrong for a DBMS to do this. Some DBMSs (such as Oracle) do this quite well. But again, the developers of the DBMS also take performance into consideration when making these design and implementation decisions.
The bottom line is that MySQL does provide you with a way to make happen what you want to happen. It's just that the syntax in your procedure, what you are expecting to make it happen, doesn't actually make it happen.
first of all it is a temp table so really should not be expected to be there, 2nd - it is dropped
I think you are reading something different into the "TEMPORARY" keyword than is defined in the specification. A TEMPORARY table is really just like a regular table, except that it is visible only to the session that created it, and it will be automatically dropped when the MySQL session ends. (We also note that a TEMPORARY table is not displayed by a SHOW TABLES command, and does not appear in the information_schema views.)
As to which tables (TEMPORARY or otherwise) MySQL should expect "to be there", I don't believe the documentation really addresses that, except noting that when a SQL statement is executed, and that statement references an object that does not exist, MySQL will throw an exception.
The same behavior you observe with a TEMPORARY table, you will also observe with a non-TEMPORARY table. The issue is not related to whether the table is defined as TEMPORARY or not.
where does SELECT * compare to PREPARE s1 FROM SELECT *
Those two forms effectively follow the same code path. The first execution of a static SELECT * is effectively equivalent to:
PREPARE s1 FROM 'SELECT *';
EXECUTE s1;
(Note the absence of a DEALLOCATE statement following the exeuction.) On a subsequent execution, the statement is already prepared, so it's effectively equivalent to:
EXECUTE s1;
This is similar to what would happen if you were coding in PHP mysqli
$s1 = $mysqli->prepare("SELECT * FROM mydata");
$mysqli->execute($s1);
/* rename the columns in the mydata table */
$mysqli->execute($s1);
I understand this is relatively old (+6 months) but I encountered this problem in prepared statements, and the only way I got around it was to concatenate the field names and use a prepared statement that effectively calls "select *" but uses the actual field names. I'm using prepared statements to create a temporary table, and the only standard field is the first one, while the rest causes the caching problem on "Select *".
Select fields we want to use into a temp table
Iterate through table rows of field names, and on each iteration:
set sql01 = concat(sql01,',',sFieldName,''); (just fields) and: set sql02 = concat(sql02,',',sFieldName,' varchar(50) '); (fields + field type only, for create table statement)
create the output table:
set #sql = concat('CREATE TEMPORARY TABLE tOutput(FirstField varchar(50), ',sql02,');'); PREPARE STMT FROM #sql; EXECUTE STMT; DEALLOCATE PREPARE STMT;
At the end:
set #sql = concat('SELECT FirstField,',sql01,' FROM tOutput;'); PREPARE STMT FROM #sql; EXECUTE STMT; DEALLOCATE PREPARE STMT;
I am setting a variable, and then I want to use the USE command to change the database. Unfortunately USE doesn't evaluate the variable.
SET #con="testDB";
Query OK, 0 rows affected (0.00 sec)
select #con;
+--------+
| #con |
+--------+
| testDB |
+--------+
1 row in set (0.00 sec)
use #con;
ERROR 1049 (42000): Unknown database '#con'
So the value of the variable is not evaluated when I try to connect. Any ideas?
I have found my own solution, so I am going to answer my own question, in case anybody is interested.
#outis you are right, it's not possible to use a variable with the command USE.
However, due to the fact that I want to create a table in a database specified at runtime, my solution would be to use dynamic SQL:
set #schema="testDB";
set #front="CREATE TABLE IF NOT EXISTS ";
set #endpart=".`TEST1` (DIM_ID INT(16)) ENGINE = InnoDB DEFAULT CHARACTER SET = utf8 COLLATE = utf8_unicode_ci;";
set #stat=concat(#front,#schema,#endpart);
prepare command from #stat;
execute command;
So basically this solution builds up the statement, prepares it and executes it.
#schema parameter can even be past down to the script. This way I dynamically build the create statement.
I don't think it's possible to USE #var; in some way. But in some cases doing it the other way around might be an option.
USE testDB;
SET #schema = DATABASE();
When I created a user defined variable using
SET #a =10;
also i checked
SET #a := 10;
the above query executed successfully. while accessing the variable it gives me NULL value instead of 10. I accessed the defined variables using this query
SELECT #a;
The only way this can happen (in a client session) - and the way it happens for me from time to time - is you get bit by a short timeout on the client connection. It goes like this:
mysql> set #a = 10;
mysql> [wait for N+1 minutes, where N is the client timeout]
mysql> select #a;
+------+
| NULL |
+------+
| NULL |
+------+
1 row in set (0.00 sec)
You have to initialize your variables and use them within a contiguous client session. When the session goes away, you lose all your variables.
The other explanation, as pointed out by others in the comments, is that the commands are hitting the server from different connections; your problem might not be a timeout, but that you are originating the "SET ..." and "SELECT ..." commands in different connections. User variables are not shared across different connections.