i have 10 tables with same structure except table name.
i have a sp (stored procedure) defined as following:
select * from table1 where (#param1 IS NULL OR col1=#param1)
UNION ALL
select * from table2 where (#param1 IS NULL OR col1=#param1)
UNION ALL
...
...
UNION ALL
select * from table10 where (#param1 IS NULL OR col1=#param1)
I am calling the sp with the following line:
call mySP('test') //it executes in 6,836s
Then I opened a new standard query window. I just copied the query above. Then replaced #param1 with 'test'.
This executed in 0,321s and is about 20 times faster than the stored procedure.
I changed the parameter value repeatedly for preventing the result to be cached. But this did not change the result. The SP is about 20 times slower than the equivalent standard query.
Please can you help me to figure out why this is happening ?
Did anybody encounter similar issues?
I am using mySQL 5.0.51 on windows server 2008 R2 64 bit.
edit: I am using Navicat for test.
Any idea will be helpful for me.
EDIT1:
I just have done some test according to Barmar's answer.
At finally i have changed the sp like below with one just one row:
SELECT * FROM table1 WHERE col1=#param1 AND col2=#param2
Then firstly i executed the standart query
SELECT * FROM table1 WHERE col1='test' AND col2='test' //Executed in 0.020s
After i called the my sp:
CALL MySp('test','test') //Executed in 0.466s
So i have changed where clause entirely but nothing changed. And i called the sp from mysql command window instead of navicat. It gave same result. I am still stuck on it.
my sp ddl:
CREATE DEFINER = `myDbName`#`%`
PROCEDURE `MySP` (param1 VARCHAR(100), param2 VARCHAR(100))
BEGIN
SELECT * FROM table1 WHERE col1=param1 AND col2=param2
END
And col1 and col2 is combined indexed.
You could say that why dont you use standart query then? My software design is not proper for this. I must use stored procedure. So this problem is highly important to me.
EDIT2:
I have gotten query profile informations. Big difference is because of "sending data row" in SP Profile Information. Sending data part takes %99 of query execution time. I am doing test on local database server. I am not connecting from remote computer.
SP Profile Informations
Query Profile Informations
I have tried force index statement like below in my sp. But same result.
SELECT * FROM table1 FORCE INDEX (col1_col2_combined_index) WHERE col1=#param1 AND col2=#param2
I have changed sp like below.
EXPLAIN SELECT * FROM table1 FORCE INDEX (col1_col2_combined_index) WHERE col1=param1 AND col2=param2
This gave this result:
id:1
select_type=SIMPLE
table:table1
type=ref
possible_keys:NULL
key:NULL
key_len:NULL
ref:NULL
rows:292004
Extra:Using where
Then i have executed the query below.
EXPLAIN SELECT * FROM table1 WHERE col1='test' AND col2='test'
Result is:
id:1
select_type=SIMPLE
table:table1
type=ref
possible_keys:col1_co2_combined_index
key:col1_co2_combined_index
key_len:76
ref:const,const
rows:292004
Extra:Using where
I am using FORCE INDEX statement in SP. But it insists on not using index. Any idea? I think i am close to end :)
Just a guess:
When you run the query by hand, the expression WHERE ('test' IS NULL or COL1 = 'test') can be optimized when the query is being parsed. The parser can see that the string 'test' is not null, so it converts the test to WHERE COL1 = 'test'. And if there's an index on COL1 this will be used.
However, when you create a stored procedure, parsing occurs when the procedure is created. At that time, it doesn't know what #param will be, and has to implement the query as a sequential scan of the table.
Try changing your procedure to:
IF #param IS NULL
THEN BEGIN
SELECT * FROM table1
UNION ALL
SELECT * FROM table2
...
END;
ELSE BEGIN
SELECT * FROM table1 WHERE col1 = #param
UNION ALL
SELECT * FROM table2 WHERE col1 = #param
...
END;
END IF;
I don't have much experience with MySQL stored procedures, so I'm not sure that's all the right syntax.
Possible character set issue? If your table character set is different from your database character set, this may be causing a problem.
See this bug report: http://bugs.mysql.com/bug.php?id=26224
[12 Nov 2007 21:32] Mark Kubacki Still no luck with 5.1.22_rc - keys
are ingored, query takes within a procedure 36 seconds and outside
0.12s.
[12 Nov 2007 22:30] Mark Kubacki After having changed charsets to UTF-8 (especially for the two used), which is used for the
connection anyways, keys are taken into account within the stored
procedure!
The question I cannot answer is: Why does the optimizer treat charset
conversions an other way within and outside stored procedures?
(Indeed, I might be wrong asking this.)
Interesting question, because I am fond of using stored procedures. Reason is maintenance and the encapsulation principle.
This is information I found:
http://dev.mysql.com/doc/refman/5.1/en/query-cache-operation.html
It states that the query cache is not used for queries that
1. are a subquery that belong to an outer query, and
2. are executed within the body of a stored procedure, trigger or event.
This implies that it works as designed.
I had seen this behavior, but it wasn't related to the character set.
I had a table that held self-referencing hierarchical data (a parent with children, and some children had children of their own, etc.). Since the parent_id had to reference the primary id's (and the column specified a constraint to that effect), I couldn't set the parent id to NULL or 0 (zero) to disassociate a child from a parent, so I simply referenced it to itself.
When I went to run a stored procedure to perform the recursive query to find all children (at all levels) of a particular parent, the query took between 30 & 40 times as long to run. I found that altering the query used by the stored procedure to make sure it excluded the top-level parent record (by specifying WHERE parent_id != id) restored the performance of the query.
The stored procedure I'm using is based on the one shown in:
https://stackoverflow.com/questions/27013093/recursive-query-emulation-in-mysql.
Related
I've a problem with MySql, here is the details :
I created a new schema/database, executed (only) this queries :
create table mytable (
id varchar(50) not null,
name varchar(50) null default '',
primary key (id));
create view myview as
select id,name from mytable;
insert into mytable values ('1','aaa');
insert into mytable values ('2','bbb');
insert into mytable values ('3','ccc');
and then, if I run these queries :
select * from mytable;
select * from myview;
prepare cmd from 'select id,name from mytable where id=?';
set #param1 = '2';
execute cmd using #param1;
the queries give the correct result (3 rows,3 rows,1 row).
but, the problem exists if I run this query:
prepare cmd from 'select id,name from myview where id=?';
set #param1 = '2';
execute cmd using #param1;
ERROR: #1615 - Prepared statement needs to be re-prepared
I've done some research and found that the increment of configurations below "may" solve the problem :
increase table_open_cache_instances value
increase table_open_cache value
increase table_definition_cache value
As far as I know, the queries above are the common and standard MySql queries, so I think there is no problem with the syntax.
I'm on a shared webhosting and using MySql version is 5.6.22
But the things that make me confused is, it only contain 1 schema/database, with 1 table with 3 short records and 1 view,
and I executed a common and standard MySql select query,
does the increment of values above really needed?
is there anyone with the same problem had increase the values and really solve the problem?
or, perhaps do you have any other solution which you think may or will works to solve this problem?
ps: it does not happen once or twice in a day (which assumed caused by some backup or related), but in all day (24 hours).
Thank you.
Do you do this after each execute?
deallocate prepare cmd;
The closest guess until now is some other shared members on the server dont write their code quite well (because it is a shared webhosting), either doing large alter while doing the large select, or dont deallocate the prepared statement after using it, like Rick James said. (want to make the post usefull, but I dont have the reputation, sorry Rick)
I can not make sure if the increment of "table_definition_cache" will works because the system administrator still wont change the value until now, but incase you having the same problem and you can modify it, it worth to try.
My current solution is I change all my views in my query strings into non-view or subqueries, it works for me, but the problem is still in the air.
eg. from
select myview.id, myview.name
from myview
inner join other_table on ...
where myview.id=?
into
select x.id, x.name
from (select id,name from mytable) x
inner join other_table on ...
where x.id=?
so if i have a stored procedure that contains selects how to suppress the results from these selects from appearing ?
for example if i have
create procedure xyz
begin
select * from table_name #I don't want this to be seen in the console.
do other stuff
end;
You can set the required params into variables like
SET #foo = (SELECT foo_coloumn FROM foo_table);
This will suppress the select result set.
The accepted answer didn't work for me since the resultset was multirow. I did figure out a solution, though:
CREATE TEMPORARY TABLE silence_output SELECT * FROM foo;
DROP TEMPORARY TABLE silence_output;
This avoids any multirow/multicolumn issues that might be encountered by dumping the resultset into a table that is then trashed. Hope this helps someone else. You can name the temporary table anything.
I was struggling to find a way to suppress multirow output in a stored procedure while at the same time locking all applicable rows with "FOR UPDATE".
First attempt was "SELECT ... INTO" with a "LIMIT" clause but that would only lock 1 row.
Using GROUP_CONCAT solved it for me:
SELECT GROUP_CONCAT(column) INTO #variable
FROM table
WHERE column = value
FOR UPDATE
Doing that in a transaction locked all affected rows, prevented an insert, and suppressed output.
Ok, First off, I am not a mysql guru. Second, I did search, but saw nothing relevant related to mysql, and since my DB knowledge is limited, guessing syntactical differences between two different Database types just isn't in the cards.
I am trying to determine if a particular value already exists in a table before inserting a row. I've decided to go about this using two Stored procedures. The first:
CREATE PROCEDURE `nExists` ( n VARCHAR(255) ) BEGIN
SELECT COUNT(*) FROM (SELECT * FROM Users WHERE username=n) as T;
END
And The Second:
CREATE PROCEDURE `createUser` ( n VARCHAR(255) ) BEGIN
IF (nExists(n) = 0) THEN
INSERT INTO Users...
END IF;
END
So, as you can see, I'm attempting to call nExists from createUser. I get the error that no Function exists with the name nExists...because it's a stored procedure. I'm not clear on what the difference is, or why such a difference would be necessary, but I'm a Java dev, so maybe I'm missing some grand DB-related concept here.
Could you guys help me out by any chance?
Thanks
I'm not sure how it helped you, but...
why SELECT COUNT(*) FROM (SELECT * FROM Users WHERE username=n) and not just SELECT COUNT(*) FROM Users WHERE username=n?
Just make the user name (or whatever the primary application index is) a UNIQUE index and then there is no need to test: Just try to insert a new record. If it already exists, handle the error. If it succeeds, all is well.
It can (and should) all be one stored procedure.
I am working on a WPF application which installs if MySQL is installed,
so before installation I want to check whether mysql.proc table exists or not.
I googled about it and ended up with a query
select * from information_schema.Tables
where Table_schema = Schema() and Table_Name = 'mysql.proc'
This query returns an empty row.
I also tried simple select statement
select * from mysql.proc,
and this returned a table with the names of all the stored procedures, but if this table didn't exists then it throws an exception in the c# code.
So is there any way that I can fire a query from c# and get a boolean value depending on whether mysql.proc table exists or not?
Try SHOW TABLES FROM mysql LIKE 'proc'. If there are no result rows, the table doesn't exist. If there is one row, the table exists. Note that this approach isn't portable across RDBMSs, though that doesn't seem to be a concern of yours.
As for your first query, SCHEMA() returns the default database, so if it's not "mysql", the query will fail. Likewise, data in the Table_Name column doesn't include the database name, so comparing to 'mysql.proc' will always fail.
I have a MySQL table of tasks to perform, each row having parameters for a single task.
There are many worker apps (possibly on different machines), performing tasks in a loop.
The apps access the database using MySQL's native C APIs.
In order to own a task, an app does something like that:
Generate a globally-unique id (for simplicity, let's say it is a number)
UPDATE tasks
SET guid = %d
WHERE guid = 0 LIMIT 1
SELECT params
FROM tasks
WHERE guid = %d
If the last query returns a row, we own it and have the parameters to run
Is there a way to achieve the same effect (i.e. 'own' a row and get its parameters) in a single call to the server?
try like this
UPDATE `lastid` SET `idnum` = (SELECT `id` FROM `history` ORDER BY `id` DESC LIMIT 1);
above code worked for me
You may create a procedure that does it:
CREATE PROCEDURE prc_get_task (in_guid BINARY(16), OUT out_params VARCHAR(200))
BEGIN
DECLARE task_id INT;
SELECT id, out_params
INTO task_id, out_params
FROM tasks
WHERE guid = 0
LIMIT 1
FOR UPDATE;
UPDATE task
SET guid = in_guid
WHERE id = task_id;
END;
BEGIN TRANSACTION;
CALL prc_get_task(#guid, #params);
COMMIT;
If you are looking for a single query then it can't happen. The UPDATE function specifically returns just the number of items that were updated. Similarly, the SELECT function doesn't alter a table, only return values.
Using a procedure will indeed turn it into a single function and it can be handy if locking is a concern for you. If your biggest concern is network traffic (ie: passing too many queries) then use the procedure. If you concern is server overload (ie: the DB is working too hard) then the extra overhead of a procedure could make things worse.
I have the exact same issue. We ended up using PostreSQL instead, and UPDATE ... RETURNING:
The optional RETURNING clause causes UPDATE to compute and return value(s) based on each row actually updated. Any expression using the table's columns, and/or columns of other tables mentioned in FROM, can be computed. The new (post-update) values of the table's columns are used. The syntax of the RETURNING list is identical to that of the output list of SELECT.
Example: UPDATE 'my_table' SET 'status' = 1 WHERE 'status' = 0 LIMIT 1 RETURNING *;
Or, in your case: UPDATE 'tasks' SET 'guid' = %d WHERE 'guid' = 0 LIMIT 1 RETURNING 'params';
Sorry, I know this doesn't answer the question with MySQL, and it might not be easy to just switch to PostgreSQL, but it's the best way we've found to do it. Even 6 years later, MySQL still doesn't support UPDATE ... RETURNING. It might be added at some point in the future, but for now MariaDB only has it for DELETE statements.
Edit: There is a task (low priority) to add UPDATE ... RETURNING support to MariaDB.
I don't know about the single call part, but what you're describing is a lock. Locks are an essential element of relational databases.
I don't know the specifics of locking a row, reading it, and then updating it in MySQL, but with a bit of reading of the mysql lock documentation you could do all kinds of lock-based manipulations.
The postgres documenation of locks has a great example describing exactly what you want to do: lock the table, read the table, modify the table.
UPDATE tasks
SET guid = %d, params = #params := params
WHERE guid = 0 LIMIT 1;
It will return 1 or 0, depending on whether the values were effectively changed.
SELECT #params AS params;
This one just selects the variable from the connection.
From: here