I have a stored procedure that is similar to below
SELECT *
FROM Table1
WHERE Tag IN (ids)
here Tag is an Integer column.
I tired to pass in comma separated values as string into the stored procedure but it does not work. Then I used stored procedure like below
SELECT *
FROM Table1
WHERE FIND_IN_SET(Tag, ids)
This works very well, the only problem is my table is very big - millions of rows and using FIND_IN_SET takes too long compared to IN when running a direct SQL statement.
What would be the best performance optimized option to use?
Is there a split function that can convert the ids into integer and parse it ready for IN clause? I think that would be the best option. Any suggestions or ideas?
You can prepare a statement and then execute it:
set #sql = concat('select * from table1 where tag in (', ids, ')');
PREPARE q FROM #sql;
execute q;
This constructs the string for each execution, so you can use in. The resulting execute should be able to use an index on tag, which should speed things up considerably.
Related
I have table1 which has only 2 columns: id and condition. For example:
id condition
--------------------
100 caption LIKE "%xyz%"
200 tag=5
300 color>153
...
The user sends an id to the server, and a specific select query must be run on table2 based on the condition of that id. For example, if id 100 is sent to server, then this query must be run:
SELECT * FROM table2 WHERE caption LIKE "%xyz%"
How to get the condition from table1 and run the query with that condition? I have already tried this:
SELECT * FROM table2 INNER JOIN table1 AS t1 ON t1.id=... WHERE t1.condition
However, I get the following warning with no result.
Warning: #1292 Truncated incorrect DOUBLE value: 'caption LIKE "%xyz%"'
This is possible in MySQL 5+ using prepared statements. You can create a procedure with a condition id as an argument:
DELIMITER //
CREATE PROCEDURE get_from_table2_by_condition_id(IN conditionId bigint)
BEGIN
SET #condition = NULL;
SELECT cond
INTO #condition
FROM table1
WHERE id = conditionId;
SET #sql = CONCAT('SELECT * FROM table2 WHERE ', COALESCE(#condition, 'FALSE'));
PREPARE stmt FROM #sql;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END//
DELIMITER ;
CALL get_from_table2_by_condition_id(100);
See a working example in this fiddle
I don't particularly recommend storing SQL code as you are describing.
But you have a client and a server. The solution is simply to query table1 for the condition and then construct the query that you want from that. This requires two queries.
You could set up a stored procedure inside the database that uses dynamic SQL for the same purpose.
There are several reasons why this is not desirable:
The code can introduce syntax errors which are quite hard to debug.
The code is subject to SQL injection, depending on the security features around table1.
Changes to the underlying tables might invalidate the conditions.
What is an alternative? One possibility is to create separate views for the different conditions. Or, just create one query and pass in parameters:
select t2.*
from table2 t2
where (caption like :caption or :caption is null) and
(tag = :tag or :tag is null) and
(color > :color or :color is null);
That is not possible, and if it where possible that would be a huge stored SQL injection security hole. To be able to do this, you will need something that will parse the stored condition and evaluate the condition on the fly. As far as I'm aware, MySQL has nothing built in for this.
This question already has answers here:
Parameterize an SQL IN clause
(41 answers)
Closed 9 years ago.
I have a stored procedure with one input parameter called "#IDs". This gets populated from my application which will populate it in the following format: '2, 30, 105'. The number of values inside this parameter will differ of course (For example: sometimes #IDs will be '100, 2005, 2, 510') My stored procedure is very simple. I have a table called "Persons". I'm trying to write this query:
Select * From Persons Where P_Id in (#IDs)
P_ID is the primary key in my table. The error I get is 'Conversion failed when converting the varchar value '2, 3, 4' to data type int.' Any suggestions will be greatly appreciated. Thanks.
One way is use dynamic SQL. That is generate the SQL as a string and then execute it.
An easier way (perhaps) is to use like:
where concat(', ', #IDS, ', ') like concat('%, ', id, ', %')
Note that this puts the separator at the beginning and end of the expressions, so "10" won't match "11010".
you might need to do a prepared statement. The idea is to build the select sentence and then execute it. Here's an example on how to do it...
USE mydb;
DROP PROCEDURE IF EXISTS execSql;
DELIMITER //
CREATE PROCEDURE execSql (
IN sqlq VARCHAR(5000)
) COMMENT 'Executes the statement'
BEGIN
SET #sqlv=concat(concat('select abc from yourtable where abc in (',sqll),')');
PREPARE stmt FROM #sqlv;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
END //
DELIMITER ;
Just change the query for the one you want to execute.
1) Show your code.
2) You've probably tried to pass in all the values as one parameter. That doesn't work. You have to list them as separate parameters and then bind them as separate parameters.
Yes, this makes it hard to use stored procedures when the number of in parameters may change.
I want to select from a table where a field is in a given csv string passed to a stored proc.
What is the fastest way to do this? Should I insert the values into a temporary table and join to that? Or is there a way to do it with one statement?
Thanks
Did some searching around and found a great answer.:)
Use MySql's string function FIND_IN_SET.
Example use:
SET #csvStr = "val1,val2,val3";
SELECT Col1
FROM Table1
WHERE FIND_IN_SET(Col2, #csvStr) > 0;
I edited the current answer with the "New Answer" and "Old Answer" pointing out that the new answer opens up your system to a SQL Injection vulnerability.
NEW ANSWER -
Well you have to do something like this as given below.
I am assuming your csv string would be as given in variable #str below. Else you need to make sure that your string (or arraystring) should have this format with single quotes for every element -
set #str = "'some1','some2','some3'";
set #qry1 = CONCAT('select * from testing where col1 in (',#str,')');
prepare stmt1 from #qry1;
execute stmt1;
deallocate prepare stmt1;
OLD ANSWER -
I assume that you will pass the csv file path to stored proc and read the lines in csv in that stored proc. So basically you can store all those csv field values in a temp table and write query using IN -
select * from sourceTable
where fieldValue in (select csvFieldValue from #temptable)
I have a procedure SelectProc which contains a SELECT statement. I want to add a procedure param LimitRowsCount and use it as following:
CREATE PROCEDURE SelectProc (IN LimitRowsCount INTEGER UNSIGNED)
BEGIN
SELECT (...)
LIMIT LimitRowsCount;
END
but this approach doesn't work.
The SELECT itself contains nested subqueries so I can't create view from it. Is there a way more proper than dynamic SQL (prepared statements)?
CREATE PROCEDURE SelectProc (IN LimitRowsCount INT)
BEGIN
SET #LimitRowsCount1=LimitRowsCount;
PREPARE STMT FROM "SELECT (...) LIMIT ?";
EXECUTE STMT USING #LimitRowsCount1;
END
From the manual:
The LIMIT clause can be used to constrain the number of rows
returned by the SELECT statement. LIMIT takes one or two numeric
arguments, which must both be nonnegative integer constants
(except when using prepared statements).
MySQL Manual - 12.2.8. SELECT Syntax
So that's a no - you cannot.
i have a column named "name" which is present in all tables in mysql database.
I wanted to list all the names in all tables so i used the following query
select name from (SELECT table_name FROM information_schema.tables WHERE table_type='BASE TABLE') as abc
But it did not work for me and instead it returned me the table_name column alone.
Then i used show tables and stored the output in another table called table_list then i executed the following query
select name from (select table_name from table_list) as abc
This also returned me the same result i.e. all table names.
Can i know what is that i am doing wrong and what is the right way to do it?
I am using MySQL 5.4 and i want to either write a subquery or a procedure or a function purely in mysql.
There is PREPARE and EXECUTE which can run a sql statement from inside a user variable, so could probably use something similar to (untested!)
SET #a = "";
SELECT #a := CONCAT('(select name from ',GROUP_CONCAT(table_name SEPARATOR ') UNION (select name from '),')') FROM information_schema.tables WHERE table_type='BASE TABLE' GROUP BY 1;
PREPARE stmt FROM #a;
EXECUTE stmt;
DEALLOCATE PREPARE stmt;
What you need is a way to have variables in SQL (so you can say select * from $name where $name in (select ...)). SQL doesn't allow that.
My suggestion is to split the process up:
First run this query:
select 'select distinct name from ' || table_name || ' union'
from select table_name from table_list
That'll give you the selects to run. Put them into a small script, remove the last "union" and run that script.
[EDIT] If MySQL supports an "eval" operator in stored procedures (i.e. where you can build SQL from parts and then run it), you could do this. I checked the docs and it doesn't look like there is an eval. You could also write an extension in C (see chapter 22) that either implements the lookup or an "eval" function.
But my guess is that your database won't change all the time. So the most simple solution would be to write a small SQL script that creates the code for a view (that is a string; it doesn't actually create the view ). Every time the DB changes, you simply run the script to recreate the view and afterwards, you can run the query against the view to get a list of all names.