I have a Stored Procedure in mySQL that takes a subset of data from a table and performs some analysis on that subset within a temp table. Here is my code:
CREATE PROCEDURE GetPortfolioStats
(
InIdx_i INT,
InStart_i INT,
InEnd_i INT
)
BEGIN
DECLARE myLimit INT;
DECLARE myOffset INT;
SET myLimit = InEnd_i - InStart_i + 1;
SET myOffset = InStart_i - 1;
CREATE TEMPORARY TABLE IF NOT EXISTS myTmpTable AS (SELECT * FROM Leases WHERE Portfolio_i = InIdx_i ORDER BY Index_i LIMIT myLimit OFFSET myOffset);
SET #Additive_i := (SELECT COUNT(Index_i) FROM myTmpTable WHERE ReviewType_vc = 'Additive');
DROP TABLE myTmpTable;
SELECT #Additive_i;
END; GO
This works fine. However, the problem I have is that this is a multi-threaded application and when multiple threads are calling this stored proc, they start sharing the same temp table, which messes up the Stats I'm trying to compile.
Is there a way to either apply a unique table name to each call of the stored proc or limit the scope of the temp table to just that instance of the stored proc?
To answer the specific question: the easiest solution would be to use a different database connection per thread because temporary tables are session (connection) specific:
You can use the TEMPORARY keyword when creating a table. A TEMPORARY table is visible only to the current session, and is dropped automatically when the session is closed. This means that two different sessions can use the same temporary table name without conflicting with each other or with an existing non-TEMPORARY table of the same name.
However, after checking out the actual code, I would suggest not to use a temporary table at all, use a single query with a subquery:
SELECT COUNT(Index_i)
FROM
(SELECT Index_i, ReviewType_vc
FROM Leases
WHERE Portfolio_i = InIdx_i
ORDER BY Index_i
LIMIT myLimit OFFSET myOffset) t
WHERE ReviewType_vc = 'Additive'
Related
I'm having issues where an update statement should (to my knowledge) update 5 rows (I have selected 5 rows into a temp table and used an INNER JOIN in the update statement)
however when it comes to running the update statement it updates anything that could have been selected into the temp table not just the joined contents of the temp table itself.
I'm using the FOR UPDATE code in the selection statement to lock the rows, (as I'm expecting multiple queries to be aimed at this table at one time, (NOTE removing that does not change the error effect)
I've generalised the entire code base and it still has the same effect, I've been on this for the past few days and I'm sure that its just something silly I must be doing
Code description
TABLE `data`.`data_table Table to store data and to show it has been taken my the external program.
Stored Procedure `admin`.`func_fill_table debug code to populate the above table.
Stored Procedure `data`.`func_get_data Actual code designed to retrieve a batch size records, mark them as picked up and then return them to an external application.
Basic Setup Code
DROP TABLE IF EXISTS `data`.`data_table`;
DROP PROCEDURE IF EXISTS `admin`.`func_fill_table`;
DROP PROCEDURE IF EXISTS `data`.`func_get_data`;
DROP SCHEMA IF EXISTS `data`;
DROP SCHEMA IF EXISTS `admin`;
CREATE SCHEMA `admin`;
CREATE SCHEMA `data`;
CREATE TABLE `data`.`data_table` (
`identification_field_1` char(36) NOT NULL,
`identification_field_2` char(36) NOT NULL,
`identification_field_3` int(11) NOT NULL,
`information_field_1` int(11) NOT NULL,
`utc_action_time` datetime NOT NULL,
`utc_actioned_time` datetime DEFAULT NULL,
PRIMARY KEY (`identification_field_1`,`identification_field_2`,`identification_field_3`),
KEY `NC_IDX_data_table_action_time` (`utc_action_time`)
);
Procedure Creation
DELIMITER //
CREATE PROCEDURE `admin`.`func_fill_table`(
IN records int
)
BEGIN
IF records < 1
THEN SET records = 50;
END IF;
SET #processed = 0;
SET #action_time = NULL;
WHILE #processed < records
DO
SET #action_time = DATE_ADD(now(), INTERVAL FLOOR(RAND()*(45)-10) MINUTE);#time shorter for temp testing
SET #if_1 = UUID();
SET #if_2 = UUID();
INSERT INTO data.data_table(
identification_field_1
,identification_field_2
,identification_field_3
,information_field_1
,utc_action_time
,utc_actioned_time)
VALUES (
#if_1
,#if_2
,FLOOR(RAND()*5000+1)
,FLOOR(RAND()*5000+1)
,#action_time
,NULL);
SET #processed = #processed +1;
END WHILE;
END
//
CREATE PROCEDURE `data`.`func_get_data`(
IN batch int
)
BEGIN
IF batch < 1
THEN SET batch = 1; /*Minimum Batch Size of 1 */
END IF;
DROP TABLE IF EXISTS `data_set`;
CREATE TEMPORARY TABLE `data_set`
SELECT
`identification_field_1` as `identification_field_1_local`
,`identification_field_2` as `identification_field_2_local`
,`identification_field_3` as `identification_field_3_local`
FROM `data`.`data_table`
LIMIT 0; /* Create a temp table using the same data format as the table but insert no data*/
SET SESSION sql_select_limit = batch;
INSERT INTO `data_set` (
`identification_field_1_local`
,`identification_field_2_local`
,`identification_field_3_local`)
SELECT
`identification_field_1`
,`identification_field_2`
,`identification_field_3`
FROM `data`.`data_table`
WHERE
`utc_actioned_time` IS NULL
AND `utc_action_time` < NOW()
FOR UPDATE; #Select out the rows to process (up to batch size (eg 5)) and lock those rows
UPDATE
`data`.`data_table` `dt`
INNER JOIN
`data_set` `ds`
ON (`ds`.`identification_field_1_local` = `dt`.`identification_field_1`
AND `ds`.`identification_field_2_local` = `dt`.`identification_field_2`
AND `ds`.`identification_field_3_local` = `dt`. `identification_field_3`)
SET `dt`.`utc_actioned_time` = NOW();
# Update the table to say these rows are being processed
select ROW_COUNT(),batch;
#Debug output for rows altered (should be maxed by batch number)
SELECT * FROM
`data`.`data_table` `dt`
INNER JOIN
`data_set` `ds`
ON (`ds`.`identification_field_1_local` = `dt`.`identification_field_1`
AND `ds`.`identification_field_2_local` = `dt`.`identification_field_2`
AND `ds`.`identification_field_3_local` = `dt`. `identification_field_3`);
# Debug output of the rows that should have been modified
SELECT
`identification_field_1_local`
,`identification_field_2_local`
,`identification_field_3_local`
FROM
`data_set`; /* Output data to external system*/
/* Commit the in process field and allow other processes to access thoese rows again */
END;
//
Run Code
call `admin`.`func_fill_table`(5000);
call `data`.`func_get_data`(5);
You are misusing the sql_select_limit-setting:
The maximum number of rows to return from SELECT statements.
It only applies to select statements (to limit results sent to the client), not to insert ... select .... It is intended as a safeguard to prevent users to be accidentally flooded with millions of results, not as another limit function.
While in general, you cannot use a variable for limit, you can do it in a stored procedure (for MySQL 5.5+):
The LIMIT clause can be used to constrain the number of rows returned by the SELECT statement. LIMIT takes one or two numeric arguments, which must both be nonnegative integer constants, with these exceptions: [...]
Within stored programs, LIMIT parameters can be specified using integer-valued routine parameters or local variables.
So in your case, you can simply use
...
FROM `data`.`data_table`
WHERE `utc_actioned_time` IS NULL AND `utc_action_time` < NOW()
LIMIT batch
FOR UPDATE;
I have the following stored procedure in a MYSQL database. The stored procedure gets lon,lat and than I'm doing an update on a different database and table.
DROP PROCEDURE IF EXISTS annuals.updateSalesFlorida;
CREATE DEFINER=`dddd`#`%` PROCEDURE `updateSales`(propertyId int)
BEGIN
DECLARE lat FLOAT;
DECLARE lon FLOAT;
SELECT SitusLongitude,SitusLatitude
INTO lon,lat
FROM annuals.florida
WHERE PropertyID=propertyId
LIMIT 1 FOR UPDATE;
UPDATE sales.florida
SET
`SitusLongitude` = lon,
`SitusLatitude` = lat
WHERE PROPERTYUNIQUEID=propertyId;
END;
Every time I run the stored procedure the SitusLongitude and SitusLatitude columns are 0. I know for a fact that the previous selected SitusLongitude and SitusLatitude have actual values in there. For some reason the variables are not being set in lat,lon. Any idea what I'm dong wrong?
The problem is that the procedure parameter has the same name as a column in the tables. When you refer to propertyid in the queries, it uses the column, not the parameter. Column and variable names are case-insensitive, so it doesn't matter that you spelled one of them PropertyID and the other propertyId.
Use a different name for the parameter, e.g. p_propertyId
Also, there's no need for two queries, you can do it in one with a JOIN.
UPDATE sales.florida AS s
CROSS JOIN (
SELECT *
FROM annuals.florida
WHERE propertyId = p_propertyId
LIMIT 1) AS a
SET s.SitusLongitude = a.SitusLongitude, s.SitusLatitude = a.SitusLatitude
WHERE s.PROPERTYUNIQUEID = p_propertyId
Note that using LIMIT without ORDER BY means that the row that's selected will be unpredictable.
I have a stored procedure, internally I want to call another procedure that returns a record set, how do I get an navigate the record set returned by the stored procedure via the 'CALL' ?
[edit] I've been trying to use a TEMPORARY TABLE as suggested, but having problems:
DROP TEMPORARY TABLE IF EXISTS tbl_HeadOfDepts;
CREATE TEMPORARY TABLE tbl_HeadOfDepts (biDept_id tinyint(4))
INSERT INTO tbl_HeadOfDepts CALL rsHeadOfAnyDepartments(vcCompKey, biWho_id);
I need to use CALL because 'rsHeadOfAnyDepartments' is not a function, but this will not be accepted.
Work in progress, but what I have so far that is not accepted by editor:
BEGIN
#--
# Procedure:
# rsWhoCanIaccess
#
# Parameters:
# vcCompKey, the key corresponding to the company
# biWho_id, the id of the person to check access for
#
# Returns:
# recordset containing all the people this person can access
#--
DECLARE tiSuperUser tinyint(4);
DECLARE EXIT HANDLER FOR SQLEXCEPTION
BEGIN
GET DIAGNOSTICS CONDITION 1 #sqlstate = RETURNED_SQLSTATE,
#errno = MYSQL_ERRNO, #text = MESSAGE_TEXT;
CALL procLogError(vcCompKey, CONCAT("rsWhoCanIaccess: "
,#errno, " (", #sqlstate, "): ", #text));
END;
#Is this user a super user?
SELECT tiIsSuperUser(vcCompKey, biWho_id) INTO tiSuperUser;
SET tiSuperUser = 0;#Hack for testing
IF (tiSuperUser = 1) THEN
#The user is a superuser, return everyone in the company
SELECT
t1.biPerson_id
FROM
tbl_people t1
INNER JOIN
tbl_companies t2
ON
t1.biCompany_id=t2.biCompany_id
AND
t2.vcKey=vcCompKey;
ELSE
#User is not a superuser, is the user head of any departments?
DROP TEMPORARY TABLE IF EXISTS tbl_HeadOfDepts;
CREATE TEMPORARY TABLE tbl_HeadOfDepts (biDept_id tinyint(4))
INSERT INTO tbl_HeadOfDepts CALL rsHeadOfAnyDepartments(vcCompKey, biWho_id);
SELECT * FROM tbl_HeadOfDepts;
END IF;
END
No, stored procedures can produce result sets, but not consume them directly as output from inner calls to other stored procedures. The best that you can do performance-wise is to populate a non-temporary work table and use the results.
Depending on your software and the reality of multiple callers concurrently, you might need to include a session id concept with an auto_increment (AI) column in some control table. This would ensure that with concurrency, multiple callers are not stomping on each other's rows, thus making it non-viable.
How that session would work, at a high level, is the following. The inner stored proc would be handed an AI value (theSession) from the control table, use it to populate a safely segmented session in the work table, and return as an out parameter to the outer (calling) stored proc. That outer one could then safely use those rows, and clean up at the end (delete from workTable where sessionId=theSession).
Why do I suggest a non-temporary work table? To be clear, the work table would be non-temporary. First of all there is the hassle of getting the if exists drop to work. Most importantly, though, it is about performance. DDL calls for temporary table creation are not inexpensive. You will only believe this when you do performance testing to see what I mean. It may seem trivial, but in trivial operations, those DDL calls for creation could very well account for the lion share of the time necessary for the inner stored proc to complete.
internally I want to call another procedure that returns a record set,
In your inner procedure create a TEMPORARY TABLE and populate that temp table saying insert into your_temp_table select query. then you can use that same temp table in your outer query anywhere.
It can even be a normal table as well and need not be temporary table. Also make sure to DROP the table once your procedure computation done as clean-up.
That's wrong per your comment. You should do it like below (a sample code)
create procedure rsHeadOfAnyDepartments(vcCompKey varchar(10), biWho_id int)
as
begin
DROP TEMPORARY TABLE IF EXISTS tbl_HeadOfDepts;
CREATE TEMPORARY TABLE tbl_HeadOfDepts(col1 int, col2 varchar(10), col3 varchar(30));
INSERT INTO tbl_HeadOfDepts
SELECT col1, col2, col3
FROM tblTest;
end
I'd like to create a temporary table which includes an iterator.
I would have a MySQL variable #count which includes the number of rows desired.
I want to import that number of rows into the table, with an iterator, so that I have rows 1, 2, 3, etc.
That would allow me to create the desired result set select number from tmp, and I could include other information where available using a left join.
I could even use the number and create a date. select date(now())+interval number day
Create a procedure (CREATE PROCEDURE) that takes an INT parameter.
In the stored routine, create and populate a temporary table (CREATE TEMPORARY TABLE, the MEMORY engine could be a good choice).
You'll also need a WHILE loop
All you need to do is put this all together, which should be straightforward for a seasoned MySQL user like you:
WHILE counter > 0 DO
INSERT INTO temptable SELECT counter, DATE(NOW()) + INTERVAL count DAY;
SET counter = counter -1;
END WHILE;
Background - I have a DB created from a single large flat file. Instead of creating a single large table with 106 columns. I created a "columns" table which stores the column names and the id of the table that holds that data, plus 106 other tables to store the data for each column. Since not all the records have data in all columns, I thought this might be a more efficient way to load the data (maybe a bad idea).
The difficulty with this was rebuilding a single record from this structure. To facilitate this I created the following procedure:
DROP PROCEDURE IF EXISTS `col_val`;
delimiter $$
CREATE PROCEDURE `col_val`(IN id INT)
BEGIN
DROP TEMPORARY TABLE IF EXISTS tmp_record;
CREATE TEMPORARY TABLE tmp_record (id INT(11), val varchar(100)) ENGINE=MEMORY;
SET #ctr = 1;
SET #valsql = '';
WHILE (#ctr < 107) DO
SET #valsql = CONCAT('INSERT INTO tmp_record SELECT ',#ctr,', value FROM col',#ctr,' WHERE recordID = ',#id,';');
PREPARE s1 FROM #valsql;
EXECUTE s1;
DEALLOCATE PREPARE s1;
SET #ctr = #ctr+1;
END WHILE;
END$$
DELIMITER ;
Then I use the following SQL where the stored procedure parameter is the id of the record I want.
CALL col_val(10);
SELECT c.`name`, t.`val`
FROM `columns` c INNER JOIN tmp_record t ON c.ID = t.id
Problem - The first time I run this it works great. However, each subsequent run returns the exact same record even though the parameter is changed. How does this persist even when the stored procedure should be dropping and re-creating the temp table?
I might be re-thinking the whole design and going back to a single table, but the problem illustrates something I would like to understand.
Unsure if it matters but I'm running MySQL 5.6 (64 bit) on Windows 7 and executing the SQL via MySQL Workbench v5.2.47 CE.
Thanks,
In MySQL stored procedures, don't put an # symbol in front of local variables (input parameters or locally declared variables). The #id you used refers to a user variable, which is kind of like a global variable for the session you're invoking the procedure from.
In other words, #id is a different variable from id.
That's the explanation of the immediate problem you're having. However, I would not design the tables as you have done.
Since not all the records have data in all columns, I thought this might be a more efficient way to load the data
I recommend using a conventional single table, and use NULL to signify missing data.