I am using function to update to one column , like
DetailedStatus = dbo.fn_GetProcessStageWiseStatus(PR.ProcessID, PR.ProcessRunID, getdate())
Here 500,000 records are continuously UPDATED in this line. Its like like a loop
So using this function for few records its executing fast but when its 500,000 records executing it becomes very slow...
What can I do to make this execute faster using many records?
Any measures to be taken or any split to be used?
Function:
CREATE FUNCTION [dbo].[fn_GetProcessStageWiseStatus]
(
#ProcessID INT
,#ProcessRunID INT
,#SearchDate SMALLDATETIME
)
RETURNS VARCHAR(100)
AS
BEGIN
DECLARE
#iLoopCount SMALLINT
,#iRowCount SMALLINT
,#StepProgress VARCHAR(100)
,#StepCount SMALLINT
IF EXISTS(
SELECT TOP 1 1
FROM dbo.Step S WITH(NOLOCK)
JOIN dbo.vw_FileGroup FG
ON S.FileConfigGroupID = FG.FileConfigGroupID
WHERE S.ProcessID = #ProcessID
AND S.Active = 1
AND FG.FileConfigGroupActive = 1
AND FG.Direction = 'Inbound'
)
BEGIN
SET #StepProgress = 'Not Received'
END
ELSE
BEGIN
SET #StepProgress = 'Not Started'
END
DECLARE #StepRunDetailsTable TABLE
(
KeyNo INT IDENTITY(1,1)
,StepID INT
,StepStartTime SMALLDATETIME
,StepEndTime SMALLDATETIME
,SourceEnv VARCHAR(100)
,DestEnv VARCHAR(100)
)
INSERT INTO #StepRunDetailsTable
SELECT
S.StepID
,MAX(isnull(SR.StepStartTime, '06/06/2079'))
,MAX(isnull(SR.StepEndTime, '06/06/2079'))
,isnull(SENV.EnvironmentName, '')
,isnull(DENV.EnvironmentName, '')
FROM dbo.ProcessRun PR WITH(NOLOCK)
JOIN dbo.StepRun SR WITH(NOLOCK)
ON SR.ProcessRunID = PR.ProcessRunID
JOIN dbo.vw_StepHierarchy SH
ON SR.StepID = SH.StepID
AND SH.Active = 1
JOIN dbo.Step S WITH(NOLOCK)
ON SH.StepID = S.StepID
JOIN dbo.WorkFlow WF WITH(NOLOCK)
ON S.WorkFlowID = WF.WorkFlowID
AND WF.Active = 1
JOIN dbo.Environment SENV WITH(NOLOCK)
ON SENV.EnvironmentID = WF.SourceEnvironmentID
AND SENV.Active = 1
JOIN dbo.Environment DENV WITH(NOLOCK)
ON DENV.EnvironmentID = WF.DestinationEnvironmentID
AND DENV.Active = 1
WHERE PR.ProcessRunID = #ProcessRunID
GROUP BY S.StepID, SENV.EnvironmentName, DENV.EnvironmentName, SH.StepOrder
ORDER BY SH.StepOrder ASC
SELECT #StepCount = COUNT(*)
FROM dbo.ProcessRun PR WITH(NOLOCK)
JOIN dbo.Step S WITH(NOLOCK)
ON PR.ProcessID = S.ProcessID
AND PR.ProcessRunID = #ProcessRunID
AND S.Active = 1
SELECT #iRowCount = COUNT(DISTINCT StepID) FROM #StepRunDetailsTable
SET #iLoopCount = 0
WHILE (#iRowCount > #iLoopCount)
BEGIN
SET #iLoopCount = #iLoopCount + 1
SELECT
#StepProgress =
CASE
--WHEN #SearchDate BETWEEN StepStartTime AND StepEndTime
WHEN #SearchDate >= StepStartTime AND #SearchDate <= StepEndTime
THEN DestEnv + ' Load in Progress'
WHEN #SearchDate > StepEndTime AND #iLoopCount < #StepCount
THEN 'Waiting on next step - Loaded to ' + DestEnv
WHEN #SearchDate > StepEndTime AND #iLoopCount = #StepCount
THEN 'Completed'
WHEN #SearchDate < StepStartTime AND #iLoopCount = 1
THEN 'Load Not Started'
ELSE #StepProgress
END
FROM #StepRunDetailsTable
WHERE KeyNo = #iLoopCount
END
RETURN #StepProgress
END
Thanks in advance.
Seems like you have a change in execution plan when you try to update 500k rows.
You can try and set forceseek hint on the from clause to force using seeks instead of scans.
Also, WHILE (#iRowCount > #iLoopCount) should be replaced with if exists, because you basically check for certain conditions on the results table and you need to return as early as possible.
I see that you use nolock hint everywhere to allow dirty reads, you can set isolation level read uncommitted in the calling stored procedure and remove all of those; or consider to change the database to set read_committed_snapshot on to avoid locks.
By the way, scalar function calls in SQL Server are very expensive, so if you have some massive updates/selects happening in a loop where you call a function you have to avoid using functions as much as possible.
Related
I am learning MySQL through self-practice. In a project, I want to create a transfer module using MySQL (phpMyAdmin). Unfortunately, the WHERE conditions are not working well. I execute the query using the XAMPP application. A part of the query is -
SELECT
*
FROM
(SELECT
`emp_id`,
`emp_name`,
`present_posting`,
`curr_zone`,
`office_ID1` AS `new_office`,
`Zone1` AS `new_zone`,
`office_ID1` AS `C1`,
`post`,
`preference`,
`curr_zone_id`
FROM
`transfer_applications`
WHERE
`Mutual Accepted` != 'Mutual Accepted'
ORDER BY `apID` ASC) `aa`
LEFT JOIN
(SELECT
`zone_ID`, `ZONE`, `office_ID`, `office_Vacancy`
FROM
`vacancy`) `ab` ON `aa`.`C1` = `ab`.`office_ID`
LEFT JOIN
(SELECT
`zid`, `max_min_vacancy`
FROM
`capping_vacancy`) `ac` ON `aa`.`new_zone` = `ac`.`zid`
WHERE
((`curr_zone_id` != `new_zone`
AND (`max_min_vacancy` < 150
AND `max_min_vacancy` > 1)
AND `station_Vacancy` > 0
AND `post` = 4
AND `apID` = x + 1)
OR (`curr_zone_id` = `new_zone`
AND `station_Vacancy` > 0))
The problem is that it allows transfer even if there is no vacancy available that is the minimum capping in WHERE ( max_min_vacancy > 1 ) is not working. I am unable to find out the reason why it skips this condition while all other conditions in the WHERE are working fine. Kindly help me to find out the mistake. Thanks.
The WHERE clause is connected by an OR, so if the first Boolean expression is FALSE (ie. max_min_vacancy <=1), it is still TRUE as long as the second boolean expression returns TRUE.
...
WHERE
(
(
`curr_zone_id` != `new_zone` AND
(
`max_min_vacancy` < 150 AND
`max_min_vacancy` > 1 -- if max_min_vacancy <= 1 --> FALSE
) AND
`station_Vacancy` > 0 AND
`post` = 4 AND
`apID` = x + 1
)
OR ( -- But this condition is TRUE
`curr_zone_id` = `new_zone` AND
`station_Vacancy` > 0
)
)
I'm using a procedure to calculate the length of user 'hiatus' (aka contingencies) from the program in our system. It runs after a procedure that determines user status depending on whether they are completing their daily treatment and to what extent.
The purpose of this procedure is to log the length of a user's contingency, by adding a row to a table with the following schema:
id_contingency int(11) NOT NULL AUTO_INCREMENT,
id_user int(11) DEFAULT NULL,
date_start date DEFAULT NULL,
program_day int(11) DEFAULT NULL,
date_end date DEFAULT NULL,
total_days int(11) DEFAULT NULL,
latest_tf_id archer(255) DEFAULT NULL
I considered adding this as a trigger on the update of the user_status table, but I can't risk an error preventing that table from updating. So, this procedure first closes contingencies that were previously open, when the user first entered the hiatus, but has now resumed the program, and it later opens new contingencies for users who have now started a hiatus in their treatment for the first time. It then remains open until they resume the program, and calculates how long they were on hiatus for.
This was my original procedure, and it returned error 1109 (unknown table tbl_user_status) :
DELIMITER $$
CREATE DEFINER=CURRENT_USER PROCEDURE `proc_cont_calc`
NO SQL
BEGIN
#CLOSE OPEN CONTINGENCIES FIRST or d0 > d1
CASE
WHEN tbl_user_status.d4 = 1 AND tbl_user_status.d2 > 0 AND tbl_user_status.user_status = 'seguimiento' THEN
UPDATE tbl_user_contingency, tbl_user_status SET
tbl_user_contingency.date_end = CURRENT_DATE,
tbl_user_contingency.total_days = DATEDIFF(tbl_user_contingency.date_start, tbl_user_contingency.date_end),
tbl_user_contingency.updated_by = 'proc_cont.close'
WHERE tbl_user_contingency.date_end = '' AND tbl_user_contingency.id_smoker = tbl_user_status.id_smoker LIMIT 1;
#OPEN NEW CONTINGENCIES
WHEN tbl_user_status.d5 = 1 AND tbl_user_status.d4 = 0 AND tbl_user_status.user_status = 'contingencia' THEN
INSERT INTO tbl_user_contingency (id_smoker, roadmap_day, date_start, latest_tf_id, updated_by) SELECT
id_smoker, roadmap_day, CURRENT_DATE, latest_tf_id, 'proc_cont.open' FROM tbl_user_status;
END CASE;
END$$
DELIMITER;
So I tried this (amongst other things):
CASE
WHEN (SELECT d4 FROM tbl_user_status) = 1 AND (SELECT d2 FROM tbl_user_status) > 0 AND (SELECT user_status FROM tbl_user_status) = 'seguimiento' THEN
UPDATE tbl_user_contingency, tbl_user_status SET
tbl_user_contingency.date_end = CURRENT_DATE,
tbl_user_contingency.total_days = DATEDIFF(tbl_user_contingency.date_start, tbl_user_contingency.date_end),
tbl_user_contingency.updated_by = 'proc_cont.close'
WHERE tbl_user_contingency.id_smoker = tbl_user_status.id_smoker LIMIT 1;
#OPEN NEW CONTINGENCIES
WHEN (SELECT d5 FROM tbl_user_status) = 1 AND (SELECT d4 FROM tbl_user_status) = 0 AND (SELECT user_status FROM tbl_user_status) = 'contingencia' THEN
INSERT INTO tbl_user_contingency (id_smoker, roadmap_day, date_start, latest_tf_id, updated_by) SELECT
id_smoker, roadmap_day, CURRENT_DATE, latest_tf_id, 'proc_cont.open' FROM tbl_user_status;
END CASE;
And now I'm getting error 1242 returning multiple rows.
How can I get this procedure to run properly? Thanks!
UPDATE - I tried #P.Salmon's suggestion to simply update the rows, but not all the fields were filling out, or the update overruns previous contingencies.
Thanks!
The case statement seems unnecessary here just move the conditions to where clauses for example
UPDATE tbl_user_contingency join tbl_user_status on tbl_user_contingency.id_smoker = tbl_user_status.id_smoker
SET
tbl_user_contingency.date_end = CURRENT_DATE,
tbl_user_contingency.total_days = DATEDIFF(tbl_user_contingency.date_start, tbl_user_contingency.date_end),
tbl_user_contingency.updated_by = 'proc_cont.close'
WHERE tbl_user_contingency.date_end = '' AND
tbl_user_status.d4 = 1 AND tbl_user_status.d2 > 0 AND tbl_user_status.user_status = 'seguimiento'
;
INSERT INTO tbl_user_contingency (id_smoker, roadmap_day, date_start, latest_tf_id, updated_by)
SELECT
id_smoker, roadmap_day, CURRENT_DATE, latest_tf_id, 'proc_cont.open'
FROM tbl_user_status
where tbl_user_status.d5 = 1 AND tbl_user_status.d4 = 0 AND tbl_user_status.user_status = 'contingencia'
;
You could improve your question and get thereby a better response if you describe what it is you are trying to do instead of having us guess by reverse engineering two non working code segments, by adding your table definitions, sample data and expected output as text to your question. BTW I hope you have a mechanism that will stop this thing doing stuff more than once.
I have a stored procedure which I'm trying to call, and it takes forever to execute. I have no idea what's wrong. A similar stored procedure in another database executes perfectly. I'm not well-versed with MySQL Workbench, so I don't know if the database settings are different or something.
Following is my stored procedure:
CREATE
DEFINER = `admin`#`%`
PROCEDURE `calculate_daily_coil_moved_by_crane_data`()
BEGIN
set #curr_date = curdate();
set #pre_date = date_add(curdate(), interval -1 day);
set #a_shift_start_ts = concat(#pre_date, ' 06:00:00');
set #a_shift_end_ts = concat(#pre_date, ' 13:59:59');
set #b_shift_start_ts = concat(#pre_date, ' 14:00:00');
set #b_shift_end_ts = concat(#pre_date, ' 21:59:59');
set #c_shift_start_ts = concat(#pre_date, ' 22:00:00');
set #c_shift_end_ts = concat(#curr_date, ' 05:59:59');
SELECT #curr_date,
#pre_date,
#a_shift_start_ts,
#a_shift_end_ts,
#b_shift_start_ts,
#b_shift_end_ts,
#c_shift_start_ts,
#c_shift_end_ts;
#SET DATA
insert into daily_coil_move_by_crane_data_for_report (crane_id, crane_name, date, a_shift, b_shift, c_shift)
select cr.id, cr.name, #pre_date, 0, 0, 0
from yms_phase3.crane cr
where active = 1
order by cr.name;
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE A Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'A';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set a_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #a_shift_start_ts and #a_shift_end_ts
group by in_data.crane_name), 0)
where (a_shift is null or a_shift = 0);
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE B Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'B';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set b_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #b_shift_start_ts and #b_shift_end_ts
group by in_data.crane_name), 0)
where (b_shift is null or b_shift = 0);
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE C Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'C';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set c_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #c_shift_start_ts and #c_shift_end_ts
group by in_data.crane_name), 0)
where (c_shift is null or c_shift = 0);
#----------------------------------------------------------------------------------------------------
#INSERT ALL CRANE ENTRY
insert into daily_coil_move_by_crane_data_for_report (crane_id, crane_name, date, a_shift, b_shift, c_shift)
select -1, 'ALL', #pre_date, SUM(a_shift), sum(b_shift), sum(c_shift)
from daily_coil_move_by_crane_data_for_report
where date = #pre_date
group by date;
#UPDATE TOTAL
update daily_coil_move_by_crane_data_for_report
set total_coils_moved = (a_shift + b_shift + c_shift)
where date = #pre_date;
END
Also tried to execute the query from Java using the following:
jdbcTemplate.execute("CALL calculate_daily_coil_moved_by_crane_data;");
But it gives me the following Exception:
java.sql.SQLException: Lock wait timeout exceeded
Any workaround I can do to solve this?
Please try and edit the configuration file, also search for the same here on stack. There are certain possibilities while checking this out,
Check and edit the config file on Hard drive for MySQL increase the cache capacity and default values as the default values are in KB's the memory allocated is very less and to execute such a big procedure it should at least be some MB.
Increase the connection String timeout, that is by setting up right time in seconds. by default it is 60 seconds, which is very less for executing such a procedure, I think in c# at least we set it to '0' seconds which means that it shall not get timed-out till the query is executed.
If Any left Joins/ inner query please try and check whether the same output is produced in inner joins ? as inner joins are faster than left or right joins.
Add Indexes, have foreign key references properly mapped for faster execution of query.
Hope it works.
I have a MySQL stored procedure and in it, the following WHILE statement.
I have confirmed that #RowCnt is 1, and #MaxRows is 6090, however after further debugging, I realized that the WHILE statement is going through a single iteration and not continuing; so I'm hoping to have some light shed on what could possibly be causing this.
Full disclosure: I ported this from SQL Server to a MySQL stored procedure, something I have never taken on before. (meaning SQL Server, porting OR stored procedures..)
WHILE #RowCnt <= #MaxRows DO
SELECT #currentReadSeq:=ReadSeq, #currentReadStrength:=ReadStrength, #currentReadDateTime:=ReadDateTime, #currentReaderID:=ReaderID FROM tblTempRead WHERE rownum = #RowCnt;
IF ( ((#lastReadSeq + 10) > #currentReadSeq) AND (#lastReaderId = #currentReaderId) ) THEN
SET #lastReadSeq = #currentReadSeq, #lastReadStrength = #currentReadStrength, #lastReadDateTime = #currentReadDateTime, #lastReaderID = #currentReaderID;
ELSE
INSERT INTO tblreaddataresults (SiteID, ReadDateTimeStart, ReadDateTimeEnd, ReadSeqStart, ReadSeqEnd, ReaderID, DirectSeconds) VALUES ('1002', #saveReadDateTime, #lastReadDateTime, #saveReadSeq, #lastReadSeq, #lastReaderID, timestampdiff(SECOND,#saveReadDateTime,#lastReadDateTime));
SET #saveReadSeq = #currentReadSeq, #saveReadStrength = #currentReadStrength, #saveReadDateTime = #currentReadDateTime, #saveReaderID = #currentReaderID;
SET #lastReadSeq = #saveReadSeq, #lastReadStrength = #saveReadStrength, #lastReadDateTime = #saveReadDateTime, #lastReaderID = #saveReaderID;
END IF;
SET #RowCnt = #RowCnt+1;
END WHILE;
Try This Construct
WHILE (#RowCnt <= #MaxRows)
BEGIN
SELECT #currentReadSeq:=ReadSeq, #currentReadStrength:=ReadStrength, #currentReadDateTime:=ReadDateTime, #currentReaderID:=ReaderID FROM tblTempRead WHERE rownum = #RowCnt;
IF (((#lastReadSeq + 10) > #currentReadSeq) AND (#lastReaderId = #currentReaderId))
BEGIN
SET #lastReadSeq = #currentReadSeq, #lastReadStrength = #currentReadStrength, #lastReadDateTime = #currentReadDateTime, #lastReaderID = #currentReaderID;
END
ELSE
BEGIN
INSERT INTO tblreaddataresults (SiteID, ReadDateTimeStart, ReadDateTimeEnd,ReadSeqStart, ReadSeqEnd, ReaderID, DirectSeconds) VALUES ('1002',#saveReadDateTime, #lastReadDateTime, #saveReadSeq, #lastReadSeq, #lastReaderID,timestampdiff(SECOND,#saveReadDateTime,#lastReadDateTime));
SET #saveReadSeq = #currentReadSeq, #saveReadStrength = #currentReadStrength, #saveReadDateTime = #currentReadDateTime, #saveReaderID = #currentReaderID;
SET #lastReadSeq = #saveReadSeq, #lastReadStrength = #saveReadStrength,#lastReadDateTime = #saveReadDateTime, #lastReaderID = #saveReaderID;
END
SET #RowCnt = #RowCnt+1;
END
How can I print the rows updated by this query in this query:
update
Table1.RecommendationLeg
set
actualValue = ( leg.actualprice * str.currentSize)
from
Table1.RecommendationLeg leg
inner join Recommendation str
on leg.partofId = str.id
where
leg.actualValue = 0
and datediff( n, timeOf, CURRENT_TIMESTAMP) > 30
update
Table1.RecommendationLeg
set
actualValue = ( leg.actualprice * str.currentSize)
OUTPUT INSERTED.actualValue -- <-- this. Edit, after SET not UPDATE. Oops. Sorry.
from
Table1.RecommendationLeg leg
inner join Recommendation str
on leg.partofId = str.id
where
leg.actualValue = 0
and datediff( n, timeOf, CURRENT_TIMESTAMP) > 30
If you are on SQL Server 2005 and above, you can use the OUTPUT clause.