We are having a performance issue with a reasonable simple stored procedure, so we to a back up of the database and restored it on a simple test machine and on our replica of our production environment. The only differences in the machines and the databases is the hardware, one is in a VM with 8Gig RAM and 4 processsors, the other is a physical cluster 64Gig RAM and 8 processors.
We have the same version of SQL Server (2014 SP1) on both machines, both databases restored from the same back up. When we run, the one takes less than a second, the other takes 2 minutes 40 seconds producing the same result set of 2976 rows.
The main table involved only has circa 59K rows. The execution plans are nearly identical but but for the outputs on one of the clustered index scans
So what is driving me nuts is how / why would an identical database on identical versions of SQL Server produce two wildly different actual number of rows (174822900 vs 58764) with 2975 executions vs 4 ??
This is the code being run
DBCC FREEPROCCACHE;
DECLARE #return_value int
EXEC #return_value = [dbo].[ngage_Tree_SelectClientData]
#ParentId = 0,
#FromDate = '2016-08-04 17:16:40.367',
#UserGroupId = '1|',
#MaxToReturn = 100000
SELECT 'Return Value' = #return_value;
and this is the code for the stored procedure iteself:
CREATE PROC dbo.ngage_Tree_SelectClientData
#ParentId BIGINT
,#FromDate DATETIME2
,#UserGroupId VARCHAR(50)
,#MaxToReturn INT
AS
BEGIN
SET NOCOUNT ON;
DECLARE #parentHierarchyId HIERARCHYID = (SELECT HierarchyId
FROM Tree
WHERE Id = #ParentId);
SELECT DISTINCT TOP (#MaxToReturn)
t.Id, t.ParentId,
t.Title, t.Ordinal, t.HelpTopicId,
t.IsFolder, t.SmallIcon, t.LargeIcon,
t.CmsTemplateId, t.IsSystemFolder,
t.ContextMenu, t.DateModified,
CASE
WHEN n2.[HierarchyId] IS NULL
THEN 0
ELSE 1
END AS 'HasChildren',
t.ToolTip,
t.[HierarchyId].ToString() AS ParentIds, t.Depth
FROM
Tree t
INNER JOIN
TreeToUserGroups ttug ON ttug.TreeId = t.Id
AND ttug.ReadPermission = 1
AND ttug.UserGroupId IN (SELECT ResultList
FROM dbo.SplitCommaList(#UserGroupId, '|'))
LEFT OUTER JOIN
Tree n2 ON t.[HierarchyId] = n2.[HierarchyId].GetAncestor(1)
WHERE
t.IsDeleted = 0
AND t.IsArchived = 0
AND t.DateModified > #FromDate
AND (#ParentId = 0
OR t.[HierarchyId].IsDescendantOf(#parentHierarchyId) = 1)
AND t.Id NOT IN (SELECT Id
FROM Tree
WHERE (TreeObjectType = N'EcomProduct'
AND Title = N'(MISC) MISC')
OR (Title = N'Tag Groups'
AND TreeObjectType = N'TagRoot')
)
ORDER BY
t.DateModified ASC;
END;
https://1drv.ms/u/s!Aj3dwBusMe7yqao2igQ-JZilD4fVzw DEV Plan XML
https://1drv.ms/u/s!Aj3dwBusMe7yqao1JWD-w8sOhvrFhw VM Plan XML
Related
I have a nasty, nasty data layout that I am forced to work with. I finally got a working query using C# and a for loop executing the same query over and over but adjusting which fields are called, but now I am wondering if it is possible to do it with a while loop. I am getting an error, and I am not sure if it is because I am using Faircom / C-tree as the database, or if there is something wrong with my query. I am normally a Mysql user.
the table has 20 fields I care about and want to extract into a csv list. They are codetype1-codetype20 and I want it to be something like value1, value2, value3... where as it is now I get them all back one at a time. Trouble is that codetype1 is dependent on another field to determine where I go look for the info on that code, which is why the case statements.
DROP PROCEDURE IF EXISTS proc_loop_test;
CREATE PROCEDURE proc_loop_test()
SET #index = 1;
WHILE(#index < 21) DO
SELECT Replace(Concat(To_char(apptid), To_char(.#index) ), ' ', '') AS reference_id,
apptid AS a_reference_id,
CASE
WHEN c.ee > 0 THEN d.amt
ELSE insfee.amt
END AS amount,
CASE
WHEN c.ee > 0 THEN Rtrim(e.moneyname)
ELSE insname.namefeecatid
END AS moneyschedule_name,
CASE codetype#index
WHEN 1 THEN rtrim(a.descript)
ELSE rtrim(b.descript0)
END AS description,
CASE codetype#index
WHEN 1 THEN rtrim(a.abbrevdescript)
ELSE rtrim(b.abbrev0)
END AS abbreviated_description,
CASE codetype#index
WHEN 1 THEN rtrim(a.thiscode)
ELSE rtrim(b.thiscode0)
END AS code
FROM meetings
LEFT JOIN
(
SELECT admin.table2.procid,
admin.table2.this_code_id,
admin.v_table1.descript,
admin.v_table1.abbrevdescript,
admin.v_table1.thiscode
FROM admin.table2
INNER JOIN admin.v_table1
ON admin.table2.this_code_id = admin.v_table1.this_code_id) AS a
ON meetings.codeid#index = a.procid
LEFT JOIN
(
SELECT admin.v_table1.descript AS descript0,
admin.v_table1.abbrevdescript AS abbrev0,
admin.v_table1.thiscode AS thiscode0,
admin.v_table1.this_code_id
FROM admin.v_table1) AS b
ON meetings.codeid#index = b.this_code_id
LEFT JOIN
(
SELECT patid AS id,
ee
FROM admin.customer) AS c
ON meetings.patid = c.id
LEFT JOIN
(
SELECT this_code_id AS redid,
eecategoryid,
amt
FROM admin.eeule) AS d
ON c.ee = d.eecategoryid
AND d.redid = b.this_code_id
LEFT JOIN
(
SELECT eecategoryid AS namefeecatid,
moneyname
FROM admin.eeulenames) AS e
ON d.eecategoryid = e.namefeecatid
LEFT JOIN (SELECT pi.customer_id,
pi.primarykk_id AS picid,
pi.primarykk_name,
pi.first_name,
pi.last_name,
i.groupname,
i.ee
FROM admin.v_pir AS pi
LEFT JOIN admin.money AS i
ON pi.primarykk_id = i.insid) AS
ins
ON ins.customer_id = c.id
LEFT JOIN (SELECT this_code_id AS redid,
eecategoryid,
amt
FROM admin.eeule) AS insfee
ON ins.ee = insfee.eecategoryid
AND insfee.redid = b.this_code_id
LEFT JOIN (SELECT eecategoryid AS namefeecatid,
moneyname
FROM admin.eeulenames) AS insname
ON insfee.eecategoryid = insname.namefeecatid
WHERE codeid#index >= 1
END WHILE;
END;
I have never used a while loop, and while I understand somewhat I am supposed to select this to go INTO something, do I need to create a temp table, or can it all just be stored in memory till the end of the loop and returned.
For what it is worth, the entire SELECT query works in C# when you replace the #index with concatenating format " . index . "
Based on the information that you have provided, it is strongly suggested that you reach out to your vendor for this. You're attempting to create a stored procedure, however, using a mySQL proprietary syntax. Stored procedure support is unique to each database vendor. FairCom's c-treeACE SQL actually uses Java for cross platform support and .NET for Windows.
https://docs.faircom.com/doc/jspt/#cover.htm
Stored procedure development requires a strong knowledge of the database layout which is highly application dependent. In this case, many legacy application dependencies may be involved. Again, your best source of information will be your application vendor.
I have four tasks in sequence in a SSIS package. How to get name of currently running tasks after every 10 minutes?
Here is a script you can use to fetch running package_name with SQL Job agent. You could then make a ssis package to extract this data, and then set it up in an SQL JOB Agent which runs every 10 minutes. You can see the task name in message_source_name
Current running package with taskname:
USE SSISDB
GO
/*
Configuration
*/
-- Filter data by execution id (use NULL for no filter)
-- Show only Child Packages or everyhing
DECLARE #showOnlyChildPackages BIT = 0;
-- Show only message from a specific Message Source
DECLARE #messageSourceName NVARCHAR(MAX)= '%'
-- Filter data by project name (use % for no filter)
DECLARE #projectNamePattern NVARCHAR(100) = '%'
-- Filter data by package name (use % for no filter)
DECLARE #packageNamePattern NVARCHAR(100) = '%'
-- Filter data by execution id (use NULL for no filter)
DECLARE #executionIdFilter BIGINT = NULL;
with running as (
/*
Implementation
*/
-- Show last 15 executions
SELECT TOP 15
e.execution_id,
e.project_name,
e.package_name,
e.project_lsn,
e.status,
status_desc = CASE e.status
WHEN 1 THEN 'Created'
WHEN 2 THEN 'Running'
WHEN 3 THEN 'Cancelled'
WHEN 4 THEN 'Failed'
WHEN 5 THEN 'Pending'
WHEN 6 THEN 'Ended Unexpectedly'
WHEN 7 THEN 'Succeeded'
WHEN 8 THEN 'Stopping'
WHEN 9 THEN 'Completed'
END,
e.start_time,
e.end_time,
elapsed_time_min = datediff(mi, e.start_time, e.end_time)
FROM
catalog.executions e
WHERE
e.project_name LIKE #projectNamePattern
AND
e.package_name LIKE #packageNamePattern
AND
e.execution_id = ISNULL(#executionIdFilter, e.execution_id)
and e.status = 2
)
select * from running a
inner join
(
SELECT top 1 * FROM catalog.event_messages em
WHERE ((em.operation_id = #executionIdFilter) OR #executionIdFilter IS NULL)
--AND (em.event_name IN ('OnInformation', 'OnError', 'OnWarning'))
AND (package_path LIKE CASE WHEN #showOnlyChildPackages = 1 THEN '\Package' ELSE '%' END)
AND (em.message_source_name like #messageSourceName)
order by operation_id desc,message_time desc
)
b on a.execution_id = b.operation_id
I have a stored procedure which I'm trying to call, and it takes forever to execute. I have no idea what's wrong. A similar stored procedure in another database executes perfectly. I'm not well-versed with MySQL Workbench, so I don't know if the database settings are different or something.
Following is my stored procedure:
CREATE
DEFINER = `admin`#`%`
PROCEDURE `calculate_daily_coil_moved_by_crane_data`()
BEGIN
set #curr_date = curdate();
set #pre_date = date_add(curdate(), interval -1 day);
set #a_shift_start_ts = concat(#pre_date, ' 06:00:00');
set #a_shift_end_ts = concat(#pre_date, ' 13:59:59');
set #b_shift_start_ts = concat(#pre_date, ' 14:00:00');
set #b_shift_end_ts = concat(#pre_date, ' 21:59:59');
set #c_shift_start_ts = concat(#pre_date, ' 22:00:00');
set #c_shift_end_ts = concat(#curr_date, ' 05:59:59');
SELECT #curr_date,
#pre_date,
#a_shift_start_ts,
#a_shift_end_ts,
#b_shift_start_ts,
#b_shift_end_ts,
#c_shift_start_ts,
#c_shift_end_ts;
#SET DATA
insert into daily_coil_move_by_crane_data_for_report (crane_id, crane_name, date, a_shift, b_shift, c_shift)
select cr.id, cr.name, #pre_date, 0, 0, 0
from yms_phase3.crane cr
where active = 1
order by cr.name;
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE A Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'A';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set a_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #a_shift_start_ts and #a_shift_end_ts
group by in_data.crane_name), 0)
where (a_shift is null or a_shift = 0);
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE B Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'B';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set b_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #b_shift_start_ts and #b_shift_end_ts
group by in_data.crane_name), 0)
where (b_shift is null or b_shift = 0);
#----------------------------------------------------------------------------------------------------
#--> COILS MOVED BY CRANE C Shift <--
#----------------------------------------------------------------------------------------------------
SET #shift = 'C';
#FETCH ROW DATA
update daily_coil_move_by_crane_data_for_report
set c_shift = ifnull((select COUNT(*)
FROM yms_phase3.workorder_history in_data
where in_data.crane_id = daily_coil_move_by_crane_data_for_report.crane_id
and current_execution_status IN (6 , 7)
and in_data.pick_ts between #c_shift_start_ts and #c_shift_end_ts
group by in_data.crane_name), 0)
where (c_shift is null or c_shift = 0);
#----------------------------------------------------------------------------------------------------
#INSERT ALL CRANE ENTRY
insert into daily_coil_move_by_crane_data_for_report (crane_id, crane_name, date, a_shift, b_shift, c_shift)
select -1, 'ALL', #pre_date, SUM(a_shift), sum(b_shift), sum(c_shift)
from daily_coil_move_by_crane_data_for_report
where date = #pre_date
group by date;
#UPDATE TOTAL
update daily_coil_move_by_crane_data_for_report
set total_coils_moved = (a_shift + b_shift + c_shift)
where date = #pre_date;
END
Also tried to execute the query from Java using the following:
jdbcTemplate.execute("CALL calculate_daily_coil_moved_by_crane_data;");
But it gives me the following Exception:
java.sql.SQLException: Lock wait timeout exceeded
Any workaround I can do to solve this?
Please try and edit the configuration file, also search for the same here on stack. There are certain possibilities while checking this out,
Check and edit the config file on Hard drive for MySQL increase the cache capacity and default values as the default values are in KB's the memory allocated is very less and to execute such a big procedure it should at least be some MB.
Increase the connection String timeout, that is by setting up right time in seconds. by default it is 60 seconds, which is very less for executing such a procedure, I think in c# at least we set it to '0' seconds which means that it shall not get timed-out till the query is executed.
If Any left Joins/ inner query please try and check whether the same output is produced in inner joins ? as inner joins are faster than left or right joins.
Add Indexes, have foreign key references properly mapped for faster execution of query.
Hope it works.
Presently troubleshooting a problem where running this SQL query:
UPDATE tblBenchmarkData
SET OriginalValue = DataValue, OriginalUnitID = DataUnitID,
DataValue = CAST(DataValue AS float) * 1.335
WHERE
FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND ZEGCodeID IN
(SELECT ZEGCodeID FROM tblZEGCode
WHERE(ZEGCode = 'C004') OR
(LEFT(ZEGParentCode, 4) = 'C004'))
Results in the following error:
Msg 8114, Level 16, State 5, Line 1
Error converting data type nvarchar to float.
The really odd thing is, if I change the UPDATE to SELECT to inspect the values that are retrieved are numerical values:
SELECT DataValue
FROM tblBenchmarkData
WHERE FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND ZEGCodeID IN
(SELECT ZEGCodeID
FROM tblZEGCode WHERE(ZEGCode = 'C004') OR
(LEFT(ZEGParentCode, 4) = 'C004'))
Here are the results:
DataValue
2285260
1205310
Would like to use TRY_PARSE or something like that; however, we are running on SQL Server 2008 rather than SQL Server 2012. Does anyone have any suggestions? TIA.
It would be helpful to see the schema definition of tblBenchmarkData, but you could try using ISNUMERIC in your query. Something like:
SET DataValue = CASE WHEN ISNUMERIC(DataValue)=1 THEN CAST(DataValue AS float) * 1.335
ELSE 0 END
Order of execution not always matches one's expectations.
If you set a where clause, it generally does not mean the calculations in the select list will only be applied to the rows that match that where. SQL Server may easily decide to do a bulk calculation and then filter out unwanted rows.
That said, you can easily write try_parse yourself:
create function dbo.try_parse(#v nvarchar(30))
returns float
with schemabinding, returns null on null input
as
begin
if isnumeric(#v) = 1
return cast(#v as float);
return null;
end;
So starting with your update query that's giving an error (please forgive me for rewriting it for my own clarity):
UPDATE B
SET
OriginalValue = DataValue,
OriginalUnitID = DataUnitID,
DataValue = CAST(DataValue AS float) * 1.335
FROM
dbo.tblBenchmarkData B
INNER JOIN dbo.tblZEGCode Z
ON B.ZEGCodeID = Z.ZEGCodeID
WHERE
B.FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND (
Z.ZEGCode = 'C004' OR
Z.ZEGParentCode LIKE 'C004%'
)
I think you'll find that a SELECT statement with exactly the same expressions will give the same error:
SELECT
OriginalValue,
DataValue NewOriginalValue,
OriginalUnitID,
DataUnitID OriginalUnitID,
DataValue,
CAST(DataValue AS float) * 1.335 NewDataValue
FROM
dbo.tblBenchmarkData B
INNER JOIN dbo.tblZEGCode Z
ON B.ZEGCodeID = Z.ZEGCodeID
WHERE
B.FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND (
Z.ZEGCode = 'C004' OR
Z.ZEGParentCode LIKE 'C004%'
)
This should show you the rows that can't convert:
SELECT
B.*
FROM
dbo.tblBenchmarkData B
INNER JOIN dbo.tblZEGCode Z
ON B.ZEGCodeID = Z.ZEGCodeID
WHERE
B.FieldDataSetID = '6956beeb-a1e7-47f2-96db-0044746ad6d5'
AND (
Z.ZEGCode = 'C004' OR
Z.ZEGParentCode LIKE 'C004%'
)
AND IsNumeric(DataValue) = 0
-- AND IsNumeric(DataValue + 'E0') = 0 -- try this if the prior doesn't work
The trick in the last commented line is to tack on things to the string to force only valid numbers to be numeric. For example, if you wanted only integers, IsNumeric(DataValue + '.0E0') = 0 would show you those that aren't.
I am writing in mySQL/PHP, but have this problem I cannot get my head around. This one PHP script contains two SQL statements. What I am trying to do is update a sports league table (a SQL table called tblrank - which contains many league tables, separated by TableID), and to indicate if the team has gone up or down since the last time it was updated. This first bit of code is bit clunky and probably could've been written better (I could do it in about 6 lines in MSSQL), it works by counting the number of teams that are ranked lower than it, then adding one. It appears to work... sort of, as I will explain later.
update tblrank AS r
set Rank = 1 + (select count(*) from
(select r2.teamID
from tblrank r2
inner join tblrank r3
where r3.TableID = r2.TableID and r3.TableID = $tableid
and (r3.Points > r2.Points
or (r3.Points = r2.Points and r3.TieBreaker > r2.TieBreaker))) as duh
where duh.teamID = r.teamID
and duh.TableID = r.TableID
and r.TableID = $tableid
Then, this bit of code is run to pick the image to display.
update tblrank
set image = case when Rank < LastRank then 'up.png'
when Rank > LastRank then 'down.png'
else 'nomove.png' end
where TableID = $tableid
Now if I run this for $tableid = 1, it works just fine. But, if I run it for $tableid = 2, then all Ranks in the whole of tblRank are set to 1. (Before running it for $tableid = 2, all the ranks are 1, except for records where TableID =1 ). This is obviously not what I want.
Both statements are inside a if(mysql_query($sql)) conditional, so I can verify if they have executed or not.
I use MS SQL a lot more than mySQL so I'm not an expert - can anyone help me with this as I'm baffled! I have checked that both chunks of code do execute. There is no other SQL executed between the two.
Well, I've fixed it, I'm not sure what was wrong, but I've split the update statement up, with a temporary table called rankcount being created and populated first. I did this to make it easier to see what was going on.
create temporary table rankcount (TableID int, UserID int)");
insert into rankcount (TableID, UserID)
select r2.TableID, r2.UserID
from tblRank r2
inner join tblRank r3
where r3.TableID = r2.TableID and r3.TableID = $tableid
and (r3.Points > r2.Points
or (r3.Points = r2.Points and r3.TieBreakerOne > r2.TieBreakerOne))
Then this works...
update userEntryTableRank r
set Rank = 1 + (select count(*) from rankcount rc
where rc.UserID = r.UserID and rc.TableID = r.TableID)
where r.TableID = $tableid
:)
Here's how I'd do it. First initialize all the Rank values to zero.
UPDATE tblrank SET Rank = 0;
UPDATE tblrank r1 JOIN tblrank r2 ON r1.TableID = r2.TableID
SET r1.Rank = r1.Rank + 1
WHERE r1.Points > r2.Points
OR (r1.Points = r2.Points AND r1.TieBreaker > r2.TieBreaker)
The join naturally matches each row r1 to the set of rows r2 with the same table and team, and a lower score. Then it increments Rank by 1 for each of these matching rows.