Why is it that when I run the query in Management Studio with the following where clause, it returns the data correctly when the cash_date equals the parameter, FromDate.
FILMTRAN.cash_date >= (#FromDate) AND
However, when I copy the same query to SSRS it no longer includes the data where the #FromDate equals the cash_date. The user enters in the parameter in SSRS. I even did a test where I displayed the full date including the time and did an if statement that returned "yay" if they matched or "no" if they did not. Even according to SSRS, they match, but than why is my data not being returned when it occured on the same day?! I am dumbfounded this is not working, when it clearly works in management studio.
I have even tried adding a dateadd function into my sql statement, which again responds correctly in management studio but not ssrs.
Edit: I tested the >= using a much simpler 3 line query and it worked as intended but it is still not working for my query, which works in management studio but not SSRS. So now I am thinking there is some very specific bug associated with something in my longer query which I have attached. To test my code in management studio I declare the variables as seen below. However, I delete this code when I import my query into SSRS and instead create the parameters within SSRS(just as I have always done, with no problems).
DECLARE #FromDate AS DATETIME
SET #FromDate = '1-Oct-2013'
DECLARE #ToDate AS DATETIME
SET #ToDate = '8-Apr-2015'
DECLARE #Loan AS VARCHAR(20)
SET #Loan = 'TAX_B'
SELECT distinct LOANMAST.loan,
CARDINDX.DESCRIPTION AS BorrowerName,
CARDINDX.contact_name,
CARDINDX.in_care_of,
--loan_ipt.interest,
CARDINDX.mail_address1,
CARDINDX.mail_address2,
CARDINDX.mail_city,
CARDINDX.mail_state,
CARDINDX.mail_country,
LOANMAST.closed_date,
PROPERTY.STATE,
CARDINDX.MAIL_ZIP_CODE,
PROPERTY.address1,
PROPERTY.address2,
PROPERTY.city,
PROPERTY.ZIPCODE,
PROPERTY.country,
FILMTRAN.cash_date,
CASE FILMTRAN.TRAN_SUBTYPE WHEN 'REGINST' THEN 'Regular Collections'
WHEN 'PAC' THEN 'PAC'
WHEN 'CORREST' THEN 'Group Collection'
WHEN 'ESCROWTAX' THEN 'Tax Disbursement'
WHEN 'SUNDRY' THEN 'Tax Refund'
WHEN 'FULL PAY' THEN 'Payoff'
END AS TRANSUBTYPE,
FILMTRAN.TAX_ESCROW,
FILMTRAN.TRANSACTION079,
JK.taxescrowbalance,
CASE FILMTRAN.TRANSACTION079 WHEN 'COLLECTION' THEN FILMTRAN.TAX_ESCROW
WHEN 'DISBURSEMENT' THEN 0
END AS TAXESCROWCOLL,
CASE FILMTRAN.TRANSACTION079 WHEN 'DISBURSEMENT' THEN FILMTRAN.TAX_ESCROW *-1
WHEN 'COLLECTION' THEN 0
END AS TAXESCROWDIS
FROM LOANMAST LEFT OUTER JOIN BORROWER ON LOANMAST.loan=BORROWER.loan
LEFT OUTER JOIN LOANCOLL ON LOANMAST.loan=LOANCOLL.loan
LEFT OUTER JOIN PROPERTY ON LOANCOLL.code=PROPERTY.prop_code
LEFT OUTER JOIN BALHIST ON BALHIST.loan=LOANMAST.loan
LEFT OUTER JOIN CARDINDX ON BORROWER.SHORT_NAME=CARDINDX.SHORT_NAME
LEFT OUTER JOIN LOAN_IPT ON LOAN_IPT.short_name=CARDINDX.short_name
LEFT OUTER JOIN FILMTRAN ON FILMTRAN.loan=LOANMAST.loan
JOIN (
SELECT distinct FILMTRAN.loan, FILMTRAN.cash_date, FILMTRAN.transaction079 ,FILMTRAN.tax_escrow,
sum(FT.taxescrowcoll)+sum(FT.taxescrowdis)+isnull(BH.esc_tax_bal_p, 0) AS "TaxEscrowBalance"
from
FILMTRAN
LEFT OUTER JOIN (SELECT loan, accounting_date, esc_tax_bal_p,
rank () over (order by accounting_date desc) AS date_rank
FROM BALHIST
where loan = #Loan) BH ON BH.loan=FILMTRAN.loan
JOIN
(
SELECT loan, cash_date,
FILMTRAN.transaction079,
CASE FILMTRAN.TRANSACTION079 WHEN 'COLLECTION' THEN TAX_ESCROW
WHEN 'DISBURSEMENT' THEN 0
END AS TAXESCROWCOLL,
CASE FILMTRAN.TRANSACTION079 WHEN 'DISBURSEMENT' THEN TAX_ESCROW *-1
WHEN 'COLLECTION' THEN 0
END AS TAXESCROWDIS FROM FILMTRAN
WHERE loan = #Loan)
FT ON FT.loan=FILMTRAN.loan
where FT.cash_date <= FILMTRAN.cash_date
AND (BH.date_rank ='1' OR BH.accounting_date IS NULL)
AND (len(FT.transaction079) <= len(FILMTRAN.transaction079) OR FT.cash_date < FILMTRAN.cash_date)
GROUP BY FILMTRAN.loan,FILMTRAN.cash_date, FILMTRAN.transaction079, FILMTRAN.TAX_ESCROW, BH.esc_tax_bal_p) JK ON JK.loan=FILMTRAN.loan AND JK.cash_date = FILMTRAN.cash_date AND JK.transaction079=FILMTRAN.transaction079
WHERE
FILMTRAN.RVRS_REASON <> 'ERROR_CORR' AND
Loancoll.prim_ind = 'Y' AND
LOANMAST.loan_status <> '1_INQUIRY' AND
LOANMAST.loan_status <> '2_APP_ISS' AND
LOANMAST.loan_status <> '3_APP_ACC' AND
LOANMAST.loan_status <> '4_APPROVED' AND
LOANMAST.loan_status <> '5_COMMITTED' AND
LOANMAST.loan_status <> '9_DELN_SLF' AND
LOANMAST.loan_status <> '9_REJ_BRWR' AND
FILMTRAN.cash_date >= (#FromDate)
AND FILMTRAN.cash_date <= (#ToDate)
AND LOANMAST.loan = #Loan
AND FILMTRAN.tax_escrow <> 0
You have a date formatting issue.
Hopefully you are calling a stored procedure to generate your dataset.
If not, you should be!
To see what is happening you might want to output the dates that are getting passed through into a table - then have a look at that table once you have run the report. Obviously you will drop this table once your debugging is complete.
Make sure you store the dates in this table as text rather than DateTime.
Have a look at the diffrences when you run from SSRS and Management Studio.
I was unable to replicate this issue. In my system, using SSRS-2008, the results in SSRS are the same as in SQl-Server. I'm using a where clause with an SSRS parameter and a >= operator. I tested with both Text and Date/Time parameter types, and both worked fine.
I would suggest that you make sure that the datasource is the same as what you are querying in SQL server. Also, try entering the date without a time, if you are entering a time in the parameter.
For further assistance, please post your complete query (or a facsimile) and information on your table and parameter settings in SSRS.
EDIT:
Just taking a look at your query here, and I have a feeling that this may be the problem area:
where FT.cash_date <= FILMTRAN.cash_date
AND (BH.date_rank ='1' OR BH.accounting_date IS NULL)
AND (len(FT.transaction079) <= len(FILMTRAN.transaction079) OR FT.cash_date < FILMTRAN.cash_date)
Possibly could be stemming from confusion over the use of Filmtran without aliasing within the subquery, when it is already included in the main query? I would run the query without the two sub-queries in the From clause and see if the dates are as you'd expect, the add back in the subqueries one by one to determine the location of the issue.
Ok so the solution to my problem was dumber than I would like to admit. So in addition to stating my solution I will also suggest potential problems/solutions that relate to dates in SSRS/Management Studio:
Check the filters on the dataset within SSRS! This was my problem since I used a report template that had filters already on the dataset, even though I usually do all filters by hand in SQL management studio. (See picture below)
Make sure the time is correct, not just the date. For example, 10/1/2013 might appear but the actual value behind that date is something like 10/1/2013 5:43 PM. So then doing <= will not work if you are trying to include values that happened after 10/1/2013 5:43 PM but before 10/2/2013.
Make sure the regional date settings are the same for SSRS and SQL Server. For example, if one machine views dates in the European format of DD/MM/YYYY then it could cause problems if another user's machine is expecting the date to be in MM/DD/YYYY format.
Related
reposting this question as I am still struggling to find the right solution.
I was able to isolate the 'OA' only scenarios and flag them appropriately as NA. However I am having trouble isolating the 'I' only events as well as properly aging the EquipID's that have both I and OA events.
This is the logic I am using for aging right now and it works great for 'I' only events, but properly isolating the 'I' and 'OA' EquipID's and then getting the difference between the two dates is eluding me.
datediff(curdate(),concat(left(eventDateTime,4),'-',mid(eventDateTime,5,2),'-',mid(eventDateTime,7,2))) as age_in_depot
I tried grouping by the EquipID but of course that gives me only one eventDateTime result and it is always the first result, which in the case of an 'I' and 'OA' it's only the 'I' eventDateTime.
I also have logic to identify the 'I' and 'OA' events, but this only works correctly if I am intending on grouping the EquipmentID's together to one line, which noted above, drops the 2nd eventDateTime entry needed to properly age these EquipID's.
GROUP_CONCAT( DISTINCT StatusCD) as StatusCD
Any help or insight is greatly appreciated. Once I have this 'I' only event flagged and aging I can finally finish out this dashboard I am building for our team.
Thank you in advance.
J
Pivot the table so you get the in and out dates into separate columns for the same EquipID. Then you can subtract them, and apply appropriate defaults for the missing dates.
SELECT EquipID, InDate, OutDate, StatusCD_N,
CASE WHEN InDate AND OutDate THEN DATEDIFF(OutDate, InDate)
WHEN InDate THEN DATEDIFF(CURDATE(), InDate)
ELSE 'N/A' -- only OutDate
END AS AgeInDepot
FROM (
SELECT EquipID, GROUP_CONCAT(StatusCD) AS StatsuCD_N,
MAX(IF(StatusCD = 'I', DATE(STR_TO_DATE(eventDateTime, '%Y%m%d%H%i')), NULL) AS InDate,
MAX(IF(StatusCD = 'OA', DATE(STR_TO_DATE(eventDateTime, '%Y%m%d%H%i')), NULL) AS OutDate
FROM yourTable
GROUP BY EquipID
) AS x
I'm trying to inherit value from previous row (based on correct subscription_id + checking for IS NULL subscription_status), but something goes wrong and I get incorrect value.
Take a look at screenshot.
If I'm not mistaken it also called last non-null puzzle, but examples of possible solution for other DB provide window function with IGNORE NULLS.
But, I'm using MySQL 8.x and it doesn't support this function.
I'm sorry, but SQL fiddle doesn't provide correct text-value for variables in my code :(
https://www.db-fiddle.com/f/wHanqoSCHKJHus5u6BU4DB/4
Or, you can see mistakes here:
SET #history_subscription_status = NULL;
SET #history_subscription_id = 0;
SELECT
c.date,
c.user_id,
c.subscription_id,
sd.subscription_status,
(#history_subscription_id := c.subscription_id) as 'historical_sub_id',
(#history_subscription_status := CASE
WHEN #history_subscription_id = c.subscription_id AND sd.subscription_status IS NULL
THEN #history_subscription_status
ELSE
sd.subscription_status
END
) as 'historical'
FROM
calendar c
LEFT JOIN
subscription_data sd ON sd.date = c.date AND sd.user_id = c.user_id AND sd.subscription_id = c.subscription_id
ORDER BY
c.user_id,
c.subscription_id,
c.date
I expect to get results for this query in this way:
IMPORTANT: I'm going to use this code for a lot of data (about 1 mln rows), so it very important for me to avoid additional select or subquery that can slow down the execution of the query.
I have two servers running MySQL. Both are on windows. One, is my local machime (Windows 7, MySQL 5.6.25, 32bit) and the other is my production webserver (Windows 2012, MySQL 5.7.11-log, 64bit (that's what show variables showed me).
The data is identical between the two. I backed the data up from the windows 7 (using MySQL Workbench) and restored it on the 2012 machine.
I am running a query on both machine but I am getting different results. I have two tables, projects and projectsnotes with a 1:m relationship between them related on projects.id to projectsnotes.idProject. Each note is marked with a date (dComment). The goal of the query is to retrieve project information and the latest comment only.
Here's the query:
select space(1) as cAction,
p.id,
p.iNum,
p.cStatus,
p.cName,
p.cDesc,
ifnull(pl.cNickName, 'UNASSIGNED') as cProjectLeader,
IFNULL(concat (
date_format(pn.dComment, '%Y-%m-%d'),
': ',
pn.cComment
), '') as cComment,
date_format(p.dRequested, '%Y-%m-%d') as dRequested,
date_format(p.dRequired, '%Y-%m-%d') as dRequired,
format(p.nPercentComplete, 2) as nPercentComplete,
p.tLastUpdated,
p.bCompleted,
p.idProjectLeader
from projects p
left outer join projectleaders pl on p.idProjectLeader = pl.id
left outer join (
select idProject,
dComment,
cComment
from projectnotes
order by dComment desc,
tLastUpdated desc
) pn on p.id = pn.idProject
where p.cInstallCode = 'ITM'
and cStatus in ('Pending', 'Active', 'On Hold', 'Completed', 'Cancelled')
and bCompleted = 0
group by iNum
order by iNum;
Now, here's the weird part. When I run this on my Windows 7 machine, I get the right value for cComment. Specifically:
2017-03-28: Text from note replace
That is the latest note. When I run it on the 2012 server:
2016-05-17: Text from this note replaced too
If I run the subquery alone on the 2012 server, I get the right values (namely, a list of all the notes in the reverse order.
Oh, and this note is neither the first nor the last in the notes for this project.
So I am honestly wondering what is going on. Any thoughts on this would be greatly appreciated.
Thanks in advance.
This is expected behavior.
select ...
from projects p
left outer join projectleaders pl on p.idProjectLeader = pl.id
left outer join (...) pn on p.id = pn.idProject
where ...
group by iNum
order by iNum;
Due to MySQL's peculiar handling of GROUP BY, it will not report an error on this query. However, you must keep in mind that, since you use no aggregates, and the GROUP BY will eliminate lots of rows, the rows that are kept in the final result set are determined by rather obscure criteria...
For example:
SELECT a,b FROM t GROUP BY a
Which b will be returned? In some MySQL versions, this will be the first value of b that is found in table t. If table t is ordered in a certain way, this can be exploited. But I would definitely not trust that behavior to stay unchanged between versions... Also, remember MySQL is free to change your join order...
OK. I think I have a solution to this. Instead of doing it with a join I wrote a function that returned the value I needed as follows:
DROP FUNCTION if exists f_lastprojectnote;
DELIMITER $$
CREATE FUNCTION f_lastprojectnote(tidProject varchar(36))
RETURNS varchar(1000) DETERMINISTIC
BEGIN
DECLARE cRetVal VARCHAR(1000);
SELECT concat(date_format(pn.dComment, '%Y-%m-%d'), ': ', pn.cComment) INTO cRetVal
FROM projectnotes pn
WHERE idProject = tidProject
ORDER BY dComment DESC, tLastUpdated DESC
LIMIT 1;
RETURN cRetVal;
END$$
DELIMITER ;
It works...
I am trying to build an access query with multiple criteria. The table to be queried is "tblVendor" which has information about vendor shipment data as shown below:
The second table is "tblSchedule" which has the schedule for each Vendor cutoff date. This table has cutoff dates for data analysis.
For each vendor, I need to select records which have the ShipDate >= CutoffDate. Although not shown in the data here, it may be possible that multiple vendors have same CutoffDate.
For small number of records in "tblCutoffdate", I can write a query which looks like:
SELECT tblVendors.ShipmentId, tblVendors.VendorNumber, tblVendors.VendorName,
tblVendors.Units, tblVendors.ShipDate
FROM tblVendors INNER JOIN tblCutoffDate ON tblVendors.VendorNumber =
tblCutoffDate.VendorNumber
WHERE (((tblVendors.VendorNumber) In (SELECT VendorNumber FROM [tblCutoffDate] WHERE
[tblCutoffDate].[CutoffDate] = #2/1/2014#)) AND ((tblVendors.ShipDate)>=#2/1/2014#)) OR
(((tblVendors.VendorNumber) In (SELECT VendorNumber FROM [tblCutoffDate] WHERE
[tblCutoffDate].[CutoffDate] = #4/1/2014#)) AND ((tblVendors.ShipDate)>=#4/1/2014#));
As desired, the query gives me a result which looks like:
What concerns me now is that I have a lot of records being added to the "tblCutoffDate" which makes it difficult for me to hardcode the dates in the query. Is there a better way to write the above SQL statement without any hardcoding?
You might try something like -- this should handle vendors having no past cutoff,
or those having no future cutoff
"today" needs a suitable conversion to just date w/o time
comparison "=" may go on both, or one, or none Max/Min
"null" may be replaced by 1/1/1900 and 12/31/3999 in Max/Min
SELECT tblvendors.shipmentid,
tblvendors.vendornumber,
tblvendors.vendorname,
tblvendors.units,
tblvendors.shipdate
FROM tblvendors
LEFT JOIN
( SELECT vendornum,
Max( iif cutoffdate < today, cutoffdate, null) as PriorCutoff,
Min( iif cutoffdate >= today, cutoffdate, null) as NextCutoff
FROM tblcutoffdate
GROUP BY vendornum
) as VDates
ON vendornumber = vendornum
WHERE tblvendors.shipdate BETWEEN PriorCutoff and NextCutoff
ORDER BY vendornumber, shipdate, shipmentid
A simpler WHERE clause should give you what you want.
SELECT
v.ShipmentId,
v.VendorNumber,
v.VendorName,
v.Units,
v.ShipDate
FROM
tblVendors AS v
INNER JOIN tblCutoffDate AS cd
ON v.VendorNumber = cd.VendorNumber
WHERE v.ShipDate >= cd.CutoffDate;
I have what appears to be a corrupt index?
Here is what is happening. I have two table-functions which the first is a set of cases and the second is a set of aware dates. These two sets have a 1 (case) to 0 or 1 (aware date) relationship. Normally I query them like;
SELECT c.CaseID, a.AwareDate
FROM Cases(#date) AS c
LEFT JOIN AwareDates(#date) AS a ON c.CaseID = a.CaseID;
The trouble is that not all of the rows from AwareDates which match seem to be JOIN'd. If I add a join hint, they then do. say;
SELECT c.CaseID, a.AwareDate
FROM Cases(#date) AS c
LEFT MERGE JOIN AwareDates(#date) AS a ON c.CaseID = a.CaseID;
What I notice from the query plan is that adding the join hint adds a sort of the AwareDate data before the join which is not there otherwise. Also, the query planner flips the join to a RIGHT OUTER JOIN when there is no hint, and of course keeps the LEFT JOIN where the hint is present.
I've done the following with no errors detected;
DBCC UPDATEUSAGE (0) WITH INFO_MESSAGES, COUNT_ROWS;
EXECUTE sp_updatestats 'resample';
DBCC CHECKDB (0) WITH ALL_ERRORMSGS, EXTENDED_LOGICAL_CHECKS;
I'm stumped... any ideas?
Here are the UDF definitions
ALTER FUNCTION dbo.Cases( #day date ) RETURNS TABLE
WITH SCHEMABINDING
AS RETURN (
SELECT
CaseID -- other 42 columns ommitted
FROM (
SELECT
ROW_NUMBER() OVER (PARTITION BY CaseID ORDER BY UpdateDate DESC, UpdateNumber DESC) AS RecordAge,
CaseID,
Action
FROM
dbo.CaseAudit
WHERE
convert(date,UpdateDate) <= #day
) AS History
WHERE
RecordAge = 1 -- only the most current record version
AND isnull(Action,'') != N'DEL' -- only include cases that have not been deleted
)
ALTER FUNCTION dbo.AwareDates( #day date ) RETURNS TABLE
WITH SCHEMABINDING
AS RETURN (
WITH
History AS (
SELECT row_number() OVER (PARTITION BY CaseID, ContactID ORDER BY UpdateDate DESC, UpdateNumber DESC) AS RecordAge,
CaseID, InfoReceived, ReceiveDate, ResetClock, Action
FROM dbo.ContactLogAudit WITH (NOLOCK)
WHERE convert(date,UpdateDate) <= #day
),
Notes AS (
SELECT
CaseID,
convert(date,ReceiveDate,112) AS ReceiveDate,
ResetClock
FROM History
WHERE RecordAge = 1 -- only the most current record version
AND isnull(Action,'') != N'DEL' -- only include notes that have not been deleted
AND InfoReceived = N'Y' -- only include notes that have Info Rec'd checked
AND len(ReceiveDate) = 8 AND isnumeric(ReceiveDate) = 1 AND isdate(ReceiveDate) = 1 -- only include those with a valid aware date
),
Initials AS (
SELECT CaseID, min(ReceiveDate) AS ReceiveDate
FROM Notes
GROUP BY CaseID
),
Resets AS (
SELECT CaseID, max(ReceiveDate) AS ReceiveDate
FROM Notes
WHERE ResetClock = N'Y'
GROUP BY CaseID
)
SELECT
i.CaseID AS CaseID,
i.ReceiveDate AS InitialAwareDate, -- the oldest valid aware date value (must have AE Info Reveived checked and a received date)
coalesce(r.ReceiveDate,i.ReceiveDate) AS AwareDate -- either the newest valid aware date value with the Reset Clock checked, otherwise the initial aware date value
FROM Initials AS i
LEFT JOIN Resets AS r
ON i.CaseID = r.CaseID
);
I have further found that if I drop the "WITH (NOLOCK)" table hint, I get correct results. Also if add a join hint to the AwareDates UTF or even add a COLLATE Latin1_General_BIN on the LEFT JOIN relation between Initials and Resets.
Query plan row counts -- without join hint (broken)
Cases { Actual: 25,891, Estimate: 19,071.9 }
AwareDates { Actual: 24,693, Estimated: 1,463.09 }
Initials { Actual: 24,693, Estimated: 1,463.09 }
Rests { Actual: 985, Estimated: 33.2671 }
AwareDates matches 8,108 of the Cases rows in the join'd result-set
Query plan row counts -- with join hint (working)
Cases { Actual: 25,891, Estimate: 19,071.9 }
AwareDates { Actual: 24,673, Estimated: 1,837.67 }
Initials { Actual: 24,673, Estimated: 1,837.67 }
Rests { Actual: 982, Estimated: 42.6238 }
AwareDates matches 24,673 of the Cases rows in the join'd result-set
I have further whittled down the scope of the issue. I can;
SELECT * FROM AwareDate(#date);
and
SELECT * FROM AwareDate(#date) ORDER BY CaseID;
With different row counts.
You don't specify the specific version of SQL (##version), but this seems suspiciously like a bug that was fixed in Cumulative Update 6 for SQL 2008 R2 (apparently it also applies to SQL 2008).
KB 2433265
FIX: You may receive an incorrect result when you run a query that uses the
ROW_NUMBER function together with a left outer join in SQL Server 2008
The example in the article specifies DISTINCT. The article, however, is worded ambiguously -- it's not clear whether you NEED a distinct or if DISTINCT is one of the triggers.
Your example doesn't have a distinct like the article, but it appears modified for the sake of asking the question(i.e. 42 columns missing). Is there a distinct? Also in the AwareDates udf by the time i get down to the Initials CTE you do a GROUP BY which could have the same effect as a DISTINCT.
UPDATE
#Dennis from your comment I still can't tell if you're using SQL 20080 or 2008 R2.
If you're running 2008, the KB article says "The fix for this issue was first released in Cumulative Update 11 for SQL Server 2008 Service Pack 1." So, post SP1.
On the other hand, if you're using SQL 2008 R2, you are correct that this was fixed in CU 6, which was part of SP1. But this bug appears to have resurfaced. Look at Cumulative update package 4 for SQL Server 2008 R2 Service Pack 1 -- released post SP1.
970198 FIX: You receive an incorrect result when you run a
query that uses the row_number function in SQL Server 2008
or in SQL Server 2008 R2
In the associated KB article MS dropped the reference to distinct:
Consider the following scenario. You run a query against a table that has a
clustered index in Microsoft SQL Server 2008 or in Microsoft SQL Server 2008
R2. In the query, you use the row_number function. In this scenario, you
receive an incorrect result when a parallel execution plan is used for the
query. If you run the query many times, you may receive different results.
This seems to confirm my earlier reading of KB 2433265 -- the phrasing suggests distinct is just one of many conditions that can cause the behavior. It seems that a parallel execution plan is the culprit this time around.