MS Access: Left outer self join with inequality operator - ms-access

Question

I have a single table in Access 2010:
TMP [CUST,ITEM,START_PD]
I want to get the END_PD for each CUST / ITEM.
END_PD being defined as the period before the next higher START_PD for the same CUST / ITEM.
So I perform a left join of the table to itself, using an inequality operator on START_PD as below.
SELECT s.CUST, s.ITEM, s.START_PD, Min(e.START_PD-1) AS END_PD
FROM TMP AS s
LEFT JOIN TMP AS e ON s.CUST=e.CUST AND s.ITEM=e.ITEM AND e.START_PD>s.START_PD
GROUP BY s.CUST, s.ITEM, s.START_PD
ORDER BY s.CUST, s.ITEM, s.START_PD
The base table has 46,556 rows. I would expect the query result to have the same, but the query only returns 14,967 rows.
Even when I try to return all records in the left join, I get far less than the base table. See below:
SELECT s.*,e.*
FROM TMP AS s
LEFT JOIN TMP AS e ON s.ITEM = e.ITEM AND s.CUST = e.CUST AND e.START_PD>s.START_PD
The above query only returns 19,014 records... less than the base table.
This is holding up a major project, and I'd appreciate any help. So far, it's looking like an Access bug. Any workarounds?
EDIT:
I have tried testing a small sample of the data by including WHERE s.CUST='WALMART' AND s.ITEM='0001H'. This fails by excluding the last START_PD, for which there is none greater.
CUST ITEM START_PD END_PD
WALMART 0001HAC 20694 20696
WALMART 0001HAC 20697 20704
WALMART 0001HAC 20705 20706
Strangely, if I select that same sample of the data (WALMART/0001H) into a separate table and run the EXACT SAME query on that smaller table (changing only the table name), it works fine as below. That is why I tend to think this is a bug.
CUST ITEM START_PD END_PD
WALMART 0001HAC 20694 20696
WALMART 0001HAC 20697 20704
WALMART 0001HAC 20705 20706
WALMART 0001HAC 20707

Answer 1

To answer my own question, I'd like to:
Demonstrate that this is a bug
Provide a workaround
Proof that this is a bug.
Sub Test()
Dim db As DAO.Database, rst As DAO.Recordset
Set db = CurrentDb
On Error Resume Next
db.QueryDefs.Delete "TMP_EXTENDED"
db.QueryDefs.Delete "RAW_LEFT_JOIN"
db.TableDefs.Delete "TMP"
On Error GoTo 0
'Table definition.
strSql = _
"CREATE TABLE TMP ( " & vbCrLf & _
" CUST VARCHAR(10), " & vbCrLf & _
" ITEM VARCHAR(10), " & vbCrLf & _
" START_PD LONG, " & vbCrLf & _
" PRIMARY KEY (CUST,ITEM,START_PD) " & vbCrLf & _
");"
db.Execute strSql, dbFailOnError
'Populate with data.
Set rst = db.OpenRecordset("TMP")
For custNo = 1 To 25 'change to affect final row count
For itemNo = 1 To 100 'change to affect final row count
For pdNo = 1 To 3 'change to affect final row count
strCust = "CUST" & custNo
strItem = "ITEM" & itemNo
rst.AddNew
rst("CUST") = strCust
rst("ITEM") = strItem
rst("START_PD") = pdNo
rst.Update
If rst.RecordCount Mod 1000 = 0 Then Debug.Print rst.RecordCount 'just to monitor.
Next
Next
Next
Debug.Print "TMP Table Row Count is: " & DCount("*", "TMP")
'Test query to find end period for each CUST/ITEM/START_PD.
Dim qdf As New QueryDef
qdf.Name = "TMP_EXTENDED"
qdf.SQL = "SELECT s.CUST, s.ITEM, s.START_PD, Min(e.START_PD-1) AS END_PD " & vbCrLf & _
"FROM TMP AS s " & vbCrLf & _
"LEFT JOIN TMP AS e ON (s.CUST=e.CUST AND s.ITEM=e.ITEM AND e.START_PD>s.START_PD) " & vbCrLf & _
"GROUP BY s.CUST, s.ITEM, s.START_PD " & vbCrLf & _
"ORDER BY s.CUST, s.ITEM, s.START_PD"
db.QueryDefs.Append qdf
Debug.Print "TMP_EXTENDED Row Count is: " & DCount("*", "TMP_EXTENDED")
'Test query to just perform the left join.
Set qdf = New QueryDef
qdf.Name = "RAW_LEFT_JOIN"
qdf.SQL = "SELECT s.*,e.* " & vbCrLf & _
"FROM TMP AS s " & vbCrLf & _
"LEFT JOIN TMP AS e ON s.ITEM = e.ITEM AND s.CUST = e.CUST AND e.START_PD>s.START_PD"
db.QueryDefs.Append qdf
Debug.Print "RAW_LEFT_JOIN Row Count is: " & DCount("*", "RAW_LEFT_JOIN")
RefreshDatabaseWindow
End Sub
When run as written, the above code returns:
TMP Table Row Count is: 7500
TMP_EXTENDED Row Count is: 5000
RAW_LEFT_JOIN Row Count is: 7500
Obviously this is wrong because a left join should always return all the records in the left table. In this case, TMP_EXTENDED should have returned 7500, and RAW_LEFT_JOIN should have returned more than 7500.
The loop bounds for custNo, itemNo, and pdNo can be changed to adjust the record count in table TMP. If you do this, you will see that the query works until the record count reaches about 7000, then it fails.
The same problem does not appear to exist when the join is only performed on one 'entity' column. For example, I modified the above code to use a table with only a CUST column and START_PD column, and got both queries to work properly on a table record count of 600,000.
Workaround
Since posting this, I have found another quite similar post in which a decent workaround is presented. I have modified it as below. I do not know HOW reliable it is going to be, but I'm going with it for now.
SELECT s.CUST, s.ITEM, s.START_PD, Min(e.START_PD) AS END_PD
FROM TMP AS s INNER JOIN TMP AS e ON s.ITEM = e.ITEM AND s.CUST = e.CUST
WHERE e.START_PD>s.START_PD
GROUP BY s.CUST, s.ITEM, s.START_PD
UNION ALL
SELECT CUST, ITEM, Max(START_PD), Null
FROM TMP
GROUP BY CUST, ITEM, Null
ORDER BY CUST,ITEM,START_PD,END_PD

Answer 2

Any "atypical" (not only table1.field = table2.field, but anything with constants, calculations, or comparison operators other than equality) join in Access should have it's ON clause surrounded by braces:
SELECT s.*,e.*
FROM TMP AS s
LEFT JOIN TMP AS e ON (s.ITEM = e.ITEM AND s.CUST = e.CUST AND e.START_PD>s.START_PD)
However, usually, these kinds of joins return a Join expression not supported error instead of giving incorrect results. I don't know why this one doesn't.