How does warp divergence manifest in SASS? - cuda

Question

When different threads in a warp execute divergent code, divergent branches are serialized, and inactive warps are "disabled."
If the divergent paths contain a small number of instructions, such that branch predication is used, it's pretty clear what "disabled" means (threads are turned on/off by the predicate), and it's also clearly visible in the sass dump.
If the divergent execution paths contain larger numbers of instructions (exact number dependent on some compiler heuristics) branch instructions are inserted to potentially skip one execution path or the other. This makes sense: if one long branch is seldom taken, or not taken by any threads in a certain warp, it's advantageous to allow the warp to skip those instructions (rather than being forced to execute both paths in all cases as for predication).
My question is: How are inactive threads "disabled" in the case of divergence with branches? The slide on page 2, lower left of this presentation seems to indicate that branches are taken based on a condition and threads that do not participate are switched off via predicates attached to the instructions at the branch targets. However, this is not the behavior I observe in SASS.
Here's a minimal compilable sample:
#include <stdio.h>
__global__ void nonpredicated( int* a, int iter )
{
if( a[threadIdx.x] == 0 )
// Make the number of divergent instructions unknown at
// compile time so the compiler is forced to create branches
for( int i = 0; i < iter; i++ )
{
a[threadIdx.x] += 5;
a[threadIdx.x] *= 5;
}
else
for( int i = 0; i < iter; i++ )
{
a[threadIdx.x] += 2;
a[threadIdx.x] *= 2;
}
}
int main(){}
Here's the SASS dump showing that the branch instructions are predicated, but the code at the branch targets is not predicated. Are the threads that did not take the branch switched off implicitly during execution of those branch targets, in some way that is not directly visible in the SASS? I often see terminology like "active mask" alluded to in various Cuda documents, but I'm wondering how this manifests in SASS, if it is a separate mechanism from predication.
Additionally, for pre-Volta architectures, the program counter is shared per-warp, so the idea of a predicated branch instruction is confusing to me. Why would you attach a per-thread predicate to an instruction that might change something (the program counter) that is shared by all threads in the warp?
code for sm_20
Function : _Z13nonpredicatedPii
.headerflags #"EF_CUDA_SM20 EF_CUDA_PTX_SM(EF_CUDA_SM20)"
/*0000*/ MOV R1, c[0x1][0x100]; /* 0x2800440400005de4 */
/*0008*/ S2R R0, SR_TID.X; /* 0x2c00000084001c04 */
/*0010*/ MOV32I R3, 0x4; /* 0x180000001000dde2 */
/*0018*/ IMAD.U32.U32 R2.CC, R0, R3, c[0x0][0x20]; /* 0x2007800080009c03 */
/*0020*/ IMAD.U32.U32.HI.X R3, R0, R3, c[0x0][0x24]; /* 0x208680009000dc43 */
/*0028*/ LD.E R0, [R2]; /* 0x8400000000201c85 */
/*0030*/ ISETP.EQ.AND P0, PT, R0, RZ, PT; /* 0x190e0000fc01dc23 */
/*0038*/ #P0 BRA 0xd0; /* 0x40000002400001e7 */
/*0040*/ MOV R4, c[0x0][0x28]; /* 0x28004000a0011de4 */
/*0048*/ ISETP.LT.AND P0, PT, R4, 0x1, PT; /* 0x188ec0000441dc23 */
/*0050*/ MOV R4, RZ; /* 0x28000000fc011de4 */
/*0058*/ #P0 EXIT; /* 0x80000000000001e7 */
/*0060*/ NOP; /* 0x4000000000001de4 */
/*0068*/ NOP; /* 0x4000000000001de4 */
/*0070*/ NOP; /* 0x4000000000001de4 */
/*0078*/ NOP; /* 0x4000000000001de4 */
/*0080*/ IADD R4, R4, 0x1; /* 0x4800c00004411c03 */
/*0088*/ IADD R0, R0, 0x2; /* 0x4800c00008001c03 */
/*0090*/ ISETP.LT.AND P0, PT, R4, c[0x0][0x28], PT; /* 0x188e4000a041dc23 */
/*0098*/ SHL R0, R0, 0x1; /* 0x6000c00004001c03 */
/*00a0*/ #P0 BRA 0x80; /* 0x4003ffff600001e7 */
/*00a8*/ ST.E [R2], R0; /* 0x9400000000201c85 */
/*00b0*/ BRA 0x128; /* 0x40000001c0001de7 */
/*00b8*/ NOP; /* 0x4000000000001de4 */
/*00c0*/ NOP; /* 0x4000000000001de4 */
/*00c8*/ NOP; /* 0x4000000000001de4 */
/*00d0*/ MOV R0, c[0x0][0x28]; /* 0x28004000a0001de4 */
/*00d8*/ MOV R4, RZ; /* 0x28000000fc011de4 */
/*00e0*/ ISETP.LT.AND P0, PT, R0, 0x1, PT; /* 0x188ec0000401dc23 */
/*00e8*/ MOV R0, RZ; /* 0x28000000fc001de4 */
/*00f0*/ #P0 EXIT; /* 0x80000000000001e7 */
/*00f8*/ MOV32I R5, 0x19; /* 0x1800000064015de2 */
/*0100*/ IADD R0, R0, 0x1; /* 0x4800c00004001c03 */
/*0108*/ IMAD R4, R4, 0x5, R5; /* 0x200ac00014411ca3 */
/*0110*/ ISETP.LT.AND P0, PT, R0, c[0x0][0x28], PT; /* 0x188e4000a001dc23 */
/*0118*/ #P0 BRA 0x100; /* 0x4003ffff800001e7 */
/*0120*/ ST.E [R2], R4; /* 0x9400000000211c85 */
/*0128*/ EXIT; /* 0x8000000000001de7 */
.....................................

Answer 1

Are the threads that did not take the branch switched off implicitly during execution of those branch targets, in some way that is not directly visible in the SASS?
Yes.
There is a warp execution or "active" mask which is separate from the formal concept of predication as defined in the PTX ISA manual.
Predicated execution may allow instructions to be executed (or not) for a particular thread on an instruction-by-instruction basis. The compiler may also emit predicated instructions to enact a conditional jump or branch.
However the GPU also maintains a warp active mask. When the machine observes that thread execution within a warp has diverged (for example at the point of a predicated branch, or perhaps any predicated instruction), it will set the active mask accordingly. This process isn't really "visible" at the SASS level. AFAIK the low level execution process for a diverged warp (not via predication) isn't well specified, so questions around how long the warp stays diverged and the exact mechanism for re-synchronization aren't well specified, and AFAIK can be affected by compiler choices, on some architectures. This is one recent discussion (note particularly the remarks by #njuffa).
Why would you attach a per-thread predicate to an instruction that might change something (the program counter) that is shared by all threads in the warp?
This is how you perform a conditional jump or branch. Since all execution is lock-step, if we are going to execute a particular instruction (regardless of mask status or predication status) the PC had better point to that instruction. However, the GPU can perform instruction replay to handle different cases, as needed at execution time.
A few other notes:
a mention of the "active mask" is here:
The scheduler dispatches all 32 lanes of the warp to the execution units with an active mask. Non-active threads execute through the pipe.
some NVIDIA tools allow for inspection of the active mask.