Does PTX actually have a 64-bit warp shuffle instruction? - cuda

Question

I noticed that in the docs for __shfl_sync() and it's relatives that 64-bit datatypes (long, double) are supported.
Does this mean that hardware/PTX natively supports 64-bit warp shuffles, or are these broken down into a pair of 32-bit shuffles when the code is compiled?

Answer 1

Currently, there is no 64-bit shuffle instruction in PTX. The basic register unit in all current CUDA GPUs are 32-bits. 64-bit quantities have no corresponding 64-bit registers, but instead occupy a pair of 32-bit registers. The warp shuffle operation at the machine level operates on 32-bit registers.
The compiler processes 64-bit operands to the shfl intrinsics for CUDA C++ by emitting 2 PTX (or SASS) instructions. This is readily discoverable/confirmable using the CUDA binary utilities.
Example:
$ cat t45.cu
typedef double mt;
__global__ void k(mt *d){
mt x = d[threadIdx.x];
x = __shfl_sync(0xFFFFFFFF, x, threadIdx.x+1);
d[threadIdx.x] = x;
}
$ nvcc -c t45.cu
$ cuobjdump -ptx t45.o
Fatbin elf code:
================
arch = sm_30
code version = [1,7]
producer = cuda
host = linux
compile_size = 64bit
Fatbin ptx code:
================
arch = sm_30
code version = [6,2]
producer = cuda
host = linux
compile_size = 64bit
compressed
.version 6.2
.target sm_30
.address_size 64
.visible .entry _Z1kPd(
.param .u64 _Z1kPd_param_0
)
{
.reg .pred %p<3>;
.reg .b32 %r<9>;
.reg .f64 %fd<3>;
.reg .b64 %rd<5>;
ld.param.u64 %rd1, [_Z1kPd_param_0];
cvta.to.global.u64 %rd2, %rd1;
mov.u32 %r5, %tid.x;
mul.wide.u32 %rd3, %r5, 8;
add.s64 %rd4, %rd2, %rd3;
ld.global.f64 %fd1, [%rd4];
add.s32 %r6, %r5, 1;
mov.b64 {%r1,%r2}, %fd1;
mov.u32 %r7, 31;
mov.u32 %r8, -1;
shfl.sync.idx.b32 %r4|%p1, %r2, %r6, %r7, %r8;
shfl.sync.idx.b32 %r3|%p2, %r1, %r6, %r7, %r8;
mov.b64 %fd2, {%r3,%r4};
st.global.f64 [%rd4], %fd2;
ret;
}
$