Inconsistent behavior for negative zeroes in Debug vs Release builds - cuda

Question

I was under impression that adding a positive zero to negative zero should produce a positive zero. To quote IEEE 754 2008:
When the sum of two operands with opposite signs (or the difference of two operands with like signs) is exactly zero, the sign of that sum (or difference) shall be +0 in all rounding-direction attributes except roundTowardNegative; under that attribute, the sign of an exact zero sum (or difference) shall be −0. However, x + x = x − (−x) retains the same sign as x even when x is zero.
However, in case of CUDA, it looks like compiler is being too aggressive in optimizing away addition of a positive zero in Release builds. Plain C/C++ (or C#/.NET) are working as expected. I’ve looked at PTX code produced by the compiler for different builds, and add.f32 instruction is indeed missing in Release build.
Am I missing anything here?
__global__ void convertToPositiveZero(float* dst, int size)
{
int index = blockIdx.x * blockDim.x + threadIdx.x;
if (index < size)
{
dst[index] += 0;
}
}
// Host code
int size = 100;
float* zzh = (float*)malloc(size * sizeof(float));
zzh[0] = -0.0f;
zzh[1] = 0.0f;
assert(0x80000000 == *((int*)&zzh[0]));
if (0x80000000 != *((int*)&zzh[0]))
{
printf("Expected negative zero.\n");
exit(-1);
}
assert(0x00000000 == *((int*)&zzh[1]));
float* zzd;
cudaMalloc(&zzd, size * sizeof(float));
cudaMemcpy(zzd, zzh, size * sizeof(float), cudaMemcpyHostToDevice);
convertToPositiveZero<<<1, 100>>>(zzd, size);
cudaMemcpy(zzh, zzd, size * sizeof(float), cudaMemcpyDeviceToHost);
//zzh[0] += 0.0f;
assert(0x00000000 == *((int*)&zzh[0]));
if (0x00000000 != *((int*)&zzh[0]))
{
printf("Expected positive zero.\n");
exit(-1);
}
assert(0x00000000 == *((int*)&zzh[1]));
printf("Done.\n");

Answer 1

Your problem seems to be due to the optimizations carried out by nvcc when fusing FADD and FMUL into FMAD operations.
I was able to reproduce your problem under a Release modality. The resulting disassembled code, compiled by CUDA 5.5 and for a sm=2.1, is
code for sm_21
Function : _Z21convertToPositiveZeroPfi
.headerflags #"EF_CUDA_SM20 EF_CUDA_PTX_SM(EF_CUDA_SM20)
/*0000*/ MOV R1, c[0x1][0x100];
/*0008*/ S2R R0, SR_CTAID.X;
/*0010*/ S2R R2, SR_TID.X;
/*0018*/ IMAD R0, R0, c[0x0][0x8], R2;
/*0020*/ ISETP.GE.AND P0, PT, R0, c[0x0][0x28], PT;
/*0028*/ #P0 BRA.U 0x60;
/*0030*/ #!P0 MOV32I R3, 0x4;
/*0038*/ #!P0 IMAD R2.CC, R0, R3, c[0x0][0x20];
/*0040*/ #!P0 IMAD.HI.X R3, R0, R3, c[0x0][0x24];
/*0048*/ #!P0 LD.E R0, [R2];
/*0050*/ #!P0 F2F.F32.F32 R0, R0;
/*0058*/ #!P0 ST.E [R2], R0;
/*0060*/ EXIT ;
As you also noticed from the PTX file, there is no floating point add operations. Now, if you compile with -fmad=false option, the disassembled code becomes
code for sm_21
Function : _Z21convertToPositiveZeroPfi
.headerflags #"EF_CUDA_SM20 EF_CUDA_PTX_SM(EF_CUDA_SM20)
/*0000*/ MOV R1, c[0x1][0x100];
/*0008*/ S2R R0, SR_CTAID.X;
/*0010*/ S2R R2, SR_TID.X;
/*0018*/ IMAD R0, R0, c[0x0][0x8], R2;
/*0020*/ ISETP.GE.AND P0, PT, R0, c[0x0][0x28], PT;
/*0028*/ #P0 BRA.U 0x60;
/*0030*/ #!P0 MOV32I R3, 0x4;
/*0038*/ #!P0 IMAD R2.CC, R0, R3, c[0x0][0x20];
/*0040*/ #!P0 IMAD.HI.X R3, R0, R3, c[0x0][0x24];
/*0048*/ #!P0 LD.E R0, [R2];
/*0050*/ #!P0 FADD R0, R0, RZ;
/*0058*/ #!P0 ST.E [R2], R0;
/*0060*/ EXIT ;
As you can see, the presence of a FADD operation is restored and the "correct" sign of 0 is restored as well.