In CUDA why can't I allocate 2d shared memory dynamically? - cuda

Question

The following works fine;
__extern__ float dyanimicSh1D[];
But the following does not work:
__extern__ float dyanimicSh2D[][];
I want to understand why it is so?

Answer 1

You can't do it because the compiler needs the width information for the array to generate code that does proper indexing.
If you allocate shared memory in a static fashion like this:
__shared__ float sarr[24][12];
Then not only are you telling how much memory to allocate/provide, you are also giving the width of the array (12 in this example). This is important, because a static 2D array of this type is not treated under the hood as an array of pointers, but instead it is a flat allocation, with indexing created by the compiler, at compile-time.
so that later when you do something like this:
float val = sarr[y][x];
the compiler will take the sarr pointer, and do pointer arithmetic to add x + (y*12) to it, before dereferencing that pointer to retrieve the value. The 12 in that calculation is discovered at compile-time and used by the compiler in generating the code to do the indexing.
Doing something like this:
extern __shared__ float sarr[][];
doesn't supply the array width information to the compiler, so it cannot generate the indexing needed at compile time, and is not allowed.
By the way, this works:
extern __shared__ float sarr[][12];
Here is an example:
$ cat t46.cu
#include <cstdio>
__global__ void k(int x, int y){
extern __shared__ float sarr[][12];
for (int i = 0; i < 32; i ++)
for (int j = 0; j < 12; j++)
sarr[i][j] = i * 256 + j;
float val = sarr[y][x];
printf("%f\n", val);
}
int main(){
k<<<1,1,128*12>>>(3,2);
cudaDeviceSynchronize();
}
$ nvcc -o t46 t46.cu
$ cuda-memcheck ./t46
========= CUDA-MEMCHECK
515.000000
========= ERROR SUMMARY: 0 errors
$