Determine dimGrid in CUDA - cuda

Question

I'm tiling a 2D Matrix into blocks of fixed size BLOCK_DIM 16*16. Then I found that dimGrid is (from internet):
dim3 dimGrid((NColumns - 1)/16 + 1, (NRows - 1)/16 + 1).
Isn't this reversed? Shouldn't it be Nrows first?

Answer 1

If I were writing the code, I would probably write it the way you have shown.
I think of x,y cartesian space this way:
Y
^
|
|
+------->X
That is, the "X" axis is the "horizontal" axis and the "Y" axis is the vertical axis. There is no reason it has to be this way -- it's just a mental model. But I think it's fairly common.
Now, If the x,y space is used to represent a 2D image, then as I move from right to left (i.e. along the horizontal axis) I am moving from one column to another in the image. As I move up and down (i.e. along the vertical axis) I am moving from one row to another in the image.
Therefore, with this mental model, the Y coordinate indicates the row and the X coordinate indicates the column of the image. The X coordinate will therefore have a maximum (logical) value equal to the number of columns in the image, and the Y coordinate will have a maximum value equal to the number of rows in the image. For the proposed dimGrid variable definition:
dim3 dimGrid((NColumns - 1)/16 + 1, (NRows - 1)/16 + 1).
since the x grid dimension appears first, we see that this "mental model" is consistent with the definition of dimGrid.
This sort of usage also would typically mean that for an image-processing algorithm in CUDA, adjacent threads in X would have "naturally"-calculated 2D indices:
int idx = threadIdx.x + blockDim.x*blockIdx.x;
int idy = threadIdx.y + blockDim.y*blockIdx.y;
or:
int col = threadIdx.x + blockDim.x*blockIdx.x;
int row = threadIdx.y + blockDim.y*blockIdx.y;
such that they would have adjacent X values in the image or adjacent "columns". In C-style row-major storage, having adjacent threads in X (in the grid) access adjacent columns in the image is usually a good recipe for achieving coalesced access in your kernel.