Is these way coalesced access? - cuda

Question

note "When a warp executes an instruction that accesses global memory, it coalesces the memory accesses of the threads within the warp into one or more of these memory transactions".
but I have some questions.
__global__ void add(double *a. double *b){
int i = blockDim.x * blockIdx.x + threadIdx.x;
i = 3 * i;
b[i] = a[i] + a[i + 1] + a[i + 2];
}
can the three accesses(a[i] , a[i + 1] , a[i + 2]) executed with only an instruction? (I mean that is it coalesced access?)
or does the coalesced only exist in the different thread（transverse） of a warp?(no exist in a thread?)
I have read the similar questionss：
From non coalesced access to coalesced memory access CUDA
But I still don't understand,so is it non-coalesced memory access?
2.
__global__ void add(double *a. double *b){
int i = blockDim.x * blockIdx.x + threadIdx.x;
b[i] = a[i] + a[i + 10] + a[i + 12];//assuming no out of indeax
}
It may can be the non-coalesced access.
so I change the code to:
__global__ void add(double *a. double *b){
int i = blockDim.x * blockIdx.x + threadIdx.x;
__shared__ double shareM[3*BLOCK_SIZE];
shareM[threadIdx.x] = a[i];
shareM[threadIdx.x + 1] = a[i + 10];
shareM[threadIdx.x + 2] = a[i + 12];
b[i] = shareM[threadIdx.x] + shareM[threadIdx.x + 1] + shareM[threadIdx.x + 2];
}
I see that coalescent access do not matter with shared memory.
but it mean that is the way below coalesced access under one thread?
shareM[threadIdx.x] = a[i];
shareM[threadIdx.x + 1] = a[i + 10];
shareM[threadIdx.x + 2] = a[i + 12];
or does the shared memory coalesced access only exist in diferent thread like the fllowing example?:
thread0:
shareM[0] = a[3]
thread1:
shareM[4] = a[23]
thread2:
shareM[7] = a[56]
3.I that don't understand "coalescent access do not matter with shared memory".
is it mean that load the data to local(or register) memory from global memory slower than load the data to shared memory from global memory ?
if it is, why we don't use the shared memory as transfer station(just only one 8bytes shared memory for one thread is enough)?
thank you.

Answer 1

can the three accesses(a[i] , a[i + 1] , a[i + 2]) executed with only an instruction? (I mean that is it coalesced access?)
When working with GPU kernels, I guess it's better to think everything in a parallel way. Every instruction is executed in a group of 32 threads, a.k.a a warp, so they are actually not just three accesses(here the word "access" is also vague, I assume you mean array accessing), they are 32 x 3 = 96 accesses in total. A more correct way to say this is that they are three array accesses per thread.
According to [1-3], the coalesced accessing pattern is a behavior in terms of a warp:
When a warp executes an instruction that accesses global memory, it coalesces the memory accesses of the threads within the warp into one or more of these memory transactions depending on the size of the word accessed by each thread and the distribution of the memory addresses across the threads.
So, we need to think respectively for these three array accesses. Let's rewrite the code as:
__global__ void add(double *a. double *b){
int i = blockDim.x * blockIdx.x + threadIdx.x;
i = 3 * i;
double ai = a[i]; // <1>
double ai1 = a[i + 1]; // <2>
double ai2 = a[i + 2]; // <3>
b[i] = ai + ai1 + ai2;
}
And it is succient to only consider the first warp with threadid range from 0 to 31.
<1>: Each thread in a warp allocates a double variable called ai in its register and wants to access a value from a based on the index i. Note the original i \in [0,31] and then it's multiped by 3, so the warp is accessing a[0], a[3], ... , a[93]. Since a is a double array(i.e. every entry is of size 8 byte), it needs to access 32 * 8 = 256 byte in total, that's two 128-byte segments that can be dealt with two 128-byte memory transactions. According to [4]:
If the size of the words accessed by each thread is more than 4 bytes, a memory request by a warp is first split into separate 128-byte memory requests that are issued independently: Two memory requests, one for each half-warp, if the size is 8 bytes, Four memory requests, one for each quarter-warp, if the size is 16 bytes.
to load these 256-byte data from global memory to register, the minimum memory request number is 2. If a can be accessed in this way, then this accessing pattern is coalescing. But apparently the pattern used in <1> is not, it's like the graph below:
<1>
t0 + t31
+---+---+---+-------------+----------------------+
| | | | ...... |
v v v v v
+---+-------+----+--------+-------+--------+-----+--+-
|segment| | | | | |
+----------------+--------+-------+--------+--------+-
a[0] a[31] a[63] a[95]
32 threads in the warp are accessing memory separately in six 128-byte segments. In the cached mode, it needs six 128-byte memory transactions at least. That's 768 bytes in total, but only 256 bytes are useful. The bus utilization is about 1/3.
<2>: This is very similar to <1>, with 1 offset from the start:
<2>
t0 + t31
+---+---+---+-------------+----------------------+
| | | | ...... |
v v v v v
++---+---+---+---+--------+-------+--------+------+-+-
|segment| | | | | |
+----------------+--------+-------+--------+--------+-
a[0] a[31] a[63] a[95]
<3>: This is very similar to <1>, with 2 offset from the start:
<3>
t0 + t31
+---+---+---+-------------+----------------------+
| | | | ...... |
v v v v v
+-+---+---+---+--+--------+-------+--------+-------++-
|segment| | | | | |
+----------------+--------+-------+--------+--------+-
a[0] a[31] a[63] a[95]
I think now you already get the idea and probably think: How about loading these 768 bytes from global memory in one pass because all of them are used once, exactly. However, recall that each thread has its private registers and these registers cannot communicate with each other([5]), so this cannot be done merely with registers and that's where shared memory comes in.
(warp1) (warp2) (warp3)
+ + +
| | |
t0 | t31 | t0 | t31
+-+-+-+---+-+-+-+---------+---------+-+-+-+++-+-+-+-+
| | | | | | | | | ...... | | | | | | | | |
v v v v v v v v v v v v v v v v v v
+-+-+-+---+-+-+-++--------+-------+-+-+-+-+++-+-+-+---
|segment| | | | | |
+----------------+--------+-------+--------+--------+-
a[0] a[31] a[63] a[95]
is it mean that load the data to local(or register) memory from global memory slower than load the data to shared memory from global memory ? if it is, why we don't use the shared memory as transfer station(just only one 8bytes shared memory for one thread is enough)?
AFAICT, you cannot directly transfer data from global memory to shared memory.
References:
[1]. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#maximize-memory-throughput
[2]. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#device-memory-accesses
[3]. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#global-memory-3-0__examples-of-global-memory-accesses
[4]. https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#global-memory-3-0
[5]. I lied, there is a way to do this by using __shlf intrinsics.