cuda matrix multiplication by columns - cuda

Question

I'm trying to do matrix multiplication in cuda. My implementation is different from the cuda example.
The cuda example (from the cuda samples) performs matrix multiplication by multiplying each value in the row of the first matrix by each value in the column of the second matrix, then summing the products and storing it in an output vector at the index of the row from the first matrix.
My implementation multiplies each value in the column of the first matrix by the single value of the row of the second matrix, where the row index = column index. It then has an output vector in global memory that has each of its indices updated.
The cuda example implementation can have a single thread update each index in the output vector, whereas my implementation can have multiple threads updating each index.
The results that I get show only some of the values. For example, if I had it do 4 iterations of updates, it would only do 2 or 1.
I think that the threads might be interfering with each other since they're all trying to write to the same indices of the vector in global memory. So maybe, while one thread is writing to an index, the other might not be able to insert its value and update the index?
Just wondering if this assessment makes sense.
For example. To multiply the following two matrices:
[3 0 0 2 [1 [a
3 0 0 2 x 2 = b
3 0 0 0 3 c
0 1 1 0] 4] d]
The Cuda sample does matrix multiplication in the following way using 4 threads where a,b,c,d are stored in global memory:
Thread 0: 3*1 + 0*2 + 0*3 + 2*4 = a
Thread 1: 3*1 + 0*2 + 0*3 + 2*4 = b
Thread 2: 3*1 + 0*2 + 0*3 + 0*4 = c
Thread 3: 0*1 + 1*2 + 1*3 + 0*4 = d
My implementation looks like this:
a = b = c = d = 0
Thread 0:
3*1 += a
3*1 += b
3*1 += c
0*1 += d
Thread 1:
0*2 += a
0*2 += b
0*2 += c
1*2 += d
Thread 2:
0*3 += a
0*3 += b
0*3 += c
1*3 += d
Thread 3:
2*4 += a
2*4 += b
0*4 += c
0*4 += d
So at one time all four threads could be trying to update one of the indices.

Answer 1

In order to fix this issue, I used atomicAdd to do the += operation. When a thread performs the operation 3*1 += a (for example), it does three things.
It gets the previous value of a
It updates the value by doing 3*1 + previous value of a
It then stores the new value into a
By using atomicAdd it guarantees that these operations can occur by the thread without interruption from other threads. If atomicAdd is not used, thread0 could get the previous value of a and while thread0 is updating the value, thread1 could get the previous value of a and perform its own update. In this way a += operation would not occur because the threads aren't able to finish their operations.
If a += 3*1 is used instead of atomicAdd(&a, 3*1), then it is possible for thread1 to interfere and change the value of thread0 before thread0 finishes what it's doing. It creates a race condition.
atomicAdd is a += operation. You would use the following code to perform the operation:
__global__ void kernel(){
int a = 0;
atomicAdd(&a, 3*1); //is the same as a += 3*1
}