Speed of finding non-zero elements in (non-)diagonal matrix - octave

Question

What's happening in this example?
Looking at the full matrix, the find operation is faster on the non-diagonal matrices.
Contrarily, getting the sparse representation is faster on the diagonal matrix (which seems reasonable).
The find operation on the sparse matrices is almost equal.
For curiosity, can someone tell me whats happening under the hood here? Why is finding non-zero elements on a full matrix faster than finding them on a diagonal matrix?
printf("Diagonal Mat:\n\n")
A = eye(10000);
printf("Full mat: ")
tic
find(A);
toc
printf("Building sparse representation: ")
tic
As = sparse(A);
toc
printf("Sparse mat: ")
tic
find(As);
toc
printf("\n\nNon-Diagonally flagged Mat:\n\n")
A = A | A; # This removes the "Diagonal Matrix" flag from A
printf("Full mat: ")
tic
find(A);
toc
printf("Building sparse representation: ")
tic
As = sparse(A);
toc
printf("Sparse mat: ")
tic
find(As);
toc
printf("\n\nActually Non-Diagonal Mat:\n\n")
A(:,:) = 0;
A(:,1) = 1;
printf("Full mat: ")
tic
find(A);
toc
printf("Building sparse representation: ")
tic
As = sparse(A);
toc
printf("Sparse mat: ")
tic
find(As);
toc
Output:
Diagonal Mat:
Full mat: Elapsed time is 0.204636 seconds.
Building sparse representation: Elapsed time is 5.19753e-05 seconds.
Sparse mat: Elapsed time is 7.60555e-05 seconds.
Non-Diagonally flagged Mat:
Full mat: Elapsed time is 0.0800331 seconds.
Building sparse representation: Elapsed time is 0.0924602 seconds.
Sparse mat: Elapsed time is 7.48634e-05 seconds.
Actually Non-Diagonal Mat:
Full mat: Elapsed time is 0.0799708 seconds.
Building sparse representation: Elapsed time is 0.092248 seconds.
Sparse mat: Elapsed time is 7.70092e-05 seconds.

Answer 1

First of all, the following would be a nicer way to measure that:
for i = 1:10, find (d); endfor
t = cputime ();
for i = 1:100, find (d); endfor
cputime () -t
for i = 1:10, find (f); endfor
t = cputime ();
for i = 1:100, find (f); endfor
cputime () -t
That's a good question. Octave has an internal specialisation for diagonal matrices where only the diagonal values are stored. You can see how much less memory it uses:
octave> d = eye (10000);
octave> f = full (eye (10000));
octave> typeinfo (d)
ans = diagonal matrix
octave> typeinfo (f)
ans = matrix
octave> whos d f
Variables in the current scope:
Attr Name Size Bytes Class
==== ==== ==== ===== =====
d 10000x10000 80000 double
f 10000x10000 800000000 double
Total is 200000000 elements using 800080000 bytes
The specialisation is for reduced memory and performance for cases where diagonal matrices are common. The problem with such specialisations is that they add special cases all over the place, specially when you want to access the data directly which Octave often does.
In the case of find, it has special cases for boolean arrays, integer arrays, permutation matrices, and sparse matrices. There is no special handling for diagonal matrices so the the case for real type double precision array is used instead. This means that diagonal matrices get internally converted to full arrays when calling find anyway.
Weirdly, calling full on the diagonal matrix before calling find seems to still be more efficient, so maybe my reasoning is wrong. I have opened a performance bug report

Answer 2

It has something to do with how your computer (or in detail, your cpu) is processing and buffering values on the stack and heap. When you allocate memory on the heap for your arrays, it will do so allocating a "list" of values, one after the other. Thus, when you iterate over an array, value by value, the cpu will just jump from one value on that list to the next one (very fast) but if you jump from value i to value i + n, where n is not 1, that means the cpu has to find this next value somewhere else on that list. Therefore, the way you save values in your programming environment and the way you iterate over those values influences the speed of the following process.
This was just a short and very simple try to explain this topic. Actually, it is more complicated as more factors and different cpu and memory techniques are possible. If you are interested in this kind of things, I would suggest starting here: https://en.wikipedia.org/wiki/System_programming (To have a Top Down view at the topic).