Running Octave 6.3.0 for Windows. I need to get the smallest eigenvalue of some matrix.eigs(A,1,"sm") is supposed to do that, but I often get wrong results with singular matrices.
eigs(A) (which returns all the the first 6 eigenvalues/vectors) is correct (at least at the machine precision):
>> A = [[1 1 1];[1 1 1];[1 1 1]]
A =
1 1 1
1 1 1
1 1 1
>> [v lambda flag] = eigs(A)
v =
0.5774 -0.3094 -0.7556
0.5774 -0.4996 0.6458
0.5774 0.8091 0.1098
lambda =
Diagonal Matrix
3.0000e+00 0 0
0 -4.5198e-16 0
0 0 -1.5831e-17
flag = 0
But eigs(A,1,"sm") is not:
>> [v lambda flag] = eigs(A,1,"sm")
warning: eigs: 'A - sigma*B' is singular, indicating sigma is exactly an eigenvalue so convergence is not guaranteed
warning: called from
eigs at line 298 column 20
warning: matrix singular to machine precision
warning: called from
eigs at line 298 column 20
warning: matrix singular to machine precision
warning: called from
eigs at line 298 column 20
warning: matrix singular to machine precision
warning: called from
eigs at line 298 column 20
warning: matrix singular to machine precision
warning: called from
eigs at line 298 column 20
v =
-0.7554
0.2745
0.5950
lambda = 0.4322
flag = 0
Not only the returned eigenvalue is wrong, but the returned flag is zero, indicating that every went right in the function...
Is it a wrong usage of eigs() (but from the doc I can't see what is wrong) or a bug?
EDIT: if not a bug, at least a design issue... No problem either when requesting the 2 smallest values instead of the smallest value alone.
>> eigs(A,2,"sm")
ans =
-1.7700e-17
-5.8485e-16
EDIT 2: the eigs() function in Matlab online just runs fine and return the correct results (at the machine precision)
>> A=ones(3)
A =
1 1 1
1 1 1
1 1 1
>> [v lambda flag] = eigs(A,1,"smallestabs")
v =
-0.7556
0.6458
0.1098
lambda =
-1.5831e-17
flag =
0
After more tests and investigations I think I can answer that yes, Octave eigs() has some flaw.
eigs(A,1,"sm") likely uses the inverse power iteration method, that is repeatedly solving y=A\x, then x=y, starting with an arbitrary x vector. Obviously there's a problem if A is singular. However:
Matlab eigs() runs fine in such case, and returns the correct eigenvalue (at the machine precision). I don't know what it does, maybe adding a tiny value on the diagonal if the matrix is detected as singular, but it does something better (or at least different) than Octave.
If for some (good or bad) reason Octave's algorithm cannot handle a singular matrix, then this should be reflected in the 3rd return argument ("flag"). Instead, it is always zero as if everything went OK.
eigs(A,1,"sm") is actually equivalent to eigs(A,1,0), and the more general syntax is eigs(A,1,sigma), which means "find the closest eigenvalue to sigma, and the associated eigenvector". For this, the inverse power iteration method is applied with the matrix A-sigma*I. Problem: if sigma is already an exact eigenvalue this matrix is singular by definition. Octave eigs() fails in this case, while Matlab eigs() succeeds. It's kind of weird to have a failure when one knows in advance the exact eigenvalue, or sets it by chance. So the right thing to do in Octave is to test if (A-sigma.I) is singular, and if yes add a tiny value to sigma: eigs(A,1,sigma+eps*norm(A)). Matlab eigs() probably does something like that.
Related
I'm trying to solve the following ODE:
where R(T) is defined as:
This is my not so great attempt at using Octave:
1;
function xdot = f(t, T)
xdot = 987 * ( 0.0000696 * ( 1 + 0.0038 * ( T(t) - 25 ))) - ( 0.0168 * (T(t)-25 )) - (( 3.25 * 10 ^ (-13))) * ((T(t))^4 - (25^4));
endfunction
[x, istate, msg] = lsode( "f", 100, (t=linspace(0,3600,1000)'));
T_ref and T_infinity_sign are the same constant.
Why isn't my code correct?
If you type
debug_on_error(1)
on your octave session, and then run your code, you will see that the "f" part is called as expected, but then it fails inside lsode with the following error:
error: T(100): out of bound 1 (dimensions are 1x1)
error: called from
f at line 4 column 8
If you look at the documentation of lsode, it says it expects a function f whose first argument is a state vector x, and the second is a scalar, corresponding to time t at which that state vector occurs; f is expected to output the differential dx/dt at time t for state vector x.
From your code it seems that you are reversing the order of the arguments (and their meanings).
Therefore, when you passed T as a second argument to your function, lsode treats it like a scalar, so when you then try to evaluate T(t), it fails with an 'out_of_bounds_ error.
My advice would be, read the documentation of lsode and have a look at its examples carefully, and start playing with it to understand how it works.
PS. The debug_on_error directive launches the debugger if an error occurs during code execution. To exit the debugger type dbquit (or if you're using the GUI, click on the 'Quit Debugging Mode' button at the top right of the octave editor). If you don't know how to use the octave debugger, I recommend you spend some time to learn it, it is a very useful tool.
I'm having issues with the index for my code. I'm trying to create a code on Octave for the power method (vector iteration) and the error: 'x(4): out of bound 3' keeps popping up at line 6.
A=[6,-2,2,4;0,-4,2,2;0,0,2,-5;0,0,0,-3]
b=[12;10;-9;-3]
n=4
for i=rows(A):-1:1
for j=i+1:rows(A)
x(i)=[b(i)-A(i,j)*x(j)]/A(i,i); #error: 'x(4): out of bound 3'
endfor
endfor
x
In the following line, note that you have x appearing twice; the first seeks to assign to it, but the second simply tries to access its value:
x(i) = [ b(i) - A(i,j) * x(j) ] / A(i,i);
⬑ assignment ⬑ access
Assigning to an index that doesn't exist (yet) is absolutely fine; octave will simply fill in the intervening values with 'zeroes'. E.g.
>> clear x
>> x(3) = 1 % output: x = [0, 0, 1]
However, trying to access an index which doesn't exist yet is an error, since there's nothing there to access. This results in an "out of bound" error (and, in its error message, octave is kind enough to tell you what the last legitimate index is that you can access in that particular array).
Therefore, this is an error:
>> clear x
>> x(3) = 1 % output: x = [0, 0, 1]
>> 1 + x(4) % output: error: x(4): out of bound 3
Now going back to your specific code, you are trying to access something that doesn't exist yet. The reason it doesn't exist yet, is that you have set up your for loops such that j will achieve a higher value than i at a particular step, such that you are trying to access x(j), which does not exist yet, in order to assign it to x(i), where i < j. Therefore this results in an out of bounds error (you are trying to access index j when you only have up to i available).
In your particular case, octave informs you that this happened when j was 4, and i was 3.
PS: I will echo #HansHirse's implied warning here, that you should always pay attention to your variables, and clear them appropriately in your scripts, especially if you plan to run it many times. Never use a variable that you haven't defined (or cleared) beforehand. Otherwise, x here may not be undefined when you run your script, say, a second time. This leads to all sorts of problems, e.g., your code works but for the wrong reasons, and then fails to work again when you run it the next day and x is now undefined. In this particular example, if you had an x in your workspace which had the right number of elements in it, your code would "work" but produce the wrong result, and you wouldn't know any better.
I searched the forum and found this thread, but it does not cover my question
Two ways around -inf
From a Machine Learning class, week 3, I am getting -inf when using log(0), which later turns into an NaN. The NaN results in no answer being given in a sum formula, so no scalar for J (a cost function which is the result of matrix math).
Here is a test of my function
>> sigmoid([-100;0;100])
ans =
3.7201e-44
5.0000e-01
1.0000e+00
This is as expected. but the hypothesis requires ans = 1-sigmoid
>> 1-ans
ans =
1.00000
0.50000
0.00000
and the Log(0) gives -Inf
>> log(ans)
ans =
0.00000
-0.69315
-Inf
-Inf rows do not add to the cost function, but the -Inf carries through to NaN, and I do not get a result. I cannot find any material on -Inf, but am thinking there is a problem with my sigmoid function.
Can you provide any direction?
The typical way to avoid infinity in these cases is to add eps to the operand:
log(ans + eps)
eps is a very, very small value, and won't affect the output for values of ans unless ans is zero:
>> z = [-100;0;100];
>> g = 1 ./ (1+exp(-z));
>> log(1-g + eps)
ans =
0.0000
-0.6931
-36.0437
Adding to the answers here, I really do hope you would provide some more context to your question (in particular, what are you actually trying to do.
I will go out on a limb and guess the context, just in case this is useful. You are probably doing machine learning, and trying to define a cost function based on the negative log likelihood of a model, and then trying to differentiate it to find the point where this cost is at its minimum.
In general for a reasonable model with a useful likelihood that adheres to Cromwell's rule, you shouldn't have these problems, but, in practice it happens. And presumably in the process of trying to calculate a negative log likelihood of a zero probability you get inf, and trying to calculate a differential between two points produces inf / inf = nan.
In this case, this is an 'edge case', and generally in computer science edge cases need to be spotted as exceptional circumstances and dealt with appropriately. The reality is that you can reasonably expect that inf isn't going to be your function's minimum! Therefore, whether you remove it from the calculations, or replace it by a very large number (whether arbitrarily or via machine precision) doesn't really make a difference.
So in practice you can do either of the two things suggested by others here, or even just detect such instances and skip them from the calculation. The practical result should be the same.
-inf means negative infinity. Which is the correct answer because log of (0) is minus infinity by definition.
The easiest thing to do is to check your intermediate results and if the number is below some threshold (like 1e-12) then just set it to that threshold. The answers won't be perfect but they will still be pretty close.
Using the following as the sigmoid function:
function g = sigmoid(z)
g = 1 ./ (1 + e.^-z);
end
Then the following code runs with no issues. Choose the threshold value in the 'max' statement to be less than the expected noise in your measurements and then you're good to go
>> a = sigmoid([-100, 0, 100])
a =
3.7201e-44 5.0000e-01 1.0000e+00
>> b = 1-a
b =
1.00000 0.50000 0.00000
>> c = max(b, 1e-12)
c =
1.0000e+00 5.0000e-01 1.0000e-12
>> d = log(c)
d =
0.00000 -0.69315 -27.63102
I'm getting the warning:
warning: mx_el_eq: automatic broadcasting operation applied
From the code:
f = [1;2;3];
f == 1:3;
warning: mx_el_eq: automatic broadcasting operation applied
Can this can be done without warnings?
This is because you are comparing column vector f with row vector 1:3. In Matlab this would be an error however Octave automatically broadcasts. This means that in order to apply the == operator it will expand one of your vectors along a singleton dimension (i.e. a dimension of size 1). In you case both vectors have a singleton dimension to expand so you essentially get the equivalent of:
a1 = [1 1 1;
2 2 2;
3 3 3]; %// for f
a2 = [1 2 3
1 2 3
1 2 3]; %// for 1:3
a1 == a2
Note that in order to get the same result in Matlab you would have to directly call bsxfun
bsxfun(#eq, f, 1:3)
In order to compare you vectors elementwise without the broadcasting you just need to transpose one of them:
f' == 1:3
Automatic broadcasting was a new feature introduced in Octave 3.6. It surprised many people (which were expecting an error), so it was decided to throw a warning. To disable this warning you'll need to turn it off with:
warning ("off", "Octave:broadcast");
You can also turn it off only in the scope of your function:
warning ("off", "Octave:broadcast", "local");
However, I'd recommend you do it in your .octaverc file instead.
The problem with the decision of throwing a warning is that it sounds like you are doing something wrong when you're really not. So as of Octave 4.0, that warning got removed (it is now part of the "Octave:language-extension" warning id).
I am trying to make a simple program in PGI's fortran compiler. This simple program will use the graphics card to calculate pi using the "dart board" algorithm. After battling with this program for quite some time now I have finally got it to behave for the most part. However, I am currently stuck on passing back the results properly. I must say, this is a rather tricky program to debug since I can no longer shove any print statements into the subroutine. This program currently returns all zeros. I am not really sure what is going on, but I have two ideas. Both of which I am not sure how to fix:
The CUDA kernel is not running somehow?
I am not converting the values properly? pi_parts = pi_parts_d
Well, this is the status of my current program. All variables with _d on the end stand for the CUDA prepared device memory where all the other variables (with the exception of the CUDA kernel) are typical Fortran CPU prepared variables. Now there are some print statements I have commented out that I have already tried out from CPU Fortran land. These commands were to check if I really was generating the random numbers properly. As for the CUDA method, I have currently commented out the calculations and replaced z to statically equal to 1 just to see something happen.
module calcPi
contains
attributes(global) subroutine pi_darts(x, y, results, N)
use cudafor
implicit none
integer :: id
integer, value :: N
real, dimension(N) :: x, y, results
real :: z
id = (blockIdx%x-1)*blockDim%x + threadIdx%x
if (id .lt. N) then
! SQRT NOT NEEDED, SQRT(1) === 1
! Anything above and below 1 would stay the same even with the applied
! sqrt function. Therefore using the sqrt function wastes GPU time.
z = 1.0
!z = x(id)*x(id)+y(id)*y(id)
!if (z .lt. 1.0) then
! z = 1.0
!else
! z = 0.0
!endif
results(id) = z
endif
end subroutine pi_darts
end module calcPi
program final_project
use calcPi
use cudafor
implicit none
integer, parameter :: N = 400
integer :: i
real, dimension(N) :: x, y, pi_parts
real, dimension(N), device :: x_d, y_d, pi_parts_d
type(dim3) :: grid, tBlock
! Initialize the random number generaters seed
call random_seed()
! Make sure we initialize the parts with 0
pi_parts = 0
! Prepare the random numbers (These cannot be generated from inside the
! cuda kernel)
call random_number(x)
call random_number(y)
!write(*,*) x, y
! Convert the random numbers into graphics card memory land!
x_d = x
y_d = y
pi_parts_d = pi_parts
! For the cuda kernel
tBlock = dim3(256,1,1)
grid = dim3((N/tBlock%x)+1,1,1)
! Start the cuda kernel
call pi_darts<<<grid, tblock>>>(x_d, y_d, pi_parts_d, N)
! Transform the results into CPU Memory
pi_parts = pi_parts_d
write(*,*) pi_parts
write(*,*) 'PI: ', 4.0*sum(pi_parts)/N
end program final_project
EDIT TO CODE:
Changed various lines to reflect the fixes mentioned by: Robert Crovella. Current status: error caught by cuda-memcheck revealing: Program hit error 8 on CUDA API call to cudaLaunch on my machine.
If there is any method I can use to test this program please let me know. I am throwing darts and seeing where they land for my current style of debugging with CUDA. Not the most ideal, but it will have to do until I find another way.
May the Fortran Gods have mercy on my soul at this dark hour.
When I compile and run your program I get a segfault. This is due to the last parameter you are passing to the kernel (N_d):
call pi_darts<<<grid, tblock>>>(x_d, y_d, pi_parts_d, N_d)
Since N is a scalar quantity, the kernel is expecting to use it directly, rather than as a pointer. So when you pass a pointer to device data (N_d), the process of setting up the kernel generates a seg fault (in host code!) as it attempts to access the value N, which should be passed directly as:
call pi_darts<<<grid, tblock>>>(x_d, y_d, pi_parts_d, N)
When I make that change to the code you have posted, I then get actual printed output (instead of a seg fault), which is an array of ones and zeroes (256 ones, followed by 144 zeroes, for a total of N=400 values), followed by the calculated PI value (which happens to be 2.56 in this case (4*256/400), since you have made the kernel basically a dummy kernel).
This line of code is also probably not doing what you want:
grid = dim3(N/tBlock%x,1,1)
With N = 400 and tBlock%x = 256 (from previous code lines), the result of the calculation is 1 (ie. grid ends up at (1,1,1) which amounts to one threadblock). But you really want to launch 2 threadblocks, so as to cover the entire range of your data set (N = 400 elements). There's a number of ways to fix this, but for simplicity let's just always add 1 to the calculation:
grid = dim3((N/tBlock%x)+1,1,1)
Under these circumstances, when we launch grids that are larger (in terms of total threads) than our data set size (512 threads but only 400 data elements in this example) it's customary to put a thread check near the beginning of our kernel (in this case, after the initialization of id), to prevent out-of-bounds accesses, like so:
if (id .lt. N) then
(and a corresponding endif at the very end of the kernel code) This way, only the threads that correspond to actual valid data are allowed to do any work.
With the above changes, your code should be essentially functional, and you should be able to revert your kernel code to the proper statements and start to get an estimate of PI.
Note that you can check the CUDA API for error return codes, and you can also run your code with cuda-memcheck to get an idea of whether the kernel is making out-of-bounds accesses. Niether of these would have helped with this particular seg fault, however.