Linear Regression using fminunc Implementation - octave

Question

I'm trying to implement linear regression with only one feature using fminunc in Octave.
Here is my code.
x = load('/home/battousai/Downloads/ex2Data/ex2x.dat');
y = load('/home/battousai/Downloads/ex2Data/ex2y.dat');
m = length(y);
x = [ones(m , 1) , x];
theta = [0 , 0]';
X0 = [x , y , theta];
options = optimset('GradObj' , 'on' , 'MaxIter' , 1500);
[x , val] = fminunc(#computeCost , X0 , options)
And here is the cost function which returns the gradient as well as the value of the cost function.
function [J , gradient] = computeCost(x , y , theta)
m = length(y);
J = (0.5 / m) .* (x * theta - y )' * (x * theta - y );
gradient = (1/m) .* x' * (x * theta - y);
end
The length of the data set is 50, i.e., the dimensions are 50 x 1. I'm not getting the part that how should I pass X0 to the fminunc.
Updated Driver Code:
x = load('/home/battousai/Downloads/ex2Data/ex2x.dat');
y = load('/home/battousai/Downloads/ex2Data/ex2y.dat');
m = length(y);
x = [ones(m , 1) x];
theta_initial = [0 , 0];
options = optimset('Display','iter','GradObj','on' , 'MaxIter' , 100);
[X , Cost] = fminunc(#(t)(computeCost(x , y , theta)), theta_initial , options)
Updated Code for Cost function:
function [J , gradient] = computeCost(x , y , theta)
m = length(y);
J = (1/(2*m)) * ((x * theta) - y )' * ((x * theta) - y) ;
gradient = (1 / m) .* x' * ((x * theta) - y);
end
Now I'm getting values of theta to be [0,0] but when I used normal equation, values of theta turned out to be [0.750163 , 0.063881].

Answer 1

From the documentation for fminunc:
FCN should accept a vector (array) defining the unknown variables
and
X0 determines a starting guess.
Since your input is a cost function (i.e. it associates your choice of parameter vector with a cost), the input argument to your cost function, that needs to be optimised via fminunc is only theta, since x and y (i.e. your observations and your targets) are considered 'given' aspects of the problem and are not things you're trying to optimise. So you either declare x and y global and access them from your function like so:
function [J , gradient] = computeCost(theta_0)
global x; global y;
% ...
and then call fminunc as: fminunc (#computeCost, t_0, options)
or, keep your computeCost function as computeCost(x, y, theta), and change your fminunc call to something like this:
[x , val] = fminunc(# (t) computeCost(x, y, t) , t0 , options)
UPDATE Not sure what you were doing wrong. Here is the full code and an octave session running it. Seems fine.
%% in file myscript.m
x = load('ex2x.dat');
y = load('ex2y.dat');
m = length(y);
x = [ones(m , 1) , x];
theta_0 = [0 , 0]';
options = optimset('GradObj' , 'on' , 'MaxIter' , 1500);
[theta_opt, cost] = fminunc(# (t) computeCost(x,y,t) , theta_0 , options)
%% in file computeCost.m
function [J , gradient] = computeCost(x , y , theta)
m = length(y);
J = (0.5 / m) .* (x * theta - y )' * (x * theta - y );
gradient = (1/m) .* x' * (x * theta - y);
end
%% in the octave terminal:
>> myscript
theta_opt =
0.750163
0.063881
cost = 9.8707e-04