I'm trying to make my own project, learning about linear regression.
For some reasone I got wrong linear regression.
I've tried changing alfa values and iterations but it didnt help.
I was checking also for other advices here but I couldnt make it.
GRADIENT DESCENT
function [theta, J_history] = gradientDescent(X, y, theta, alpha, num_iters)
%GRADIENTDESCENT Performs gradient descent to learn theta
% theta = GRADIENTDESCENT(X, y, theta, alpha, num_iters) updates theta by
% taking num_iters gradient steps with learning rate alpha
% Initialize some useful values
m = length(y); % number of training examples
J_history = zeros(num_iters, 1);
for iter = 1:num_iters
x = X(:,2);
h = theta(1) + (theta(2) * x);
theta_zero = theta(1) - alpha * (1/m) * sum(h-y); %a-step*gradient(1)
theta_one = theta(2) - alpha * (1/m) * sum((h - y) .* x); %b-step*gardient(2)
theta = [theta_zero; theta_one];
J_history(iter) = computeCost(X, y, theta);
end
end
MAIN CODE
clear ; close all; clc
%% ======================= Part 2: Plotting =======================
fprintf('Plotting Data ...\n')
data = csvread('Fish.txt');
X = data(:, 1); y = data(:, 2);
m = length(y); % number of training examples
figure;
plot(X, y, 'dp', 'MarkerSize', 5);
title('Wykres')
fprintf('Program paused. Press enter to continue.\n');
pause;
%% =================== Part 3: Gradient descent ===================
fprintf('Running Gradient Descent ...\n')
X = [ones(m, 1), data(:,1)]; % Add a column of ones to x
%disp(X)
theta = zeros(size(X, 2), 1); % initialize fitting parameters
% Some gradient descent settings
iterations = 1500;
alpha = 0.000001;
% compute and display initial cost
computeCost(X, y, theta)
% run gradient descent
theta = gradientDescent(X, y, theta, alpha, iterations);
% print theta to screen
fprintf('Theta found by gradient descent: ');
fprintf('%f %f \n', theta(1), theta(2));
% Plot the linear fit
hold on; % keep previous plot visible
plot(X(:,2), X*theta, '-')
legend('Training data', 'Linear regression')
hold off % don't overlay any more plots on this figure
FILE
Fish.txt
First column is weight, second is length
242,23.2
290,24
340,23.9
363,26.3
430,26.5
450,26.8
500,26.8
390,27.6
450,27.6
500,28.5
475,28.4
500,28.7
500,29.1
340,29.5
600,29.4
600,29.4
700,30.4
700,30.4
610,30.9
650,31
575,31.3
685,31.4
620,31.5
680,31.8
700,31.9
725,31.8
720,32
714,32.7
850,32.8
1000,33.5
920,35
955,35
925,36.2
975,37.4
950,38
40,12.9
69,16.5
78,17.5
87,18.2
120,18.6
0,19
110,19.1
120,19.4
150,20.4
145,20.5
160,20.5
140,21
160,21.1
169,22
161,22
200,22.1
180,23.6
290,24
272,25
390,29.5
270,23.6
270,24.1
306,25.6
540,28.5
800,33.7
1000,37.3
55,13.5
60,14.3
90,16.3
120,17.5
150,18.4
140,19
170,19
145,19.8
200,21.2
273,23
300,24
5.9,7.5
32,12.5
40,13.8
51.5,15
70,15.7
100,16.2
78,16.8
80,17.2
85,17.8
85,18.2
110,19
115,19
125,19
130,19.3
120,20
120,20
130,20
135,20
110,20
130,20.5
150,20.5
145,20.7
150,21
170,21.5
225,22
145,22
188,22.6
180,23
197,23.5
218,25
300,25.2
260,25.4
265,25.4
250,25.4
250,25.9
300,26.9
320,27.8
514,30.5
556,32
840,32.5
685,34
700,34
700,34.5
690,34.6
900,36.5
650,36.5
820,36.6
850,36.9
900,37
1015,37
820,37.1
1100,39
1000,39.8
1100,40.1
1000,40.2
1000,41.1
200,30
300,31.7
300,32.7
300,34.8
430,35.5
345,36
456,40
510,40
540,40.1
500,42
567,43.2
770,44.8
950,48.3
1250,52
1600,56
1550,56
1650,59
6.7,9.3
7.5,10
7,10.1
9.7,10.4
9.8,10.7
8.7,10.8
10,11.3
9.9,11.3
9.8,11.4
12.2,11.5
13.4,11.7
12.2,12.1
19.7,13.2
19.9,13.8
Thank you in advance!
I tried using standard deviation and Ive replaced
theta = zeros(2, 1);
for
theta = zeros(size(X, 2), 1);
I've also changed values of alpha
My (PyTorch) sum of model’s output isn’t 1. And this is the structure of model.
LSTM(4433, 64)
LSTM(64, 64)
Linear(64, 4433)
Sigmoid()
And this is the predicted output of the model.
Input
[1, 0, 0, …, 0, 0]
Output
[.7842, .5, .5, …, .5, .5]
Do you know any function that can make its sum 1?
Sigmoid activation function maps every input to a value between [0, 1], without taking into account other elements in the input vector. However, Softmax does a similar transformation but the output vector sums 1.
TL;DR: use softmax instead of sigmoid.
I have a question regarding using the sine function. When I entered the number 4, 8, .., etc., really I supposed to get the number somewhere very close to zero, but not exactly (Ex. 0.001, 0.0003, etc). However, I got the number y = 1.224 when x = 4, and y = -2.449 when x = 8. This should be incorrect. I don’t understand the problem here. Does anyone know what is going on here?
[Photo of my code and a sin graph - Link]
https://ibb.co/6YCWW90
[Code]
import math
import matplotlib.pyplot as plt
import numpy as np
x = [0, 1, 2, 3, 4, 5, 6, 7, 8]
y = [math.sin(0.25 * math.pi * i) for i in x]
print(y)
plt.plot(x, y)
plt.show()
Everything works.
When i is 0, then "0.25 * math.pi * i" is precisely 0 and when you calculate the sine, you get exactly 0.0.
When i is 4, then calculating "0.25 * math.pi * i" results in a number very close to PI, but the accuracy is limited. If you calculate the sine, you get a number which is very, very close to zero, but because of limited accuracy, not exactly zero. The result is 1.2246467991473532e-16. NOTE: it is 0.00000000000000012246467991473532, not 1.224 what your wrote in your question.
Similarly rounding errors result in -2.4492935982947064e-16 for i equal to 8. The argument is not exactly 2 PI and rounding errors result in a value slightly different than 0.0.
Again -2.4492935982947064e-16 is -0.00000000000000024492935982947064 and not -2.449 as you wrote in your question.
julia language deep learning framework,
This is a quick start for Knet.jl,
https://denizyuret.github.io/Knet.jl/latest/tutorial/#Tutorial
ENV ["COLUMNS"] = 72
using Knet, MLDatasets, IterTools
struct Conv; w; b; f; end
(c :: Conv) (x) = c.f. (pool (conv4 (c.w, x). + C.b))
Conv (w1, w2, cx, cy, f = relu) = Conv (param (w1, w2, cx, cy), param0 (1,1, cy, 1), f);
The complex type Conv has three fields, w, b, and f.
The Conv type c (x) function broadcasts the next function with the f function.
The inner product of the w matrix and the x matrix is calculated with conv4 (c.w, x), and the addition with c.b is performed with. +.
I don't know what the pool is looking for in that matrix.
This (pool (conv4 ...)) is passed through the relu activation function.
At the last Conv (w1, w2, cx, cy, f = relu) = Conv (param (w1, w2, cx, cy), param0 (1,1, cy, 1), f);
I don't know what I'm trying to do.
This is the situation of understanding.
What are you trying to do, especially in the pool?
Why are there two params on the 5th line?
I do not know.
Actually, the layer does a convolution followed by max pooling:
Pooling layers reduce the dimensions of the data by combining the outputs of neuron clusters at one layer into a single neuron in the next layer. Local pooling combines small clusters, typically 2 x 2. Global pooling acts on all the neurons of the convolutional layer. There are two common types of pooling: max and average. Max pooling uses the maximum value of each cluster of neurons at the prior layer, while average pooling instead uses the average value. (source: https://en.wikipedia.org/wiki/Convolutional_neural_network#Pooling_layers)
There are two params on the 5th line, because a convolutional layers has two trainable parameters: the kernel weights w and the bias b. The function param (and param0) initialize them with the correct size and mark them as trainable parameters that will be updated during the optimization.
To learn neural networks, I found these examples: linear regression and a simple feed-forward network (multilayer perceptron) quite useful.
So I'm trying to knock out this last problem, and I'm following my teacher's guide but my graph seems to still be off, the problem is:
Use the FindRoot command in Mathematica to define an inverse function g(y) to y = f(x) = 3x + tan(x) with the restriction ‑pi/2 < x < pi/2. Use x = tan-1(y) as a starting value. Then use the Plot command to make a graph of g(y).
This is how I wrote it out:
g[y_] := x /. FindRoot[3 x + Tan[x] == y, {x, ArcTan[y]}]
Plot[g[y], {y, (-Pi/2), (Pi/2)}]
I'm not sure exactly what the problem is, but it shows the graph as just being a straight line through the origin. I'm not sure if this is how it's supposed to be (which I assume it's not), but any and all help would be much appreciated!
Having your equation,
3 x + Tan[x] == y
You can check the correctness of the plot of g(y) by plotting y(x):
Plot[3 x + Tan[x], {x, -.4, .4}]
As you can easily see, it is a straight line through the origin. g(y) is inverse of y(x) by definition, so you can get a plot of g(y) it just by exchanging the y and x axes:
Plot[3 x + Tan[x], {x, -.4, .4},
PlotRange -> All] /. {x_Real, y_Real} :> {y, x}