which python regression function to use for linear regression curve - regression

Im trying to replicate a function in python and was able to code the following using multiple columns in a dataframe but was wondering if there is a python regression function that would do this more effectively. Here is the link to the description of the function. Sorry in advanced not really a stats guy. :)
http://tlc.thinkorswim.com/center/reference/thinkScript/Functions/Statistical/Inertia.html
It states that its the linear regression curve using the least-squares method to approximate data for each set of bars.
input y = close;
input n = 20;
def x = x[1] + 1; # previous value + 1
def a = (n * Sum(x * y, n) - Sum(x, n) * Sum(y, n) ) / ( n * Sum(Sqr(x), n) -Sqr(Sum(x, n)));
def b = (Sum(Sqr(x), n) * Sum(y, n) - Sum(x, n) * Sum(x * y, n) ) / ( n * Sum(Sqr(x), n) - Sqr(Sum(x, n)));
plot InertiaTS = a * x + b;
Thanks
Updated
here is the pandas columns and function. I first defined the xValue and yValue columns and then the following which is the raw calculation:
df['ind1']= ((10 * (df['xValue']*df['ysValue']).rolling(10, min_periods=10).sum() - df['xValue'].rolling(10, min_periods=10).sum()*df['ysValue'].rolling(10, min_periods=10).sum())/ (10 * (df['xValue'] ** 2).rolling(10, min_periods=10).sum() - (df['xValue'].rolling(10, min_periods=10).sum())**2)) * df['xValue'] + (((df['xValue'] ** 2).rolling(10, min_periods=10).sum()*df['ysValue'].rolling(10, min_periods=10).sum() - df['xValue'].rolling(10, min_periods=10).sum()*(df['xValue']*df['ysValue']).rolling(10, min_periods=10).sum())/(10 * (df['xValue'] ** 2).rolling(10, min_periods=10).sum() - (df['xValue'].rolling(10, min_periods=10).sum())**2))

It's not really clear whether you are just looking for a way to perform regression in python or you want to code the algorithm yourself.
if you want a package to do the regression, you can look at scikit-learn
Using,
from sklearn import linear_model
linear_model.LinearRegression()
If you want to code your own algorithm, you can look at gradient descent. you can look at a video by Andrew Ng on coursera - https://www.coursera.org/learn/machine-learning/lecture/GFFPB/gradient-descent-intuition. It's fairly intuitive to code the algorithm, the steps are as follows,
i. define a cost function - this is based on OLS(ordinary least squares) and looks like,
J = 1/2 (h(x) - y ) ^2
ii. take a partial derivative of cost function wrt each feature or j. Here X is the input vector comprised of n features one of which is j.
iii. Update the feature vector using gradient descent -
theta = theta - alpha * (partial derivative)
You can find the details here from Andrew Ng's papaper: http://cs229.stanford.edu/notes/cs229-notes1.pdf
sorry, it's difficult to put latex on SO

Related

Kriging spherical model not working in QGIS 3.24 (SagaGIS 7.8.2)

my work is based on QGIS 3.24, and the algorithm i am trying to make work is "Ordinary Kriging" from SagaGIS 7.8.2.
As some may recall, SagaGIS used to offer different models (equations) for fitting the Variogram (VAR_MODEL). In my case i have to use the spherical model which used to look like this:
a + b * ifelse(x > c, 1, 1.5 * x / c - 0.5 * x^3 / c^3)
where a is the Nugget, b is the difference between Sill and Nugget and C is the Range.
After few unsuccesfull errors i understood that QGIS doesnt like the power expressed with this symbol "^", so i worked around it substituting with a simple moltiplication ( x^3 -> x * x * x ). So my model looks like this now:
a + b * ifelse(x > c, 1, 1.5 * x / c - 0.5 * x * x * x / c * c * c)
Now, i noted that the ifelse statement, which sound R-ish to me, is not working, but no particoular error is arised. Clearly, removing the ifelse statement with the Test and True condition, which in our case is ifelse(x>c,1, makes the code works, but the results are absourdly wrong, clearly.
Can anyone help me out with this problem? Is there any workaround?
Davide

Using linear approximation to perform addition and subtraction | error barrier

I'm attempting my first solo project, after taking an introductory course to machine learning, where I'm trying to use linear approximation to predict the outcome of addition/subtraction of two numbers.
I have 3 features: first number, subtraction/addition (0 or 1), and second number.
So my input looks something like this:
3 0 1
4 1 2
3 0 3
With corresponding output like this:
2
6
0
I have (I think) successfully implemented logistic regression algorithm, as the squared error does gradually decrease, but in 100 values, ranging from 0 to 50, the squared error value flattens out at around 685.6 after about 400 iterations.
Graph: Squared Error vs Iterations
.
To fix this, I have tried using a larger dataset for training, getting rid of regularization, and normalizing the input values.
I know that one of the steps to fix high bias is to add complexity to the approximation, but I want to maximize the performance at this particular level. Is it possible to go any further on this level?
My linear approximation code in Octave:
% Iterate
for i = 1 : iter
% hypothesis
h = X * Theta;
% reg theta prep
regTheta = Theta;
regTheta(:, 1) = 0;
% cost calc
J(i, 2) = (1 / (2 * m)) * (sum((h - y) .^ 2) + lambda * sum(sum(regTheta .^ 2,1),2));
% theta calc
Theta = Theta - (alpha / m) * ((h - y)' * X)' + lambda * sum(sum(regTheta, 1), 2);
end
Note: I'm using 0 for lambda, as to ignore regularization.

Matlab plotting the shifted logistic function

I would like to plot the shifted logistic function as shown from Wolfram Alpha.
In particular, I would like the function to be of the form
y = exp(x - t) / (1 + exp(x - t))
where t > 0. In the link, for example, t is 6. I had originally tried the following:
x = 0:.1:12;
y = exp(x - 6) ./ (1 + exp(x - 6));
plot(x, y);
axis([0 6 0 1])
However, this is not the same as the result from Wolfram Alpha. Here is an export of my plot.
I do not understand what the difference is between what I am trying to do here vs. plotting shifted sin and cosine functions (which works using the same technique).
I am not completely new to Matlab but I do not usually use it in this way.
Edit: My values for x in the code should have been from 0 to 12.
fplot takes as inputs a function handle and a range to plot for:
>> fplot(#(x) exp(x-6) / (1 + exp(x-6)), [0 12])
The beauty of fplot in this case is you don't need to spend time calculating y-values beforehand; you could also extract values from the graph after the fact if you want (by getting the line handle's XData and YData properties).
Your input to Wolfram Alpha is incorrect. It is interpreted as e*(x-6)/(1-e*(x-6)). Use plot y = exp(x - 6) / (1 + exp(x - 6)) for x from 0 to 12 in Wolfram Alpha (see here) for the same results as in MATLAB. Also use axis([0 12 0 1]) (or no axis statement at all on a new plot) to see the full results in MATLAB.
In reply to your comment: use y = exp(1)*(x - 6) ./ (1 + exp(1)*(x - 6)); to do in MATLAB what you were doing in Wolfram Alpha.

How can I find the smallest difference between two angles around a point?

Given a 2D circle with 2 angles in the range -PI -> PI around a coordinate, what is the value of the smallest angle between them?
Taking into account that the difference between PI and -PI is not 2 PI but zero.
An Example:
Imagine a circle, with 2 lines coming out from the center, there are 2 angles between those lines, the angle they make on the inside aka the smaller angle, and the angle they make on the outside, aka the bigger angle.
Both angles when added up make a full circle. Given that each angle can fit within a certain range, what is the smaller angles value, taking into account the rollover
This gives a signed angle for any angles:
a = targetA - sourceA
a = (a + 180) % 360 - 180
Beware in many languages the modulo operation returns a value with the same sign as the dividend (like C, C++, C#, JavaScript, full list here). This requires a custom mod function like so:
mod = (a, n) -> a - floor(a/n) * n
Or so:
mod = (a, n) -> (a % n + n) % n
If angles are within [-180, 180] this also works:
a = targetA - sourceA
a += (a>180) ? -360 : (a<-180) ? 360 : 0
In a more verbose way:
a = targetA - sourceA
a -= 360 if a > 180
a += 360 if a < -180
x is the target angle. y is the source or starting angle:
atan2(sin(x-y), cos(x-y))
It returns the signed delta angle. Note that depending on your API the order of the parameters for the atan2() function might be different.
If your two angles are x and y, then one of the angles between them is abs(x - y). The other angle is (2 * PI) - abs(x - y). So the value of the smallest of the 2 angles is:
min((2 * PI) - abs(x - y), abs(x - y))
This gives you the absolute value of the angle, and it assumes the inputs are normalized (ie: within the range [0, 2π)).
If you want to preserve the sign (ie: direction) of the angle and also accept angles outside the range [0, 2π) you can generalize the above. Here's Python code for the generalized version:
PI = math.pi
TAU = 2*PI
def smallestSignedAngleBetween(x, y):
a = (x - y) % TAU
b = (y - x) % TAU
return -a if a < b else b
Note that the % operator does not behave the same in all languages, particularly when negative values are involved, so if porting some sign adjustments may be necessary.
An efficient code in C++ that works for any angle and in both: radians and degrees is:
inline double getAbsoluteDiff2Angles(const double x, const double y, const double c)
{
// c can be PI (for radians) or 180.0 (for degrees);
return c - fabs(fmod(fabs(x - y), 2*c) - c);
}
See it working here:
https://www.desmos.com/calculator/sbgxyfchjr
For signed angle:
return fmod(fabs(x - y) + c, 2*c) - c;
In some other programming languages where mod of negative numbers are positive, the inner abs can be eliminated.
I rise to the challenge of providing the signed answer:
def f(x,y):
import math
return min(y-x, y-x+2*math.pi, y-x-2*math.pi, key=abs)
For UnityEngine users, the easy way is just to use Mathf.DeltaAngle.
Arithmetical (as opposed to algorithmic) solution:
angle = Pi - abs(abs(a1 - a2) - Pi);
I absolutely love Peter B's answer above, but if you need a dead simple approach that produces the same results, here it is:
function absAngle(a) {
// this yields correct counter-clock-wise numbers, like 350deg for -370
return (360 + (a % 360)) % 360;
}
function angleDelta(a, b) {
// https://gamedev.stackexchange.com/a/4472
let delta = Math.abs(absAngle(a) - absAngle(b));
let sign = absAngle(a) > absAngle(b) || delta >= 180 ? -1 : 1;
return (180 - Math.abs(delta - 180)) * sign;
}
// sample output
for (let angle = -370; angle <= 370; angle+=20) {
let testAngle = 10;
console.log(testAngle, "->", angle, "=", angleDelta(testAngle, angle));
}
One thing to note is that I deliberately flipped the sign: counter-clockwise deltas are negative, and clockwise ones are positive
There is no need to compute trigonometric functions. The simple code in C language is:
#include <math.h>
#define PIV2 M_PI+M_PI
#define C360 360.0000000000000000000
double difangrad(double x, double y)
{
double arg;
arg = fmod(y-x, PIV2);
if (arg < 0 ) arg = arg + PIV2;
if (arg > M_PI) arg = arg - PIV2;
return (-arg);
}
double difangdeg(double x, double y)
{
double arg;
arg = fmod(y-x, C360);
if (arg < 0 ) arg = arg + C360;
if (arg > 180) arg = arg - C360;
return (-arg);
}
let dif = a - b , in radians
dif = difangrad(a,b);
let dif = a - b , in degrees
dif = difangdeg(a,b);
difangdeg(180.000000 , -180.000000) = 0.000000
difangdeg(-180.000000 , 180.000000) = -0.000000
difangdeg(359.000000 , 1.000000) = -2.000000
difangdeg(1.000000 , 359.000000) = 2.000000
No sin, no cos, no tan,.... only geometry!!!!
A simple method, which I use in C++ is:
double deltaOrientation = angle1 - angle2;
double delta = remainder(deltaOrientation, 2*M_PI);

Convert a quadratic curve points to polynomial representation?

I have the X,Y of 2 end points and 1 bezier point, of a Quadratic Bezier curve.
Using this data, how can I derive the polynomial representation of the curve?
(source: euclidraw.com)
B(t) = (1-t) * (1-t) * B0 + 2 * (1-t) * t * B1 + t * t * B2
Oog. That would be tricky. Beziers are parametrized curves, namely:
x = f(t)
y = g(t)
where t=0 yields one endpoint and t=1 yields the other.
You could technically figure out how to eliminate "t" and get an equation in x and y, but it would not be a polynomial like y = a + bx + cx2 ...; it would be an equation h(x,y) = 0 where h is probably somewhat ugly.
Wikipedia has a section about this. Perhaps this helps.