In Kalman Filtering, there is an observation model for describing the relationship between the state x_t and the observation y_t: y_t = H*x_t+r_t, with w_t~N(0,R_t). The covariance of the observation noise is generally assumed fixed.
Is it possible to use a state-dependent covariance ( R_t = f(x_t) ) ? It will affects the maths behind the theory, but if it's a more accurate covariance, I guess the results should be better.
Related
I am beginner in deep learning.
I am using this dataset and I want my network to detect keypoints of a hand.
How can I make my output layer's nodes to be in range [-1, 1] (range of normalized 2D points)?
Another problem is when I train for more than 1 epoch the loss gets negative values
criterion: torch.nn.MultiLabelSoftMarginLoss() and optimizer: torch.optim.SGD()
Here u can find my repo
net = nnModel.Net()
net = net.to(device)
criterion = nn.MultiLabelSoftMarginLoss()
optimizer = optim.SGD(net.parameters(), lr=learning_rate)
lr_scheduler = torch.optim.lr_scheduler.ExponentialLR(optimizer=optimizer, gamma=decay_rate)
You can use the Tanh activation function, since the image of the function lies in [-1, 1].
The problem of predicting key-points in an image is more of a regression problem than a classification problem (especially if you're making your model outputs + targets fall within a continuous interval). Therefore, I suggest you use the L2 Loss.
In fact, it could be a good exercise for you to determine which loss function that is appropriate for regression problems provides the lowest expected generalization error using cross-validation. There's several such functions available in PyTorch.
One way I can think of is to use torch.nn.Sigmoid which produces outputs in [0,1] range and scale outputs to [-1,1] using 2*x-1 transformation.
I'm trying to achieve an algorithm to efficiently calculate a combination of convolution and correlation such as following :
c(x,y)=(sum of i, (sum of j, a(x-i,y+j)*b(i,j)))
I have known that 1-D convolution or correlation can be solved by
a conv b = ifft(fft(a).*fft(b))
a corr b = ifft(fft(a).*conjg(fft(b)))
But I have no idea about the combination of them in 2-D or N-D problems. I think it is similar to 2-D convolution, but I don't know the specific deduction process.
The correlation can be written in terms of the convolution by reversing one of the arguments:
corr(x(t),y(t)) = conv(x(t),y(-t))
Thus, if you want the x-axis to behave like a convolution and the y-axis to behave like a correlation, reverse the y-axis only and compute the convolution. It doesn’t matter if you use a spatial or frequency domain implementation.
I'm teaching myself how to use extended kalman filters and have written a filter where both the error and covariance are converging. However, the covariance drops to nearly its convergent value immediately.
During the first iteration, at the covariance update step ( P = (I-KH)Pminus), some of the diagonals of KH become 1. This causes the corresponding diagonals of the new covariance P to be nearly zero.
Is this a "normal" occurrence with EK filters or a bug?
I've tried increasing the covariance noise but this doesn't seem to have much of an impact. Additionally, I've been going over my H matrix but cannot find any errors. Am I right to assume that K is less likely to have an error, since both the state estimate and covariance converge?
In the Kalman fit, the measurements make the entries of the state covariance shrink via the intermediate of the Kalman gain in the update step, and the prediction step makes your state covariance entries grow via the process noise term.
If your fit is implemented correctly, and the entries of KH become close to one, then this means that the entries of the measurement covariance matrix are really small compared to the entries of the state covariance (where the comparison of relative sizes has to take into account whatever happens in you H, say, a change of units from parsec to millimeter).
The easiest way to check your implementation is to implement a measurement where one of your state variables is measured directly, i.e. H is of the form H = { 0, ..., 0, 1, 0, 0, ... }. If you now put the same value in both the corresponding diagonal entry of the covariance matrix, keeping it diagonal to avoid complications from covariances, and in your (1D) measurement covariance, then the updated state should fall right in the middle of the state before the update (i.e. the predicted state) and the measured value, and the covariance entry should shrink in half.
I was asked to create a leg follower robot (I already did it) and in the second part of this assignment I have to develop a Kalman filter in order to improve the following process of the robot. The robot gets from the person the distance where she is to the robot and also the angle (it is a relative angle, because the reference is the robot itself, not absolute x-y coordinates)
About this assignment I have a serious doubt. Everything I have read, every sample I have seen about kalman filter has been in one dimension (a car running distance or a rock falling from a building) and according to the task I would have to apply it in 2 dimensions. Is it possible to apply a kalman filter like this?
If it is possible to calculate kalman filter in 2 dimensions then I would understand that what is asked to do is to follow the legs in a linnearized way, despite a person walks weirdly (with random movements) --> About this I have the doubt of how to establish the function of the state matrix, could anyone please tell me how to do it or to tell me where I can find more information about this?
thanks.
Well you should read up on Kalman Filter. Basically what it does is estimate a state through its mean and variance separately. The state can be whatever you want. You can have local coordinates in your state but also global coordinates.
Note that the latter will certainly result in nonlinear system dynamics, in which case you could use the Extended Kalman Filter, or to be more correct the continuous-discrete Kalman Filter, where you treat the system dynamics in a continuous manner and the measurements in discrete time.
Example with global coordinates:
Assuming you have a small cubic mass which can drive forward with velocity v. You could simply model the dynamics in local coordinates only, where your state s would be s = [v], which is a linear model.
But, you could also incorporate the global coordinates x and y, assuming we are moving on a plane only. Then you would have s = [x, y, phi, v]'. We need phi to keep track of the current orientation since the cube can only move forward in respect to its orientation of course. Let's define phi as the angle between the cube's forward direction and the x-axis. Or in other words: With phi=0 the cube would move along the x-axis, with phi=90° it would move along the y-axis.
The nonlinear system dynamics with global coordinates can then be written as
s_dot = [x_dot, y_dot, phi_dot, v_dot]'
with
x_dot = cos(phi) * v
y_dot = sin(phi) * v
phi_dot = ...
v_dot = ... (Newton's Law)
In EKF (Extended Kalman Filter) Prediction step you would use the (discretized) equations above to predict the mean of the state in the first step of and the linearized (and discretized) equations for prediction of the Variance.
There are two things to keep in mind when you decide what your state vector s should look like:
You might be tempted to use my linear example s = [v] and then integrate the velocity outside of the Kalman Filter in order to obtain the global coordinate estimates. This would work, but you would lose the awesomeness of the Kalman Filter since you would only integrate the mean of the state, not its variance. In other words, you would have no idea what the current uncertainties for your global coordinates are.
The second step of the Kalman Filter, the measurement or correction update, requires that you can describe your sensor output as a function of your states. So you may have to add states to your representation just so that you can express your measurements correctly as z[k] = h(s[k], w[k]) where z are measurements and w is a noise vector with Gaussian distribution.
I implemented an EKF. The algorithm works very well but I need a criterion to detected when the filter is converged after initialisation. What is the best / most common way to do this. I have two ideas:
1.) When innovation has reached a pre defined limit.
2.) When estimated variance has reached a pre defined limit.
Any suggestions ?
The most common error when dealing with Kalman filters (and especially EKF) is to think that the convergence of the P matrix is equivalent to real convergence of the estimation.
You need to look at the normalized innovations.
The innovation is the difference between the expected (predicted measurement) to the actual one:
Innov = y - h(x_predicted)
The normalized innovation is the Mahalanobis distance of the innovation, which correlates with the convergence once the P matrix is small:
d^2 = Innov.transpose * Cov(Innov).inverse * Innov
Where Cov(Innov) = Cov(y - h(x_predicted)) = R + H * P_predicted * H.Transpose
I usually look at the derivative of the estimated variance. If it is not changing any more, the filter has converged.
This usually works quite well.