A special loss function in caffe - caffe

I have a kind of euclidean loss function which is:
\sum_{i,j} c_i*max{0,y_{ji}-k_{ji}} + p_i*max{0,k_{ji}-y_{ji}}
which y_{ji} are the output of caffe and k_{ji} are the real output value, i is the index of the items and j is index of samples.
The issue is about getting the values of parameters c_i and p_i.
When I have c_i = c_q for all i \neq q, and similarly for p_i, I simply get the values of them as parameters of the loss layer (I added two new parameters in the caffe.proto). However, the problems is that now I have around 300 items so that it is not reasonable to get them as loss layer parameters.
I tried to get their values in the loss layer, I mean I tried to add another bottom layer for loss layer, but it gave an error.
I am stuck here!
Please guide me how I can solve this issue.
Thanks in advance,
Afshin

Related

Amplitude Spectrum of a function

My question is related to plotting amplitude spectrum.
Problem 1: (I have solved it) I have to represent the following function as a discrete set of N=100 numbers separated by time increment of 1/N:
e(t) = 3sin2.1t + 2sin1.9t
I did it using stem function in matlab and plotted it.
Problem 2: (I have question about it) The next thing was to repeat the same above all, using dataset of 200 points with time increment of 1/N and 1/2N.
My question is a bit basic but I just want to clear if I am following the right path to solve my problem.
I want to ask that for problem 2, for both 1/N and 1/2N, should I use N=200 (as I believe it is separate problem)?
A few of my mates have suggested using N=100 for 1/N and N=200 for 1/2N.
which one is the right thing?
Any help will be highly appreciated. Thanks

Use LSTM to forecast Precipitation

I build a LSTM to forecast Precipitation, but it doesn't work well.
My code is very simple and data is very short only contains 720 points.
i use MinMaxScale to scale the data.
this is my code, seq_len = 12
model = Sequential([
layers.LSTM(2, input_shape=(SEQ_LEN, 1),
layers.Dense(1)])
my data is like this
and the output compares with true value like this
I use adam and mae loss function, epoch=10
is it underfitting? or is this simple net can't do this work?
r2_score is no more than 0.55
please tell me how to adjust it. thanks
there are so many options;
first of all it would be better to define the optimized window size by changing the periods of the sequences
The second option would be changing the batch-size of the dataset
Change optimizer into SGD cause of few datapoints and before training model define the best values for learning rate by setting Learning Rate Schedule callback
Try another model architecture with convolution layers and etc
Sometimes it would be a trick to help model performance by setting lambda layer after the last layer to scale up values cause of lstm default activation function is tanh.

How to implement soft-argmax in caffe?

In Caffe deep-learning framework there is an argmax layer which is not differentiable and hence can not be used for end to end training of a CNN.
Can anyone tell me how I could implement the soft version of argmax which is soft-argmax?
I want to regress coordinates from heatmap and then use those coordinates in loss calculations. I am very new to this framework therefore no idea how to do this. any help will be much appreciated.
I don't get exactly what you want, but there are following options:
Use L2 loss to train regression task (EuclideanLoss). Or SmoothL1Loss (from SSD Caffe by Wei Lui), or L1 (don't know were you get it).
Use softmax with cross-entropy loss (SoftmaxWithLoss) to train classification task with classes corresponding to the possible values of x or y coordinate. For example, one loss layer for x, and one for y. SoftmaxWithLoss accepts label as a numeric value, and casts it to int with static_cast(). But take into account that implementation doesn't check that the casted value is within 0..(num_classes-1) range, so you have to be careful.
If you want something more unusual, you'll have to write you own layer in C++, C++/CUDA or Python+NumPy. This is very often the case unless you are already using someone other's implementation.

keras: trainable_weight issue

In one particular application, I need to train only bias of the convolutional operation. Therefore I removed W parameter from trainable_weight. that looks like this:
self.trainable_weights = [ self.b]
I save model at 0 epoch and after 200 epoch and I found that W is somehow able to learn. Do not know what is going on. When I saw model.summary() it showing the correct number of learning parameters. Can anyone tell me what is wrong here?

How to get the predicted values in training data set for Least Squares Support Vector Regression

I would like to make a prediction by using Least Squares Support Vector Machine for Regression, which is proposed by Suykens et al. I am using LS-SVMlab, which you can find the MATLAB toolbox here. Let's consider I have an independent variable X and a dependent variable Y, that both are simulated. I am following the instructions in the tutorial.
>>X = linspace(-1,1,50)’;
>>Y = (15*(X.^2-1).^2.*X.^4).*exp(-X)+normrnd(0,0.1,length(X),1);
>>type = ’function estimation’;
>>[gam,sig2] = tunelssvm({X,Y,type,[], [],’RBF_kernel’},’simplex’,...’leaveoneoutlssvm’,’mse’});
>>[alpha,b] = trainlssvm({X,Y,type,gam,sig2,’RBF_kernel’});
>>plotlssvm({X,Y,type,gam,sig2,’RBF_kernel’},{alpha,b});
The code above finds the best parameters using simplex method and leave-one-out cross validation and trains the model and give me alphas (support vector values for all the data points in the training set) and b coefficients. However, it does not give me the predictions of the variable Y. It only draws the plot. In some articles, I saw plots like the one below,
As I said before, the LS-SVM toolbox does not give me the predicted values of Y, it only draws the plot but no values in the workspace. How can I get these values and draw a graph of predicted values together with actual values?
There is one solution that I think of. By using X values in the training set, I re-run the model and get the prediction of values Y by using simlssvm command but it does not seem reasonable to me. Any solution that you can offer? Thanks in advance.
I am afraid you have answered your own question. The only way to obtain the prediction for the training points in LS-SVMLab is by simulating the training points after training your model.
[yp,alpha,b,gam,sig2,model] = lssvm(x,y,'f')
when u use this function yp is the predicted value