I am writing an encoder/decoder model very similar to https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html
The only difference is, here, the words are represented by some indices. I want to show them based on another metric, which are represented by flaot numbers.
The loss function nn.criterion = nn.NLLLoss(), seems to be working for times we are only workin with classes.
If my output array is not an array of integers, but an array of float numbers, what kind of loss function I can use? Considering all other parts are similar to the tutorial?
Thanks in advance.
Related
I have dataset (sequence to sequence), each sample input is seq of charterers (combination from from 20 characters and max length 2166) and out is list of charterers (combination of three characters G,H,B). for example OIREDSSSRTTT ----> GGGHHHHBHBBB
I would like to do simple pytorch model that work in that type of dataset. Model that can predict sequence of classes. I would appreciate any suggestions or links for simple mode that do the same?
Thanks
If the output sequence always has the same length as the input sequence, you might want to use transformer encoder, because it basically transforms the inputs with attention to the context. Also you can try to use anything that is used to tagging: BiLSTM, BiGRU, etc.
If you want your model to be able to predict sequences of different length (not necessary the same as input length), look at some encoder-decoder models, such as vanilla transformer.
You can start with the sequence tagging model from PyTorch tutorial https://pytorch.org/tutorials/beginner/nlp/sequence_models_tutorial.html .
As #Ilya Fedorov said, you can move to transformer models for potentially better performance.
I need to map a series of characters to integers, and I already know which characters map into which integers. I want to interpret that as a function from naturals to naturals (using the character's ASCII codes). What would be the best way to find that function?
Additional information: the function is constant throughout the program, the number of given points I have is small (less than 10), the integers I'm mapping to are sequential from 0 (that is, indexes of an array).
Obs. I am asking this question here because I think this has to be found by a program, but maybe I should be asking on the math site? Thank you.
If I use
float sum = thrust::transform_reduce(d_a.begin(), d_a.end(), conditional_operator(), 0.f, thrust::plus<float>());
I get the sum of all elements meeting a condition provided by conditional_operator(), as in Conditional reduction in CUDA.
But what can I sum only the elements d_a[0], d_a[2], d_a[4], d_a[6], ..... ?
I thought of changing the conditional operator, but it works on on elements in the array without any reference to the index.
What can I do for that?
There are two approaches I can think of for solving this sort of problem:
Use the thrust zip operator to combine a counting iterator with the input data and modify your existing functor to accept tuples of (index, data). You can have the functor return the data when the index matches your criteria, and zero otherwise. This will work correctly with scan and reduction algorithms
Use a thrust permutation iterator to gather the data which you want to sum and pass it to the standard reduce algorithm. The thrust developers have an example strided iterator which you can use to solve the problem of only processing every nth entry in an input iterator.
It might be worth implemented both and benchmarking them to see which approach is faster.
I need to write my own function which has the form f(x,y)=Integrate(g(x,y,z),z from 0 to inf). so the code I used was:
function y=f(x,y)
g=#(z)exp(-z.^2)./(z.^x).*(z.^2+y.^2).^(x/2);% as a function of x,y and z
y=quadgk(g,0,inf)
and if I call it for a single value like f(x0,y0), it works but if I try to calculate something like f([1:10],y0), then the error message says that there is something wrong with the times and dimension. In principle I can use for loops but then my code slows down and takes forever. Is there any help I can get from you guys? or references?
I'm trying to avoid the for loop since in matlab it's much faster to use matrix computation than to use for loop. I wonder if there is any trick that I can take advantage of this feature.
Thanks for any help in advance,
Lynn
Perhaps you can try to transpose the interval, creating row based values instead of column based f([1:10]',y0). Otherwise something in your function might be wrong, for example to get x^y to work with lists as input, you have to prefix with a dot x.^y. The same for mulitply and division I think..
If loop is no problem for you, you should do something like:
function y2=f(x,y)
y2=zeros(size(x));
for n=1:numel(x)
g=#(z)exp(-z.^2)./(z.^x(n)).*(z.^2+y.^2).^(x(n)/2);% as a function of x,y and z
y2(n)=quadgk(g,0,inf)
end
The problem here is that quadk itself uses vectors as argument for g. Then you have in g somethink like z.^x, which is the power of two vectors that is only defined if z and x have the same dimension. But this is not what you want.
I assume that you want to evaluate the function for all arguments in x and that the output vector has the same dimension as x. But this does not seem to be possible since even this simple example
g=#(x)[x;x.^2]
quad(g,0,1)
does not work:
Error using quad (line 79)
The integrand function must return an output vector of the same length as the
input vector.
A similar error shows when using quadgk. The documentation also says that this routine works only for scalar functions and this is not surprising since an adaptive quadrature rule would in general use different points for each function to evaluate the integral.
You have to use quadvinstead, which can integrate vector valued functions. But this gives wrong results since your function is integrated in the interval [0,\infty).
How to find the following Maximum or supremum by computer software such as Mathematica and Matlab: $\sup\frac{(1+s)^{4}+(s+t)^{4}+t^{4}}{1+s^{4}+t^{4}}$?
Instead of numerical approximation, what is the accurate maximum?
Thanks.
Since the question seems a bit like homework, here's an answer that starts a bit like a lecture:
ask yourself what happens to the function as s and t go to small and to large positive and negative values; this will help you to identify the range of values you should be examining; both Mathematica and Matlab can help your figure this out;
draw the graph of your function over the range of values of interest, develop a feel for its shape and try to figure out where it has maxima; for this the Mathematic Plot3D[] function and the Matlab plot() function will both be useful;
since this is a function of 2 variables, you should think about plotting some of its sections, ie hold s (or t) constant, and make a 2D plot of the section function; again, develop some understanding of how the function behaves;
now you should be able to do some kind of search of the s,t values around the maxima of the function and get an acceptably accurate result.
If this is too difficult then you could use the Mathematica function NMaximize[]. I don't think that Matlab has the same functionality for symbolic functions built-in and you'll have to do the computations numerically but the function findmax will help.
In Matlab, one would create a vector/matrix with s and t values, and a corresponding vector with the function values. Then you can pinpoint the maximum using the function max
In Mathematica, use FindMaximum like this:
f[s_,t_]:= ((1+s)^4 + (s+t)^4 + t^4)/(1+s^4+t^4)
FindMaximum[ f[s,t],{s,0},{t,0} ]
This searches for a maximum starting from (s,t)=(0,0).
For more info, see http://reference.wolfram.com/mathematica/ref/FindMaximum.html