I am tentatively trying to train a deep reinforcement learning model the maze escaping task, and each time it takes one image as the input (e.g., a different "maze").
Suppose I have about 10K different maze images, and the ideal case is that after training N mazes, my model would do a good job to quickly solve the puzzle in the rest 10K - N images.
I am writing to inquire some good idea/empirical evidences on how to select a good N for the training task.
And in general, how should I estimate and enhance the ability of "transfer learning" of my reinforcement model? Make it more generalized?
Any advice or suggestions would be appreciate it very much. Thanks.
Firstly,
I strongly recommend you to use 2D arrays for the maps of the mazes instead of images, it would do your model a huge favor, becuse it's a more feature extracted approach. try using 2D arrays in which walls are demonstrated by ones upon the ground of zeros.
And about finding the optimized N:
Your model architecture is way more important than the share of training data in all of the data or the batch sizes. It's better to make a well designed model and then to find the optimized amount of N by testing different Ns(becuse it is only one variable, the process of optimizing N can be easily done by you yourself).
Related
I am working on a long-term time series (wind speed) forecasting model with different deep learning algorithms. I am using MLP, CNN, and LSTM. I have several questions, and I would appreciate it if you could answer them.
-Do I have to do any preprocessing for seasonality for these deep learning models?
Why is my R-square so bad and sometimes negative?
When I plot the predicted model on the train or test, it is obvious that the model is not good since it is like a straight line and does not capture the trend. However, my evaluation parameters are really good. For example, the RMSE, MAE, and MAPE are 0.77, 0.67, and 0.1, respectively. So is it enough to just report these parameters as many articles have?
And the last one, is it possible to use the proposed model for different datasets? Is it reasonable if I use another city wind speed dataset with a different pattern and trend on this model? Because I have seen many articles that have done it and my models are not working on different datasets.
I am a student and currently studying deep learning by myself. Here I would like to ask for clarification regarding the transfer learning.
For example MobileNetv2 (https://keras.io/api/applications/mobilenet/#mobilenetv2-function), if the weights parameter is set to None, then I am not doing transfer learning as the weights are random initialized. If I would like to do transfer learning, then I should set the weights parameter to imagenet. Is this concept correct?
Clarification and explanation regarding deep learning
Yes, when you initialize the weights with random values, you are just using the architecture and training the model from scratch. The goal of transfer learning is to use the previously gained knowledge by another trained model to get better results or to use less computational resources.
There are different ways to use transfer learning:
You can freeze the learned weights of the base model and replace the last layer of the model base on your problem and just train the last layer
You can start with the learned weights and fine-tune them (let them change in the learning process). Many people do that because sometimes it makes the training faster and gives better results because the weights already contain so much information.
You can use the first layers to extract basic features like colors, edges, circles... and add your desired layers after them. In this way, you can use your resources to learn high-level features.
There are more cases, but I hope it could give you an idea.
I am new to deep learning (I just finished to read deep learning with pytorch), and I was wondering what is the best neural network architecture for my case.
I have a large multiclass classification problem (user identification problem), about 1000 classes in which each class is a user. I have about 2000 features for each user after one-hot encoding and cleaning. Data are highly imbalanced, but I can always use oversampling/downsampling techniques.
I was wondering what is the best architecture to implement for my case. I've always seen deep learning applied to time series or images, so I'm not sure about what to use in this case. I was thinking about a multi-layer perceptron but maybe there are better solutions.
Thanks for your tips and help. Have a nice day!
You can try triplet learning instead of simple classification.
From your 1000 users, you can make, c * 1000 * 999 / 2 pairs. c is the average number of samples per class/user.
https://arxiv.org/pdf/1412.6622.pdf
When using the reinforcement learning model ddpg, the input data are sequence data, high-dimensional (21 dimensional) state and low dimensional (1-dimensional) action. Does this have any negative impact on the training of the model? How to solve it
In general in any machine learning scenario, dimensionality per se is not a problem, it is mostly a matter of how much variability there is the input data. Of course, higher dimensional data can have much higher variability than lower dimensional one.
Even considering this, the problem can "easily" be solved by feeding more data to the ML algorithm and increasing the complexity that it is allowed to represent (i.e. more nodes and/or layers in a neural network).
In RL, this is even less of a problem because you don't really have a restriction on how much data you actually have. You can always run your agent some more on the environment to get more sample trajectories to train on. The only issue you might find here is that your computing time grows a lot (depending on how much more you need to train on the environment for this problem).
I am trying to train a learning model to recognize one specific scene. For example, say I would like to train it to recognize pictures taken at an amusement park and I already have 10 thousand pictures taken at an amusement park. I would like to train this model with those pictures so that it would be able to give a score for other pictures of the probability that they were taken at an amusement park. How do I do that?
Considering this is an image recognition problem, I would probably use a convolutional neural network, but I am not quite sure how to train it in this case.
Thanks!
There are several possible ways. The most trivial one is to collect a large number of negative examples (images from other places) and train a two-class model.
The second approach would be to train a network to extract meaningful low-dimensional representations from an input image (embeddings). Here you can use siamese training to explicitly train the network to learn similarities between images. Such an approach is employed for face recognition, for instance (see FaceNet). Having such embeddings, you can use some well-established methods for outlier detections, for instance, 1-class SVM, or any other classifier. In this case you also need negative examples.
I would heavily augment your data using image cropping - it is the most obvious way to increase the amount of training data in your case.
In general, your success in this task strongly depends on the task statement (are restricted to parks only, or any kind of place) and the proper data.