When I finished training the YOLO V5 model, evaluation metrics such as precision, recall, and AP (average precision) of the model on the validation data were shown. Is it possible to view those evaluation metrics from a trained model against a new images? (note: the new image has its annotation file)
Sorry if this is a dumb question. Thanks!
To evaluate on you existing data:
python val.py --weights yolov5s.pt --data your_dataset.yaml --img 640
To evaluate on new data:
python val.py --weights yolov5s.pt --data your_new_dataset.yaml --img 640
Related
For language identification, I am using the following tutorial :
Fasttext language detection tutorial
After executing the command as in tutorial:
./fasttext test langdetect.bin valid.txt
I have the following the output:
N 10000
P#1 0.967
R#1 0.967
after this, which commands will predict the language? how to enter the text in other languages?
I am very new to this language detection. I could find ample tutorials for python prediction but not in linux command line.
Thanks in advance.
Language detection is a particular case of text classification using supervised models (here you can find the tutorial).
According to the tutorial, you can predict on new examples, by typing:
./fasttext predict-prob langdetect.bin - -1 0.5
(we want as many prediction as possible (argument -1) and we want only labels with probability higher or equal to 0.5)
and then typing the sentence.
If you have a txt file with sentences to be classified, you can type:
$ ./fasttext predict-prob langdetect.bin test.txt k
where k is the number of classes to show.
This cheatsheet may also be useful.
Question: How do I print/return the softmax layer for a multiclass problem using Keras?
my motivation: it is important for visualization/debugging.
it is important to do this for the 'training' setting. ergo batch normalization and dropout must behave as they do in train time.
it should be efficient. calling vanilla model.predict() every now and then is less desirable as the model I am using is heavy and this is extra forward passes. The most desirable case is finding a way to simply display the original network output which was calculated during training.
it is ok to assume that this is done while using Tensorflow as a backend.
Thank you.
You can get the outputs of any layer by using: model.layers[index].output
For all layers use this:
from keras import backend as K
inp = model.input # input placeholder
outputs = [layer.output for layer in model.layers] # all layer outputs
functor = K.function([inp]+ [K.learning_phase()], outputs ) # evaluation function
# Testing
test = np.random.random(input_shape)[np.newaxis,...]
layer_outs = functor([test, 1.])
print layer_outs
Background
I used Python and Keras to implement the model of [1].
The model's structure is described in Fig.3 of this paper:
[1] Zheng, Y.: Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks, 2014
Problem
The trained model gives predictions of only 1 class out of 4 classes. For example, [3,3,3,...,3] (= all 3's)
My code at Github
Run main_q02.py
The model is defined in mcdcnn_3.py
Utility functions are defined in utils.py and PAMAP2Utils.py
Dataset Download
The code requires only two files:
PAMAP2_Dataset/Protocol/subject101.dat
PAMAP2_Dataset/Protocol/subject102.dat
About the dataset
The dataset classes are NOT balanced.
class: 0, 1, 2, 3
number of samples (%): 28.76%, 36.18%, 18.42%, 16.64%
Note: computed over all 7 subjects
Does one dominate? Classes 0 and 1 dominate around 65% of all samples.
class 0: 28.76%
class 1: 36.18%
Additional details
Operating system: Ubuntu 14.04 LTS
Version of python packages:
Theano (0.8.2)
Keras (1.1.0)
numpy (1.13.0)
pandas (0.20.2)
Details of the model (from the paper):
"separate multivariate time series into univariate ones and perform feature learning on each univariate series individually." [1]
"adopt sigmoid function in all activation layers" [1]
"utilize average pooling without overlapping" [1]
use stochastic gradient descent (SGD) for learning
parameters: momentum = 0.9, decay = 0.0005, learning rate = 0.01
I am fairly new to keras and DL and I am trying to build a loss function but I have questions about how the data from my network is passed through y_pred and y_true of the loss function.
As an example, my network has 3 different outputs here is one:
SEC5 = merge( [SEC1_up, SEC2_up, SEC3_up, SEC4_up], mode='concat', concat_axis=1 )
SEC5 = Convolution2D( 2,1,1, subsample=(1, 1), border_mode='same', activation="sigmoid" )( SEC5 )
SEC5 is now a 2 channel tensor that is predicting edges in one channel and non-edges in the other.
My model is created with the following line:
model = Model( input=inputs, output=[Final, ILLP2, SEC1, SEC2, SEC3, SEC4, SEC5] )
Where I perform binary cross entropy on Final, Squared loss on ILLP2, and then a custom loss for each of the SEC layers. When building the custom loss I have come across something that I don't understand. How are multiple channel layers (like SEC5) passed to the loss function? This is particularly important in my edge loss as I need to calculate the number of edges in the edge layer, and the number of non edges in the non edge layer.
What I don't understand is the actual variable in the loss function (y_true and y_pred) when I do this:
print 'y_true data'
print y_true.ndim
print y_true.type
print 'y_pred data'
print y_pred.ndim
print y_pred.type
I get the following values:
y_true data
2
TensorType(float32, matrix)
y_pred data
2
TensorType(float32, matrix)
And this is where i get really confused by everything. As I understand it, tensortypes of matrix can only be 2 dimensional, but I essentially have 3 dimensions? How does it deal with this information?
I feel like I should understand this before I go making elaborate loss functions of my own, any information you could provide me with would be greatly appreciated.
Cheers,
Michael
I am using Pylearn2 OR Caffe to build a deep network. My target is ordered nominal. I am trying to find a proper loss function but cannot find any in Pylearn2 or Caffe.
I read a paper "Loss Functions for Preference Levels: Regression with Discrete Ordered Labels" . I get the general idea - but I am not sure I understand what will the thresholds be, if my final layer is a SoftMax over Logistic Regression (outputting probabilities).
Can some help me by pointing to any implementation of such a loss function ?
Thanks
Regards
For both pylearn2 and caffe, your labels will need to be 0-4 instead of 1-5...it's just the way they work. The output layer will be 5 units, each is a essentially a logistic unit...and the softmax can be thought of as an adaptor that normalizes the final outputs. But "softmax" is commonly used as an output type. When training, the value of any individual unit is rarely ever exactly 0.0 or 1.0...it's always a distribution across your units - which log-loss can be calculated on. This loss is used to compare against the "perfect" case and the error is back-propped to update your network weights. Note that a raw output from PL2 or Caffe is not a specific digit 0,1,2,3, or 5...it's 5 number, each associated to the likelihood of each of the 5 classes. When classifying, one just takes the class with the highest value as the 'winner'.
I'll try to give an example...
say I have a 3 class problem, I train a network with a 3 unit softmax.
the first unit represents the first class, second the second and third, third.
Say I feed a test case through and get...
0.25, 0.5, 0.25 ...0.5 is the highest, so a classifier would say "2". this is the softmax output...it makes sure the sum of the output units is one.
You should have a look at ordinal (logistic) regression. This is the formal solution to the problem setup you describe ( do not use plain regression as the distance measures of errors are wrong).
https://stats.stackexchange.com/questions/140061/how-to-set-up-neural-network-to-output-ordinal-data
In particular I recommend looking at Coral ordinal regression implementation at
https://github.com/ck37/coral-ordinal/issues.