I'm using the Caffenet model. I did a classification task with 9 classes successfully. Then I tried to change it to a regression network by preparing another LMDB file with labels ranging from 700 to 1400. I changed the original training code and replaced the softmax with EuclideanLoss and num_outputs to 1. I did the same for testing and got this error:
"Check failed: ExactNumBottomBlobs() == bottom.size() (2 vs. 1) EuclideanLoss Layer takes 2 bottom blob(s) as input. * Check failure stack trace: * Aborted (core dumped)"
So I commented out EuclideanLoss layer:
layer { name: "prob" type: "EuclideanLoss" bottom: "fc8-cats-dogs-n" top: "prob" }
but now I get:
File "testr.py", line 86, in pred_probas = out['prob'] KeyError: 'prob'
Can anyone help me with this please?
Related
I am trying to do pixel-wise classification with caffe, so need to provide a ground truth image the size of the input image. There is several ways of doing this, and I decided to set up my input as a 4-channel LMDB (according to the 2nd point of this answer). This requires me to add a Slice layer after my input, which is also outlined in the same answer.
I keep getting Unknown blob input data_lmdb to layer 0 as an error message (data_lmdb is supposed to be my very bottom input layer). I found that unknown blob (be it top or bottom) error is mostly caused by forgetting to define something in one of the TRAIN / TEST phases while defining it in the other (e.g. this question, or this one). But, I am using a combination of train.prototxt, inference.prototxt and solver.prototxt files that I have previously used, just replacing the input layers from HD5 to LMDB (for a bit of practice), so everything should be defined.
Can anybody see why I am getting the Unknown blob input data_lmdb to layer 0 error? From the train log files I can see that it crashes as soon as it reads the train.prototxt file (it doesn't even reach the Creating layer part).
My prototxt files are as follows:
solver.prototxt
net: "train.prototxt" # Change this to the absolute path to your model file
test_initialization: false
test_iter: 1
test_interval: 1000000
base_lr: 0.01
lr_policy: "fixed"
gamma: 1.0
stepsize: 2000
display: 20
momentum: 0.9
max_iter: 10000
weight_decay: 0.0005
snapshot: 100
snapshot_prefix: "set_snapshot_name" # Absolute path to output solver snapshots
solver_mode: GPU
train.prototxt (first two layers only; they are followed by a LNR normalization layer and then a Convolution layer):
name: "my_net"
layer {
name: "data_lmdb"
type: "Data"
top: "slice_input"
data_param {
source: "data/train"
batch_size: 4
backend: LMDB
}
}
layer{
name: "slice_input"
type: "Slice"
bottom: "data_lmdb" # 4-channels = rgb+truth
top: "data"
top: "label"
slice_param {
axis: 1
slice_point: 3
}
}
The first few layer definitions in inference.prototxt are identical to train.prototxt (which shouldn't matter anyway as it is not used in training) except the following:
in data_lmdb the source path is different (data/test)
in data_lmdb layer uses batch_size: 1
Please do let me know if I need to include any more information or layers. I was trying to keep it brief, which didn't really work out in the end.
The message Unknown blob input points on non-existent blob that some layer wants to have as input. Your slice_input layer specified data_lmdb as input blob, but there is no such a blob in your network. Instead, you have a layer with such a name. Blob names are defined by the top field, which is slice_input in this case.
You shoud either change top: "slice_input" to top: "data_lmdb" in your data_lmdb layer, or use bottom: "slice_input" # 4-channels = rgb+truth.
However, for more clear naming I would offer you the following:
name: "my_net"
layer {
name: "data"
type: "Data"
top: "data_and_label"
data_param {
source: "data/train"
batch_size: 4
backend: LMDB
}
}
layer{
name: "slice_input"
type: "Slice"
bottom: "data_and_label" # 4-channels = rgb+truth
top: "data"
top: "label"
slice_param {
axis: 1
slice_point: 3
}
}
I'm trying to create a single multi-class and multi-label net configuration in caffe.
Let's say classification of dogs:
Is the dog small or large? (class)
What color is it? (class)
is it have a collar? (label)
Is this thing possible using caffe?
What is the proper way to do so?
What is the right way to build the lmdb file?
All the publications about multi-label classification are from around 2015, something in this subject changed since then?
Thanks.
The problem with Caffe's LMDB interface is that it only allows for a single int label per image.
If you want multiple labels per image you'll have to use a different input layer.
I suggest using "HDF5Data" layer:
This allows for more flexibility setting the input data, you may have as many "top"s as you want for this layer. You may have multiple labels per input image and have multiple losses for your net to train on.
See this post on how to create hdf5 data for caffe.
Thanks Shai,
Just trying to understand the practical way..
After creating 2 .text files (one for training and one for validation) containing all the tags of the images, for example:
/train/img/1.png 0 4 18
/train/img/2.png 1 7 17 33
/train/img/3.png 0 4 17
Running the py script:
import h5py, os
import caffe
import numpy as np
SIZE = 227 # fixed size to all images
with open( 'train.txt', 'r' ) as T :
lines = T.readlines()
# If you do not have enough memory split data into
# multiple batches and generate multiple separate h5 files
X = np.zeros( (len(lines), 3, SIZE, SIZE), dtype='f4' )
y = np.zeros( (len(lines),1), dtype='f4' )
for i,l in enumerate(lines):
sp = l.split(' ')
img = caffe.io.load_image( sp[0] )
img = caffe.io.resize( img, (SIZE, SIZE, 3) ) # resize to fixed size
# you may apply other input transformations here...
# Note that the transformation should take img from size-by-size-by-3 and transpose it to 3-by-size-by-size
# for example
transposed_img = img.transpose((2,0,1))[::-1,:,:] # RGB->BGR
X[i] = transposed_img
y[i] = float(sp[1])
with h5py.File('train.h5','w') as H:
H.create_dataset( 'X', data=X ) # note the name X given to the dataset!
H.create_dataset( 'y', data=y ) # note the name y given to the dataset!
with open('train_h5_list.txt','w') as L:
L.write( 'train.h5' ) # list all h5 files you are going to use
And creating train.h5 and val.h5 (is X data set containing the images and Y contain the labels?).
Replace my network input layers from:
layers {
name: "data"
type: DATA
top: "data"
top: "label"
data_param {
source: "/home/gal/digits/digits/jobs/20181010-191058-21ab/train_db"
backend: LMDB
batch_size: 64
}
transform_param {
crop_size: 227
mean_file: "/home/gal/digits/digits/jobs/20181010-191058-21ab/mean.binaryproto"
mirror: true
}
include: { phase: TRAIN }
}
layers {
name: "data"
type: DATA
top: "data"
top: "label"
data_param {
source: "/home/gal/digits/digits/jobs/20181010-191058-21ab/val_db"
backend: LMDB
batch_size: 64
}
transform_param {
crop_size: 227
mean_file: "/home/gal/digits/digits/jobs/20181010-191058-21ab/mean.binaryproto"
mirror: true
}
include: { phase: TEST }
}
to
layer {
type: "HDF5Data"
top: "X" # same name as given in create_dataset!
top: "y"
hdf5_data_param {
source: "train_h5_list.txt" # do not give the h5 files directly, but the list.
batch_size: 32
}
include { phase:TRAIN }
}
layer {
type: "HDF5Data"
top: "X" # same name as given in create_dataset!
top: "y"
hdf5_data_param {
source: "val_h5_list.txt" # do not give the h5 files directly, but the list.
batch_size: 32
}
include { phase:TEST }
}
I guess HDF5 doesn't need a mean.binaryproto?
Next, how the output layer should change in order to output multiple label probabilities?
I guess I need cross- entropy layer instead of softmax?
This is the current output layers:
layers {
bottom: "prob"
bottom: "label"
top: "loss"
name: "loss"
type: SOFTMAX_LOSS
loss_weight: 1
}
layers {
name: "accuracy"
type: ACCURACY
bottom: "prob"
bottom: "label"
top: "accuracy"
include: { phase: TEST }
}
I'm trying to feed a CSV file to Kur, but I don't know how to specify more than one column in the input without the program crashing. Here's a small example:
model:
- input:
- SepalWidthCm
- SepalLengthCm
- dense: 10
- activation: tanh
- dense: 3
- activation: tanh
name: Species
train:
data:
- csv:
path: Iris.csv
header: yes
epochs: 1000
weights: best.w
log: tutorial-log
loss:
- target: Species
name: mean_squared_error
The error:
File "/Users/bytter/.pyenv/versions/3.5.2/bin/kur", line 11, in <module>
sys.exit(main())
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/__main__.py", line 269, in main
sys.exit(args.func(args) or 0)
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/__main__.py", line 48, in train
func = spec.get_training_function()
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/kurfile.py", line 282, in get_training_function
model = self.get_model(provider)
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/kurfile.py", line 148, in get_model
self.model.build()
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/model/model.py", line 282, in build
self.build_graph(input_nodes, output_nodes, network)
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/model/model.py", line 356, in build_graph
for layer in node.container.build(self):
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/containers/container.py", line 281, in build
self._built = list(self._build(model))
File "/Users/bytter/.pyenv/versions/3.5.2/lib/python3.5/site-packages/kur/containers/layers/placeholder.py", line 122, in _build
'Placeholder "{}" requires a shape.'.format(self.name))
kur.containers.parsing_error.ParsingError: Placeholder "..input.0" requires a shape.
Using - input: SepalWidthCm works as expected.
The problem with your approach is that Kur doesn't know how you want the inputs concatenated. Should your input become 2D tensor of dimensions (2, N) (where N is the number of data points in your CSV file), like this?
[
[SepalWidthCm_0, SepalWidthCm_1, ...],
[SepalLengthCm_0, SepalLengthCm_1, ...]
]
(N.B., that example isn't a very deep-learning friendly structure.) Or should it be combined into a tensor of dimensions (N, 2), like this?
[
[SepalWidthCm_0, SepalLengthCm_0],
[SepalWidthCm_1, SepalLengthCm_1],
...
]
Or maybe you want to apply the same operations to each column in parallel? Regardless, this problem gets a lot harder / more ambiguous to answer when your input data is multi-dimensional (e.g., instead of scalars like length or width, you have vectors or even matrices).
Instead of trying to guess what you want (and possibly getting it wrong), Kur expects each input to be a single data source, which you can then combine however you see fit.
Here are a couple ways you might want your data combined, and how to do it in Kur.
Row-wise Combination. This is the second example above, where we want to combine "rows" of CSV data into tuples, so that the input tensor has dimensionality (batchSize, 2). Then your Kur model would look like:
model:
# Define the model inputs.
- input: SepalWidthCm
- input: SepalLengthCm
# Concatenate the inputs.
- merge: concat
inputs: [SepalWidthCm, SepalLengthCm]
# Do processing on these "vectorized" inputs.
- dense: 10
- activation: tanh
- dense: 1
- activation: tanh
# Output
- output: Species
Independent Processing, and then Combining. This is the setup where you do some operations on each input column independently, and then you merge them together (potentially with some more operations afterwards). In ASCII-art, this might look like:
INPUT_1 --> dense, activation --\
+---> dense, activation --> OUTPUT
INPUT_2 --> dense, activation --/
In this case, you would have a Kur model that looks like this:
model:
# First "branch" of processing.
- input: SepalWidthCm
- dense: 10
- activation: tanh
name: WidthBranch
# Second "branch" of processing.
- input: SepalLengthCm
- dense: 10
- activation: tanh
name: LengthBranch
# Fuse things together.
- merge:
inputs: [WidthBranch, LengthBranch]
# Continue some processing
- dense: 1
- activation: tanh
# Output
- output: Species
Keep in mind that the merge layer has been around since Kur 0.3, so make sure you using a recent version.
(Disclaimer: I am the core maintainer of Kur.)
I am using caffe and it doesn't have a locally connected layer. So any example on how to use im2col layer, reshape layer and inner product layer to implement locally connected layer? Thanks
Personal View of Point:
I have also tried to use Crop, Im2col, Reshape and InnerProduct layer to implement locally connected layer but failed.
Because when I want to implement a convolution operation using InnerProduct layer, I find that in InnerProductLayer<Dtype>::Forward_cpu() function:
caffe_cpu_gemm<Dtype>(CblasNoTrans, transpose_ ? CblasNoTrans : CblasTrans,
M_, N_, K_, (Dtype)1.,
bottom_data, weight, (Dtype)0., top_data);
and in BaseConvolutionLayer<Dtype>::forward_cpu_gemm() function:
caffe_cpu_gemm<Dtype>(CblasNoTrans, CblasNoTrans, conv_out_channels_ /
group_, conv_out_spatial_dim_, kernel_dim_,
(Dtype)1., weights + weight_offset_ * g, col_buff + col_offset_ * g,
(Dtype)0., output + output_offset_ * g);
the weight(s), which should be used as convolution kernels, are passed to different arguments of caffe_cpu_gemm().
So I can't implement a convolution operation using InnerProductLayer<Dtype>::Forward_cpu() function and thus can't implement a local connected layer(I mean local convolution here) using Crop, Im2col, Reshape and InnerProduct layers.
My solution:
However, I implemented a local convolution layer here and its idea is to divide input feature maps into N*N grid(even with overlap) and performs convolution on each of the grid using different kernels. For example, the input feature maps have a shape (2, 3, 8, 8) and you want to divide the spatial feature map 8*8 into 16 2*2 local regions and then perform convolution on each local region with different bank of kernels, you can write a prototxt like this:
layer {
name: "local_conv"
type: "LocalConvolution"
bottom: "bottom" # shape (2,3,8,8)
top: "top"
param {
lr_mult: 1
decay_mult: 1
}
param {
lr_mult: 2
decay_mult: 0
}
local_conv_param {
local_region_number_h: 4
local_region_number_w: 4
local_region_ratio_h: 0.3 # determin the height/width of local regions
local_region_ratio_w: 0.3 # local_region_size = floor(local_region_ratio * input_size)
local_region_step_h: 2 # step between local regions on the top left part
# and other regions will lie in the axial symmetry positions
# automatically
local_region_step_w: 2
num_output: 5
kernel_h: 3
kernel_w: 1
stride: 1
pad: 0
weight_filler {
type: "xavier"
}
bias_filler {
type: "constant"
}
}
}
You can easily add this layer to your caffe and the related files are:
include/caffe/layers/local_conv_layer.hpp
src/caffe/layers/local_conv_layer.cpp(cu)
and you should also add message LocalConvolutionParameter, optional LocalConvolutionParameter local_conv_param from src/caffe/proto/caffe.proto to your caffe.proto.
.
I am trying to compute the test loss in my own training loop in python. Calling solver.test_nets[0].forward() seems to update the score blob but not the loss one. Any idea how to get it updated?
I am using the following solver config:
net: "/tmp/tmp8ikb9sg2/train.prototxt"
test_net: "/tmp/tmp8ikb9sg2/test.prototxt"
test_iter: 1
test_interval: 2147483647
base_lr: 0.1
lr_policy: "fixed"
test_initialization: false
and train and test.prototxt are exactly the same except for the phase definition at the top of the file:
name: "pycaffenet"
state {
phase: TRAIN # set TEST in test.prototxt
}
...
layer {
name: "loss"
type: "SoftmaxWithLoss"
bottom: "score"
bottom: "output"
top: "loss"
}
It was actually a different issue that what I thought. The loss blob was being updated but it remained the same because the weights of solver.test_nets[0] were not changing. It looks like they not automatically shared with solver.net. This can be done by simply calling:
solver.test_nets[0].share_with(solver.net)