Is there a simple way of renaming layers in a caffe network by using the pycaffe interface?
I have looked through the net surgery example, but I cannot find an example of what I need.
For example, I would like to load a trained Caffe model and change the name of conv1 layer and its corresponding blob to new-conv1.
I don't know a direct way to do it, but here is a workaround:
Given a pretrained Caffe model my_model.caffemodel and its net architecture net.prototxt. Make a copy of net.prototxt (say net_new.prototxt), and change the name of conv1 layer to new-conv1 (you can change the names of bottom and top if you want).
import caffe
net_old = caffe.Net('net.prototxt','my_model.caffemodel',caffe.TEST)
net_new = caffe.Net('net_new.prototxt','my_model.caffemodel',caffe.TEST)
net_new.params['new-conv1'][0].data[...] = net_old.params['conv1'][0].data[...] #copy filter across 2 nets
net_new.params['new-conv1'][1].data[...] = net_old.params['conv1'][1].data[...] #copy bias
net_new.save('my_model_new.caffemodel')
Related
I am working on Object detection in document images. I have a model working with DETR-Resnet-50 on custom data
I also found out that Layout Parser Faster RCNN - Mask RCNN models use Resnet 50 as a backbone. And since these are already trained on document images, fine tuning these would give better results.
I think one way to do this would be to do like:
model = torch.hub.load('facebookresearch/detr:main', 'detr_resnet50', pretrained=True)
for name, param in model.named_parameters():
print(name, param.shape)
#param.data = weight
and when the layers name seem similar, you change the weights manually.
Now the problem is that Layout Parser models are in detectron2 and my DETR models are in HuggingFace. How can I change the weights of backbone (resnet50) in DETR so that they are initialised with those weights instead of imagenet ones?
I'm using Resnet50 model to classify images into two classes: normal cells and cancer cells.
so I want to to increase the accuracy but i don't know what to modify.
# we are using resnet50 for transfer learnin here. So we have imported it
from tensorflow.keras.applications import resnet50
# initializing model with weights='imagenet'i.e. we are carring its original weights
model_name='resnet50'
base_model=resnet50.ResNet50(include_top=False, weights="imagenet",input_shape=img_shape, pooling='max')
last_layer=base_model.output # we are taking last layer of the model
# Add flatten layer: we are extending Neural Network by adding flattn layer
flatten=layers.Flatten()(last_layer)
# Add dense layer
dense1=layers.Dense(100,activation='relu')(flatten)
# Add dense layer to the final output layer
output_layer=layers.Dense(class_count,activation='softmax')(flatten)
# Creating modle with input and output layer
model=Model(inputs=base_model.inputs,outputs=output_layer)
model.compile(Adamax(learning_rate=.001), loss='categorical_crossentropy', metrics=['accuracy'])
There were 48 errors in 534 test cases Model accuracy= 91.01 %
Also what do you think about the results of the graph?
this is the classification report
i got good results but is there a possibility to increase accuracy more than that?
This is a broad question as there are many ways one can attempt to generally improve the network's accuracy. some of which may be
Increase the dimension of the layers that are learned in transfer learning (make sure not to overfit)
Use transfer learning with Convolution layers and not MLP
let the optimization algorithm choose the learning rate on its own
Play with additional augmentations to the dataset
and the list goes on.
Also, if possible, I would suggest comparing your results to other publicly available benchmarks - by doing so you might understand the upper bounds of the accuracies better
I would like to create a fully convolution network for binary image classification in pytorch that can take dynamic input image sizes, but I don't quite understand conceptually the idea behind changing the final layer from a fully connected layer to a convolution layer. Here and here both state that this is possible by using a 1x1 convolution.
Suppose I have a 16x16x1 image as input to the CNN. After several convolutions, the output is a 16x16x32. If using a fully connected layer, I can produce a single value output by creating 16*16*32 weights and feeding it to a single neuron. What I don't understand is how you would get a single value output by applying a 1x1 convolution. Wouldn't you end up with 16x16x1 output?
Check this link: http://cs231n.github.io/convolutional-networks/#convert
In this case, your convolution layer should be a 16 x 16 filter with 1 output channel. This will convert the 16 x 16 x 32 input into a single output.
Sample code to test:
from keras.layers import Conv2D, Input
from keras.models import Model
import numpy as np
input = Input((16,16,32))
output = Conv2D(1, 16)(input)
model = Model(input, output)
print(model.summary()) # check the output shape
output = model.predict(np.zeros((1, 16, 16, 32))) # check on sample data
print(f'output is {np.squeeze(output)}')
This approach of Fully convolutional networks are useful in segmentation tasks using patch based approaches since you can speed up prediction(inference) by feeding a bigger portion of the image.
For classification tasks, you usually have a fc layer at the end. In that case, a layer like AdaptiveAvgPool2d is used which ensures the fc layer sees a constant input feature size irrespective of the input image size.
https://pytorch.org/docs/stable/nn.html#adaptiveavgpool2d
See this pull request for torchvision VGG: https://github.com/pytorch/vision/pull/747
In case of Keras, GlobalAveragePooling2D. See the example, "Fine-tune InceptionV3 on a new set of classes".
https://keras.io/applications/
I hope you are familier with keras. Now see your image is of 16*16*1. Image will pass to the keras convoloutional layer but first we have to create the model. like model=Sequential() by this we are able to get keras model instance. now we will give our convoloutional layer with our parameters like
model.add(Conv2D(20,(2,2),padding="same"))
now here we are adding 20 filters to our image. and our image becomes 16*16*20 now for more best features we add more conv layers like
model.add(Conv2D(32,(2,2),padding="same"))
now we add 32 filters to your image after this your image will be size of 16*16*32
dont forgot to put activation after conv layers. If you are new than you should study about activations, Optimization and loss of the network. these are the basic part of neural Networks.
Now its time to move towards fully connected layer. First we need to flatten our image because fully connected layer only works on 2d vectors (no_of_ex,image_dim) in your case
imgae diminsion after applying flattening will be (16*16*32)
model.add(Flatten())
after flatening our image your network will give it to fully connected layers
model.add(Dense(32))
model.add(Activation("relu"))
model.add(Dense(8))
model.add(Activation("relu"))
model.add(Dense(2))
because you are having a problem of binary classification if you have to classify 3 classes than last layer will have 3 neuron if you have to classify 10 examples than your last dense layer willh have 10 neuron.
model.add(Activation("softmax"))
model.compile(loss='binary_crossentropy',
optimizer=Adam(),
metrics=['accuracy'])
return model
after this you have to fit this model.
estimator=model()
estimator.fit(X_train,y_train)
full code:
def model (classes):
model=Sequential()
# conv2d set =====> Conv2d====>relu=====>MaxPooling
model.add(Conv2D(20,(5,5),padding="same"))
model.add(Activation("relu"))
model.add(Conv2D(32,(5,5),padding="same"))
model.add(Activation("relu"))
model.add(Flatten())
model.add(Dense(32))
model.add(Activation("relu"))
model.add(Dense(8))
model.add(Activation("relu"))
model.add(Dense(2))
#now adding Softmax Classifer because we want to classify 10 class
model.add(Dense(classes))
model.add(Activation("softmax"))
model.compile(loss='categorical_crossentropy',
optimizer=Adam(lr=0.0001, decay=1e-6),
metrics=['accuracy'])
return model
You can take help from this kernal
I would like to modify the ImageNet caffe model as described bellow:
As the input channel number for temporal nets is different from that
of spatial nets (20 vs. 3), we average the ImageNet model filters of
first layer across the channel, and then copy the average results 20
times as the initialization of temporal nets.
My question is how can I achive the above results? How can I open the caffe model to be able to do those changes to it?
I read the net surgery tutorial but it doesn't cover the procedure needed.
Thank you for your assistance!
AMayer
The Net Surgery tutorial should give you the basics you need to cover this. But let me explain the steps you need to do in more detail:
Prepare the .prototxt network architectures: You need two files: the existing ImageNet .prototxt file, and your new temporal network architecture. You should make all layers except the first convolutional layers identical in both networks, including the names of the layers. That way, you can use the ImageNet .caffemodel file to initialize the weights automatically.
As the first conv layer has a different size, you have to give it a different name in your .prototxt file than it has in the ImageNet file. Otherwise, Caffe will try to initialize this layer with the existing weights too, which will fail as they have different shapes. (This is what happens in the edit to your question.) Just name it e.g. conv1b and change all references to that layer accordingly.
Load the ImageNet network for testing, so you can extract the parameters from the model file:
net = caffe.Net('imagenet.prototxt', 'imagenet.caffemodel', caffe.TEST)
Extract the weights from this loaded model.
conv_1_weights = old_net.params['conv1'][0].data
conv_1_biases = old_net.params['conv1'][1].data
Average the weights across the channels:
conv_av_weights = np.mean(conv_1_weights, axis=1, keepdims=True)
Load your new network together with the old .caffemodel file, as all layers except for the first layer directly use the weights from ImageNet:
new_net = caffe.Net('new_network.prototxt', 'imagenet.caffemodel', caffe.TEST)
Assign your calculated average weights to the new network
new_net.params['conv1b'][0].data[...] = conv_av_weights
new_net.params['conv1b'][1].data[...] = conv_1_biases
Save your weights to a new .caffemodel file:
new_net.save('new_weights.caffemodel')
I'm trying to use Keras's Siamese layer in conjunction with a shared Convolution2D layer.
I don't need the input to pass through any other layers before the Siamese layer but the Siamese layer requires that input layers be specified. I can't figure out how to create the input layers to match the input of the conv layer. The only concrete example of the Siamese layer being used I could find is in the tests where Dense layers (with vector inputs) are used as input. Basically, I want an input layer that allows me to specify the image dimensions as input so they can be passed on to the shared conv layer.
In code I have something like the following:
img_rows = 28
img_cols = 28
img_input_shape = (1, img_rows, img_cols)
shared = Sequential()
shared.add(Convolution2D(nb_filters, nb_conv, nb_conv,
border_mode='valid',
input_shape=img_input_shape))
shared.add(Activation('relu'))
# .... more layers, etc.
right_input_layer = SomeInputLayer(input_shape=img_input_shape) # what should SomeInputLayer be?
left_input_layer = SomeInputLayer(input_shape=img_input_shape)
siamese = Siamese(shared, [left_input_layer, right_input_layer], merge_mode='concat')
model = Sequential()
model.add(siamese)
# ...
model.compile(loss=contrastive_loss, optimizer='rmsprop')
What should SomeInputLayer be? Or is my appraoch in general incorrect?
Okay, I figured it out. The "abstract" Layer class is basically a pass through layer which is just what I need. I was also making a mistake where I thought Siamese could take an entire model (i.e. multiple layers) but it in fact only takes a single layer. To make the creation of these Siamese layers less painful there is a add_shared_layer helper function.
I should also point out this pull request that would allow a shared layer to the first layer in a model, exactly what I am trying to do.