I'm trying to extract the features of images using VGG19 network (the output should be of dim : [1 , 7 , 7 , 512] per frame
here is the code I have used :
deep_net = models.vgg19(pretrained=True).cuda()
deep_net = nn.Sequential(*list(deep_net.children())[:-2])
deep_net.eval()
save_file_sample_path = '/media/data1/out.npy'
input_image = torch.zeros(1, 3, 224, 224)
output_feat = np.zeros(shape=[1, 49, 512])
with torch.no_grad():
im = default_loader('/media/data1/images/frame612.jpg')
im = transform(im)
input_image[0, :, :] = im
input_image = input_image.cuda()
output_feat = deep_net(input_image)
output_feat = output_feat.features[:-2].view(1, 512, 49).transpose(1, 2)
But I get the following error :
AttributeError: 'Sequential' object has no attribute 'features'
At the line :
output_feat = output_feat.features[:-2].view(1, 512, 49).transpose(1, 2)
Any idea why this does not work anymore? and how to fix?
Thanks!
It's because you are rebuilding deep_net with nn.Sequential so it loses the attribute features.
deep_net = models.vgg19(pretrained=True)
deep_net.features
Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(1): ReLU(inplace=True)
...
(36): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
)
deep_net = nn.Sequential(*list(deep_net.children())[:-2])
deep_net.features
AttributeError: 'Sequential' object has no attribute 'features'
The equivalent you want now is this:
list(deep_net.children())[0][:-2]
Related
I am trying to implement the dot product and general implementation of calculating similarity scores from encoder and decoder output and hidden states respectively in keras.
I have got the idea to do the product of tf.keras.layers.dot(encoder_output,decoder_state) for calculating product score but there is error in multiplication of these two values.
class Attention(tf.keras.Model):
def __init__(self,units):
super().__init__()
self.units = units
def call(self, decoder_state, encoder_output):
score = tf.keras.layers.dot([encoder_output,decoder_state], axes=[2, 1])
attention_weights = tf.nn.softmax(score, axis=1)
context_vector = attention_weights * encoder_output
context_vector = tf.reduce_sum(context_vector, axis=1)
return context_vector, attention_weights
batch_size = 16
units = 32
input_length = 20
decoder_state = tf.random.uniform(shape=[batch_size, units])
encoder_output = tf.random.uniform(shape=[batch_size, input_length, units])
attention = Attention(units)
context_vector, attention_weights = attention(decoder_state, encoder_output)
I am getting the following error:
/usr/local/lib/python3.6/dist-packages/six.py in raise_from(value, from_value)
InvalidArgumentError: Incompatible shapes: [16,20] vs. [16,20,32] [Op:Mul]
It is a very simple fix but as I am new to this I am not able to get the exact method needed to be called here.
I have tried reshaping the values of encoder_output but still this does not work.
Request to help me fix this.
I am just putting #Ayush Srivastava's comment as a response so that the post gets an answer.
Basically, the error occurs because you are trying to multiply 2 tensors (namely attention_weights and encoder_output) with different shapes, so you need to reshape the decoder_state.
Here is the full answer:
class Attention(tf.keras.Model):
def __init__(self,units):
super().__init__()
self.units = units
def call(self, decoder_state, encoder_output):
decoder_state = tf.keras.layers.Reshape((decoder_state.shape[1], 1))(decoder_state)
score = tf.keras.layers.dot([encoder_output, decoder_state],[2, 1])
attention_weights = tf.nn.softmax(score, axis=1)
context_vector = attention_weights * encoder_output
context_vector = tf.reduce_sum(context_vector, axis=1)
return context_vector, attention_weights
Shapes:
decoder_state before reshape: (16, 32)
decoder_state after reshape: (16, 32, 1)
enc_output: (16, 20, 32)
score: (16, 20, 1)
attention_weights: (16, 20, 1)
context_vector before sum: (16, 20, 32)
I Want to Combine Two CNN Into Just One In Keras, What I Mean Is that I Want The Neural Network To Take Two Images And Process Each One in Separate CNN, and Then Concatenate Them Together Into The Flattening Layer and Use Fully Connected Layer to Do The Last Work, Here What I Did:
# Start With First Branch ############################################################
branch_one = Sequential()
# Adding The Convolution
branch_one.add(Conv2D(32, (3,3),input_shape = (64,64,3) , activation = 'relu'))
branch_one.add(Conv2D(32, (3, 3), activation='relu'))
# Doing The Pooling Phase
branch_one.add(MaxPooling2D(pool_size=(2, 2)))
branch_one.add(Dropout(0.25))
branch_one.add(Flatten())
# Start With Second Branch ############################################################
branch_two = Sequential()
# Adding The Convolution
branch_two.add(Conv2D(32, (3,3),input_shape = (64,64,3) , activation = 'relu'))
branch_two.add(Conv2D(32, (3, 3), activation='relu'))
# Doing The Pooling Phase
branch_two.add(MaxPooling2D(pool_size=(2, 2)))
branch_two.add(Dropout(0.25))
branch_two.add(Flatten())
# Making The Combinition ##########################################################
final = Sequential()
final.add(Concatenate([branch_one, branch_two]))
final.add(Dense(units = 128, activation = "relu"))
final.add(Dense(units = 1, activation = "sigmoid"))
# Doing The Compilation
final.compile(loss = 'binary_crossentropy', optimizer = 'adam', metrics = ['accuracy'])
# Adding and Pushing The Images to CNN
# use ImageDataGenerator to preprocess the data
from keras.preprocessing.image import ImageDataGenerator
# augment the data that we have
train_datagen = ImageDataGenerator(rescale = 1./255,
shear_range = 0.2,
zoom_range = 0.2,
horizontal_flip = True)
test_datagen = ImageDataGenerator(rescale = 1./255)
# prepare training data
X1 = train_datagen.flow_from_directory('./ddsm1000_resized/images/train',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
X2 = train_datagen.flow_from_directory('./ddsm1000_resized_canny/images/train',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
# prepare test data
Y1 = test_datagen.flow_from_directory('./ddsm1000_resized/images/test',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
Y2 = test_datagen.flow_from_directory('./ddsm1000_resized_canny/images/test',
target_size = (64, 64),
batch_size = 32,
class_mode = 'binary')
final.fit_generator([X1, X2], steps_per_epoch = (8000 / 32), epochs = 1, validation_data = [Y1,Y2], validation_steps = 2000)
Keras Telling Me
RuntimeError: You must compile your model before using it.
I Think That is The CNN Does not the shapes of input data, so what Can I Do Here ?? Thanks
Make the change as pointed below:
from keras.layers import Merge
...
...
# Making The Combinition ##########################################################
final = Sequential()
final.add(Merge([branch_one, branch_two], mode = 'concat'))
...
...
I am having trouble reshaping the layer before feeding it through deconvolution. I dont know how to reverse the flatten layer in convolution.
Thanks for the help!
def build_deep_autoencoder(img_shape, code_size):
H,W,C = img_shape
encoder = keras.models.Sequential()
encoder.add(L.InputLayer(img_shape))
encoder.add(L.Conv2D(32, (3,3), padding = 'same', activation = 'elu', name='layer_1'))
encoder.add(L.MaxPooling2D((3,3), padding = 'same',name = 'max_pooling_1'))
encoder.add(L.Conv2D(64, (3,3), padding = 'same', activation = 'elu', name='layer_2'))
encoder.add(L.MaxPooling2D((3,3),padding = 'same',name = 'max_pooling_2'))
encoder.add(L.Conv2D(128, (3,3), padding = 'same', activation = 'elu', name='layer_3'))
encoder.add(L.MaxPooling2D((3,3),padding = 'same',name = 'max_pooling_3'))
encoder.add(L.Conv2D(256, (3,3), padding = 'same', activation = 'elu', name='layer_4'))
encoder.add(L.MaxPooling2D((3,3),padding = 'same',name = 'max_pooling_4'))
encoder.add(L.Flatten())
encoder.add(L.Dense(256))
# decoder
decoder = keras.models.Sequential()
decoder.add(L.InputLayer((code_size,)))
decoder.add(L.Dense(256))
decoder.add(L.Conv2DTranspose(filters=128, kernel_size=(3, 3), strides=2, activation='elu', padding='same'))
decoder.add(L.Conv2DTranspose(filters=64, kernel_size=(3, 3), strides=2, activation='elu', padding='same'))
decoder.add(L.Conv2DTranspose(filters=32, kernel_size=(3, 3), strides=2, activation='elu', padding='same'))
decoder.add(L.Conv2DTranspose(filters=3, kernel_size=(3, 3), strides=2, activation='none', padding='same'))
return encoder, decoder
In your encoder, use the following instead of adding dense layer of 256:
decoder.add(L.Dense(2*2*256)) #actual encoder
decoder.add(L.Reshape((2,2,256))) #un-flatten
I have tried to extract feature in the keras ( with tensorflow and as well as with theano background). But became not successful.
My code is :
model = Sequential()
model.add(Convolution2D(64, 5, 5, input_shape=(3,img_width, img_height)))
model.add(LeakyReLU(alpha=.01))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Convolution2D(64, 2, 2))
model.add(LeakyReLU(alpha=.01))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Dropout(0.1))
model.add(Flatten())
model.add(Dense(128))
model.add(Activation('relu'))
model.add(Dense(8))
model.add(ActivityRegularization(l2=0.01))
model.add(Activation('relu'))
model.add(Dropout(0.25))
model.add(Dense(1))
model.add(Activation('sigmoid'))
test_datagen = ImageDataGenerator(rescale=1./255)
test_generator = test_datagen.flow_from_directory(
test_data_dir,
target_size=(img_width, img_height),
batch_size=16,
class_mode='binary')
scores = model.evaluate_generator(test_generator, 237)
print("Accuracy = ", scores[1])
I have used fit_generator for train, validation and test cases.
My code for getting output from e.g., layer 3 was:
get_activations = theano.function([model.layers[0].input],
model.layers[3].output(train=False), allow_input_downcast=True)
activations = get_activations(test_generator)
But after excuting it I am getting an error :
File "test.py", line 96, in <module>
get_activations = theano.function([model.layers[0].input],
model.layers[3].output(train=False), allow_input_downcast=True)
TypeError: 'TensorVariable' object is not callable
How can I do it In theano or in the tensorflow mode ( any of them or both). I have used fit_generator in my code for image augmentation.
Please help.
First of all, I recommend you to switch to Keras-2.x for a better experience. Keras-2.x is more well-organized.
Secondly, using theano codes like theano.function is not recommended, you should use keras.backend.function instead.
The error is raised because model.layers[3].output is a tensor, you cannot call a tensor like model.layers[3].output(train=False).
My code to extract features usually looks like:
model = Sequential()
model.add(Convolution2D(64, 5, 5, input_shape=(3,img_width, img_height)))
model.add(Convolution2D(64, 2, 2))
# blablabla...
model.add(Dense(8, name='feature'))
model.add(Dropout(0.25))
model.add(Dense(1))
model.add(Activation('sigmoid'))
feature_network = Model(model.input, model.get_layer('feature').output)
feature = feature_network.predict(your_data_here)
I use feature to name the target feature layer and use the get_layer function to retrieve that layer.
I've created a CNN designed to recognize objects.
from keras.preprocessing.image import img_to_array, load_img
img = load_img('newimage.jpg')
x = img_to_array(img)
x = x.reshape( (1,) + x.shape )
scores = model.predict(x, verbose=1)
print(scores)
However I'm getting:
expected convolution2d_input_1 to have shape (None, 3, 108, 192) but got array with shape (1, 3, 192, 108)
My model:
def create_model():
model = Sequential()
model.add(Convolution2D(32, 3, 3, input_shape=(3, img_width, img_height)))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Convolution2D(32, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Convolution2D(64, 3, 3))
model.add(Activation('relu'))
model.add(MaxPooling2D(pool_size=(2, 2)))
model.add(Flatten())
model.add(Dense(64))
model.add(Activation('relu'))
model.add(Dropout(0.5))
model.add(Dense(1))
model.add(Dense(3, activation='softmax'))
model.compile(loss='categorical_crossentropy',
optimizer='rmsprop',
metrics=['accuracy'])
return model
I've looked at related answers and the documentation, but at a loss as to how to reshape the array to match what's expected?
I guess the problem is with setting up the image width and height. As the error says:
expected convolution2d_input_1 to have shape (None, 3, 108, 192) # expected width = 108 and height = 192
but got array with shape (1, 3, 192, 108) # width = 192, height = 108
Update: I tested your code with a small change and it worked!
I am giving just changed lines:
img_width, img_height = 960, 717
model.add(Convolution2D(32, 3, 3, input_shape=(img_height, img_width, 3)))
This is the main change - input_shape=(img_height, img_width, 3)
The image i used to run this code was of width = 960 and height = 717. I have updated my previous answer as some part of the answer was wrong! Sorry for that.