model.fit(X_train, y_train, batch_size = 16, initial_epoch=10,epochs=15, verbose = 1, validation_data = (X_test, y_test))
I have trained a model using LeNet architecture for some symbol images. I have to detect these symbols using sliding window cnn algorithm on different img. How can I do that?
Can anyone help
You can use model.predict() function on the test dataset for prediction.
Like this:
predictions = model.predict(X_test)
Related
I'm currently learning how to use pytorch to model NNs and did the "Getting Started" Session on the PyTorch Website.
I tried to train a PyTorch NN to apply the function e.g. f(x)=2x-1 to a given input integer list but my model is far apart from learning the right thing.
How can I model and train a PyTorch model to learn a given mathematical function f(x) ?
I've tried this model and trained it with 10 random numbers with labels generated by the 'myFunc' function to learn the function 2x-1.
Thanks for your help.
batch_size = 10
def myFunc(a):
#y = 2x-1
return 2*a-1
class NeuralNetwork(nn.Module):
def __init__(self):
super().__init__()
self.lin1 = nn.Linear(batch_size,1)
self.lin2 = nn.Linear(1,batch_size)
def forward(self, x):
x = self.lin1(x)
x = F.relu(x)
x = self.lin2(x)
return x
model = NeuralNetwork()
Theoretically for your example of an affine-linear function over a bounded interval you need only
linear(bias) -> relu -> linear(bias)
with one node per linear layer. Or just one linear layer without activation.
For more general functions, you will need larger layers in the construction of the first type, with one node for every piece in a piece-wise approximation. The last layer always needs to be linear without activation. Using more layers might give more pieces with less total nodes.
I am trying to build a LSTM Autoendoer for anomaly detection.
But the model seems not work for my data.
Here is the normal data that I use it for training.
And here is abnormal data that I use it for validation.
If model works, its loss should be high at #200000~#500000.
Unfortunately, here is the result I put the valid data to the model:
In the abnormal interval, the loss still low.
Here is my code of training model.
I would greatly appreciate it if you kindly give me any suggestions.
scaler = MinMaxScaler(feature_range=(0, 1))
scaler.fit(healthy_data)
data_scaled = scaler.transform(healthy_data)
data_broken_scaled = scaler.transform(broken_data)
timesteps=32
data = data_scaled
dim = 1
data.shape = (-1,timesteps,dim)
lr = 0.0001
Nadam = optimizers.Nadam(lr=lr)
model = Sequential()
model.add(LSTM(50,input_shape=(timesteps,dim),return_sequences=True))
model.add(Dense(dim))
model.compile(loss='mae', optimizer=Nadam ,metrics=['mse'])
EStop = EarlyStopping(monitor='val_loss', min_delta=0.001,patience=150, verbose=2, mode='auto',restore_best_weights=True)
history = model.fit(data,data,validation_data=(data,data),epochs=3000,batch_size=72,verbose=2,shuffle=False,callbacks=[EStop]).history
pred_broken = model.predict(data_broken_scaled)
loss_broken = np.mean(np.abs(pred_broken-data_broken_scaled),axis=1)
fig, ax = plt.subplots(figsize=(20, 6), dpi=80, facecolor='w', edgecolor='k')
ax.plot(range(0,len(loss_broken)), loss_broken, '-', color='red', animated = True, linewidth=1)
I Think, it may better to use Fourier transform to detect anomaly in frequency view.
I means, the train and test data will be converted to frequency domain via Fourier transform. Also I recommend to use Time-windows.
Most AI will act better if they have enough good train data. Your anomaly points should be labeled to 1(by pre-processing step of the data) and it should have high frequency at your time-step(time-windows).
In short, I think your train data may not sufficiently representative for anomaly now.
Pre-trained deep learning models have 2 types of weights associated with them,with top(meaning the dense fully connected layer) and without_top(removing the dense fully connected layer). I want to know how to train a model for the no_top weights
for eg: I have an architecture and I want to create weights for it trained on cifar10 data.So how to train for that?
Since you are talking about CIFAR10, I will give you an example for image classification.
First you get your pre-trained model from Keras without the top layers.
from keras.applications.vgg16 import VGG16
base_model = VGG16(weights='imagenet', include_top=False, input_shape=(48, 48, 3))
Then you add your new layers for the classification tasks.
x = base_model.output
x = Flatten()(x)
x = Dense(512, activation='elu', kernel_initializer='he_normal')(x)
prediction = Dense(10, activation='softmax', kernel_initializer='he_normal')(x)
The last layer has 10 units since CIFAR10 has 10 classes.
You create your model.
model = Model(inputs=base_model.input, outputs=prediction)
You get the CIFAR10 data.
from keras.datasets import cifar10
(Xtrain, Ytrain), (Xtest, Ytest) = cifar10.load_data()
Ytrain = to_categorical(Ytrain)
Ytest = to_categorical(Ytest)
Compile and train.
model.compile(Adam(), loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(Xtrain, Ytrain, batch_size=64, epochs=1, validation_data=(Xtest, Ytest))
Note: Here the training will change the weights of all your layers (pre-trained layers + new layers). You might want to freeze the pre-trained layers before training to keep their nicely trained weights.
I am working on an images classification using Keras.
There is my model:
model = Sequential()
model.add(Conv2D(filters = 8, kernel_size = (3,3),padding = 'Same',
activation ='relu', input_shape = (64,64,3)))
model.add(MaxPool2D(pool_size=(2,2)))
model.add(Conv2D(filters = 16, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model.add(MaxPool2D(pool_size=(2,2)))
model.add(Conv2D(filters = 32, kernel_size = (3,3),padding = 'Same',
activation ='relu'))
model.add(MaxPool2D(pool_size=(2,2)))
model.add(Dropout(0.2))
model.add(Flatten())
model.add(Dense(32, activation='relu'))
model.add(Dropout(0.1))
model.add(Dense(3, activation = "sigmoid"))
I would like to avoid Flatten() since in this case we are losing some spatial information. I looked some tutorials, but all of them used Flatten(). Is it possible to use some thing like deconvolution instead?
Flatten is fine.
The "spatial" relations of Flatten going into a Dense layer are, in a sense preserved. As all values from a particular position map to the same weight in the dense layer. So every point is mapped consistently across the dataset. The "spatial" relations mapped in the convolutional layers are looking for localized patterns, and in those layers keeping the input unaltered is important.
I know how to merge different models into one in Keras.
first_model = Sequential()
first_model.add(LSTM(output_dim, input_shape=(m, input_dim)))
second_model = Sequential()
second_model.add(LSTM(output_dim, input_shape=(n-m, input_dim)))
model = Sequential()
model.add(Merge([first_model, second_model], mode='concat'))
model.fit([X1,X2])
I am not sure how to do this in TensorFlow though.
I have two LSTM models and want to merge those (in the same way as in above Keras example).
outputs_1, state_1 = tf.nn.dynamic_rnn(stacked_lstm_1, model_input_1)
outputs_2, state_2 = tf.nn.dynamic_rnn(stacked_lstm_2, model_input_2)
Any help would be much appreciated!
As was said in the comment, I believe the simplest way to do this is just to concatenate the outputs. The only complication that I've found is that, at least how I made my LSTM layers, they ended up with the exact same names for their weight tensors. This led to an error because TensorFlow thought the weights were already made when I tried to make the second layer. If you have this problem, you can solve it using a variable scope, which will apply to the names of the tensors in that LSTM layer:
with tf.variable_scope("LSTM_1"):
lstm_cells_1 = tf.contrib.rnn.MultiRNNCell(tf.contrib.rnn.LSTMCell(256))
output_1, state_1 = tf.nn.dynamic_rnn(lstm_cells_1, inputs_1)
last_output_1 = output_1[:, -1, :]
# I usually work with the last one; you can keep them all, if you want
with tf.variable_scope("LSTM_2"):
lstm_cells_2 = tf.contrib.rnn.MultiRNNCell(tf.contrib.rnn.LSTMCell(256))
output_2, state_2 = tf.nn.dynamic_rnn(lstm_cells_2, inputs_2)
last_output_2 = output_2[:, -1, :]
merged = tf.concat((last_output_1, last_output_2), axis=1)