https://github.com/playerkk/face-py-faster-rcnn
the link above has indicated that a pretrained model is available.
enter image description here
After you download the pretrained weights ( a .caffemodel file), you can instantiate a caffe.Net object with the network definition (.prototxt file - from the repository you referred, test.prototxt), e.g.
net = caffe.Net(prototxt, caffemodel, caffe.TEST)
(I guess you would like to use the pretrained model for inference, if you would like to do transfer-learning on your data you should use caffe.TRAIN).
Then you should load the image, feed it into the input blobs, run net.forward on the image and extract the results from the output blobs - e.g. net.blobs['cls_score'].data, net.blobs['cls_prob'].data and net.blobs['bbox_pred'].data.
You can use the original py-faster-rcnn's demo with minor adjustments.
Good luck!
Related
I want to convert this NSFW model to CoreML model. What I did:
Download Anaconda 2.7
Install coremltools
Convert this yahoo nsfw model from here - https://github.com/yahoo/open_nsfw/tree/master/nsfw_model but I am not sure it’s Caffe v1 because Apple documentation says that only this version supported. Anyway…
I use this commands for conversion and it converted without any warnings.
coreml_model = coremltools.converters.caffe.convert(('resnet_50_1by2_nsfw.caffemodel', 'deploy.prototxt'), image_input_names='data')
coreml_model.save(’nsfw2.mlmodel')
I imported this model to my project and again all looks fine.
I prepared 224x224 images and use Vision framework like VNImageRequestHandler with cgImage and etc.
But!
All images return the same result
[<VNCoreMLFeatureValueObservation: 0x281b1daa0> 2E00F417-95C0-4AA1-A621-A0945BB5E095 requestRevision=1 confidence=1.000000 "prob" - "MultiArray : Double 1 x 1 x 2 x 1 x 1 array" (1.000000)]
How can I debug this issue and found out what’s wrong?
Maybe you're looking only at naughty images? ;-)
It's probably the image preprocessing. You didn't specify any preprocessing options while Caffe models usually normalize using ImageNet mean/std. Refer to my blog post for more info: https://machinethink.net/blog/help-core-ml-gives-wrong-output/
However, I don't see any normalization options in your deploy.prototxt, so perhaps it's not that.
How I would debug this: remove everything but the first layer from the Caffe model and convert to Core ML. Run this one-layer model in both Caffe and Core ML and compare the outputs. If they are different, something is up with how you're loading or preprocessing the input data.
Below are the various transforms I have found so far using NOP_VIEWER.model.getData().
I'm using the transforms to bring a position into viewer space, and I haven't been able to find any good documentation describing what they all do. My hope here is that this question can help by providing some documentation of the role of these transforms and how/when to use them.
The model originally comes from Revit.
GlobalOffset (Vector3)
placementWithOffset (Matrix4) - seems to be just the inverse of GlobalOffset as a matrix?
placementTransform (Matrix4) - undefined in all models I've tested, I've seen some hints that this is a user defined matrix.
refPointTransform (Matrix4)
Also, there are some transforms in the NOP_VIEWER.model.getData().metadata. These may be Revit specific:
metadata.georeference.positionLL84 (Array[3]) - this is where the model's GPS coords are stored
metadata.georeference.refPointLMV (Array[3]) - no idea what this is, and it has huge and seemingly random values on many models. For example, on my current model it is [-17746143.211481072, -6429345.318822183, 27.360225423452952]
metadata.[custom values].angleToTrueNorth - I guess this is specifying whether the model is aligned to true or magnetic north?
metadata.[custom values].refPointTransform - (Array[12]) - data used to create the refPointTransform matrix above
Can someone help by documenting what these transforms do?
Related: Place a custom object into viewer space using GPS coords
As an alternative solution, the Viewer works with extensions. The Autodesk.Geolocation extension provides a few methods to handle the data structure you mentioned:
Load extension:
let geoExt;
NOP_VIEWER.loadExtension('Autodesk.Geolocation').then((e) => {geoExt = e});
Or get already loaded extension:
let geoExt = NOP_VIEWER.getLoadedExtensions()['Autodesk.Geolocation']
Then use the methods to convert the coordinates
geoExt.lmvToLonLat
geoExt.lonLatToLmv
Here is a quick article on it.
You may .activate() the extension to see additional information on the model geo location.
I am trying to deploy a tf.keras image classification model to Google CloudML Engine. Do I have to include code to create serving graph separately from training to get it to serve my models in a web app? I already have my model in SavedModel format (saved_model.pb & variable files), so I'm not sure if I need to do this extra step to get it to work.
e.g. this is code directly from GCP Tensorflow Deploying models documentation
def json_serving_input_fn():
"""Build the serving inputs."""
inputs = {}
for feat in INPUT_COLUMNS:
inputs[feat.name] = tf.placeholder(shape=[None], dtype=feat.dtype)
return tf.estimator.export.ServingInputReceiver(inputs, inputs)
You are probably training your model with actual image files, while it is best to send images as encoded byte-string to a model hosted on CloudML. Therefore you'll need to specify a ServingInputReceiver function when exporting the model, as you mention. Some boilerplate code to do this for a Keras model:
# Convert keras model to TF estimator
tf_files_path = './tf'
estimator =\
tf.keras.estimator.model_to_estimator(keras_model=model,
model_dir=tf_files_path)
# Your serving input function will accept a string
# And decode it into an image
def serving_input_receiver_fn():
def prepare_image(image_str_tensor):
image = tf.image.decode_png(image_str_tensor,
channels=3)
return image # apply additional processing if necessary
# Ensure model is batchable
# https://stackoverflow.com/questions/52303403/
input_ph = tf.placeholder(tf.string, shape=[None])
images_tensor = tf.map_fn(
prepare_image, input_ph, back_prop=False, dtype=tf.float32)
return tf.estimator.export.ServingInputReceiver(
{model.input_names[0]: images_tensor},
{'image_bytes': input_ph})
# Export the estimator - deploy it to CloudML afterwards
export_path = './export'
estimator.export_savedmodel(
export_path,
serving_input_receiver_fn=serving_input_receiver_fn)
You can refer to this very helpful answer for a more complete reference and other options for exporting your model.
Edit: If this approach throws a ValueError: Couldn't find trained model at ./tf. error, you can try it the workaround solution that I documented in this answer.
I've been looking at Laurent Luce's' blog on sentiment analysis. Unfortunately I have not been able to follow his blog so I cannot ask him a question directly. Here is the link to the blog: http://www.laurentluce.com/posts/twitter-sentiment-analysis-using-python-and-nltk/
Pretty much everything works nicely. However I cannot figure out how to get this part to work - text below is pasted from the link. My question is how do I look inside the classifier train method? It will help my understanding if I could do that.
"Let’s take a look inside the classifier train method in the source code of the NLTK library. ‘label_probdist’ is the prior probability of each label and ‘feature_probdist’ is the feature/value probability dictionary. Those two probability objects are used to create the classifier."
def train(labeled_featuresets, estimator=ELEProbDist):
...
# Create the P(label) distribution
label_probdist = estimator(label_freqdist)
...
# Create the P(fval|label, fname) distribution
feature_probdist = {}
...
return NaiveBayesClassifier(label_probdist, feature_probdist)
NLTK is open source. You can find the source-code here. Github has solid search functionality, so you should be able to find whatever it is you need using that. Specifically, the naive bayes classifier is here
and you can see the train method for it in that file.
My network contains some specific layers which are not supported by current tensorRT. so I want to run the conv layers and pooling layers on tensorRT and then use the output from tensorRT as the input of my caffe model which contains some specific layers. Is there some API or example code thar I can refer to? Thanks
See the source code in the samples directory of your TensorRT installation.
For those stumbling now on this issue I got this to work by making the input and output of TensorRT inference the mutable_gpu_data from caffe blobs:
auto* gpuImagePtr = inputBlob->mutable_gpu_data();
cudaMemcpy(gpuImagePtr, inputData, mNetInputMemory, cudaMemcpyHostToDevice);
std::vector<void*> buffers(2);
buffers[0] = gpuImagePtr;
buffers[1] = outputBlob->mutable_gpu_data();
cudaContext->enqueue(batchSize, &buffers[0], stream, nullptr);