I want to convert this NSFW model to CoreML model. What I did:
Download Anaconda 2.7
Install coremltools
Convert this yahoo nsfw model from here - https://github.com/yahoo/open_nsfw/tree/master/nsfw_model but I am not sure it’s Caffe v1 because Apple documentation says that only this version supported. Anyway…
I use this commands for conversion and it converted without any warnings.
coreml_model = coremltools.converters.caffe.convert(('resnet_50_1by2_nsfw.caffemodel', 'deploy.prototxt'), image_input_names='data')
coreml_model.save(’nsfw2.mlmodel')
I imported this model to my project and again all looks fine.
I prepared 224x224 images and use Vision framework like VNImageRequestHandler with cgImage and etc.
But!
All images return the same result
[<VNCoreMLFeatureValueObservation: 0x281b1daa0> 2E00F417-95C0-4AA1-A621-A0945BB5E095 requestRevision=1 confidence=1.000000 "prob" - "MultiArray : Double 1 x 1 x 2 x 1 x 1 array" (1.000000)]
How can I debug this issue and found out what’s wrong?
Maybe you're looking only at naughty images? ;-)
It's probably the image preprocessing. You didn't specify any preprocessing options while Caffe models usually normalize using ImageNet mean/std. Refer to my blog post for more info: https://machinethink.net/blog/help-core-ml-gives-wrong-output/
However, I don't see any normalization options in your deploy.prototxt, so perhaps it's not that.
How I would debug this: remove everything but the first layer from the Caffe model and convert to Core ML. Run this one-layer model in both Caffe and Core ML and compare the outputs. If they are different, something is up with how you're loading or preprocessing the input data.
Related
I am trying to run some example python3 code
https://docs.databricks.com/applications/deep-learning/distributed-training/horovod-runner.html
on databricks GPU cluster (with 1 driver and 2 workers).
Databricks environment:
ML 6.6, scala 2.11, Spark 2.4.5, GPU
It is for distributed deep learning model training.
I just tried a very simple example at first:
from sparkdl import HorovodRunner
hr = HorovodRunner(np=2)
def train():
print('in train')
import tensorflow as tf
print('after import tf')
hvd.init()
print('done')
hr.run(train)
But, the command is alway running without any progress.
HorovodRunner will stream all training logs to notebook cell output. If there are too many
logs, you
can adjust the log level in your train method. Or you can set driver_log_verbosity to
'log_callback_only' and use a HorovodRunner log callback on the first worker to get concise
progress updates.
The global names read or written to by the pickled function are {'print', 'hvd'}.
The pickled object size is 1444 bytes.
### How to enable Horovod Timeline? ###
HorovodRunner has the ability to record the timeline of its activity with Horovod Timeline.
To
record a Horovod Timeline, set the `HOROVOD_TIMELINE` environment variable to the location
of the
timeline file to be created. You can then open the timeline file using the chrome://tracing
facility of the Chrome browser.
Do I miss something or need to set up something to make it work ?
Thanks
your code does no actual training within it.. you might have better luck with the better example code
https://docs.databricks.com/applications/machine-learning/train-model/distributed-training/mnist-pytorch.html
https://github.com/playerkk/face-py-faster-rcnn
the link above has indicated that a pretrained model is available.
enter image description here
After you download the pretrained weights ( a .caffemodel file), you can instantiate a caffe.Net object with the network definition (.prototxt file - from the repository you referred, test.prototxt), e.g.
net = caffe.Net(prototxt, caffemodel, caffe.TEST)
(I guess you would like to use the pretrained model for inference, if you would like to do transfer-learning on your data you should use caffe.TRAIN).
Then you should load the image, feed it into the input blobs, run net.forward on the image and extract the results from the output blobs - e.g. net.blobs['cls_score'].data, net.blobs['cls_prob'].data and net.blobs['bbox_pred'].data.
You can use the original py-faster-rcnn's demo with minor adjustments.
Good luck!
I've been looking at Laurent Luce's' blog on sentiment analysis. Unfortunately I have not been able to follow his blog so I cannot ask him a question directly. Here is the link to the blog: http://www.laurentluce.com/posts/twitter-sentiment-analysis-using-python-and-nltk/
Pretty much everything works nicely. However I cannot figure out how to get this part to work - text below is pasted from the link. My question is how do I look inside the classifier train method? It will help my understanding if I could do that.
"Let’s take a look inside the classifier train method in the source code of the NLTK library. ‘label_probdist’ is the prior probability of each label and ‘feature_probdist’ is the feature/value probability dictionary. Those two probability objects are used to create the classifier."
def train(labeled_featuresets, estimator=ELEProbDist):
...
# Create the P(label) distribution
label_probdist = estimator(label_freqdist)
...
# Create the P(fval|label, fname) distribution
feature_probdist = {}
...
return NaiveBayesClassifier(label_probdist, feature_probdist)
NLTK is open source. You can find the source-code here. Github has solid search functionality, so you should be able to find whatever it is you need using that. Specifically, the naive bayes classifier is here
and you can see the train method for it in that file.
I want to persist a trained model in CNTK and found the 'persist' functionality after some amount of searching. However, there seems to be some error in importing it.
from cntk import persist
This is throwing ImportError.
Am I doing something the wrong way? Or is this no longer supported? Is there an alternate way to persist a model?
persist is from an earlier beta. save_model is now a method of every CNTK function. So instead of doing save_model(z, filename) you do z.save_model(filename). Load_model works the same as before but you import it from cntk.ops.functions. For an example, see: https://github.com/Microsoft/CNTK/blob/v2.0.beta7.0/Tutorials/CNTK_203_Reinforcement_Learning_Basics.ipynb or https://github.com/Microsoft/CNTK/blob/v2.0.beta7.0/bindings/python/cntk/tests/persist_test.py
The functionality has moved to cntk functions. The new way is mynetwork.save_model(...) where mynetwork represents the root of your computation (typically the prediction). For loading the model you can just say mynetwork = C.load_model(...)
on perl5 if someone want to parse binary file he has the pack/unpack utiltiy where he can convert binary structure to perl variables and vice verca ,
is there now a production equivlant for pack/unpack on perl6 ,as from the documentation i found that there are pack/unpack methods for Perl6 but they are experimental ,
does anyone know the status of those functions and if there are alternative to parse binary file which contains a list of records on perl6 ?
You are correct, the pack/unpack methods are experimental; there is currently no other method that is recommended in their place, however.
The experimental flag indicates that the Perl 6 dev team may change the interface. pack & unpack were marked in this way because there was not enough time to review and update the interface before the Christmas release in 2015-12.
I am using rakudo:
use experimental :pack;
pack("C*", [1, 2, 3]); => Buf:0x<01>
I am not sure this is correct use. I expect all bytes get packed in.