My network contains some specific layers which are not supported by current tensorRT. so I want to run the conv layers and pooling layers on tensorRT and then use the output from tensorRT as the input of my caffe model which contains some specific layers. Is there some API or example code thar I can refer to? Thanks
See the source code in the samples directory of your TensorRT installation.
For those stumbling now on this issue I got this to work by making the input and output of TensorRT inference the mutable_gpu_data from caffe blobs:
auto* gpuImagePtr = inputBlob->mutable_gpu_data();
cudaMemcpy(gpuImagePtr, inputData, mNetInputMemory, cudaMemcpyHostToDevice);
std::vector<void*> buffers(2);
buffers[0] = gpuImagePtr;
buffers[1] = outputBlob->mutable_gpu_data();
cudaContext->enqueue(batchSize, &buffers[0], stream, nullptr);
Related
I am using the pretrained PyTorchVideo model slowfast_r50_detection as shown here. I want to retrain this model with a different private dataset that I have and use it in a similar way as shown in the example. I am new to PyTorch and am not sure how to start retraining such a model. Any pointers would be very helpful.
You can simply load your model first, and than use load_state_dict() function to load the pre-trained model
path_to_saved_model = "Directory/directory/your_saved_model.tar"
video_model = slow_r50_detection(True)
video_model.load_state_dict(torch.load(path_to_saved_model)['model_state_dict'])
device = "cuda:0" if torch.cuda.is_available() else "cpu"
video_model = video_model.to(device)
The model loads the pre-trained weights from the saved model, and anything you run post the load_state_dict() line, the model uses previously trained weights.
I want to convert this NSFW model to CoreML model. What I did:
Download Anaconda 2.7
Install coremltools
Convert this yahoo nsfw model from here - https://github.com/yahoo/open_nsfw/tree/master/nsfw_model but I am not sure it’s Caffe v1 because Apple documentation says that only this version supported. Anyway…
I use this commands for conversion and it converted without any warnings.
coreml_model = coremltools.converters.caffe.convert(('resnet_50_1by2_nsfw.caffemodel', 'deploy.prototxt'), image_input_names='data')
coreml_model.save(’nsfw2.mlmodel')
I imported this model to my project and again all looks fine.
I prepared 224x224 images and use Vision framework like VNImageRequestHandler with cgImage and etc.
But!
All images return the same result
[<VNCoreMLFeatureValueObservation: 0x281b1daa0> 2E00F417-95C0-4AA1-A621-A0945BB5E095 requestRevision=1 confidence=1.000000 "prob" - "MultiArray : Double 1 x 1 x 2 x 1 x 1 array" (1.000000)]
How can I debug this issue and found out what’s wrong?
Maybe you're looking only at naughty images? ;-)
It's probably the image preprocessing. You didn't specify any preprocessing options while Caffe models usually normalize using ImageNet mean/std. Refer to my blog post for more info: https://machinethink.net/blog/help-core-ml-gives-wrong-output/
However, I don't see any normalization options in your deploy.prototxt, so perhaps it's not that.
How I would debug this: remove everything but the first layer from the Caffe model and convert to Core ML. Run this one-layer model in both Caffe and Core ML and compare the outputs. If they are different, something is up with how you're loading or preprocessing the input data.
https://github.com/playerkk/face-py-faster-rcnn
the link above has indicated that a pretrained model is available.
enter image description here
After you download the pretrained weights ( a .caffemodel file), you can instantiate a caffe.Net object with the network definition (.prototxt file - from the repository you referred, test.prototxt), e.g.
net = caffe.Net(prototxt, caffemodel, caffe.TEST)
(I guess you would like to use the pretrained model for inference, if you would like to do transfer-learning on your data you should use caffe.TRAIN).
Then you should load the image, feed it into the input blobs, run net.forward on the image and extract the results from the output blobs - e.g. net.blobs['cls_score'].data, net.blobs['cls_prob'].data and net.blobs['bbox_pred'].data.
You can use the original py-faster-rcnn's demo with minor adjustments.
Good luck!
I am trying to deploy a tf.keras image classification model to Google CloudML Engine. Do I have to include code to create serving graph separately from training to get it to serve my models in a web app? I already have my model in SavedModel format (saved_model.pb & variable files), so I'm not sure if I need to do this extra step to get it to work.
e.g. this is code directly from GCP Tensorflow Deploying models documentation
def json_serving_input_fn():
"""Build the serving inputs."""
inputs = {}
for feat in INPUT_COLUMNS:
inputs[feat.name] = tf.placeholder(shape=[None], dtype=feat.dtype)
return tf.estimator.export.ServingInputReceiver(inputs, inputs)
You are probably training your model with actual image files, while it is best to send images as encoded byte-string to a model hosted on CloudML. Therefore you'll need to specify a ServingInputReceiver function when exporting the model, as you mention. Some boilerplate code to do this for a Keras model:
# Convert keras model to TF estimator
tf_files_path = './tf'
estimator =\
tf.keras.estimator.model_to_estimator(keras_model=model,
model_dir=tf_files_path)
# Your serving input function will accept a string
# And decode it into an image
def serving_input_receiver_fn():
def prepare_image(image_str_tensor):
image = tf.image.decode_png(image_str_tensor,
channels=3)
return image # apply additional processing if necessary
# Ensure model is batchable
# https://stackoverflow.com/questions/52303403/
input_ph = tf.placeholder(tf.string, shape=[None])
images_tensor = tf.map_fn(
prepare_image, input_ph, back_prop=False, dtype=tf.float32)
return tf.estimator.export.ServingInputReceiver(
{model.input_names[0]: images_tensor},
{'image_bytes': input_ph})
# Export the estimator - deploy it to CloudML afterwards
export_path = './export'
estimator.export_savedmodel(
export_path,
serving_input_receiver_fn=serving_input_receiver_fn)
You can refer to this very helpful answer for a more complete reference and other options for exporting your model.
Edit: If this approach throws a ValueError: Couldn't find trained model at ./tf. error, you can try it the workaround solution that I documented in this answer.
I'm very new to optix and cuda.
I'm trying to modify optix SDK example to present a 3D model with ray tracing. I modified the "progressivePhotonMap" example. Because of lacking of optix/cuda knowledge, I don't know how to draw texture on the 3D model, can anyone who is familiar with SDK example could help me?
I read other draw texture examples like "swimmingShark" or "cook" and try to find out clue to use. However, those examples seem has different way to draw texture.
From now on, I know i have to load texture in cpp file
GeometryInstance instance = m_context->createGeometryInstance( mesh, &m_material, &m_material+1 );
instance["diffuse_map"]->setTextureSampler(loadTexture( m_context, ... );
and create TextureSampler in cuda file
rtTextureSampler<float4, 2> diffuse_map; // Corresponds to OBJ mtl params
,and give them texcoord to draw, like this,
float3 Kd = make_float3( tex2D( diffuse_map, texcoord.x*diffuse_map_scale, texcoord.y*diffuse_map_scale ) );
However, I cannot found where the texcoord get the texture coordinate data in cuda file.
It seems there should be some code like this in .cpp file
GI["texcoord"]->setBuffer(texcoord)
Could anyone teach me where texcoord get the texture coordinate data, and how to match coordinate data and texure to present 3D model with ray tracing?
I can't find tutorial in google, I really need help or direction to reach my goal. Thank you.
You should read up on the OptiX documentation first. Specifically the paragraph regarding Attribute variables.
IIRC the texcoord variable is an attribute of the form
rtDeclareVariable( float3, texcoord, attribute texcoord );
that is computed in the intersection program and passed along to the closest hit program (attributes are designed to pass data from the intersection point to the shading points).
Short answer: it is set into another CUDA function which, conceptually, computes some data needed by that line.