void SilenceLayer<Dtype>::Backward_cpu(const vector<Blob<Dtype>*>& top,
const vector<bool>& propagate_down, const vector<Blob<Dtype>*>& bottom) {
for (int i = 0; i < bottom.size(); ++i) {
if (propagate_down[i]) {
caffe_set(bottom[i]->count(), Dtype(0),
bottom[i]->mutable_cpu_diff());
}
}
}
It just sets the diff to zero.
What is the use of this layer?
The use of this layer is simply to avoid that the output of unused blobs is reported in the log. Being an output manager layer, it is obviously zero its gradient.
For instance, let us assume we are using AlexNet and we change the bottom of the 'fc7' layer to 'pool5' instead of 'fc6'. If we do not delete the 'fc6' blob declaration, this layer is not used anymore but its ouput will be printed in stderr: it is considered as an output of the whole architecture. If we want to keep 'fc6' for some reasons, but without showing its values, we can use the 'SilenceLayer'.
http://caffe.berkeleyvision.org/tutorial/layers/silence.html
See also caffe.help.
Related
Recently, I am using the plugin "hole filling" of OpenFlipper, and have entirely compiled the OpenFlipper. However, the new mesh has a large number of duplicate vertices, when I tried to add the filling patch to the original mesh. I used the following codes to perform the adding operation:
// filling_patch: newly created filling mesh
// mesh_ori: the original mesh before hole filling
class MeshT::FaceHandle fh;
class MeshT::FaceIter f_it, f_end;
class MeshT::FaceVertexIter fv_it;
for(f_it = filling_patch->faces_begin(), f_end = fill_patch ->faces_end(); f_it != f_end; f_it++)
{
// ith face
fh = *f_it;
// Check whether it is valid
if(!fh.is_valid())
{
return;
}
// Store its three vertices
std::vector<class MeshT::VertexHandle> face_vhandles;
face_vhandles.clear();
// Iterate each vertex of this face
for(fv_it = mesh_ori->fv_iter(fh); fv_it.is_valid(); fv_it++)
{
// Get the 3D point
class MeshT::Point p = filling_patch->point(*fv_it);
// Add this point to original mesh. Note: vh is a new vertevHandle, differ to *fv_it
class MeshT::VertexHandle vh = mesh_ori->add_vertex(p);
face_vhandles.push_back(vh);
}
// Save the face to mesh
mesh_ori->add_face(face_vhandles);
}
So, I am not sure whether there is an existing function that can be used to fix this problem in OpenMesh.
Does someone give me some advice?
Thanks a lot.
In caffe.proto
// Set clip_gradients to >= 0 to clip parameter gradients to that L2 norm,
// whenever their actual L2 norm is larger.
optional float clip_gradients = 35 [default = -1];
I am having trouble setting the clipping_gradient, I think it should be dynamic anyway but if we are to chose a fixed number, how should we chose it? Is caffe setting it to 35? What does it mean?? I have experimented with a number of fixed choices but I see not much of a difference. I understand the exploding gradients / gradient clipping concept in the broad sense, however I am not sure how I should chose a fixed number in the solver.
You can print out the sum of the sum squared gradients for some iteration to get an idea about clip_gradients. This can be done this way:
net_->forward();
net_->backward();
const vector<Blob<Dtype>*>& net_params = net_->learnable_params();
float sumsq_diff = 0;
for (int i = 0; i < net_params.size(); ++i) {
sumsq_diff += net_params[i]->sumsq_diff();
}
std::cout<<"sum of gradient: "<<std::sqrt(sumsq_diff)<<"\n";
net_->update();
For details about how clip_gradients is used see solver.cpp.
Here is my javascript code, it is pretty simple:
console.profile();
var count = 1000;
var fn1 = function () {
for (var i = 0; i < count; i++) {
console.log("THIS IS FN1");
}
}
var fn2 = function () {
for (var i = 0; i < count; i++) {
console.log("THIS IS FN2");
}
fn1();
}
fn2();
console.profileEnd();
and this is my profile screenshot:
Why there are some gap in the image, just like my black rectangle marked?
What does this gap mean?
You see this non-uniform sequence of gaps between log calls on top of fn2 and fn1 because the profiler is sampling and gives you only statistical information. It will stop JS thread and capture current call stack roughly once per 1ms (100us in high res mode) and width of each bar is proportional to the number of consecutive samples where we've seen the same call stack.
The split of fn2 is a bug. Since we stop JS thread in random state it is not always possible to iterate JS call stack because e.g. top frame may be half-constructed. We do our best to determine current state of the VM and crawl call stack but sometimes our heuristics fail in which case we may end up capturing incomplete stack like in your case.
I have been working on creating an assets class that can generate dynamic TextureAtlas objects whenever I need them. The specific method is Assets.generateTextureAtlas() and I am trying to optimise it as much as possible as I quite frequently need to regenerate texture atlas's and was hoping to get a better time than my 53ms average.
53ms is currently costing me about 3 frames which can add up quickly the more items I need to pack inside my texture atlas and the frequency I need to generate them. So an answer to all the pitfalls within my code would be great.
The entire class code is available here in a github gist.
The RectanglePacker class is simply used to pack rectangles as close together as possible (similar to Texture Packer) and can be found here.
For reference, here is the method:
public static function generateTextureAtlas(folder:String):void
{
if (!_initialised) throw new Error("Assets class not initialised.");
if (_renderTextureAtlases[folder] != null)
{
(_renderTextureAtlases[folder] as TextureAtlas).dispose();
}
var i:int;
var image:Image = new Image(_blankTexture);
var itemName:String;
var itemNames:Vector.<String> = Assets.getNames(folder + "/");
var itemsTexture:RenderTexture;
var itemTexture:Texture;
var itemTextures:Vector.<Texture> = Assets.getTextures(folder + "/");
var noOfRectangles:int;
var rect:Rectangle;
var rectanglePacker:RectanglePacker = new RectanglePacker();
var texture:Texture;
noOfRectangles = itemTextures.length;
if (noOfRectangles == 0)
{
return;
}
for (i = 0; i < noOfRectangles; i++)
{
rectanglePacker.insertRectangle(Math.round(itemTextures[i].width), Math.round(itemTextures[i].height), i);
}
rectanglePacker.packRectangles();
if (rectanglePacker.rectangleCount != noOfRectangles)
{
throw new Error("Only " + rectanglePacker.rectangleCount + " out of " + noOfRectangles + " rectangles packed for folder: " + folder);
}
itemsTexture = new RenderTexture(rectanglePacker.width, rectanglePacker.height);
itemsTexture.drawBundled(function():void
{
for (i = 0; i < noOfRectangles; i++)
{
itemTexture = itemTextures[rectanglePacker.getRectangleId(i)];
rect = rectanglePacker.getRectangle(i, rect);
image.texture = itemTexture;
image.readjustSize();
image.x = rect.x + itemTexture.frame.x;
image.y = rect.y + itemTexture.frame.y;
itemsTexture.draw(image);
}
});
_renderTextureAtlases[folder] = new TextureAtlas(itemsTexture);
for (i = 0; i < noOfRectangles; i++)
{
itemName = itemNames[rectanglePacker.getRectangleId(i)];
itemTexture = itemTextures[rectanglePacker.getRectangleId(i)];
rect = rectanglePacker.getRectangle(i);
(_renderTextureAtlases[folder] as TextureAtlas).addRegion(itemName, rect, itemTexture.frame);
}
}
Well reading the project & finding what all can be optimized would sure take time.
Start by removing multiple calls to rectanglePacker.getRectangle(i) inside loops.
For example :
itemName = itemNames[rectanglePacker.getRectangleId(i)];
itemTexture = itemTextures[rectanglePacker.getRectangleId(i)];
rect = rectanglePacker.getRectangle(i);
perhaps, could have been:
rect = rectanglePacker.getRectangle(i);
itemName = itemNames[rect];
itemTexture = itemTextures[rect];
If getRectangle does indeed just 'get a rectangle' & not set anything.
I think the bigger issue at hand is this, why oh why do you HAVE to do this during run-time, in a situation when this can't take more time? This IS an expansive operation, no matter how much you optimize this you will probably end up with it taking about 40ms or similar when done in AS3.
This is why these kind of operations should be done during compile time or during "loading screens" or other "transitions" when frame-rate is not critical and when you can afford it.
Alternatively create another system in c++ or some other language which can actually handle the number-crunching that gives you the finished result.
Also, when it comes to checking performance, yes the entire function takes 53ms, BUT, where are those milliseconds used? 53ms says nothing and is only the "overhead profiling thing" where you found the culprit, you need to break it down into smaller chunks to gather some reliable information about what it is that ACTUALLY takes time, inside that function.
I mean, inside that function, you have 3 for loops, several calls to other classes, casts, deletes, creations. It's not like you are doing one thing, that function probably results in ~500 lines of code and a bazillion cpu operations. And, you have no idea where it is used. I would guess that it is the rectanglePacker.packRectangles(); that takes 60% of that time, but without profiling, you and we don't know on what to optimize, we simply don't have sufficient data.
If you HAVE to do this during run-time in AS3, I would recommend doing this spread out during several frames and distributing workload evenly during 10 frames or so. You could also doing it with help of another thread and workers. But most of all, this seems like a design error since this could probably be done at another time. And if not, then in another language which is better at these kind of operations.
The easiest way to profile this is to add a couple of timestamps similar to:
var timestamps:Array = [];
And then push getTimer() at different places in code, and then print them out when function is done
As others said, it's unlikely that the reason of bad performance is non-optimized AS code. Output from the profiler (Scout, for example) wold be very helpful. However, if your purpose is just adding new textures, I can suggest several optimizations:
Why would you need to re-generate the whole atlas every time (calling Assets.getTextures() and creating new render texture)? Why don't you just add new items to the existing atlas? Creation of a new RenderTexture (and, thus, a new texture in GPU memory) is very costly operation, because it requires sync between CPU and GPU. On the other hand, drawing into RenderTexture is carried out entirely inside GPU, so it takes much less time.
If you place every item on a grid, then you can avoid using RectanglePacker as all of your rectangles can have the same dimensions matching the dimensions of a grid.
Edit:
To clarify, some time ago I had a similar problem: I had to add new items to the existing atlas on a regular basis. And the performance of this operation was quite acceptable (about 8ms on iPad3 using 1024x1024 dynamic texture). But I used the same RenderTexture and the same Sprite object that contained my dynamic atlas items. When I need to add a new item, I just create new Image with desired texture (stand-alone or from another static atlas), then place it inside the Sprite container, and then redraw this container to the RenderTexture. Similarly with deletion/modification of an item.
My code is as follows:
void Scene::copy(Scene const & source)
{
maxnum=source.maxnum;
imagelist = new Image*[maxnum];
for(int i=0; i<maxnum; i++)
{
if(source.imagelist[i] != NULL)
{
imagelist[i] = new Image;
imagelist[i]->xcoord = source.imagelist[i]->xcoord;
imagelist[i]->ycoord = source.imagelist[i]->ycoord;
(*imagelist[i])=(*source.imagelist[i]);
}
else
{
imagelist[i] = NULL;
}
}
}
A little background: The Scene class has a private int called maxnum and an dynamically allocated Array of Image pointers upon construction. These pointers point to images. The copy constructor attempts to make a deep copy of all of the images in the array. Somehow I'm getting a Segfault, but I don't see how I would be accessing an array out of bounds.
Anyone see something wrong?
I'm new to C++, so its probably something obvious.
Thanks,
I would suggest that maxnum (and maybe imagelist) become a private data member and implement const getMaxnum() and setMaxnum() methods. But I doubt that is the cause of any segfault the way you described this.
I would try removing that const before your reference and implement const public methods to extract data. It probably compiles since it is just a reference. Also, I would try switching to a pointer instead of pass by reference.
Alternatively, you can create a separate Scene class object and pass the Image type data as an array pointer. And I don't think you can declare Image *imagelist[value];.
void Scene::copy(Image *sourceimagelist, int sourcemaxnum) {
maxnum=sourcemaxnum;
imagelist=new Image[maxnum];
//...
imagelist[i].xcoord = sourceimagelist[i].xcoord;
imagelist[i].ycoord = sourceimagelist[i].ycoord;
//...
}
//...
Scene a,b;
//...
b.Copy(a.imagelist,a.maxnum);
If the source Image had maxnum set higher than the actual number of items in its imagelist, then the loop would run past the end of the source.imagelist array. Maybe maxnum is getting initialized to the value one while the array starts out empty (or maxnum might not be getting initalized at all), or maybe if you have a Scene::remove_image() function, it might have removed an imagelist entry without decrementing maxnum. I'd suggest using an std::vector rather than a raw array. The vector will keep track of its own size, so your for loop would be:
for(int i=0; i<source.imagelist.size(); i++)
and it would only access as many items as the source vector held. Another possible explanation for the crash is that one of your pointers in source.imagelist belongs to an Image that was deleted, but the pointer was never set to NULL and is now a dangling pointer.
delete source.imagelist[4];
...
... // If source.imagelist[4] wasn't set to NULL or removed from the array,
... // then we'll have trouble later.
...
for(int i=0; i<maxnum; i++)
{
if (source.imagelist[i] != NULL) // This evaluates to true even when i == 4
{
// When i == 4, we're reading the xcoord member from an Image
// object that no longer exists.
imagelist[i]->xcoord = source.imagelist[i]->xcoord;
That last line will access memory that it shouldn't. Maybe the object still happens to exist in memory because it hasn't gotten overwritten yet, or maybe it has been overwritten and you'll retrieve an invalid xcoord value. If you're lucky, though, then your program will simply crash. If you're dealing directly with new and delete, make sure that you set a pointer to NULL after you delete it so that you don't have a dangling pointer. That doesn't prevent this problem if you're holding a copy of the pointer somewhere, though, in which case the second copy isn't going to get set to NULL when you delete-and-NULL the first copy. If you later try to access the second copy of the pointer, you'll have no way of knowing that it's no longer pointing to a valid object.
It's much safer to use a smart pointer class and let that deal with memory management for you. There's a smart pointer in the standard C++ library called std::auto_ptr, but it has strange semantics and can't be used in C++ containers, such as std::vector. If you have the Boost libraries installed, though, then I'd suggest replacing your raw pointers with a boost::shared_ptr.