I am trying to build a CV model for detecting objects in videos. I have about 6 videos that have the content I need to train my model. These are things like lanes, other vehicles, etc. that I’m trying to detect.
I’m curious about the format of the dataset I need to train my model with. I can have each frame of each video turn into images and create a large repository of images to train with or I can use the videos directly. Which way do you think is better?
I apologize if this isn't directly a programming question. I'm trying to assemble my data and I couldn't make up my mind about this.
Yolo version 3 is a good starting point. The trained model will have a .weight file and a .cfg file which can be used to detect object from webcam, video in computer or, in Android with opencv.
In opencv python, cv.dnn.readNetFromDarknet("yolov3_tiny.cfg", "CarDetector.weights") can be used load the trained model.
In android similar code,
String tinyYoloCfg = getPath("yolov3_tiny.cfg", this);
String tinyYoloWeights = getPath("CarDetector.weights", this);
Net tinyYolo = Dnn.readNetFromDarknet(tinyYoloCfg, tinyYoloWeights);
Function reference can be found here,
https://docs.opencv.org/4.2.0/d6/d0f/group__dnn.html
Your video frames need to be annotated with a tool that generates bounding boxes in yolo format and there are quite a few available. In order to train custom model this repository contains all necessary information,
https://github.com/AlexeyAB/darknet
Related
I have a dxf file that I have uploaded to oss and translated it to svf.
How can I fetch the layer states in my 2D model using forgeviewer?
In autocad, I have these layer states Screenshot for autocad layer states.
Namely:
F1 Component Plan
F2 Electrical Plan
F3 Bracket Plan
But when in the forgeviewer, I cant find those lawyer states (Grouping).
I'm afraid this type of information may not be available. Generally, the Forge Model Derivative service always tries to extract a "balanced" amount of information from design files (enough for previewing purposes, but not too much, to make sure the output keeps a reasonable size).
When you load your DXF file into the viewer, you can try querying its metadata using viewer.model.getData(), and see if you find the layer state there. I did try that with one of my testing DXFs, and didn't see this information there.
Finally, if you really do need the layer state data, there's another option - you could use Design Automation for AutoCAD, with a custom AutoCAD plugin that would extract all the information you need directly from the source file.
I’m new to deep learning and I was reading some state of art papers and I found that mask r-cnn is utterly used in segmentation and classification of images. I would like to apply it to my MSc project but I got some questions that you may be able to answer. I apologize if this isn’t the right place to do it.
First, I would like to know what are the best strategy to get the annotations. It seems kind of labor intensive and I’m not understanding if there is any easy way. Following that, I want to know if you know any annotation tool for mask r-cnn that generates the binary masks that are manually done by the user.
I hope this can turn into a productive and informative thread so any suggestion, experience would be highly appreciated.
Regards
You can use MASK-RCNN, I recommend it, is a two-stage framework, first you can scan the image and generate areas likely contain an object. And the second stage classifies the proposal drawing bounding boxes.
But the two-big question
how to train a model from scratch? And What happens when we want to
train our own dataset?
You can use annotations downloaded from the internet, or you can start creating your own annotations, this takes a lot of time!
You have tools like:
VIA GGC image annotator
http://www.robots.ox.ac.uk/~vgg/software/via/via_demo.html
it's online and you don't have to download any program. It is the one that I recommend you, save the images in a .json file, and so you can use the class of ballons that comes by default in SAMPLES in the framework MASK R-CNN, you would only have to put your json file and your images and to train your dataset.
But there are always more options, you have labellimg which is also used for annotation and is very well known but save the files in xml, you will have to make a few changes to your Class in python. You also have labelme, labelbox, etc.
I currently have 6 animated models each between 5Mb and 8Mb each. When using each one individually it can take a little time to download them. I need to load these models up as and when I need them and not in the preloader of libGdx. All 6 models will take a long time to download in the preloader so I'm trying to bypass it.
I've used both GWT and libGdx many times but not together. This project is purely a html one and I'm familiar with classes that need to be serialised so data can be transferred back and forth using GWT's RPC methods.
So far I've come up with 2 ideas of doing this:
Transfer the model data and rebuild the mesh from scratch. This
would take a lot of time and just wouldn't work. Plus it's likely I'll lose such data like animations.
Using LibGdx's
ModelData class which would work perfect, but unfortunately non of
the main class and sub classes are serialised.
The current project has interfaces that bridge the platform specific from the 'core' to the 'html' which can then async the RPC calls. These work with libGdx just great.
Is there a way of skipping the preloader and loading the models on demand when they are needed?
If you need anymore information I'll be glad to add that in.
Take a look at the Dynamic Asset Loading with libGDX and GWT example by MonsterOfCookie: https://github.com/MonsterOfCookie/libGDXGwtHtmlExample
Disadvantage is that you have to compile your own libGDX fork because Monster's PR was not merged. (But for working seriously with libGDX' GWT backend you'll probably need your own fork anyway)
General question: What is a stable JSON format for loading 3d models that is currently widely used?
Extended question:
I'm a programmer doing some WebGL stuff, particularly with the Elm programming language.
I've been looking at different model formats, and it seems like using the three.js JSON format as a kind of standard makes a lot of sense for my project (and potentially for the langauge in general).
However, on the three.js github, it says that version 3 of the model langauge is soon to be depricated.
So, I'd like to know the current status of the model format before I convert. Is version 3 stable for the time being? Has version 4 been released? Will there be a version 4 model format, or is it being replaced by the geometry format? Are the upcoming changes so breaking that adopting the format at this point in time is a bad idea?
I've seenthe new ObjectLoader classes, how do these relate to the Json mesh formats? Is it a format that I can convert .obj to?
To follow up on my github post:
To be honest I don't think its safe to say that version 3 was never too stable. 3 has always had issues and the SceneLoader class that supports grew to be kind of unfriendly to maintain. Now 4 is pretty stable just lacking in support for textures. It is fine for objects, geometry, and materials but there is no exporter yet (that I am aware of).
Now what I think you are most curious about is the actual model format, which is this:
https://github.com/mrdoob/three.js/wiki/JSON-Geometry-format-4#example-of-geometry
To be honest the actual geometry format hasn't really changed much that I can tell. The big change between 3 and 4 (so far) is the scene formatting. In fact geometry is parsed with the JSONLoader class. In fact a couple days I committed, to dev branch, a new example file for msgpack compressed JSON scenes.
https://github.com/mrdoob/three.js/blob/dev/examples/webgl_loader_msgpack.html
msgpack is just JSON compression, so when it is decoded it is a JSON object. This msgpack file was converted from three.js/blob/dev/examples/scenes/robo_pigeon.js
This scene is a version 4 scene format. Each entry in the "geometries" table is actually an embedded geometry format. This format can also live in an external file. If you compare it to the first link you will see the formats are the same. Geometry files can be individually loaded into a scene with JSONLoader.
Now you asked about converters: glancing at convert_obj_three.py it says "JSON model version," in the documentation so I am going to guess it spits out a basic geometry model format and not a scene format so this may be usable. Even the blender exporter can still export compatible geometry scenes (leave the "Scene" option checked off). How do I know? Because the geometry I used for robo_pigeon.js came from that exporter, I just had to construct the version 4 scene by hand.
Does this begin to answer your question?
According to mrdoob he is planning to change the geometry format but as of this very moment the version 3 model format works fine in a version 4 scene because ObjectLoader passes those geometry (model) definitions to JSONLoader. So until a new format is actually spec'd out and JSONLoader is updated the version 3 model is the current one.
One more note: scene loaders (SceneLoader, ObjectLoader) don't natively load geometry. They always dispatch the tasks to the correct class. Not sure if it is supported yet in version 4 but in version 3 you could directly link the scene to OBJ files. And speaking of OBJ files, if you are just starting to poke at three.js and have assets in OBJ then have you considered just working with the OJBLoader directly?
I was consulting several references to discover how I may output trained Weka models into Java source code so that I may use the classifiers I am training in actual code for research applications I have been developing.
As I was playing with Weka 3.7, I noticed that while it does output Java code to its main text buffer when use simpler classification (supervised in my case this time) methods such as J48 decision tree, it removes the option (rather, it voids it by removing the ability to checkmark it and fades the text) to output Java code for RandomTree and RandomForest (which are the ones that give me the best performance in my situation).
Note: I am clicking on the "More Options" button and checking "Output source code:".
Does Weka not allow you to output RandomTree or RandomForest as Java code? If so, why? Or if it does and just doesn't put it in the output buffer (since RF is multiple decision trees which I imagine it doesn't want to waste buffer space), how does one go digging up where in the file system Weka outputs java code by default?
Are there any tricks to get Weka to give me my trained RandomForest as Java code? Or is Serialization of the output *.model files my only hope when it comes to RF and RandomTree?
Thanks in advance to those who provide help.
NOTE: (As an addendum to the answer provided below) If you run across a similar situation (requiring you to use your trained classifier/ML model in your code), I recommend following the links posted in the answer that was provided in response to my question. If you do not specifically need the Java code for the RandomForest, as an example, de-serializing the model works quite nicely and fits into Java application code, fulfilling its task as a trained model/hardened algorithm meant to predict future unlabelled instances.
RandomTree and RandomForest can't be output as Java code. I'm not sure for the reasoning why, but they don't implement the "Sourceable" interface.
This explains a little about outputting a classifier as Java code: Link 1
This shows which classifiers can be output as Java code: Link 2
Unfortunately I think the easiest route will be Serialization, although, you could maybe try implementing "Sourceable" for other classifiers on your own.
Another, but perhaps inconvenient solution, would be to use Weka to build the classifier every time you use it. You wouldn't need to load the ".model" file, but you would need to load your training data and relearn the model. Here is a starters guide to building classifiers in your own java code http://weka.wikispaces.com/Use+WEKA+in+your+Java+code.
Solved the problem for myself by turning the output of WEKA's -printTrees option of the RandomForest classifier into Java source code.
http://pielot.org/2015/06/exporting-randomforest-models-to-java-source-code/
Since I am using classifiers with Android, all of the existing options had disadvantages:
shipping Android apps with serialized models didn't reliably work across devices
computing the model on the phone took too much resources
The final code will consist of three classes only: the class with the generated model + two classes to make the classification work.