LIDC-IDRI Lung CT image dataset preprocessing steps and code (Python) - deep-learning

I am new to the lung ct image processing domain. While working with the LIDC-IDRI dataset, I found different methods/steps given on the corresponding website. The content does not share any code snippets or examples for the same.
Could anyone share any related resource (steps/code/verified GitHub repo/...) for preprocessing of the LIDC-IDRI lung ct image dataset for segmentation and/or classification of lung nodules?
I tried to get code for lung ct image processing in the LIdC-IDRI web page but didn't get one. Could anyone please share any related resource (steps/code (python)/verified GitHub repo/...) for preprocessing of the LIDC-IDRI lung ct image dataset for segmentation and/or classification of lung nodules?

Related

Missing custom Inventor materials in SVF/SVF2 models using Autodesk FORGE Model Derivative

A customer has created a custom Inventor Material Library: all his parts are defined to use these custom materials in order to obtain a very realistic representation of model.
When I try to convert his assembly files (exported with Pack & Go) with Autodesk FORGE Model Derivative, the output models (no different between SVF and SVF2 format) contains some white surfaces associated to the parts defined with custom materials.
Interessant: in developer console of browser I've found an error HTTP 400 related to the Galvinized_2_svf_tex_mod.png.
What can I do in order to obtain a better SVF model?
I've found the cause: see Appearances with enabled Self-Illumination are translated to white when creating a shared view from Inventor
The Material Library of customer has the "Self Illumination" active!
I just had a similar issue. I removed the "Self Illumination" and the color came through. I then added "Self Illumination" back to the color, but added the assembly file to my zip with copy and paste in leu of Pack and Go. And the color still works in the viewer.
Don't know if that always works, but if anyone else needs SI, it might be worth a try.
P.S. Good to know that shared views can be a quick test of the Viewer. Hadn't thought of that before.

Autodesk-Forge configurator live sample with geometry change

I want to build enterprise solution for configurator base on Autodesk Inventor models and drawings. I want to have ability to change dimension of assembly (witdh, depth, length and another). I want to see changes on model in real time (something like here but with changes dimensions in custom values: https://tylko.com/shelf/bookcases/1429438/?cv=0&board=cat_all__type_all__id_4267)
As a result I want to have fully documented model with drawings.
Can Forge do that? Is there any demo?
Yes, here's a sample application that does something similar.
https://forge-rimconfigurator-inventor.herokuapp.com/
You can find the source code of the above application here:
https://github.com/sajith-subramanian/forge-rimconfigurator-inventor
There also is another intricate sample available. You can find it here:
http://inventor-config-demo.autodesk.io/
The source code of the above sample is available here:
https://github.com/Autodesk-Forge/forge-configurator-inventor
Thank you for response. Provided samples are different than:
https://tylko.com/shelf/bookcases/1429438/?cv=0&board=cat_all__type_all__id_4267
My sample is dynamic. Is there a possibility to not wait for change update. Same as you can see change directly in Inventor.

BLEU score from Microsoft Translator Hub is 0.00

I just started exploring Microsoft Translator Hub and my question is if i use dictionary to preserve word from being translated from english to korean, then steps for it will be :
Inside my excel, for the first row i will add the language code
For the second row, it will be the value to preserve. For example: Under 'en' column, i will put 'BACK' as the value and under 'ko' column, i will also put 'BACK' as the value.
Upload it as a document.
Uncheck all the other document under training tab, check the document just uploaded under dictionary tab.
Start training the document.
So, ive done all of this steps but the BLEU score still comes out 0.00. Am i doing it wrong? Did i understand the use of Translator Hub wrongly also?
Thank you very much in advanced.
Creating a dictionary-only training in the Microsoft Translator Hub does not produce a BLEU score. This is expected. You didn't upload any training, tuning or test set for this type of training. Refer to the Translator Hub User Guide section 2.6. Dictionary-only training.
To answer your second question, if the training is successful, the 'Evaluate Results' tab shows the machine translation of sentences that were a part of the test dataset. Refer to the Hub User Guide, section 3.3.5. Evaluate Results.
As there was no test set uploaded, the tab shows nothing. This is expected.

Using bvlc_googlenet as pretrained model in digits - errors

digits 4.0 0.14.0-rc.3 /Ubuntu (aws)
training a 5 class GoogLenet model with about 800 training samples in each class. I was trying to use the bvlc_imagent as pre-trained model. These are the steps I took:
downloaded imagenet from http://dl.caffe.berkeleyvision.org/bvlc_googlenet.caffemodel and placed it in /home/ubuntu/models
2.
a. Pasted the "train_val.prototxt" from here https://github.com/BVLC/caffe/blob/master/models/bvlc_reference_caffenet/train_val.prototxt into the custom network tab and
b. '#' commented out the "source" and "backend" lines (since it was complaning about them)
In the pre-trained models text box pasted the path to the '.caffemodel'. in my case: "/home/ubuntu/models/bvlc_googlenet.caffemodel"
I get this error:
ERROR: Cannot copy param 0 weights from layer 'loss1/classifier'; shape mismatch. Source param shape is 1 1 1000 1024 (1024000); target param shape is 6 1024 (6144). To learn this layer's parameters from scratch rather than copying from a saved net, rename the layer.
I have pasted various train_val.prototext from github issues etc and no luck unfortunately,
I am not sure why this is getting so complicated, in older versions of digits, we could just enter the path to the folder and it was working great for transfer learning.
Could someone help?
Rename the layer from "loss1/classifier" to "loss1/classifier_retrain".
When fine-tuning a model, here's what Caffe does:
# pseudo-code
for layer in new_model:
if layer.name in old_model:
new_model.layer.weights = old_model.layer.weights
You're getting an error because the weights for "loss1/classifier" were for a 1000-class classification problem (1000x1024), and you're trying to copy them into a layer for a 6-class classification problem (6x1024). When you rename the layer, Caffe doesn't try to copy the weights for that layer and you get randomly initialized weights - which is what you want.
Also, I suggest you use this network description which is already set up as an all-in-one network description for GoogLeNet. It will save you some trouble.
https://github.com/NVIDIA/DIGITS/blob/digits-4.0/digits/standard-networks/caffe/googlenet.prototxt

How to extract data from an embed Raphael dataset to CSV?

Attempting to extract the data from this Google Politics Insights webpage from "Jan-2012 to the Present" for Mitt Romney and Barack Obama for the following datasets:
Search Trends Based on volume
Google News Mentions Mentions in articles and blog posts
YouTube Video Views Views from candidate channels
For visual example, here's what I mean:
Using Firebug I was able to figure out the data is stored in a format readable by Raphael 2.1.0; looked at the dataset and nothing strikes me as a simple way to convert the data to CSV.
How do I convert the data per chart per presidential candidate into a CSV that has a table for "Search Trends", "Google News Mentions", and "YouTube Video Views" broken down by the smallest increment of time with the results measured in the graph are set to a value of "0.0 to 1.0"? (Note: The reason for "0.0 to 1.0" is the graphs do not appear to give volume info, so the volume is relative to the height of the graph itself.)
Alternatively, if there's another source for all three datasets in CSV, that would work too.
First thing to do is to find out where the data comes from, so I looked up the network traffic in my developer console, and found it very soon: The data is stored as json here.
Now you've got plenty of data for each candidate. I don't know exactly in what relation these numbers are but they definitely are used for their calulation in the graph. I found out that the position in the main.js is on line 392 where they calculate the data with this expression:
Math.log(dataPoints[i][j] * 100.0) / Math.log(logScaleBase);
My guess is: Without the logarithm and a bit exponential calculation you should get the right results.