I'm attempting to perform diffeomorphic demon registration of two MRI images. The processing pipeline has been this so far:
Skull-stripping
Anisotropic Diffusion (smoothing)
Histogram Matching
Affine registration
Diffeomorphic Demons Registration
I didn't come up with the process, just fixing and cleaning up a Slicer software tool several researchers have built. I'm a summer student and quite unfamiliar with domain, however I've started to more less understand what's going on. The problem I've been unable to solve for the past several weeks is this error:
itk::ExceptionObject (0x1231130)
Location: "void itk::ImageToImageFilter<TInputImage, TOutputImage>::VerifyInputInformation() [with TInputImage = itk::Image<itk::Vector<float, 3u>, 3u>; TOutputImage = itk::Image<itk::Vector<float, 3u>, 3u>]"
File: /home/parallels/Desktop/Slicer-SuperBuild/ITKv4/Modules/Core/Common/include/itkImageToImageFilter.hxx
Line: 241
Description: itk::ERROR: DiffeomorphicDemonsRegistrationFilter(0x13870b0): Inputs do not occupy the same physical space!
InputImage Origin: [7.9639916e+01, -1.1042095e+02, -1.0426932e+02], InputImageMovingImage Origin: [-8.8678563e+01, -1.4923204e+02, 1.2193930e+02]
Tolerance: 1.5000000e-05
InputImage Spacing: [1.5000000e+01, 1.5000000e+01, 1.9199951e+01], InputImageMovingImage Spacing: [1.5154560e+01, 1.5108180e+01, 1.9319538e+01]
Tolerance: 1.5000000e-05
InputImage Direction: 1.5926319e-08 1.4070701e-08 -1.0000000e+00
9.9237583e-01 -1.2324859e-01 1.4070700e-08
1.2324859e-01 9.9237583e-01 1.5926320e-08
, InputImageMovingImage Direction: -0.0000000e+00 5.5205551e-10 1.0000000e+00
5.5205551e-10 1.0000000e+00 -5.5205553e-10
-1.0000000e+00 5.5205551e-10 0.0000000e+00
Tolerance: 1.0000000e-06
From what I understand, the diffeomorphic registration relies on the two images being coincident, hence the affine registration step beforehand. For some reason though, the affine transformation doesn't line up the two images properly. As a result, they don't occupy the same physical space. I'm clearly missing something but can't seem to figure out what it is.
The affine registration is performed with this file (I made some edits on my local copy to comply with the Slicer module's way of input/output but they're nearly identical). The transform that was created had the following characteristics:
Optimizer stop condition: RegularStepGradientDescentOptimizerv4: Step too small after 33 iterations. Current step (9.76563e-05) is less than minimum step (0.0001).
Result =
Center X = -1.95155
Center Y = 11.6381
Center Z = 36.5165
Translation X = 1.09423
Translation Y = 0.021133
Translation Z = -0.0154539
Iterations = 34
Metric value = 8974.52
Scale 1 = 1.15384
Scale 2 = 1.08962
Angle (degrees) = -5.6116
The following code was used to save the transform to a file:
// Write the transform to a file
itk::TransformFileWriter::Pointer transformWriter = itk::TransformFileWriter::New();
transformWriter->SetInput( registration->GetOutput()->Get() );
transformWriter->SetFileName( outputMatrix.c_str() );
transformWriter->Update();
And the demon registration is performed with this file (Made a few edits but mostly the same as well). The affine transform is loaded at line 799.
Reading transform from transform input file.
Found: AffineTransform (0x1e0c0b0)
RTTI typeinfo: itk::AffineTransform<double, 3u>
Reference Count: 3
Modified Time: 1322
Debug: Off
Object Name:
Observers:
none
Matrix:
1.01338 0.0887047 0.0223631
-0.11891 1.09423 0.021133
-0.0154539 0.0302253 1.14062
Offset: [-0.256888, -34.7809, -17.895]
Center: [-1.95155, 11.6381, 36.5165]
Translation: [1.56597, -32.6804, -12.3781]
Inverse:
0.977286 -0.0787352 -0.0177019
0.105999 0.905809 -0.0188607
0.0104321 -0.0250698 0.876975
Singular: 0
Edit: Commenting out the following line does not change anything in the resulting output. ie the initial displacement is not being applied at all. Still don't know why.
typedef typename itk::MultiResolutionPDEDeformableRegistration <ImageType, ImageType, DeformationFieldType, PixelType > MultiResRegistrationFilterType;
typename MultiResRegistrationFilterType::Pointer multires = MultiResRegistrationFilterType::New();
// Commenting out this line doesn't change anything
multires->SetArbitraryInitialDisplacementField ( inputDefField );
It looks like the Diffeomorphic Demons Registration code you are using requires that Fixed and Moving images have the same physical space, meaning:
Origin
Spacing
Dimensions
Did you try to resample your moving image to the fixed image space, and then call the Diffeomorphic Demons Registration with this resampled moving image and fixed image?
I didn't know this EZminc code but it looks like it requires both inputs to be in the same space somehow.
Related
I'd like to extract the "col" color value from this function to be used to paint plots or candle colors. But everything I try creates one error or another. I checked the Script Reference. Shouldn't there be some way to "return" a value, as is usually the case with most functions?
lset(l,x1,y1,x2,y2,col)=>
line.set_xy1(l,x1,y1)
line.set_xy2(l,x2,y2)
line.set_width(l,5)
line.set_style(l, line.style_solid)
line.set_color(l,y2 > y1 ? #ff1100 : #39ff14) //red : green
temp = line.get_price(l,bar_index) // another value to extract
The documentation is showing it like this:
line.new(x1, y1, x2, y2, xloc, extend, color, style, width) → series line
So in your code it's looking differently and also the "new" is missing.
Scrolling a bit up on the linked page shows that there exist indeed methods to retrieve some properties of the line object:
Lines are managed using built-in functions in the line namespace. They include:
line.new() to create them.
line.set_*() functions to modify the properties of an line.
line.get_*() functions to read the properties of an existing line.
line.copy() to clone them.
line.delete() to delete them.
The line.all array which always contains the IDs of all
the visible lines on the chart. The array’s size will depend on
the maximum line count for your script and how many of those you
have drawn. aray.size(line.all) will return the array’s size.
The most simple usage is to instantiate a line object with the correct values directly, like shown here:
//#version=5
indicator("Price path projection", "PPP", true, max_lines_count = 100)
qtyOfLinesInput = input.int(10, minval = 1)
y2Increment = (close - open) / qtyOfLinesInput
// Starting point of the fan in y.
lineY1 = math.avg(close[1], open[1])
// Loop creating the fan of lines on each bar.
for i = 0 to qtyOfLinesInput
// End point in y if line stopped at current bar.
lineY2 = open + (y2Increment * i)
// Extrapolate necessary y position to the next bar because we extend lines one bar in the future.
lineY2 := lineY2 + (lineY2 - lineY1)
lineColor = lineY2 > lineY1 ? color.lime : color.fuchsia
line.new(bar_index - 1, lineY1, bar_index + 1, lineY2, color = lineColor)
Getting the line color from outside is difficult or impossible though as there never exists a method to retrieve it while for other properties those methods exist.
So the most simple way is to create the same funcionality, to get the color that exists inside the line-object, outside too, or only outside.
currentLineColor = y2 > y1 ? #ff1100 : #39ff14
You could try to extend the line-object somehow like this:
line.prototype.get_color = function() {
return this.color;
};
console.log(line.get_color())
I'm not sure if the approach with the prototype is working but it's worth it to try if you need it.
So I grabbed latest code from https://github.com/bulletphysics/pybullet_robots
I modified laikago.py example to the following, I'm only controlling one joint using torque.
Strangely enough poor robot flies away from the scene :)
There are two ways to fix it: either decrease simulation time step (line 10) or change torque value to 60 (line 33).
Any ideas what am I doing wrong?
import pybullet as p
import time
import os
os.chdir(os.path.dirname(os.path.abspath(__file__)))
p.connect(p.GUI)
plane = p.loadURDF("data/plane.urdf")
p.setGravity(0,0,-9.8)
timeStep = 1./300 # HERE!!! changing it to 1./400 fixes the problem
p.setTimeStep(timeStep)
urdfFlags = p.URDF_USE_SELF_COLLISION
quadruped = p.loadURDF("laikago/laikago_toes.urdf",[0,0,.5],[0,0.5,0.5,0], flags = urdfFlags,useFixedBase=False)
#enable collision between lower legs (2,5,8 and 11)
lower_legs = [2,5,8,11]
for l0 in lower_legs:
for l1 in lower_legs:
if (l1>l0):
p.setCollisionFilterPair(quadruped, quadruped, l0, l1, 1)
#set damping
for j in range (p.getNumJoints(quadruped)):
p.changeDynamics(quadruped, j, linearDamping=0, angularDamping=100)
p.setJointMotorControl2(quadruped, j, p.VELOCITY_CONTROL, force=0)
#run the simulation
p.setRealTimeSimulation(0)
for i in range (1000):
p.setJointMotorControl2(quadruped, 13, p.TORQUE_CONTROL, force=62) # HERE!!! changing it to force=60 fixes the problem
p.stepSimulation()
time.sleep(timeStep)
I am a bit late to the party, but maybe you are still in search of an answer. The reason is given here: https://pybullet.org/Bullet/phpBB3/viewtopic.php?t=12644 and a note was added to: https://docs.google.com/document/d/10sXEhzFRSnvFcl3XxNGhnD4N2SedqwdAvK3dsihxVUA/edit#heading=h.jxof6bt5vhut
The reason: "You need to first unlock/disable the default velocity/position motor. [...] Otherwise, your force/torque has to exceed the default motor." (see first link).
To fix do (same link):
pybullet.setJointMotorControl2(objUid, linkIndex, p.VELOCITY_CONTROL, force=0)
I'm using RL4J (the Reinforcement Learning framework integrated in DeepLearning4J) for making a car complete the lap in a track for a videogame.
I save the the model after the training with this code:
QLearningDiscreteConv<ScreenFrameState> dql = new QLearningDiscreteConv(mdp, RACING_NET_CONFIG, RACING_HP, RACING_QL, manager);
dql.train();
dql.getNeuralNet().save(model);
After the model is saved, I'd like to see how it behaves, and so I load it to play it:
DQN load = DQN.load(model);
QLearningDiscreteConv<ScreenFrameState> dql = new QLearningDiscreteConv(mdp, load, RACING_HP, RACING_QL, manager);
dql.getPolicy().play(mdp);
but it fails while loading with this error:
org.deeplearning4j.exception.DL4JInvalidInputException: Cannot do forward pass in Convolution layer (layer name = layer0, layer index = 0): input array depth does not match CNN layer configuration (data input depth = 109, [minibatch,inputDepth,height,width]=[1, 109, 150, 3]; expected input depth = 10) (layer name: layer0, layer index: 0)
at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.preOutput(ConvolutionLayer.java:294)
at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.preOutput(ConvolutionLayer.java:248)
at org.deeplearning4j.nn.layers.convolution.ConvolutionLayer.activate(ConvolutionLayer.java:392)
at org.deeplearning4j.nn.layers.AbstractLayer.activate(AbstractLayer.java:309)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.activationFromPrevLayer(MultiLayerNetwork.java:789)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.feedForwardToLayer(MultiLayerNetwork.java:929)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.feedForward(MultiLayerNetwork.java:870)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.feedForward(MultiLayerNetwork.java:861)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.silentOutput(MultiLayerNetwork.java:1906)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:1898)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:1871)
at org.deeplearning4j.nn.multilayer.MultiLayerNetwork.output(MultiLayerNetwork.java:1952)
at org.deeplearning4j.rl4j.network.dqn.DQN.output(DQN.java:49)
at org.deeplearning4j.rl4j.policy.DQNPolicy.nextAction(DQNPolicy.java:32)
at org.deeplearning4j.rl4j.policy.DQNPolicy.nextAction(DQNPolicy.java:18)
at org.deeplearning4j.rl4j.policy.Policy.play(Policy.java:72)
at org.deeplearning4j.rl4j.policy.Policy.play(Policy.java:27)
at me.andreaiacono.racinglearning.rl.QLearning.race(QLearning.java:81)
at me.andreaiacono.racinglearning.core.player.QLearningPlayer.race(QLearningPlayer.java:19)
at me.andreaiacono.racinglearning.gui.GameWorker.doInBackground(GameWorker.java:56)
at me.andreaiacono.racinglearning.gui.GameWorker.doInBackground(GameWorker.java:11)
at javax.swing.SwingWorker$1.call(SwingWorker.java:295)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at javax.swing.SwingWorker.run(SwingWorker.java:334)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:748)
The input is correct: my screen is 150 * 109 pixel with 3 color channels; why does it expect a size of 10 when loading? What am I missing?
Thanks,
Andrea
(data input depth = 109, [minibatch,inputDepth,height,width]=[1, 109, 150, 3]; expected input depth = 10)
That looks like you set inputDepth to 109, whereas it should be set to 3 (number of channels). I'm not familiar with dl4j personally, so not sure why it would say "expected input depth = 10", but I suppose you could at least try switching around the order in which you gave those arguments.
What is the version that you use? Soetimes if you use the snapshot repository, a temporary mistake is possible, but the guys fix it really quick. So by chance you might have taken code from snapshot at this moment. Take s stable version.
I want to load data from CSV file like this:
var format = new CSVFormat('.', ' ');
IVersatileDataSource source = new CSVDataSource(filename, false, format);
var data = new VersatileMLDataSet(source); ...
Then I have two options:
Use EncogModel
var model = new EncogModel(data);
model.SelectMethod(data, MLMethodFactory.TypeFeedforward); ...
Make own network
var network = new BasicNetwork();
network.AddLayer(new BasicLayer(null, true, 11));
network.AddLayer(new BasicLayer(new ActivationSigmoid(), true, 8));
network.AddLayer(new BasicLayer(new ActivationTANH(), true, 5));
...
IMLDataSet trainingSet = new BasicMLDataSet(input, output);
I don't know how to set number of layers, neurons and activation functions with first option (Encog Model). All I get is some default feedforward network with one hidden layer only.
I don't know how can get easily input and output arrays separately for my own network (second option) from VersatileMLDataSet. I can get whole array (input + output), but there must be a way how to get only input array or output array.
I found answer in documentation (Encog Method & Training Factories, page 75), with EncogModel is possible customize network like this:
var methodFactory = new MLMethodFactory();
var method = methodFactory . Create(
MLMethodFactory .TYPEFEEDFORWARD,
”?:B−>SIGMOID−>4:B−>SIGMOID−>?”,
2,
1);
The above code creates a neural network with two input neurons and one
output neuron. There are four hidden neurons. Bias neurons are placed
on the input and hidden layers. As is typical for neural networks,
there are no bias neurons on the output layer. The sigmoid activation
function is used between both the input and hidden neuron, as well
between the hidden and output layer. You may notice the two question
marks in the neural network architecture string. These will be filled
in by the input and output layer sizes specified in the create method
and are optional. You can hard-code the input and output sizes. In
this case the numbers specified in the create call will be ignored.
I have something like 40 million TIFF documents, all 1-bit single page duplex. In about 40% of cases, the back image of these TIFFs is 'blank' and I'd like to remove them before I do a load to a CMS to reduce space requirements.
Is there a simple method to look at the data content of each page and delete it if it falls under a preset threshold, say 2% 'black'?
I'm technology agnostic on this one, but a C# solution would probably be the easiest to support. Problem is, I've no image manipulation experience so don't really know where to start.
Edit to add: The images are old scans and so are 'dirty', so this is not expected to be an exact science. The threshold would need to be set to avoid the chance of false positives.
You probably should:
open each image
iterate through its pages (using Bitmap.GetFrameCount / Bitmap.SelectActiveFrame methods)
access bits of each page (using Bitmap.LockBits method)
analyze contents of each page (simple loop)
if contents is worthwhile then copy data to another image (Bitmap.LockBits and a loop)
This task isn't particularly complex but will require some code to be written. This site contains some samples that you may search for using method names as keywords).
P.S. I assume that all of images can be successfully loaded into a System.Drawing.Bitmap.
You can do something like that with DotImage (disclaimer, I work for Atalasoft and have written most of the underlying classes that you'd be using). The code to do it will look something like this:
public void RemoveBlankPages(Stream source stm)
{
List<int> blanks = new List<int>();
if (GetBlankPages(stm, blanks)) {
// all pages blank - delete file? Skip? Your choice.
}
else {
// memory stream is convenient - maybe a temp file instead?
using (MemoryStream ostm = new MemoryStream()) {
// pulls out all the blanks and writes to the temp stream
stm.Seek(0, SeekOrigin.Begin);
RemoveBlanks(blanks, stm, ostm);
CopyStream(ostm, stm); // copies first stm to second, truncating at end
}
}
}
private bool GetBlankPages(Stream stm, List<int> blanks)
{
TiffDecoder decoder = new TiffDecoder();
ImageInfo info = decoder.GetImageInfo(stm);
for (int i=0; i < info.FrameCount; i++) {
try {
stm.Seek(0, SeekOrigin.Begin);
using (AtalaImage image = decoder.Read(stm, i, null)) {
if (IsBlankPage(image)) blanks.Add(i);
}
}
catch {
// bad file - skip? could also try to remove the bad page:
blanks.Add(i);
}
}
return blanks.Count == info.FrameCount;
}
private bool IsBlankPage(AtalaImage image)
{
// you might want to configure the command to do noise removal and black border
// removal (or not) first.
BlankPageDetectionCommand command = new BlankPageDetectionCommand();
BlankPageDetectionResults results = command.Apply(image) as BlankPageDetectionResults;
return results.IsImageBlank;
}
private void RemoveBlanks(List<int> blanks, Stream source, Stream dest)
{
// blanks needs to be sorted low to high, which it will be if generated from
// above
TiffDocument doc = new TiffDocument(source);
int totalRemoved = 0;
foreach (int page in blanks) {
doc.Pages.RemoveAt(page - totalRemoved);
totalRemoved++;
}
doc.Save(dest);
}
You should note that blank page detection is not as simple as "are all the pixels white(-ish)?" since scanning introduces all kinds of interesting artifacts. To get the BlankPageDetectionCommand, you would need the Document Imaging package.
Are you interested in shrinking the files or just want to avoid people wasting their time viewing blank pages? You can do a quick and dirty edit of the files to rid yourself of known blank pages by just patching the second IFD to be 0x00000000. Here's what I mean - TIFF files have a simple layout if you're just navigating through the pages:
TIFF Header (4 bytes)
First IFD offset (4 bytes - typically points to 0x00000008)
IFD:
Number of tags (2-bytes)
{individual TIFF tags} (12-bytes each)
Next IFD offset (4 bytes)
Just patch the "next IFD offset" to a value of 0x00000000 to "unlink" pages beyond the current one.