Skip to content

GitLab

Commit 92873447 authored by Jean-Loup Haberbusch's avatar Jean-Loup Haberbusch
Browse files

ADDED Create manual segmentation tool

parent 2e3466bc
Pipeline #15686 failed with stage
in 3 minutes and 43 seconds
Hide whitespace changes
Inline Side-by-side
docs/Getting Started/Examples and Tutorials/Create_a_manual_segmentation_tool.md 0 → 100644
View file @ 92873447
CamiTK is a great tool that brings you the visualization power of VTK,
image processing power of ITK and GUI handling capabilities of Qt
letting you only concentrate on your work. There are many ways to go
about implementing what you need in CamiTK. In this tutorial, you will
be presented with on such way: **development in increments**.
For this tutorial, let’s create a **segmentation tool** that lets you
manually segment an image by drawing along the contours of the objects
of interest similar to the ITK-SNAP tool[^1] using the polygon mode.
First, let’s define the required functionality of this tool.
1. Draw contours in a given orthogonal viewer (axial, coronal or
sagittal) of CamiTK-imp (as different objects can be easily seen in
different views)
2. Copy an already drawn contour and paste it on another slice (as
modifying a contour may be faster than drawing it from scratch)
3. Visually see the segmentation in all four default viewers (axial,
coronal, sagittal and 3D) of CamiTK-imp
4. Save the segmentation as an image
5. Make the tool less error prone
And let’s do the implementation **in
increments**.
!!! warning
This tutorial was created and tested on CamiTK 4.0.4 using VTK 6.3 and QT 5.8. Depending on your configuration there may be slight changes or issues. |
## Step 01: Use the wizard to create a CEP
We will use this tool to perform some operations on images. Hence, we
only need to create a CamiTK Action that acts on CamiTK ImageComponents.
But following the good practices of CamiTK let us create a CEP called
Segmentor. The Segmentor CEP will have only one action also called
Segmentor.
Let’s create this CEP using the CamiTK-Wizard. While creating the action
let’s add two parameters to it.
1. Contour Color: the color of the contour
2. Contour Width: the line width of the contour
The autogenerated code skeleton can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/01.zip).
And let’s use this code base to develop our tutorial.
Build the CEP and add the Segmentor action to CamiTK-imp. Load an image
to CamiTK-imp and you will find the **Segmentor** action under
**Tutorial** action family. When you apply the action, it will create a
copy of the current image by executing the automatically generated
code.
## Step 02: Remove the unnecessary code snippets and create the necessary GUI components
Usually, an action is executed when the user clicks the **Apply**
button. This behavior is suitable when you perform only one operation on
the data component. Since we are planning to use this tool to perform
multiple operations on an image (draw contours, paste the last drawn
contour, create segmented image, *etc.*), using only the Apply button is
not adequate. Consequently, we should disable the default Apply button
and provide our own GUI widgets to handle our needs.
We can disable the default Apply button in the following
manner:
```c++
dynamic_cast<ActionWidget*>(Action::getWidget())->setButtonVisibility(false);
```
Let us determine the additional GUI widgets that we require for this
tool.
- Radio buttons to specify on which viewer the contours are drawn
- Single slice related widgets
- Push button to complete the current contour
- Push button to reset the current contour
- Push button to accept the current drawn contour as the final
contour
- Push button to paste the last drawn contour on the selected
slice
- Whole image related widgets
- Reset everything
- Save the segmentation as an image
The above mentioned widgets will be created and added to a **QFrame**.
The default widget created by the Action will also be added to this
QFrame. Ultimately, this QFrame will be returned by the overloaded
`getWidget( )` method of the action.
Finally, let’s remove the content of the `
apply( )` method and delete the `
process( )` method given by default.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/02.zip).
When you load the action and click on the provided buttons nothing
executes as we have not specified what to do when the user interacts
with those
widgets.
## Step 03: Draw contours in a given orthogonal viewer (axial, coronal or sagittal) of CamiTK-imp
As the viewer on which the contours are drawn is going to be used
throughout the implementation, we will create an ENum for the viewers.
We will update the currently selected viewer depending on the radio
button selected in the GUI. We could listen to the `Clicked( )` signal of the radio
button and set the currently selected viewer accordingly. In order to do
this, we need to access the GUI widgets. Hence, we should set the
objects names of the three radio buttons using the
`setObjectName( )` method. Since we
are going to use the signals and slots mechanism, we have to include the
`Q_OBJECT` macro in the class header
file.
In order to draw contours on a viewer we can use the
`vtkContourWidget`[^2]. We will
create a member variable that will hold an instance of a
vtkContourWidget. The contour should be drawn on the selected slice of
the selected viewer. In order to calculate the plane where the contour
should be drawn, we require the information of the ImageComponent on
which the contours are drawn. We will use another member variable to
hold the currently selected ImageComponent call it
`currentImageComp`. We should
set/check `currentImageComp` every
time the `QWidget` of the action is
requested. If the selected ImageComponent is different from the
previously selected one, then we have to handle the changes. We will see
how this is done at a later step. For the moment, let’s assume that the
actions will be carried out on the same ImageComponent.
We should draw on only one slice at a time. Hence, every time we want to
draw contours on a different viewer or a different slice of the same
viewer we have to reset the contours. Let’s define the
`resetCurrentContourWidget( )` method
that handles the above mentioned requirements. And let’s setup this
method to be executed when the **Reset** button is clicked and also when
the selected viewer is changed.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/03.zip).
Once the action is loaded, you will be able to draw contours on any of
the slices of the
image.
## Step 04: Update the color and width of the contour and implement automatic contour completion
Try changing the color or width of the contour. You will observe that
nothing happens although we update the color and width of the contour
inside the `resetCurrentContourWidget( )` method. That is because the values of properties of an action
are updated only when the **Apply** button is clicked. But we have
disabled the **Apply** button of the action previously. In order to
correct this behavior all we need to do is to set the
`setAutoUpdateProperty( )` to true of
the `ActionWidget` when we generate
our custom `QWidget` of the action.
The `vtkContourWidget` comes with a
handy method to complete the currently drawn contour. In order to have a
closed loop we have to make sure at least two points are already drawn
when completing the contour. Let’s implement that in a method called
`completeCurrentContour( )` and set
it up with the `clicked( )` signal of
the Complete button.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/04.zip).
Now you will be able to complete the contour
drawn.
## Step 05: Build the segmentation image and show the region inside the contour in the orthogonal viewers
!!! note
This is the heart of our tool and there are numerous ways of tackling this problem. Our solution is inspired by the approach used in ITK-SNAP.
First we are going to create an image that will hold information about
the development of the segmentation. Let’s define a member variable to
hold this image called `segmentedImage`. The first time the `currentImageComp` is assigned, we create
`segmentedImage` and set every voxel
of it to black. Every time a new region is added this image will be
updated to reflect the changes. In other words,
`segmentedImage` will work as a mask
to represent the segmentation.
Now that we have our `segmentedImage`,
we can move onto showing the region inside the contour on the respective
orthogonal viewers. It is done in two steps.
1. Find the voxels that correspond to the region inside the contour and
set those voxels of `segmentedImage
` as white (done by `updateSegmentedImage( )` method)
2. Find the intersection between the white voxels of
`segmentedImage` and the
currently displayed slice of the `currentImageComp` and draw an overlay on each orthogonal
viewer (done by `drawOverlaysOn2DViewers( )` method)
First step will be implemented in the `updateSegmentedImage( )` method. The heart of this step is the
use of `vtkPolyDataToImageStencil`[^3] filter that converts the PolyData
into an image stencil. The PolyData from the
`contourWidget` will first be
triangulated and then extruded using the `vtkContourTriangulator` and `vtkLinearExtrusionFilter` filters respectively. With contours,
`vtkPolyDataToImageStencil` only
works if the image planes are aligned with the axial planes (see [^4]).
Consequently, if the contours are drawn on the sagittal or coronal
viewers we have to transform the relevant plane to be along the axial
plane. For the sagittal viewer the **original x,y,z** coordinates will
become **y,z,x** and for the coronal viewer the **original x,y,z**
coordinates will become **z,x,y**. Once the PolyData is transformed into
an image stencil, it will be cut using another temporary while image in
order to find the corresponding selected voxels from the original region
inside the contour. Finally, these voxels will be marked as white in the
`segmentedImage`. In order to achieve
this, we do not have to check the whole `segmentedImage` but only the corresponding slice along the
relevant direction.
In the second step we will implement the `drawOverlaysOn2DViewers( )` method. We will have to interact with
all 2D orthogonal viewers during this step. In order to do that let’s
create an actor and a mapper for each orthogonal viewer and add that to
each `RendererWidget` or the
corresponding viewer. We do not have to do this every time the
`drawOverlaysOn2DViewers( )` method
is called. Instead we will do this once when the
`QWidget` of the action is created
and make the actors and mappers available as members of the class. Since
`vtkImageData` is a structured point
dataset, we can use `vtkImageDataGeometryFilter` to extract a single slice from the
segmentedImage. Then, we can use `vtkThresholdPoints` to retrieve only the points that are white.
Finally, we can give these white points as the input for the mapper of
the corresponding viewer which in turn will take care of drawing the
overlay.
These steps will be executed with the `clicked( )` signal of the **Accept** button. Let’s call the
corresponding slot `acceptCurrentContour(
)`.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/05.zip).
Now when you click the **Accept** button the region inside the contour
will be highlighted and the corresponding regions of other viewers will
also be highlighted.
## Step 06: Auto update the segmented regions with slice sliders
Draw contours on more than one slice and you will observe that the
previously drawn contours become invisible when you move back to the
previous slice.
That is normal as we have to update the 2D overlay when the slices are
changed. We can do that by listening to the `selectionChanged( )` signal of the 2D viewers. We only have to
update the corresponding 2D viewer. Let’s name the corresponding slot as
`updateOverlayOnSingle2DViewer( )`
method. Since repeating the same lines of code is not a good coding
practice, we will modify the `drawOverlaysOn2DViewers( )` method to handle this situation as
well.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/06.zip).
Now when you change the viewer sliders the segmented regions will be
correctly updated.
## Step 07: Show the segmented regions in the 3D viewer
We will create another member variable to hold the segmented mesh and
call it `segmentedMesh`. As we already
have an updated `segmentedImage`, it
is very easy to get a surface mesh of the segmented regions using
`vtkMarchingCubes` filter[^5]. This
behavior is implemented in the `updateSegmentedMesh( )` method.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/07.zip).
Now when you add a new region to the segmentation, it will immediately
be shown in the 3D
viewer.
## Step 08: Correcting for any arbitrary positioning in the world coordinates
After opening an image in `CamiTK-imp`,
use the **Edit Frame** action under **Frame** to give an arbitrary
transform to the image in the world coordinates. And then start drawing
contours on the image. You will observe that the 3D mesh drawn by our
tool is not placed correctly.
In order to correct this mishap we need to transform all
`PolyData` belonging to the
`segmentedMesh` from the image to the
world coordinates. The `updateSegmentedMesh(
)` method is modified accordingly.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/08.zip).
Now even if you draw on an image which is subjected to an arbitrary
transformation the segmentation mesh will be correctly shown in the 3D
viewer.
## Step 09: Paste the last contour on another slice
In order to paste the last contour on another slice, all we need to do
is to calculate the translation of the last drawn contour along the
corresponding axis and then transform the `
PolyData` of the contour to the new location. We also need to
clean the `PolyData` as otherwise too
many points will be generated as the use of this function increases.
Then, we have to reset the `contourWidget
` at the new location but this time we should initialize it with
the `PolyData` from the previous
contour. This functionality is implemented in the
`pasteLastContour( )` method and it
will be triggered when the `clicked( )
` signal is emitted from the **Paste Last** button.
Similar to step 06, we will refrain from copying the same lines of code
to reset the contour. Instead, lets modify the already implemented
`resetCurrentContourWidget( )` method
and bring the common commands to a common method.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/09.zip).
Now you can use **Paste Last** button to paste the last drawn contour on
another image.
## Step 10: Implement Reset All and Save As Image methods
In order to reset everything all we need to do is reset the
`segmentedImage` and then update 3D
viewer and 2D viewers. Let’s implement this behavior in a method called
`resetEverything( )` and set this to
be executed when `clicked( )` signal
of **Reset** button is emitted.
In order to save the segmented image, we create a new
`ImageComponent` and save is using
the action **Save**. This behavior is implemented in the
`saveSegmentedImage( )` method and is
executed when `clicked( )` signal of
**Save As Image** button is emitted.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/10.zip).
Now the **Reset** and **Save As Image** button are functional.
## Step 11: Make the tool less error prone
All the desired functionalities of the tool are implemented. However,
since we can click any button without any restrictions, certain click
combinations will lead the tool to crash. (*e.g.* Clicking **Paste
Last** when you haven’t drawn anything leads the tool to crash.)
If we could restrict the access to **Accept** and **Paste Last** buttons
we can make the tool more robust. **Accept** button should be enabled
when:
1. the contour is closed
2. when the last contour is pasted on another slice
**Paste Last** button should be enabled only when **Accept** button is
clicked. All other times both buttons should be disabled.
One way of completing the contour is by placing the last point of the
contour on the first point of it. We can capture this event if we listen
to `EndInteractionEvent` of the
`vtkContourWidget`. The easiest way to
do this is by making the `Segmentor`
class a subclass of `vtkCommand`
class. When we do that, we have to overload the
`Execute( )` method of the
`vtkCommand` class in order to
instantiate `Segmentor` class. Let’s
encapsulate the changing button availability in a method called
`changeButtonAvailability( )`.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/11.zip).
Certains buttons are enabled/disabled appropriately in order to make the
tool more robust.
## Step 12: Handle different ImageComponents
When the `currentImageComp` on which
the `Segmentor` action is applied
changes, we have to reset everything in order to make the tool robust.
Deleting the `segmentedImage` and
`segmentedMesh` get the job done.
Hence, users are advised to finish segmenting one image before moving on
to another.
The corresponding code skeleton of the action can be found
[here](http://prasnuts.free.fr/images/stories/site/files/CamiTK/12.zip).
Now the user can change the `ImageComponent` on which the contours are drawn.
## Almost done
It is our hope that you have gained some insight into making a CamiTK
ready tool by increments. As with any piece of code, you may run into
bugs with this tool. If that is the case, do not hesitate to let us know
and we will try to fix them asap.
[^1]: http://www.itksnap.org/pmwiki/pmwiki.php
[^2]: http://www.vtk.org/doc/release/6.3/html/classvtkContourWidget.html
[^3]: http://www.vtk.org/doc/release/6.3/html/classvtkPolyDataToImageStencil.html
[^4]: http://www.vtk.org/doc/release/6.3/html/classvtkPolyDataToImageStencil.html#details
[^5]: http://www.vtk.org/doc/release/6.3/html/classvtkMarchingCubes.html
Supports Markdown
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment