Reference frames and transformations management
As a | CamiTK developer |
I would like to | support and document reference frames, geometrical transformations and orientation conventions |
So that | CamiTK can load, register and display volumes and meshes with explicit reference frames, transformations and anatomical orientations |
Epic/Topics | Referential Management Epic |
Current state overview
Geometrical transformations in CamiTK are currently managed by:
- Frames which are attached to all components
- A transformation from each frame to its parent Frame
- A World Frame
- A default orientation for volumes (RAI convention)
- Depending on the file format used (DICOM, MHD/MHA, Nifti...) the transformation to the parent frame may or may not be loaded/saved
- Depending on the file format, the anatomical orientation may be loaded/saved or not, and conversion to RAI may or may not work.
- There is no standard way to store a registration between two images so that CamiTK reloads it when the software loads images again
Solutions
1. Documentation
- Document every detail of geometrical conventions using illustrations when needed to avoid confusion
- For DICOM, there is the position of the patient in the machine, the position of the image relative to the machine, and the convention of the order to save the pixels. Illustrations are useful.
- Matrices may be stored in row-major (Nifti) or column-major order (MHA file format)
- For each image and mesh file format, detail how we read and write it (e.g. ITK and VTK do not read and write the same information in the header and do not assume the same conventions for MHA format). This should explain how CamiTK may differ with other software and why we took those decisions.
2. Main (core) changes
- Define a CamiTK unique ID : CamiUID so that each object has a unique ID, maybe associated with a name, a description and a type ?
- Define a ReferenceFrame class (mostly a CamiUID) so that every displayable component has an instance of it
- We have to keep the anatomical orientation information. It might be stored in the Referential object and specify which axes (if known) are Right/Left, Superior/Inferior, Anterior/Posterior. This information is transitive (e.g. if there is no large rotation (> 45°) or axes flipping, the anatomical orientation is the same between two reference frames linked by a transformation). It can be set to UNKNOWN (in which case a viewer should not display the anatomical directions).
- Define a Transformation class that has an origin and a destination (from/to) ReferenceFrame, a vtkTransform (by default a linear transform represented by a 4x4 matrix), and a CamiUID. Make it generic enough so that it could manage a 4th dimension (e.g. for a time offset between videos) and non-linear transformations
- The Frame interface should be updated
- A Referential and Transformation Manager should be included in CamiTK Application to manage the list of known reference frames and transformations
- In this manager, keep the source of referentials and transformations. E.g. if a Nifti file contains two transforms, the loaded nifti component, when saved, should save again both transforms. and not other transforms to other referentials. Or maybe this information should be stored in the Image Component itself (which transformations to save).
- A CamiTK world referential will exist by default, and components that are loaded without specific transformations to CamiTK will have an identity transform from their referential to CamiTKWorldReferential
- There is no reason to force an orientation convention in CamiTK (e.g. RAI for all volumes).
3. UI changes
- An explorer for referential and transformations should be provided, (like the Component explorer, the property explorer and so on). It should be able to display/edit transformations, to link referentials with new transformations (identity, loaded from file...).
- A color code for each referential will be displayed next to each component name, and next to each viewer name, and can be changed by clicking on it to choose another one
4. I/O
- Define a way to store metadata associated to files (e.g., keep the UID of the referential of a MHD file). This is linked to a metadata format (see specific issue #158 (closed)) that should be defined for all CamiTK-related data (such as the LUT settings associated to an image, a list of actions to run for a state machine, a scene description and so on). If possible, choose a human-readable format and try to be compatible with other software or norms, e.g. the JSON format used by BIDS or in the metadata files of Brainvisa. Maybe a .camitk file.
- test
5. Viewers
- Each viewer is attached to a reference frame (e.g. all coordinates of a selected point in the viewer are expressed in this referential), and a camera position and orientation.
- Slice viewers may use their reference frame orientation information to display R/L, A/P, I/S directions
- In viewer preferences, the user can set its own convention (Right displayed on the left, or right displayed on the right)
- When adding a volume to a SliceViewer, it should get its reference frame from it, use anatomical orientation information of the frame, and user preference to orient its camera transform (no need to have RAI/LAS or other convention forced on the volume).
- Viewers should be in charge of the way they want to display a volume (e.g. no Axial View Component, just a SliceViewer with a camera orientation that matches Axial, and a volume image).
- Medical Image Viewer should manage the slices displayed in the 3D view in relation to the slices displayed in the SliceViewers by itself (not using subcomponents)
- A common cursor between viewers can be achieved by following the transformation graph from one view's reference frame to another viewer's reference frame. We can display a 3D cross with its own reference frame, and moving the cursor means changing the transformation between this frame and the selected object's frame. This allows a 3D cursor in a SliceViewer to be displayed in a 3D viewer, rotated with the angle of the slice currently being clicked on.
Acceptance tests
-
[Documentation is clear and complete] -
[Reading an image, registering it with another, saving both, then closing the software and reading both again should load all referentials and transforms so that the images are still aligned] -
[Same test with an image and a mesh segmented from it] -
[Same test with two meshes registered then saved and loaded again] -
[3D cursor synchronized between views with different referentials] -
[Displaying a volume in a SliceViewer using a referential of another volume (should display "arbitrary" slices of the volume aligned with the slices of the other volume)] -
[Loading two volumes from the same MRI acquisition should show volumes aligned because of the common scanner-based reference] -
use QUuid as a map key in TransformationManager, not QString -
Cleanup the TransformationManager API (maybe too complete/complex) -
Better API Documentation -
Remove C++23 specific code (too early)
Track
Misc
- Automatic subscription of issue creator:
If appropriate, do not forget to mark this issue as "confidential" by checking the corresponding tick box below
Edited by Manik Bhattacharjee