As the choice and availability of image modalities increase, and as images are collected from the same patient in more than one modality, or where complete 3D sequences need to be compared, prospects of image coorelation and fusion arise. In common with a number of workers to support clinical investigations using our 3D MRI, X-ray CT and PET and SPECT system we are interested in the associated algorithmic problems. This report summarises a few of the available approaches and outlines some of our local developments in these directions. [ Skip Preliminaries ].
The primary objective is to determine, by mathematical methods, the relationships between one or more 3D medical imaging modalities (usually X-Ray CT, MRI and PET/SPET) by voxel - voxel registration
The main reasons for co-registration of images are:
1. To supplement or 'fill-in' information amongst modalities, such as
thereby attempting to enhance the diagnostic information provided by one modality alone.
or
2. To allow the more effective comparison of images taken at different investigation epochs, where the need is to study pathologic changes by minimising the effect of patient movement or differential instrument artefact.
In all cases the objective is to improve the spatial or temporal quantitation or to present a more informative visual representation from images of hard and soft anatomy as well as function.
To date most work has been carried out on the brain since it provides a rigid reference framework, although as we shall see later some initial work has been carried out on flexible organs.
The general approach is to determine a transformation procedure, either based upon affine or polynomial transformations. An affine transformation is defined in terms of translation, rotation and scaling in three dimensions, and a polynomial transformation uses a power relationship of {x,y,z} to achieve co-ordinate remapping.
For all methods the common approach is:
1. Define or determine corresponding features
2. Find the matching transformation
3. Transform one (or more) to bring into spatial reference with
another
Since trigonometric functions can be approximated by polynomials, these are often seen as the unified base for transformation, however we normally need to examine the most likely physical causes of dimensional change and apply the more realistic correction model.
With an affine transformation if we start off with parallel lines in our input space these will still be parallel in our output space. With polynomial transformations the performance at points non-local to corresponding landmarks may be less well defined.
A number of different approaches are available:
1. Extracting and matching surfaces
2. Selecting anatomically similar points
3. Determining invariant moments
4. Correlating image structures
We will explore each of these in turn, showing examples.
For computational convenience the images have been reduced to
a uniform data set size of 128 x 128 x 64 slices. Many of the
algorithms are outlined in 2D form for simplicity but are, as the literature often states, naturally extendible to 3D. Frequently, this is where the challanges are. All the sectional images presented are produced by interpolation from data sets processed in 3D.
External Markers
Fiducial markers are placed at scanning time either on surface
or as components of a stereotactic frame attached to the patient's
anatomy. Inevitably these systems often lead to
Problems of repeatability, removal or fading with time or patient
distress. Such methods often lead to the highest possible registration
accuracy and may in any case be an essential feature of surgical
procedure.
Internal Markers
The strategy is to identify common anatomical objects or extremities
in the images being compared. This is often time consuming and
difficult even for experts. However the involvement and inconvenience
to the patient is minimised.
Both methods produce polynomial or affine transformations, and
one of the most popular algorithmic approaches rests on the derivation
of the normal equations derived from least squares fitting of
an over-specified set of matching points, with a controlled removal
of those points outside pre-set error limits
A number of workers, particularly Hawkes and Hill (Comp. Med.
Im.&Graph.(1993) 17,4, 357-363) have developed registration
techniques based on these and other methods.
For this work we have taken an alternative approach using orthogonal
polynomials developed from that described from Goshtasby (Im.
& Vis. Comp. (1988) 6,4,), based on the concept that these
are more flexible, less prone to singularities, and can be increased
in polynomial order to achieve a required accuracy of registration.
They also provide a framework by which point-based relaxation
can be introduced which matches the expert's point recognition
confidence.
As an example of this approach we have taken a high resolution
MRI and X-ray CT image, albeit from different subjects for the sake of this experiment
After identifying a few common landmarks registered images are
produced, slices from which have been produced using 3D sinc-based interpolation.
In this approach, initially described by Woods et al. (JCAT (1992) 16, 620-633
& JCAT (1993) 17, 536-546), the images are correlated and
a match determined when an objective function based on image similarity
is minimised. They describe success between and within modalities
on phantoms and images for translation and rotation. Optimisation
is achieved by a Newton Raphson Method, therefore having to calculate
a first derivative.
Algorithmic development
The two data sets are then subtracted to determine the accuracy
of registration.
In order to gain confidence in the correlation approach we have
perturbed the match parameters to establish how the variance of
the difference image is affected
and have experimented with artificial lesions to determine the
point at which the method begins to break down.
Multi-modality Registration
In two pieces of recent work we have taken the PET and MRI scans
from a volunteer and converted these from 512 x 512 x 24 slices
and 128 x 128 x 62 slice to our normal reference resolution.
The 3D MRI and PET images are matched via correlation without pre-
segmentation and the following sections taken.
Transverse
Transverse Sections
For the second experiment we have taken two aets of 3D MRI breast
scans, pre and post contrast enhancement during which some movement
has taken place. The first data sequence shows a few image frames
of the subtraction image before correction and the second after
correction.
Without Automatic Registration
With Automatic Registration
Some problems which may arise from the various algorithmic approaches are:
Surface Methods
Landmark methods
Principal Axes Methods
Object correspondence, false boundaries and interaction of non-isotropic
scaling
Correlation
Appropriateness and sensitivity to structural change
From these initial experiments the following observations might be proposed:
Rigid Bodies
Flexible Bodies
All processes involving matching need close attention to the objective
function and a physically realistic model of the practical outcomes,
otherwise our algorithms for matching modalities will resemble
attempts to combine objects as dissimilar as apples, plums and grapes and
result simply in well-packaged squash.
Peter E Undrill,
Methods described are surface matching , some benefits of orthogonal polynomials for point based methods, , principal components ,and robust optimisation methods, correlation of flexible anatomy and a brief discussion
Landmark Methods
Correlation Methods
Sagittal
Sagittal Sections
Flexible Objects
Discussion
Concluding Observations
Dept. BioMedical Physics and Bioengineering,
University of Aberdeen,
Foresterhill, Aberdeen, AB9 2ZD,
Scotland, UK.
