Computational Techniques: Image Registration

Image registration is a computational technique from the field of computer vision used to automati-cally interpret various types of image or video data. In the medical field, image registration is most commonly used to identify correlations, shapes or features inside image or video data with the aim to reconstruct motion or deformation or to match shapes or volumes from different data sets onto each other. Practical applications of image registration in the biomedical field are very diverse. Typically, it is a common task to match two data sets obtained from two different imaging modalities. This can also involve the matching of three-dimensional medical images.¹¹⁰ A review of image registration techniques is given byMaintz et al.⁶¹ In this thesis, image registration was used to identify and track motion inside movies showing contracting heart tissue. Generally, the registered data can be two- or three-dimensional as well as static or time-varying data. Here, all registration applications were lim-ited to dynamic two-dimensional data, see chapter 6. Image registration was used for two purposes:

(1) to register motion inside video data and (2) to subsequently reconstruct deformations of the tissue configuration and to extract quantitative information about the deformation.

3.4.1 Correlation-based Motion Tracking

Correlation-based block-based motion tracking refers to computational image registration techniques, that directly measure the correlation or similarity between neighbouring image subregions of two im-ages to compute the optical flow between two imim-ages. The techniques employ optical properties of the image data, assuming that motion is reflected by the motion of the grayvalue or intensity pat-tern within the image that i showing the motion. The optical flow consists of displacements that can be assigned to every pixel of the image, derived from the shifts that were detected in between two compared images. For instance, particle image velocimetry is a spatial correlation analysis technique used to register speckle motion in a sequence of images.¹⁵¹ The technique is usually applied in the field of fluid dynamics to detect flow inside liquids or gases. In conventional particle image velocime-try, tracer particles are introduced to a flow and a two-dimensional cross-section of the medium is illuminated with light. The flow of tracer particles within the light sheet is usually recorded with high-speed cameras and the sequence of images is then spatially correlated to each other under the assumption that tracer particles move coherently and uniformly in between two images and that their displacement is sufficiently small to be captured within subregions of the images. In the case of block-matching techniques, for evaluation of spatial correlation the images are divided into small subregions usually referred to as interrogation windows. For each interrogation window the local displacement~uis determined by means of spatially correlating the image data of the site under con-sideration with the optical properties of its surrounding. The process of interrogation is repeated for all interrogation windows of the images, such that we obtain a displacement map for each image with several hundreds to thousands of instantaneous displacement vectors. The statistical methods employ cross-correlation or minimum quadratic difference algorithms. The displacement vector ~u then points into the direction of the minimum of the 2-dimensional correlation maps. Due to the

planar illumination of the flow field only 2 in plane components of the displacement vector u can be determined. This is referred to as two-component particle image velocimetry. Spatial correlation techniques used in particle image velocimetry can be applied to many other forms of data and is not limited to particle images. In this study, it was applied to vidoe data as well as cross-sectional ultrasound B-scan speckle images showing contracting myocardial tissue. Gray value intensity dis-tributions rather than particles were tracked. In chapter 6, the local displacements~uwere calculated from interrogation windows in between 2 speckle images using the minimum quadratic differences method⁵³(MQD). The average displacement(∆x,∆y)^tof two interrogation windowsS andS⁰ can be found by determining the minimum of the quadratic difference. The location of the minimum value in the two-dimensional correlation plane given in equation (5.5.2) is used as the displacement

~

u. The accuracy of cross-correlation and MQD algorithms are similar. MQD algorithms are in general slower than cross-correlation algorithms. However, MQD algorithms may be more robust for speckle images. Cross-correlation or mean-quadratic-difference methods inherently recover only linear displacements ~u, that is the average linear shift of the intensity distributions within the in-terrogation window. Rotations or deformations can only be recovered by further processing of the displacement maps. The interrogation window size needs to be chosen sufficiently small such that second order motion effects can be neglected. The presented spatial correlation technique can track tissue motion in a sequence of speckle images, see chapter 6.

A different spatial correlation technique, which can be used for image registration and motion regis-tration of tissue was introduced by.²¹ The Lucas-Kanade method is a locally affine, globally smooth transformation based on optical flow estimation using spatial intensity gradients of images. Assuming a constant flow in a local neighborhood of a pixel, the method solves the basic optical flow equations for all the pixels in that neighbourhood by the least squares criterion. IfI0(x, y)andIt(x, y)denote a reference and a deformed image, an affine mapping is given as follows:⁷⁰

I0(x, y) = It(m1x+m2y+m5, m3x+m4y+m6) (3.4.1) where m_i with i = 1, ...,6 are parameters of the affine mapping. The following quadratic error functionE(m) then needs to be minimized in order to find the best match between reference and deformed image:

E(m) = X

x,y∈Ω

(I0(x, y)−It(m1x+m2y+m5, m3x+m4y+m6))² (3.4.2)

whereΩdefines the spatial region of interest in the image. This generally nonlinear function is lin-earized using a first-order truncated Taylor series expansion. The numerical scheme developed by⁷⁰ solves the approximate error function in Newton-Raphson iterative steps, where in each iterative step the affine parameters are estimated and the current image is repeatedly warped to the reference im-age. The Lucas-Kanade method assumes constant brightness of the images.

Determining dynamical elastic properties is generally referred to as elastography. Elastography ex-periments can generally be performed with optical imaging, ultrasound and optical coherence to-mography. Ultrasound elastography was introduced byOphir et al.³⁷ Elastography typically refers to experimental techniques, however, they are often combined with computational techniques to cap-ture elastic deformations.^{71, 91}

Chapter 3. Imaging of Electromechanical Wave Activity in the Heart

3.5 Visualization

Imaging of the heart also requires appropriate visualization of the data. The visualization proce-dure using computer graphics techniques can become a non-trivial and highly specialized task¹⁴⁹ because of the motion and deformations of the to be visualized physical activity. Visualizing three-dimensional time-varying simulation output data can be achieved using various readily available software solutions. Typically, volume data as output of numerical simulations or experimental mea-suring techniques such as computerized tomography or macroconfocal laser scanning techniques is organized in a voxel-structure on a regular rectilinear grid. The visualization of time-varying three-dimensional dynamics is then the subsequent visualization of three-dimensional voxel data or matrices for each time step, using ray-tracing or other volume rendering or visualization techniques, with the overall data corresponding to four-dimensional image data. However, the output of moving and deforming media does not necessarily provide a regular grid or voxel structure. In this thesis, a custom visualization technique had to be implemented because readily available software for the visualization of the data was not available. The procedure involved also re-rasterization and vox-elization of the simulation grid structure into regular three-dimensional image data. For this taskThe Visualization Toolkit(VTK) was used.

Chapter 4. Computational Model of Heart Tissue with Mechano-Electric Coupling

Chapter 4

Computational Model of Heart Tissue