2.6 Conclusions
Figure 2.8: Morphing path between a part of the YZ-slices 49 and 51 of the Camino dataset.
Chapter 3
Convergence of the Time Discrete Metamorphosis Model on Hadamard Manifolds
Contents
3.1 Introduction . . . 45 3.2 Review and preliminaries . . . 47 3.2.1 Hadamard manifolds . . . 48 3.2.2 Metamorphosis model in Euclidean case . . . 51 3.2.3 Manifold-valued time discrete metamorphosis model . . 55 3.3 Manifold-valued metamorphosis model . . . 57 3.4 Temporal extension operators . . . 60 3.5 Mosco convergence of time discrete geodesic paths . . . 63 3.6 Conclusion . . . 74
Abstract
Note that this chapter1 is published in [98]. Continuous image morphing is a classical task in image processing. The metamorphosis model proposed by Trouvé, Younes and coworkers [202, 264] casts this problem in the frame of Riemannian geometry and geodesic paths between images. The associated metric in the space of images incorporates dissipation caused by a viscous flow transporting image intensities and its variations along motion paths. In many applications, images are maps from the image domain into a manifold as, e.g., for diffusion tensor imaging (DTI) into the manifold of symmetric positive definite matrices with a suitable Riemannian metric. In this chapter, we propose a generalized metamorphosis model for manifold-valued images, where the image range is a finite-dimensional Hadamard manifold. A corre-sponding time discrete version was presented in [213] based on the general variational time discretization proposed in [33]. Here, we prove the Mosco convergence of the time discrete metamorphosis functional to the proposed manifold-valued metamorphosis model, which implies the convergence of time discrete geodesic paths to a geodesic path in the (time continuous) metamor-phosis model. In particular, the existence of geodesic paths is established. In fact, images as maps into Hadamard manifold are not only relevant in appli-cations, but it is also shown that the joint convexity of the distance function – which characterizes Hadamard manifolds – is a crucial ingredient to establish existence of the metamorphosis model.
1First published in [98] in 2020, published by the Society for Industrial and Applied Mathe-matics (SIAM). Copyright © by SIAM. Unauthorized reproduction of this article is prohibited.
3.1 Introduction
3.1 Introduction
Image morphing amounts to computing a visually appealing transition of two images such that image features in the reference image are mapped to corresponding image features in the target image whenever possible.
A particular model for image morphing known as image metamorphosis was proposed by Miller, Trouvé, and Younes [202, 264, 263]. It is based on the flow of diffeomorphism model and the large deformation diffeomorphic metric mapping (LDDMM), which dates back to the work of Arnold, Dupuis, Grenander and cowork-ers [9, 10, 95, 25, 168, 199, 273, 272]. From the pcowork-erspective of the flow of diffeo-morphism model, each point of the reference image is transported to the target image in an energetically optimal way such that the image intensity is preserved along the trajectories of the pixels. The metamorphosis model additionally allows for image intensity modulations along the trajectories by incorporating the mag-nitude of these modulations, which is reflected by the integrated squared material derivative of the image trajectories as a penalization term in the energy functional.
Recently, metamorphosis has been extended to images in reproducing kernel Hilbert spaces [232], to functional shapes [59], and to discrete measures [231]. For a more detailed exposition of these models we refer to [285, 201] and the references therein.
A variational time discretization of the metamorphosis model for images in L2(Ω,Rm) was proposed in [33]. Furthermore, existence of discrete geodesic paths and the Mosco convergence of the time discrete to the time continuous metamor-phosis model was proven. The time discrete metamormetamor-phosis model has successfully been applied to a variety of imaging applications like image extrapolation [99], Bézier interpolation [100], color transfer [226] or image interpolation in a medical context [32].
Throughout the past years, manifold-valued images have received increased at-tention, see [21, 72, 186, 277, 30]. Some prominent applications are linked to Hadamard manifold-valued images:
– Diffusion tensor magnetic resonance imaging is an image acquisition method that incorporates in vivo magnetic resonance images of biological tissues driven by local molecular diffusion. The range space of the resulting im-ages is frequently the space of symmetric and positive definite matrices [20, 63, 112, 267].
– Retina data is commonly modeled as images with values in the manifold of univariate non-degenerate Gaussian probability distributions endowed with the Fisher metric [8, 31]. This space is isometric to a hyperbolic space, which can be exploited numerically.
This motivates a generalization of the metamorphosis model as a Riemannian model for spaces of images. In [213], the time discrete metamorphosis model was extended to the set of imageL2(Ω,H), whereHdenotes a finite-dimensional Hadamard man-ifold. Recall that Hadamard manifolds are Hadamard spaces with a special Rie-mannian structure having non-positive sectional curvature, details are given below.
In [13], it is revealed that many concepts of Banach spaces can be generalized to Hadamard spaces, which are therefore a proper choice for the analytical treatment of algorithms for manifold-valued images. In particular, the distance in Hadamard spaces is jointly convex, which implies weak lower semi-continuity of certain func-tionals involving the distance function. Moreover, several analytic properties of Hadamard manifolds presented in Section 3.2, which are crucial for the Mosco con-vergence, cease to be valid for general manifolds.
In this chapter, we prove the Mosco convergence of the manifold-valued time dis-crete metamorphosis energy functional originally proposed in [213] to a novel (time continuous) metamorphosis energy functional on Hadamard manifolds. Moreover, we establish the convergence of manifold-valued time discrete geodesic paths to geodesic paths in the proposed manifold-valued metamorphosis model, which co-incides with the original metamorphosis energy functional in the Euclidean space.
The proof of Mosco convergence in [33] incorporates as an essential ingredient a representation formula for images via integration of the weak material derivative along motion paths for the time continuous metamorphosis model in the Euclidean setting. Here, we no longer make use of such a representation formula. Indeed, our Mosco convergence result can thus be considered as a stronger result even in the case of images as pointwise maps into a Euclidean space.
Outline The chapter is organized as follows. We start with a collection of required notation and symbols in the next paragraph, including the definition of Mosco con-vergence. In Section 3.2, we discuss the concept of Hadamard spaces and manifolds with an emphasis on important properties of the distance map. Furthermore, we review the classical flow of diffeomorphism and metamorphosis model. Here, we al-ready prove some continuity results on the Lagrange maps associated with a motion field. Finally, we pick up the time discrete metamorphosis model presented in [213].
Section 3.3 is devoted to the presentation of the manifold-valued metamorphosis model. Here, the key point is the suitable definition of a material derivative quan-tity, which is finally obtained using a variational inequality. We show that the new model for manifold-valued images coincides with the previous model in the Eu-clidean case. Section 3.4 introduces a method to extend time discrete image paths to time continuous ones as natural prerequisite to prove convergence of the en-ergy functionals on discrete paths to a limit enen-ergy functional on continuous paths.
Then, in Section 3.5, the main result of this chapter on Mosco convergence is stated and proved. In detail, we show the required liminf-inequality in Theorem 3.12 and the existence of recovery sequences in Theorem 3.14. This finally implies the con-vergence of discrete geodesic paths in Theorem 3.15 and the existence of a geodesic path for the time continuous metamorphosis model. The proofs generally follow the guideline from [33] for the classical metamorphosis model with conceptual and technical modifications in order to deal with the setup of manifold-valued images.
Notation Throughout this chapter, we assume that the image domainΩ⊂Rn is bounded with Lipschitz boundary. Henceforth, we denote time continuous operators
3.2 Review and preliminaries