DISSERTATION
zur Erlangung des akademischen Grades Doktoringenieur (Dr.-Ing.)
angenommen durch die Fakultät für Informatik der Otto-von-Guericke-Universität Magdeburg
von M.Sc. Georg Hille geb. am 20.03.1989 in Erfurt
Gutachterinnen/Gutachter Prof. Dr. Klaus-Dietz Tönnies Prof. Dr. Alexander Schlaefer
Prof. Dr. Dorit Merhof
The spine is the most common body part to develop bone metastases from various primary tumours with increasing case numbers over the last decades. The drastic effects on the quality of life evoked by spinal metastases, such as severe pain symptoms or neurological deficits due to nerve root and spinal cord compressions, demand a fast-acting, yet gentle therapeutic solution, as enabled by minimally invasive interventions like radiofrequency ablations. For this purpose, one or multiple applicators with electrode needle tips are placed within the tumour volume and necrotise the cancerous tissue by high frequency-induced tissue heating. The entire clinical workflow of such a minimally invasive intervention is based on medical imaging, starting from the initial diagnosis over image-guidance during the intervention to therapy control via follow-up scans. Computer-assisted strategies can support the radiologists to obtain more relevant information from the acquired images and to transfer these to subsequent processing steps. This enables a more sophisticated work-flow, while reducing the required time and workload of the radiologists. However, the specific image processing aspects to achieve this goal are challenging with regard to the required expert-like performance, high level of automatisation, and short computational times.
This thesis focuses on various aspects throughout the clinical workflow of radiofrequency ablations of spinal metastases. For this purpose, the thesis is structured following the chronological sequence of the clinical process and contains approaches to support the radiologist during the pre-, intra-and post-interventional phase. Limitations intra-and gaps in the existing state of the art of each aspect led to the development and implementation of novel strategies to provide suitable and applicable solutions. In detail, segmentation approaches of involved anatomical structures like vertebral bodies, metastases and resulting necrosis zones have been developed - the latter two being, to the best of the author’s knowledge, the first of their kind. Furthermore, an image registration method is presented, which is able to cope with the poor image quality of interventional imaging and the specific issue of spinal deformations due to different patient position-ing. Finally, a novel framework is proposed to automatically visualise and quantify the treatment outcome of spinal metastasis interventions. Each of the above-mentioned methods has been evaluated on a wide range of patient data in order to demonstrate robustness, reliability, accuracy, and speed to meet the clinical objectives.
Die Wirbelsäule ist die muskuloskelettale Struktur, in der sich am häu-figsten Knochenmetastasen verschiedenster Primärtumore entwickeln und dies mit stetig steigenden Fallzahlen in den letzten Jahrzehnten. Wirbelsäulenmetastasen verursachen eine drastische Beeinträchtigung der Lebensqualität der meisten Patienten, bedingt durch eine ausgeprägte Schmerzsymptomatik sowie teilweise durch neurologische Ausfallerschei-nungen aufgrund von Nervenwurzel- und Rückenmarkkompressionen. Dies wiederum erfordert eine unverzögert wirksame, aber im Hinblick auf das fortgeschrittene Alter der meisten Patienten trotz allem scho-nende therapeutische Lösung, wie sie insbesondere minimal-invasive Eingriffe, beispielsweise die Radiofrequenzablation, versprechen. Hier-bei werden ein oder mehrere Applikatoren mit Elektroden an deren Spitze in dem Tumorvolumen platziert, welche anschließend das metas-tasierte Gewebe mittels hochfrequenzinduzierter Gewebeerhitzung nekro-tisieren. Die medizinische Bildgebung spielt während des gesamten klinisch-therapeutischen Prozesses einer minimal-invasiven Intervention eine entscheidende Rolle; angefangen bei der initialen Diagnose, über die bildgestützte Durchführung des Eingriffs, bis hin zur abschließen-den Therapiekontrolle mittels Bildgebung. Computergestützte Strategien können Radiologen gezielt dabei helfen, aus den generierten Bildern zusätzlich relevante Informationen zu gewinnen und diese auch auf nach-folgende Prozessschritte zu übertragen. Dies eröffnet die Perspektive auf einen insgesamt fortschrittlicheren klinischen Arbeitsablauf und reduziert darüber hinaus die zeitliche und mentale Arbeitsbelastung der beteiligten Radiologen. Die spezifischen Bildverarbeitungsaspekte zur Erreichung dieses Ziels sind jedoch im Hinblick auf ihre Komplexität sowie die er-forderliche methodische Genauigkeit und die kurzen Berechnungszeiten, eine Herausforderung.
Diese Dissertation thematisiert verschiedene Aspekte des klinischen Ar-beitsablaufs bei der Radiofrequenzablation von Wirbelsäulenmetastasen. Hierzu folgt die Arbeit dem chronologischen Ablauf des klinischen Prozesses und beinhaltet Strategien für die zielgerichtete Unterstützung der Radiologen während der prä-, intra- und post-interventionellen Phase. Bestehende Limitationen oder Lücken im derzeitigen Stand der Tech-nik jedes einzelnen Aspektes erforderten die Entwicklung und Umset-zung neuer Lösungsstrategien, welche auf geeignete Art und Weise die klinischen und technischen Anforderungen erfüllen. Im Detail wurden Verfahren zur Segmentierung von relevanten anatomischen Strukturen, wie bspw. der Wirbelkörper, der Metastasen und der resultierenden Nekrosezonen entwickelt, sowie ein Bildregistrierungsverfahren, das der schlechteren Bildqualität interventioneller Bildgebung und dem spezifis-chen Problem der Wirbelsäulendeformationen aufgrund unterschiedlicher Patientenpositionierung gerecht wird. Abschließend wird ein Framework
nen ermöglicht. Jede der oben genannten Methoden bzw. Lösungsstrate-gien wurde mithilfe einer Vielzahl von klinischen Patientendaten evaluiert, um die benötigte Robustheit, Genauigkeit und Geschwindigkeit der Ver-fahren zu demonstrieren.
1 introduction 1 1.1 Motivation . . . 1 1.2 Current Workflow . . . 7 1.3 Intended Workflow . . . 11 1.4 Thesis Objectives . . . 14 1.5 Thesis Structure . . . 14 2 pre-interventional phase 15 2.1 Vertebral Body Segmentation . . . 15
2.1.1 Introduction . . . 15
2.1.2 State of the Art . . . 17
2.1.3 Objectives . . . 20
2.1.4 Materials and Methods . . . 21
2.1.5 Results . . . 26
2.1.6 Discussion . . . 26
2.1.7 Conclusion . . . 31
2.2 Spinal Mestastasis Segmentation . . . 31
2.2.1 Introduction . . . 31
2.2.2 State of the Art . . . 33
2.2.3 Objectives . . . 38
2.2.4 Materials and Methods . . . 38
2.2.5 Results . . . 43
2.2.6 Discussion . . . 43
2.2.7 Conclusion . . . 47
3 interventional phase 49 3.1 Multisegmental Spine Image Registration . . . 49
3.1.1 Introduction . . . 49
3.1.2 State of the Art . . . 50
3.1.3 Objectives . . . 53
3.1.4 Materials and Methods . . . 55
3.1.5 Results . . . 58
3.1.6 Discussion . . . 59
3.1.7 Conclusion . . . 63
4 post-interventional phase 65 4.1 Necrosis Zone Segmentation . . . 65
4.1.1 Introduction . . . 65
4.1.2 State of the Art . . . 67
4.1.3 Objectives . . . 68
4.1.4 Materials and Methods . . . 69
4.1.5 Results . . . 72
4.1.6 Discussion . . . 72
4.1.7 Conclusion . . . 76
4.2 Treatment Outcome Validation . . . 76
4.2.1 Introduction . . . 76
4.2.2 State of the Art . . . 77
4.2.3 Objectives . . . 79
4.2.4 Materials and Methods . . . 80
4.2.5 Results . . . 82
4.2.6 Discussion . . . 83
4.2.7 Conclusion . . . 86
5 summary 87
Figure 1.1 Classification of vertebral involvement of spinal
metastases . . . 2
Figure 1.2 Appearance differences between lytic and sclerotic metastases . . . 4
Figure 1.3 Illustration of the RFA procedure of spinal metastases 7 Figure 1.4 Current workflow of RFA of spinal metastases . . 8
Figure 1.5 Setting in the operating room . . . 9
Figure 1.6 Pre- and intra-operative imaging . . . 10
Figure 1.7 Interventional setting on a patient . . . 11
Figure 1.8 Pre- and post-interventional MRI . . . 12
Figure 1.9 Intended workflow of RFA of spinal metastases . 13 Figure 2.1 Partial volume effects in spinal MRI . . . 17
Figure 2.2 Pipeline of the vertebral body segmentation approach 24 Figure 2.3 Vertebral body segmentation results . . . 27
Figure 2.4 Appearance variability of spinal metastases . . . . 32
Figure 2.5 Applications for spinal metastasis segmentation . 33 Figure 2.6 U-net architecture . . . 41
Figure 2.7 Exemplary results of the spinal metastasis segmen-tation . . . 45
Figure 2.8 Influence of the dimensionality and MRI sequence on the results . . . 46
Figure 3.1 Motivation for a pre- and intra-interventional im-age fusion . . . 50
Figure 3.2 Comparison of native CT and Dyna-CT images . . 54
Figure 3.3 Pipeline of the image fusion approach . . . 56
Figure 3.4 Quantitative results of the pre- and intra-interventional image fusion . . . 59
Figure 3.5 Comparison between a globally rigid and a multi-segmental registration . . . 60
Figure 4.1 Necrosis zone shaping . . . 66
Figure 4.2 U-net used for the necrosis zone segmentation . . 71
Figure 4.3 Quantitative results of the necrosis zone segmen-tation . . . 74
Figure 4.4 Exemplary results of the necrosis zone segmentation 75 Figure 4.5 Illustration of the treatment outcome validation framework . . . 81
Figure 4.6 GUI of the treatment outcome validation tool . . . 86
Table 1.1 The incidence of skeletal metastases . . . 1
Table 2.1 Evaluation set used for the vertebral body segmen-tation approach . . . 22
Table 2.2 Comparison with the related work regarding ver-tebral body segmentation . . . 29
Table 2.3 Quantitative results of the spinal metastasis seg-mentation approach . . . 44
Table 3.1 Intra- and inter-reader variability of the pre- and intra-interventional image fusion . . . 59
Table 3.2 Comparison with the related work of spinal image fusion . . . 62
Table 4.1 Quantitative results of the necrosis zone segmen-tation approach . . . 73
Table 4.2 Results of a retrospective study using the treatment outcome validation framework . . . 84
Table A.1 Complete evaluation results of the vertebral body segmentation approach (Part 1 of 2) . . . 92
Table A.2 Complete evaluation results of the vertebral body segmentation approach (Part 2 of 2) . . . 93
Table A.3 Comparison of different learning rates and opti-mizers . . . 94
Table A.4 Registration accuracy of the proposed pre- and intra-interventional image fusion approach . . . . 95
RF radiofrequency
RFA radiofrequency ablation
MR magnetic resonance
MRI magnetic resonance imaging
CT computed tomography
FP-CT flat-panel detector computed tomography
PET positron emission tomography
SPECT single photon emission computed tomography
US ultrasound
HU Houndsfield unit
STIR short tau inversion recovery
SPIR spectral presaturation with inversion recovery PVE partial volume effects
SNR signal-to-noise ratio IRV inter-reader variability ROI region of interest
GTV gross tumour volume
FOV field of view
DSC Dice similarity coefficient FRE fiducial registration error
FREMS fiducial registration error of the multisegmental approach FREGR fiducial registration error of the globally rigid approach mFRE minimal fiducial registration error
FLE fiducial localisation error ASD average surface distance
HD Hausdorff distance
HD95 95th percentile Hausdorff distance
ACM active contour models
ASM active shape models
BC-HLS bias field-corrected hybrid level-sets CNN convolutional neural network SdNN Siamese deep neural network ReLU rectified linear unit
TL Tversky loss
SVM support vector machines
FLD Fisher’s linear discriminant CRF conditional random field
MI mutual information
NMI normalised mutual information
NGF normalised gradient fields MS-VB multisegmental voxel-based TPR true positive rate
TNR true negative rate
GPU graphics processing unit GUI graphical user interface SHIP Study of Health in Pomerania
BRATS brain tumour segmentation challenge LITS liver tumour segmentation challenge
1
I N T R O D U C T I O N
1.1 m o t i vat i o n
Due to the enhancement of medical treatment and diagnostic procedures, life expectancy has increased steadily over the last decades. However, this lifetime gain promotes also age-related diseases like cardiovascular diseases, as well as cancer and cancer-induced malicious metastases. Both of them are the most common causes of death nowadays. The survival time of most malicious carcinomas has increased with improved diagno-sis and treatment, though, this also promotes spreading of metastases. Besides liver and lung, bone metastases are the third likely, and thereof between two thirds (Harrington,1986; Wong et al.,1990) and over 90 % (Frangou and Fourney,2009) are located in the spine, varying according to the reference. The most common primary malignancies that lead to bone metastases are listed in Table1.1.
Table 1.1: The incidence of skeletal metastases, based on autopsy studies (Mac-cauro et al.,2011). Tumour Incidence Mammary carcinoma 73 % Prostatic carcinoma 68 % Thyroid carcinoma 42 % Bronchial carcinoma 36 % Renal carcinoma 35 % Rectal carcinoma 11 % Esophageal carcinoma 6 % Gastrointestinal carcinoma 5 %
Studies showed that most metastases occur in the thoracic spine, fol-lowed by the lumbar segment, where as the cervical region is the least involved (Klimo and Schmidt,2004). More than half of the patients with spinal metastases have lesions in multiple, partly non-contiguous spine segments (Togawa and Lewandrowski,2006). Following the mechanistic theory, tumour cells will metastasise anatomical regions near to their primary site, e.g. mammary carcinomas preferably infiltrate vertebrae of the thoracic region and prostate cancer usually metastasise the lumbar-sacral spine (Algra et al.,1992; Gilbert et al.,1978). The initial anatomical location of spinal metastases is generally the posterior portion of the vertebral body, gradually followed by the anterior body, lamina and
intra-compartmental pedicle extension epidural extension extra-compartmental involvement of multiple vertebrae paravertebral extension
Figure 1.1: Illustrations of different degrees of vertebral involvement in spinal metastases with corresponding exemplary patient cases.
cles with increasing tumour stages (Eleraky et al., 2010; Georgy,2008). The vertebral involvement in spinal lesion diseases largely influences the treatment strategy and therapeutic decision-making (see Figure1.1). Osseous metastases can be typically classified as osteolytic, i.e. with in-creased osteoclastic activity and therefore bone resorption, or osteoblastic (sclerotic), i.e. enhanced osseous tissue production, as well as a mixed combination of the two types (Eleraky et al., 2010). The infiltration of tumour cells causes an imbalance of osteoclastic and osteoblastic activity leading to a release of growth factors, which stimulate bone remodelling and further growth factor production. This results in a dire cycle of bone destruction and local tumour growth (Lipton, 2004; Yin et al.,2005). In some cases, certain types of metastases can be assigned to different pri-mary tumours, e.g. prostate or thyroid tumours predominantly develop osteoblastic metastases, bronchial and renal carcinomas often result in osteolytic types, and mammary carcinomas could lead to mixed sclerotic and lytic lesions (Yin et al., 2005).
Regarding the biomechanics of metastatic vertebrae, destabilisation due to fractures both, under traumatic or normal physiological stress is one of the most common consequences (Georgy,2008; Whyne et al., 2003), especially if osteolytic metastases weaken the internal bony matrix structures. Spinal stability in a clinical sense includes mechanical stability, as well as the absence of pain, deformity and any neurological signs (Panjabi, 2003). However, spinal metastases could tremendously affect the quality of life and the primarily therapeutic indications are vigorous pain by fractures, bruises, spinal cord and nerve root compressions and therefore, neurological deficits (Klimo and Schmidt, 2004). The latter often occure in advanced stages due to growing extravertebral masses (Guillevin et al.,2007). Once tumours spread and develop metastases, full
recovery is rarely possible and the therapeutic goal is often to stop, delay or shrink its growing masses. Although, a complete cure is often no longer possible, palliative treatment of spinal metastases is indicated with regard to pain palliation and the release of stenosis-related neurological deficits and an overall improved life quality. However, the relative survival rate at five years of metastatic tumours drastically decreases compared to non-metastatic tumours, e.g. for prostate cancer about 70 %, colon cancer about 25 % or renal tumours about 18 % (American Cancer Society,2019).
To detect spinal metastases, a wide variety of imaging methods can be applied, such as scintigraphs, X-ray radiography, computed tomogra-phy (CT), magnetic resonance imaging (MRI) as well as some functional imaging methods like positron emission tomography (PET) and single photon emission computed tomography (SPECT). Conventional radiogra-phy may be useful to first detect abnormalities due to lytic or sclerotic bone remodelling, but lacks more detailed information about shape and bone marrow integrity. Furthermore, it commonly detects metastases in later stages with advanced osseous structural loss (Shah and Salzman, 2011). CT imaging is superior in terms of a detailed morphology, its high spatial resolution and an increased soft-tissue contrast. However, similar to X-ray radiography the appearance of metastases depends on their mineralisation and therefore it requires noticeable bone remodelling to be recognised. Thus, early detection of infiltrated bone marrow is hampered andCTscans as a diagnostic imaging technique may be un-suitable. InCTimaging, lytic lesions often appear as soft tissue regions with irregular margins and hypointense image signals compared to os-seous structures. In contrast, sclerotic metastases predominantly show hyperdense bone matrix structures with bright image signals, comparable to cortical bone (see Figure 1.2). In comparison to the aforementioned imaging techniques, MRIovercomes restrictions of radiation exposure and combines detailed morphology and high soft tissue contrast as well as enhanced bone marrow visualisation, which makes it an adequate and useful technique for early stage metastasis detection. In addition, MRI
is suitable for the assessment of spinal cord compressions and thus for the clarification of symptomatic neurological deficits (Shah and Salzman, 2011). The displayed signal intensities vary with regard to the acquisition sequence and metastatic type due to their degree of mineralisation (see Figure 1.2). Sclerotic metastases predominantly appear hypointense in both T1- and T2-weighted sequences compared to healthy bone marrow image signals. Lytic lesions show mainly intermediate to hypointense signals in T1- and hyper- or isointense signals in T2-weighted sequences. Furthermore, a contrast agent-induced signal enhancement is usually present with metastases of the lytic type (Shah and Salzman,2011). The drawback ofMRIis the more challenging distinction between remaining active tumour tissue and scarred, necrotised or fractured bone tissue in post-therapeutic scans (Even-Sapir, 2005; Wong et al., 1990). These limitations can be partially compensated by contrast-enhanced sequences, e.g. with gadolinium-based contrast-enhancing agents, since it adds a dynamic component to the imaging process (Kim et al., 1982; Kim et al.,
a
b
c
d
e
Figure 1.2: Appearance differences of sclerotic (upper row) and lytic (bottom row) spinal metastases (arrows) inMRIand interventionalCT imag-ing. Displayed are two exemplary patient cases with sagittal T1
-weighted (a), T2-weighted (b), and STIR (c)MRIsequences as well
as their corresponding interventionalFP-CTscans in a mid-sagittal (d) and an axial cross-section (e). Especially in X-ray-based imaging, the different physiological processes regarding the bone alteration of both lesion types can be vividly visualised.
2003). The major advantage ofSPECTorPETimaging as functional and nuclear imaging techniques lies in their potential to detect pathological biochemical and physiological abnormalities due to carcinomas with high specificity (Van Dort et al.,2008).PETimaging with specific radioligands like [18F] flouride could furthermore support the distinction between lytic and sclerotic metastasis types (Barzilai et al.,2018; Even-Sapir,2005). The decisive disadvantage of these imaging procedures for diagnostic or interventional purposes is their comparatively limited spatial reso-lution, which could be partly overcome in hybrid imaging techniques likePET/CT,SPECT/CTor evenPET/MRI, whose limited availability in clinical routine practice is slowly growing (Shah and Salzman,2011).
The advances of treatment strategies for metastatic spinal lesions led to the development of the "NOMS" framework, comprising Neurological, Oncological, Mechanical, and Systemic assessments to support complex decision-making for therapy techniques across disciplines (Zuckerman et al.,2018). With the aid of NOMS, various key aspects like presence of epidural spine compressions, expected tumour control, vertebral stability, and the risk-benefit ratio of different treatment strategies are assembled to optimise patient care and overall survival (Barzilai et al., 2018).
Historically, the method of choice to treat osseous metastases was con-ventional external beam radiation, i.e. the target area was radiated by one or two beams. The major drawback of this treatment strategy regarding spinal metastases is the relatively widespread target area including risk structures like the spinal cord, which limits the applied radiation doses
and lead to the necessity of increased fractioning (Barzilai et al., 2018). Hence, the treatment response is commonly delayed and transient. In contrast, spine stereotactic radiosurgery can deliver high-dose ablative radiation in typically one to five fractions to the target (Huo et al.,2017). It utilises image-guided intensity-modulated radiation delivery and steep dose gradients due to highly focused beams, which results in effective doses within the target structures, while protecting adjacent organs at risk (Huo et al.,2017). Therefore, spine stereotactic radiosurgery commonly ensures a fast and durable symptomatic response, i.e. high local tumour control and pain relief. However, radiation therapies are constrained in terms of radioresistant tumours and do not address spinal instabilities, requiring adjuvant procedures like vertebroplasty or balloon kyphoplasty (Barzilai et al.,2018; Georgy,2008). Chemotherapy or hormonal therapy often tend to fail the desired relief of symptoms of osseous metastases and take time to be effective (Rosenthal and Callstrom,2012). Owing to the short life span and morbidity of most patients, surgical interventions may often be beyond dispute. These range from the resection of individual vertebrae, discs and surrounding ligaments to ensure en-bloc excisions to intra-tumoural surgeries in order to reduce compressive stress on neural structures. Since it is highly challenging to achieve satisfying resection margins, the risks associated with such procedures often contradict a surgical intervention (Barzilai et al.,2018).
Besides, percutaneous minimally invasive therapies gain reception as promising methods to treat spinal metastases or tumours. These include, among others, various thermal therapies, like microwave ablation, laser ablation, cryoablation and radiofrequency ablation (RFA), which cause necrotisation due to target tissue heating or freezing. Microwave ablation utilises electromagnetic waves with frequencies between 900 - 2450 MHz to heat up the target tissue via molecule agitation, which leads to coagu-lative necrosis. The main drawback is the low availability of commercial systems, which is probably also the reason why the related literature is rather scarce, especially for spinal metastases (Zhang, 2016). Laser ablation is based on heating up the tumour tissue by infrared light en-ergy through optical fibers and, therefore, induces tissue necrotization of smaller regions. However, there are only very few studies considering laser ablation of bone or spinal metastases (Evans et al.,2020; Rothrock et al., 2020). In contrast to the above, cryoablation rapidly cools the tis-sue to temperatures of -100◦C through partially insulated probes, which insert gaseous or liquid nitrogene or more recently argon into the target tissue (Chu and Dupuy, 2014; Skanes et al., 2004). While cryoablation has the advantage of good visibility inMRIor ultrasound (US) imaging with an easily recognisable ablation zone due to the ice ball formed at the needle tip, its applicability is limited to rather small lesions with a radius of roughly 2 - 2.5 mm, since the cooling effect rapidly declines with increasing distance from the cryogenic center (Khairy et al., 2003; Skanes et al.,2004).
Radiofrequency ablation represents an effective method to treat rela-tively small tumours and metastases, if surgical resection is inapplicable
and pain reduction is attempted within several hours or few days follow-ing the intervention. It has been used to reduce lower back pain caused by facet osteoarithritis (Cho et al.,1997) or osteoid osteoma (Rosenthal et al., 1998) and was introduced approximatley a decade ago to treat osseous spinal metastases (Dupuy et al., 2010). Overall, the number of
RFAsessions per year has steadily increased over the last decade (Starr et al.,2019) and is expected to continue to grow immensely (Transparency Market Research,2019). However,RFAtreatment of metastases is - with a few exceptions - not curative, but suitable for pain palliation, to regain lost neurological function, and to contain local tumour growth (Posteraro et al.,2004).RFAutilises frequencies within the range of 300 - 3000 kHz, with no stimulation or interference with neuromuscular or electrolytic processes (Ni et al.,2005). The basic setup consits of a radiofrequency (RF) generator as the source of theRFvoltage, a needle-like applicator with electrodes at its tip and a grounding which closes the current circuit with the patient’s body as an active element. When the generator is switched on, a high induction flux forms around the active electrodes at the needle tip due to the impedance characteristic of the target volume (Hong and Georgiades, 2010; Kline,2000).
The physics underlying theRFAis based on the reaction of the target tissue’s dipole molecules, i.e. primarily water molecules, which align in the direction of the current and begin to oscillate at the same frequency. This molecular oscillation leads to a friction-induced rise in temperature, which in the end results in coagulation and therefore, target tissue necro-tisation (see Figure1.3). It is worth mentioning, that the electrode at the needle tip itself is not hot or the thermal source of heating, but triggers ionic movement within the adjacent tissue that causes the heating (Hong and Georgiades,2010). However, this means that the electrical and thermal conductivity of the tissue is of critical importance for successful ablation procedures. In addition, it is essential that the tissue temperature is rising not too fast and not beyond 105◦C, since consequent carbonisation or vaporisation could restrict any further energy transmission and lead to incomplete ablation zones (Hong and Georgiades,2010). At temperatures above 60◦C cells start to necrotize (Carrafiello et al.,2007; Frangou and Fourney,2009) due to irreversible protein denaturation and the destruc-tion of key enzymes (Goldberg and Gazelle,2001). Furthermore, studies showed that temperatures around 45◦C could result in apoptosis, which is of crucial importance regarding the protection of surrounding risk structures like spinal cord or peripheral nerves (Vujaskovic et al., 1994; Yamane et al.,1992). Thus, a typical ablation zone consists of a core region of full necrosis around the needle tip electrodes with an adjacent area of moderate necrotisation and a further zone characterised by apoptosis. These biochemical effects ofRFAs occur within seconds to a few minutes, leaving both micro- and macroscopically visible effects, in particular a mi-crocavitation caused by the applicator access path, the ablated tumoural and peri-tumoural tissue, a necrosis-related dark rim and an outer area of inflammation or oedema (Ni et al.,2005). Regarding theRFAdevices, it can be noted that almost all medical generators use frequencies between
a b
c d
Figure 1.3: Illustration of the procedure ofRFAof spinal metastases. Subsequent to the calibration of the image-guided navigation system at the begin-ning of the intervention (a) the radiologist creates access pathways for theRFAapplicators using trocars (b). After the insertion of either single or multiple applicators, an expanding ablation zone develops w.r.t. the elapsed time and the induced energy (c). The resulting shape of the necrosis zone can be influenced by timed switching between various configurations of active pairs of electrodes (d).
450 - 600 KHz, while probe types differ more fundamentally in design, circuitry and feedback mechanism, e.g. monopolar, bipolar, internally cooled, single- or multi-array needle tips (Zhang,2016).
The following sections deal with the current clinical workflow in detail and show existing limitations and constraints, as well as the potential for improvement through computer-assisted methods.
1.2 c u r r e n t w o r k f l o w
The following description of the workflow refers primarily to clinical pro-cedures in the Department of Neuroradiology at the University Hospital of Magdeburg, but could partly be transferred to related facilities, which treat vertebral metastases byRFA. The whole clinical workflow ofRFAs of spinal metastases could be subdivided into three major phases: the pre-interventional therapy planning phase, the image-guided pre-interventional phase and the post-interventional therapy control phase (see Figure1.4).
Pre-interventional Phase Post-interventional Phase Interventional Phase
diagnostic image acquisition
(MRI, CT)
therapy decision-making
(applicability of RFA)
mental intervention planning
calibration of navigation
system (skin marker, FP-CT)
image-guided instrument
navigation by biplanar angiography and FP-CT
RFA process monitored by
power generator (impedance, time, induced energy)
follow-up MR imaging qualitative and subjective
treatment outcome validation (visual exploration, symptom reduction)
Figure 1.4: Current workflow of spinal metastasis treatment usingRFAin the Department of Neuroradiology of the University Hospital of Magde-burg.
p r e-interventional phase Patients with unresolved backpain or with suspected metastases are examined using various imaging pro-tocols. Starting with spinal MRI, containing sagittal and axial native T1- and T2-weighted sequences as well as sagittal short tau inversion recovery (STIR)/spectral presaturation with inversion recovery (SPIR) sequences to enhance oedemata typical due to cancerous and metastatic processes. Native T1- and contrast-enhanced T1-weighted sequences are commonly the most useful in terms of spinal lesion diagnosis, since intra-vertebral image signals hypointense to surrounding muscles, discs, and normal bone marrow strongly indicate abnormality and marrow re-placement (Shah and Salzman,2011). The differentMRIprotocols mainly serve the purpose of providing intra-sequence image contrasts, which are highly tissue-specific due to the biochemical composition and the characteristic differences of variousMRIsequences in displaying fat and water. AdjuvantCTimaging is used to highlight fracture patterns and bone density alterations, i.e. osteolytic or sclerotic processes due to metas-tases (Halvorson et al., 2006; Shah and Salzman, 2011). Especially, the integrity of the ventral vertebral rim is assessed in CT scans, since it affects therapeutic decision-making w.r.t. post-interventional stabilisation of the ablated vertebrae. Accordingly, only if the ventral vertebral rim is unimpaired, kyphoplasty comes into question. The circumstance that diagnostic imaging is most commonly performed in the supine position will be of great relevance for the following interventional image-guidance. In general, the current clinical diagnostic and therapeutic decision-making does not include advanced image processing or interactive procedures and is mainly based on image exploration and the experience of the radiologists involved. This applies to the decision whetherRFAis feasible and, if so, to the mentally planning of interventional access routes and
RFAneedle positioning.
i n t e r v e n t i o na l p h a s e The first step during the intervention is to place skin markers on the patient’s back, calibrate and initialise the CAScination navigation system with a flat-panel detector computed
to-Figure 1.5: Current setting in the operating room in the Department of Neu-roradiology of the University Hospital of Magdeburg. On the right hand side, the CAScination navigation system is located (with a 3D model of the patient’s thorax on screen; built from theFP-CT vol-ume). The left monitor displays angiographic images, taken regularly to track the RFA needle and guide the metastasis puncture. The power generator for the radiofrequency current is covered by the left monitor.
mography (FP-CT) scan using a rotational C-arm (Siemens Dyna-CT, Doerfler et al., 2015). Subsequently, the CAScination system creates a volume from the Dyna-CT scan for optical navigation and tracking (see Figure 1.5). For the purpose of RFAneedle placement, the radiologist hammers and/or drills a trocar into the vertebral bone structure, most commonly through the vertebra’s pedicle, i.e. transpedicular, as this is most likely to preserve the structural integrity of the vertebrae (Chen et al., 2016). This is typically feasible only for lumbar and lower thoracic verte-brae, since their pedicles ensure enough space for a stability-preserving insertion through the narrow corridor of cancellous bone tissue. Depen-dent on factors like age, sex, and height, the pedicle diameter usually varies between 3 to >10 mm from cervical to lumbar vertebrae (Charles et al.,2015; Christodoulou et al.,2005; Liu et al.,2010; Scoles et al.,1988). If spatial restrictions contradict a transpedicular pathway creation, typically for upper thoracic and cervical vertebrae, parapedicular pathways, i.e. along the outside of the pedicles, are used to access the target region (Kothe et al., 2001). For both approaches, biplanar angiographic shots and FP-CTscans are performed to track the current position. However, both image modalities do not directly display the metastases, but rather indicate their location and extent due to advanced alterations of the bone matrix structure. Moreover, metal artifacts caused by inserted instruments further aggravate an accurate tumour localisation (see Figure 1.6). Fol-lowing the resulting pathways,RFAneedles are inserted. Electrodes on the needle tip trigger molecular friction due to 300 - 500 kHz current phase changes and therefore, rising tissue temperature (see Figure1.7). The metastatic tissue is treated with temperatures above 70◦C leading to coagulation necrosis and cell death (Palussiere et al.,2012).
Owing to the poor visibility of metastatic tissue during the intervention, the radiologists have to largely infer the location of the metastases from
a
b
c
Figure 1.6: Pre-interventionalMRI(upper row) and interventionalFP-CTscans (lower row) of three exemplary patient cases (a-c) are shown. The overall image quality and the soft tissue contrast of the interven-tional scans suffer noticeably due to low-dose protocols, as well as beam hardening, streak, and dark band artifacts from the inserted applicators.
the pre-interventionally acquired MRI data and mentally match those images with the intra-interventional imaging. Thus, an exact localisation is only possible to a limited extend and this may result in suboptimal
RFAapplicator positions and time-consuming corrections.
In contrast to diagnostic imaging, each image during the intervention is moreover acquired in prone patient position, causing intervertebral joint movements and an altered spine flexion compared to the diagnostic images, particularly in thoracic and cervical spine segments. This aspect further increases the cognitive demands of the radiologists for a precise metastasis puncture. In addition, the ablation progress itself cannot be visualised on the interventional images and must be inferred fromRF
power generator parameters. With progressing ablation of the metastases the impedance will increase due to the absence of conducting tissue, and along with the overall ablation time and energy the radiologist could estimate the coagulation and ablation progress, respectively. Currently, there is no more accurate approach, such as a suitable MRI thermome-try or transferred patient-specific necrosis zone predictions as a result of pre-interventional simulations. Subsequent to the ablation process, stabilisation methods are applied if necessary, e.g. kyphoplasty or verte-broplasty (Posteraro et al.,2004).
Figure 1.7: Patient’s back with both, skin markers for the navigation and the insertedRFAapplicators. In the background stands the power gener-ator for the radiofrequency current displaying the induced energy, the ablation time and the current tissue impedance, which are the parameters to estimate the ablation progress.
p o s t-interventional phase A few days after the intervention, post-operativeMRIscans are acquired to evaluate the treatment outcome, which is currently done by separately exploring and mentally matching metastasis and necrosis zone from pre- and post-interventional images (see Figure 1.8). This, however, is challenging due to the difficulties of correlating spatial positions in both image volumes and estimating the correct spatial extension in three-dimensional space. However, various studies have shown that a reliable assessment of the ablation zone is generally possible by means of follow-up imaging, since they found a strong correspondence between macroscopically and MRI-based mea-sured ablation zones (Palussiere et al.,2012). Therefore, an image-based and quantitative assessment of the necrosis area would be convincing regarding the treatment outcome. Despite this, there are currently no image processing methods like registration or segmentation approaches involved in the post-interventional workflow, leading to time-consuming, inconvenient and rather subjective assessments without any quantitative conclusions of the treatment outcome.
1.3 i n t e n d e d w o r k f l o w
The current clinical workflow offers various possibilities to enhance or speed up processing steps by means of computer-assistance, regardless whether pre-, intra- or post-interventional (see Figure 1.9). Addition-ally, data generation associated with computer-aided methods promotes comprehensibility, reproducibility and clinical documentation.
Figure 1.8: Pre- (upper row) and post-treatment (lower row) MRI scans of three exemplary patient cases (a-c) with corresponding metastases and necrosis zones (arrows). Shown are T1-weighted (pre-RFA) and
contrast-enhanced T1-weighted (follow-up) scans.
p r e-interventional phase Following the image acquisition, diag-nosis and decision forRFAtreatment, a patient-specific simulation of the
RFAcan predict the coagulation area considering state-dependent tissue parameters, such as electric and thermoconductive properties (Kröger et al.,2006; Weihusen et al.,2010). In order to assign those tissue parameters to patient-specific anatomical structures, a preceding detection and seg-mentation step, either manual, semi-automatic or fully automatic needs to be implemented. This includes first and foremost the metastases itself, as well as the surrounding tissues like vertebral bodies, intervertebral discs and organs at risk like the spinal cord. Since manually performed segmentations tend to be highly time-consuming and tedious considering the amount of tomographic image data, automatised approaches are to be preferred. The overall goal of computer-assistance during the ther-apy planning phase is to define a patient-specific optimal intervention strategy using a numerical simulation which predicts the ablation zone, considering optimisedRFAapplicator positioning, induced energy per time, and state-dependent tissue parameters of the involved structures (Matschek et al.,2017; Weihusen et al.,2010).
i n t e r v e n t i o na l p h a s e Immediately before intervention begin, the previously created and patient-individually optimised treatment plan could be digitally accessed and thus enables a rapid mental prepara-tion to the upcoming intervenprepara-tion. Addiprepara-tionally, pre-intervenprepara-tionally
diagnostic image acquisition
(MRI, CT)
therapy decision-making
(Applicability of RFA)
mental intervention planning
patient-individual RFA simulation (applicator access path, ablation process, necrosis zone prediction)
calibration of navigation
system (skin marker, FP-CT)
pre- and intra-operative image fusion (transfer of planning data onto intra-operative images)
image-guided instrument
navigation by biplanar angiography and FP-CT
RFA process monitored by
power generator (impedance, time, induced energy)
follow-up MR imaging
quantitative and objective
treatment outcome validation
qualitative and subjective treatment outcome validation (visual exploration, symptom reduction) Pre-interventional Phase Post-interventional Phase Interventional Phase
Figure 1.9: The current clinical procedure can be improved by integrating computer-assisted methods (in green) in each of the main work-flow phases. These approaches replace in particular time-consuming tasks based on mentally demanding work (in grey). In contrast to the support during the intervention and the subsequent treatment outcome validation, this thesis will not cover the actualRFA simula-tion itself, but approaches to provide necessary prerequisites for its feasibility.
acquired information could be transferred onto intra-interventional image data via manual or automatised image fusion. Thus, overlays of previ-ously segmented metastases, risk structures, predicted ablation zones and preferable applicator pathways could be projected onto the intra-operative images, which would enhance metastasis puncture precision and speed and thus, positively affect the treatment outcome. Supporting visual con-text information could reduce the radiologists’ cognitive effort arising from the mentally mapping of diagnostic image and planning informa-tion onto intra-operative images. However, largely automatic registrainforma-tion methods are to be preferred, since manual fusion would considerably delay the start of the intervention. Considering navigation support by optimised applicator pathways, a real-time tracking of the needle tips could further reduce the required cognitive effort of the radiologists. There are various existing approaches in literature and also commercial solutions regarding this tracking task, which could also enable live up-dates of the intervention plan (Hirooka et al.,2016; Tomonari et al.,2013). Subsequent to the metastasis puncture, the ablation process is performed under consideration of the parameter settings, i.e. induced energy and ablation time, defined during the preceding simulation.
p o s t-interventional phase After theRFA, follow-upMRIscans can be used as an input for a computer-supported assessment of the treat-ment outcome. With the aid of a framework that covers every step of the post-treatment process, i.e. target structure segmentation, image fusion of pre- and post-RFAimages, and the computation of quantitative valida-tion measures, the intervenvalida-tion outcome can be reliably and objectively
evaluated. Information regarding tumour coverage and safety margins towards organs at risk, for instance, can enhance prediction performance of tumour recurrence and survival time. Furthermore, by comparison with the pre-interventionalRFAsimulations, the results of the treatment validation can contribute to the optimisation of prospective ablation zone predictions, in the sense of a feedback loop. Considering such an ad-vanced computer-supported post-treatment workflow, the framework should consist of widely automatised, precise, and fast image processing methods.
1.4 t h e s i s o b j e c t i v e s
The previous section has pointed out, that there are various indications to establish a more sophisticated and computer-assisted workflow of RFA
interventions of spinal tumours and metastases, as it is done at present. The main purpose of this thesis is to provide solutions in terms of widely automatised image processing approaches to reduce the radiologists’ workload and time needed for recurring and time-consuming tasks, to enhance precision and speed of interventional procedures, and to support clinical decision-making. It is of crucial importance that such methods are suitable to be integrated into clinical routine by meeting the requirements defined by radiologists.
Therefore, various aspects throughout the clinical workflow with promis-ing potential of improvement were identified, startpromis-ing from approaches to support therapy and intervention planning, to approaches that provide relevant assisting information during the intervention, as well as meth-ods to enable a quantitative and reliable treatment outcome validation afterwards. For this purpose, multiple objectives regarding each task were defined in cooperation with the clinical partner in order to develop suitable and adequate solutions. In addition, various aspects of this thesis can also be transferred to analogous clinical issues or interventions, which makes the findings of this work relevant beyond the particular underlying subject matter.
1.5 t h e s i s s t r u c t u r e
Since this thesis covers computer-assisted methods supporting various processing steps throughout the clinical workflow of spinal RFAs, it seemed to be most appropriate to align the following chapters with the chronological order of the clinical procedures. This implies that the author will follow the general clinical sequence of a pre-interventional phase, an interventional phase and a post-treatment phase, as introduced in Section1.3, and subdivide each of the key aspects of this thesis into the usual sections "Introduction", "State of the Art", "Materials and Methods", "Results", "Discussion", and "Conclusion", without losing sight of the overall purpose of the work. This hopefully contributes to the thematic coherence of the individual parts and the readability of the whole thesis.
2
P R E - I N T E R V E N T I O N A L P H A S E
Subsequent to the initial diagnosis of spinal metastases or lesions and the resulting decision to treat them by minimally invasive RFablations the therapy and intervention planning phase represents the first part of the treatment workflow. In terms of computer-assistance, the segmentation of relevant anatomical structures represents a pivotal step towards a prospective patient-individual ablation simulation and support during the intervention. In order to numerically simulate heat propagation and related tissue necrotisation volumetric models of all involved tissues are required, i.e. metastases, vertebral bodies and risk structures like the spinal cord. This thesis focuses on the segmentation of the first two structures, as there were still open research issues or limitations in the related literature, while for instance, spinal cord segmentation was adequately addressed in the past by several studies (De Leener et al., 2015; Prados et al., 2016). It is worth mentioning, that this chapter of the thesis does not cover the implementation and design of a numerical simulation itself, which is in development by the cooperation partner Frauenhofer MEVIS (Kröger et al., 2010; Kröger et al., 2006; Weihusen et al., 2010). However, it addresses automatised approaches to provide required prerequisites, which otherwise would have to be produced manually in a very time-consuming and tedious manner. The content of this chapter is based on Hille et al. (2018b) and Hille et al. (2020).
2.1 v e r t e b r a l b o d y s e g m e n tat i o n
2.1.1 Introduction
Although this thesis focuses in particular on minimally invasive interven-tions of spinal metastases, it is noteworthy that with advancing computer-assisted medicine the segmentation of spinal structures like the vertebral bodies becomes increasingly relevant in other medical fields, too. Prior to the relevance in spinal oncology, the quantitative recording of the spine and thus the vertebrae has a considerable impact on various orthopedic and neuroradiological diagnoses, ranging from scoliosis, stenosis and osteoporosis to vertebral fractures (Brinjikji et al.,2015; Parizel et al.,2010). Besides the diagnostic and therapeutic purposes, automatised segmenta-tion procedures become increasingly important for processing the vast amounts of image data acquired for epidemiological studies (Rak and Tönnies,2017).
Basically, segmentation methods can be categorised according to the degree of automation, starting with completely manual methods, fol-lowed by semi-automatic methods, which for example only require user initialisation, up to fully automatised solutions (Smistad et al.,2015). Each
category offers advantages, but always with the drawback of restrictions or limitations regarding other aspects. While manually performed seg-mentations most commonly represent the gold standard with regard to the segmentation accuracy and are used as a ground truth for evalua-tion purposes of more automatised methods, they come at the cost of being highly time-consuming and tedious. With increasing degree of automation, the more challenging and ambitious it becomes to formulate adaptive and generalisable model terms from a priori knowledge and available image information to provide suitable solutions for specific segmentation tasks. In general, it constitutes a balancing act between defining descriptive features as precisely as possible, while being gener-alisable and capable of representing shape and appearance variabilities of the target structures due to their natural variety as well as different imaging protocols. The goal of most automatised approaches is to reduce the required time and user effort to perform the segmentation task. This, however, usually comes at the cost of a lowered accuracy, since it is hardly possible to represent the vast appearance variability of anatomical struc-tures in a specific model. With the increasing relevance of learning-based methods, the formulation of model terms has been largely eliminated and replaced by the automatised generation of distinctive image features. This has the great advantage that even features can be extracted which would either not have been noticed as important in a manually crafted model, or the implementation of such image information as a model term was not sufficiently applicable. However, learning-based systems must be fed with sufficiently large amounts of training data to widely represent the inherent variance of the data. Since clinical image data is often highly limited, the amount of available patient cases is usually a restricting factor in learning-based segmentation strategies.
While most of the related spine segmentation approaches that focus on CT or radiography (Darwish et al., 2015; Hammernik et al., 2015; Lessmann et al., 2019) benefit from the high contrast of bone tissue as well as the mostly high spatial resolutions, diagnostic MRIbecame an indispensable technique in clinical decision-making due to amplified soft tissue contrasts and the avoidance of radiation exposure. Besides, CT
and radiographic imaging cannot adequately deal with some pathologies like bone tumours and metastases, particularly in early stages (Shah and Salzman,2011). Therefore, MRIis often essential for diagnostic purposes, for instance, for the scenario addressed in this thesis. However, some characteristics of routine spineMRItremendously hamper the automation of segmentation approaches. Firstly, highly anisotropic spatial resolution results in partial volume effects and, thus, in blurred delineations between different tissue types especially among adjacent cross-sections (see Figure
2.1). Furthermore, bias field artifacts cause non-homogenous intensities between central and marginal areas. In addition, the image quality and emphasis of different tissues is affected by various imaging parameters, since standarised measurement units like the Houndsfield unit (HU) in CTdo not exist inMRI. This means, however, that even native MRI
a
b
c
Figure 2.1: Two reconstructed axial (a and b) and one native sagittal slices (c) of a clinical routine T1-weightedMRIscan demonstrate the difficulty in
distinguishing vertebrae from adjacent structures due to the partial volume effects (PVE) caused by high anisotropy. With large sagittal slice spacing, this becomes particularly pronounced at the lateral ends of the vertebrae.
in absolute intensities when it comes to different scanner models or parameter settings. Therefore, a segmentation method that is relevant in clinical settings has to deal with a large variety of MRIsequences and imaging protocols.
2.1.2 State of the Art
There are various works in literature regarding vertebral body segmenta-tion inMRI, which differ in their applied methods and the used image data. Besides approaches that were applied to mid-sagittal 2D images (Athertya, Kumar, et al., 2016; Ghosh et al., 2014; Huang et al., 2009), which disregarded valuable spatial information of tomographic imaging, various 3D methods were presented for spinalMRIthat will be described in detail below.
Besides largely obsolete approaches based on thresholding, water-sheds, and region-growing, segmentation strategies using deformable models, e.g. active contour models (ACM) (Caselles et al., 1997; Kass et al., 1988) or active shape models (ASM) (Cootes et al., 1995) were applied to vertebral body segmentation in spinalMRI. Davatzikos et al. (2002), for instance, trained a deformable shape model to register image data with template images and achieved on average a Dice similarity coefficient (DSC) of 81.5 ± 3.6 % on routine images of 14 young healthy volunteers. They used solely T1-weightedMRIscans with a spatial reso-lution of 0.93 × 0.93 × 3 mm3.
Štern et al. (2011) also applied a model-based approach, while optimis-ing 29 shape parameters by maximisation of the dissimilarity between inner and outer object intensities driven by image gradients. Their
ap-proach was initialised by marking of each vertebrae in terms of an user input and by approximately specifying the vertebral size as a selection of the respective spine segment (upper/lower thoracic and lumbar). The applied evaluation set contained solely T2-weightedMRIsequences with in total 75 vertebral bodies of nine healthy subjects, three of them with a resolution of 0.4 × 0.4 × 3 mm3 and six with isotropic spatial reso-lution (1 × 1 × 1 mm3). Their approach resulted in an average surface distance (ASD) between the segmented object surface and ground truth points of 1.85 ± 0.47 mm. Štern et al. (2011) stated a processing time ranging from one to 15 min per vertebra.
Neubert et al. (2011) and Neubert et al. (2012) usedASMto segment both, vertebral bodies and intervertebral discs. They tested their approach on T2-weighted MRI scans of 14 healthy volunteers with in total 132 vertebrae, acquired with high resolution (0.34 × 0.34 × 1 to 1.2 mm3) and achieved a meanDSCof 91 % and a mean Hausdorff distance (HD) of 4.08 mm. However, the average run time per vertebra of 35 min was considerably long. A 10-fold processing time reduction decreased their
DSCfrom 90.8 % (Neubert et al.,2012) to 85 % (Neubert et al.,2011). Ayed et al. (2012) pursued the idea of formulating the segmentation as a distribution-matching problem. By using an augmented Lagrangian method the distribution of vertebral appearance features was matched to an a priori known reference distribution in order to classify vertebral foreground and non-vertebral background voxel. A mean mid-sagittal
2D-DSCof 85 % was achieved, which leads to the assumption that volumetric quality measures would certainly drop, since most segmentations have difficulties at the lateral ends of the vertebrae due to partial volume effects caused by low laterolateral resolutions or large slice spacings, respectively.
Kadoury et al. (2013) included shape and pose relations between var-ious vertebrae to extend the concept of statistical shape models and to avoid any ambiguities. Using non-linear manifold embeddings im-proved the shape space representations in contrast to common ASM. They achieved aASDof on average 2.93 ± 1.83 mm, while applying solely T1-weighted sequences of eight subjects with almost isotropic spatial resolution (1.3 × 0.9 × 1 mm3).
Zuki´c et al. (2014) combined edge- and intensity-based features, i.e. Canny edges and thresholded gradient magnitudes to a multifeature-based model. Their approach was initialised by a preceding vertebral center detection step using a Viola-Jones detector. The surface mesh of their model was enlarged by balloon forces and constrained by a smooth-ness term and the approximated vertebral body size. They achieved an average DSCof 79.3 % and a meanASDof 1.76 ± 0.38 mm. Their method was evaluated on clinical routine datasets consisting of a variety ofMRI se-quences including both healthy and pathological vertebrae. Therefore, in contrast to the above mentioned works, their evaluation set was designed to reflect clinical routine imaging.
Chu et al. (2015) fully automatically localised vertebral bodies to define a region of interest (ROI) for a subsequent segmentation step, where
they were using random forest classification for estimating the fore- or background likelihood of each pixel within the produced ROIs. The results were combined with a learned probability map to segment each vertebral body via thresholding. Chu et al. (2015) tested their approach on 23 T2-weighted images, without stating any pathologies, achieving an overallDSCof 88.7 %, a meanASDof 1.5 ± 0.2 mm and an averageHD
of 6.4 ± 1.2 mm. The average computational time per dataset was about 1.3 min.
Korez et al. (2016) introduced a convolutional neural network (CNN )-based approach in spineMRIsegmentation. Their method linked active shape models with likelihood maps of the vertebral bodies and achieved an overall DSCof 93.4 %, an average HDof 3.83 mm and a mean ASD
of 0.54 mm. Korez et al. trained and tested their methods on the 23 T2-weighted images made publicly available from Chu et al. (2015).
Goankar et al. (2017) presented a machine learning-based system for vertebral body segmentation on clinicalMRIscans of the lumbar spine. In contrast to Chu et al. (2015) and Korez et al. (2016) they examined the ap-plicability of their method to differentMRIsequences, though they trained only on T2-weighted images. The implementation of superpixel-based multiparameter ensemble learning was followed by some morphological post-processing to increase segmentation scores. Goankar et al. had in total 48 sagittal T2-weighted and 15 T1-weightedMRIscans and randomly selected six T2-weighted image volumes for training procedure. The spa-tial resolution varied in-plane from 0.34 × 0.34 to 1.1 × 1.1 mm and the slice thickness was between 0.5 to 5.0 mm. Applying their approach to T2-weighted images resulted in a meanDSCof 83 %. The segmentation of vertebrae in T1-weighted images after training on T2-weighted images expectably resulted in lowerDSCscores (on average 75 %).
It is noteworthy that in the time since the elaboration and publication of the work presented in this section, a few more relevant works have been published. Rak et al.,2019, which was the most promising among them, presented a whole-spine segmentation approach for MRIdata com-bining graph cuts including star-convexity constraints and convolutional neural networks, which considerably built on their previous work (Rak and Tönnies, 2017). After a required vertebral patch extraction, aCNN
provides likelihood maps in terms of appearance and shape priors, which were the input for a graph cut formulation based on encoding swaps to avoid ambiguous segmentations of neighbouring vertebrae. To evaluate their approach, two databases were used, including T1- and T2-weighted image volumes of 64 healthy subjects from an epidemiologic study and 23 T2-weighted scans from a publicly available source (Chu et al.,2015). Rak et al. (2019) reported a meanDSCof 94.9 %, an averageASDof 0.93 mm and run times below two seconds per vertebra.
The analysis of the existing literature regarding vertebral body seg-mentation showed that each of the related work was limited by either one or more of the following aspects: insufficient segmentation accuracy, long computational time, limitation to a singleMRIsequence or applying image data of higher quality for non-clinical study purposes including
healthy subjects. Accordingly, despite various existing approaches from the related literature, there was still a need for a solution strategy that addresses the specific requirements of the particular clinical purpose in this thesis.
2.1.3 Objectives
Considering the settings of clinical routine and the aforementioned goal of supporting therapy planning of spinal metastasis interventions, the following objectives were defined in cooperation with the clinical partner:
• Computational time per patient case < 1 min on current consumer hardware
• Applicable to variousMRIsequences for diagnostic purposes (T1-, T2-, contrast-enhanced T1-weighted etc.)
• Segmentation accuracy in the range of the inter-reader variability of field experts
In clinical procedures time plays an important role, expecting suit-able computerised approaches to fit into the clinical workflow without significant delay or excessive workload. Supplementary and novel ap-proaches to be integrated should optimally support physicians in clinical decision-making and with repetitive tasks, which tend to be tedious and monotonous, and thus, susceptible to errors due to fatigue. Typically, manual segmentations are repetitive tasks, especially with stacks of slices acquired during 3D tomographic imaging likeMRI. Therefore, the sup-port provided by computer-assisted and widely automatised methods in spinal segmentation can drastically reduce the required effort and time. Subsequent to the analysis of the related literature and in consideration of the required time of a manually performed vertebral body segmentation (>10 min per patient case), the objective of a computational time of less than 1 min per patient case was defined. This means a significant reduc-tion of the required time compared to a manual contouring, whereby user interaction, e.g. for initialisation purposes, is still feasible.
Furthermore, approaches with a wide-ranging applicability to clinical routine spinalMRIshould deal with various imaging sequences, parame-ter settings, spatial resolutions, spine sections and healthy vertebrae as well as pathological altered due to fractures, bruises, or metastases. In order to verify the compliance with this requirement, a comprehensive and diverse evaluation set has been compiled with particular focus on clinical data, which is usually highly anisotropic and diverse regard-ing the imagregard-ing protocols. Previous works often shifted away from the challenges of clinical settings by applying their approaches to only one particularMRIsequence or healthy subjects for study purposes.
In order to provide effective support in clinical routine of spinal inter-ventions the segmentation quality should be close to that of an experi-enced radiologist and in the range of the inter-reader variability (IRV). The
IRV represents an appropriate and robust estimate of an expert-like seg-mentation accuracy. Methods that produce significantly worse accuracies are only of limited use, since the required post-processing and correction effort would outweigh the time saved by an automatic solution.
Summarising, an approach with reasonable clinical applicability for the addressed scenario should cope with various MRI sequences and imaging protocols, highly anisotropic data, multiple pathological findings in any spine segment, it should require only short computational time, and provide a segmentation accuracy in the range of theIRV.
2.1.4 Materials and Methods
Image Data
In order to demonstrate the suitability of the proposed method, an evalu-ation set was assembled, which included image data of various clinical and research purposes. It consisted of four different databases, including 63 sagittalMRIdatasets with overall 419 vertebral bodies of the cervical, thoracic and lumbar spine. The patients or subjects differed in age, sex and presence of spinal pathologies. The evaluation data was acquired in different hospitals and research facilities with various MRIscanners and comprised multiple imaging sequences and protocols (see Table 2.1). Besides this variety, a key characteristic of most datasets was their high anisotropy factor (slice spacing divided by in-plane pixel spacing), rang-ing from 1.6 to 8.19.
The first database consisted of pre-interventionally acquiredMRIdata before RFAs of spinal metastases and was representative for the main application case which this thesis addresses. This image data included both vertebrae with metastases from different primary tumours and their adjacent healthy neighbours.
Commonly, the comparison of segmentation approaches and their results between entirely different datasets must be considered as indirect. To overcome this limitation, the evaluation set furthermore consisted of overall 39 image volumes made publicly available together with the related work of Zuki´c et al. (2014) and Chu et al. (2015). Hence, the produced results could be matched directly with those works. The data from Zuki´c et al. (2014) included both healthy and pathological spines, e.g. with scoliosis, spondylolisthesis, and vertebral fractures and consisted of variousMRIsequences. The third database, published with the work of Chu et al. (2015), comprised 23 T2-weighted magnetic resonance (MR) images of thoracolumbar spines of voluntary subjects and represented common image data used for research purposes.
The concluding database consisted of epidemiological image data from the Study of Health in Pomerania (SHIP) study (Völzke et al., 2011) and featured spineMRimages including T1- and T2-weighted sequences. Using this data in particular served the purpose to understand the limits of the presented method regarding low spatial resolution and image quality.
Table 2.1: Characterisation of all datasets used for the evaluation of the proposed method. The individual datasets differed regarding the used MRI sequences (spin echo (SE), turbo spin echo (TSE), fast spin echo (FSE), turbo inversion recovery magnitude (TIRM)), pixel spacing Px,yand
slice thickness Sz(both stated in mm), the size of the acquisition matrix
M, the anisotropy factor FA, the number of labelled vertebral bodies
#V, the displayed spine segment SpS (C cervical, T thoracic, L
-lumbar), the presence of pathology (n.s. - not stated), age, sex, and from whom the reference segmentations originated (neuroradiologists N, trained field experts T ). The horizontal lines subdivide the used datasets according to their source (first section - pre-interventionally acquired beforeRFAs, second section - publicly released by Zuki´c et al. (2014), third section - publicly released by Chu et al. (2015), fourth section - part of theSHIP(Völzke et al.,2011).
Dataset MRISeq Px,y Sz M FA #V SpS R Path. Age Sex
preRFA_1 T1TSE 0.5 3.3 640 x 640 x 20 6.6 5 C7 - T4 T - 54 F preRFA_2 T1TSE 0.78 3.3 512 x 512 x 20 4.23 7 T1 - T7 T + 70 M preRFA_3 T1TSE 0.68 3.3 512 x 512 x 20 4.85 6 T12 - L5 T + 61 M preRFA_4 T1TSE 0.49 3.3 528 x 528 x 17 6.73 7 T3 - T9 T + 76 M preRFA_5 T1TSE 0.49 3.3 528 x 528 x 15 6.73 8 T7 - L2 T + 74 M preRFA_6 T1TSE 0.46 3.3 640 x 640 x 17 7.17 5 T12 - L4 T + 76 M Aka2 T2FSE 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 21 F Aka3 T1FSE 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 21 F Aka4 TIRM 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 21 F Aks5 T2FSE 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 22 F Aks6 T1FSE 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 22 F Aks7 TIRM 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 22 F Aks8 T1FSE 0.70 4 512 x 512 x 15 5.69 8 T10 - L5 T + 22 F C002 T2TSE 1.12 3.3 448 x 448 x 31 2.96 12 T6 - L5 N + 74 F DzZ_T2 T2TSE 0.55 4.4 640 x 640 x 12 8.05 8 T10 - L5 T - 27 M DzZ_T1 T1TSE 0.68 4.4 512 x 512 x 12 6.44 8 T10 - L5 T - 27 M F02 T2SE 0.5 3.85 768 x 768 x 18 7.7 8 T10 - L5 N + 51 M F03 T2TSE 1.19 3.3 320 x 320 x 25 2.77 6 T12 - L5 N + 72 M F04 T2TSE 1.12 3 448 x 448 x 23 2.69 12 T6 - L5 N + 69 F S01 T2SE 0.47 3.85 640 x 640 x 16 8.19 6 T12 - L5 N + 65 M S02 T2SE 0.47 3.85 640 x 640 x 16 8.19 7 T11 - L5 N + 55 F St1 T2SE 0.5 3.85 704 x 704 x 20 7.7 7 T11 - L5 N + 71 M Chu (1-23) T2TSE 1.25 2.0 305 x 305 x 39 1.6 7 T11 - L5 T n.s. n.s. F, M SHIP (1-9) T1/T2TSE 1.12 4.4 448 x 448 x 15 3.67 5 L1 - L5 N, T n.s. 29-65 F, M
Methodology
The major steps of the proposed method were as followed (see also Fig-ure2.2):
1. Initially, a Gaussian filter-based intensity correction was imple-mented as a pre-processing step to deal with bias field artifacts. The filter kernel size was set to 120 × 120 × 30 mm3 and σ to 20 mm to estimate the bias field of each image volume. In order to remove it, the original image was divided by the bias field estimation. Subse-quently, each image volume has been upsampled to ensure spatial isotropy.
2. User interaction initialised the presented approach with three points in a selectable mid-sagittal cross-section to approximate the size, cen-ter and sagittal orientation of each vertebral body. For this purpose, both corners of the superior endplate as well as the posterior corner of the inferior endplate were marked. The lateral flection angle could be deduced from interpolating the landmarks’ z-coordinates (i.e. in slice direction) of consecutive vertebral bodies.
3. Intensity-based features, e.g. median and variance, were obtained from a cube within the vertebral center and with variable edge length, i.e. two fifths of the specific vertebral body height and length.
4. An abstracted vertebral body shape model was placed upon each vertebral center with the approximate vertebral body length, height and orientation.
5. Within this shape, a pre-segmentation was performed based on adaptive thresholding. The previously gained intensity-based fea-tures ensure patient andMRIsequence independence and therefore avoid common difficulties regarding thresholding in MRI. Subse-quently, the result was morphologically filtered, first by hole filling and dilating with a 3 mm diameter ball structuring element and followed by removing objects smaller than 1 cm3. To yield the vertebral body probability map P, the resulting binary image was distance-transformed by a Gaussian convolution (kernel size of 10 mm3 and σ of 2 mm) and multiplied with the original image volume. This weakened local constraints at the boundaries of the pre-segmented object and enabled level-set convergence away from disadvantageously placed shape models.
6. Boundary feature maps G of each vertebral body were computed by dilating the extracted boundaries of the fitted vertebral body shape model using a 3 mm diameter ball structuring element, sub-sequently distance-transforming them likewise the probability map and multiplying them with the gradient magnitude images. This feature ensured level-set convergence towards object boundaries
