Index

Cardiac Functional Analysis – Automated Strain Analysis of the left Ventricle using Computed Tomography

The heart is one of the most important organs within the human body. It maintains the blood circulation
which is crucial to transport substances through the body to e.g. oxygenate the brain or muscles.
Dysfunctions can lead to death. Cardiovascular and circulatory diseases are one of the most common
causes of mortality [1]. To indicate such a disease a cardiac functional analysis is often performed. A
cardiac functional analysis assesses the vitality of the heart based on objective criteria such as ejection
fraction and motion evaluation. This analysis helps to identify, localize and treat dysfunctions. Especially
tracing the dynamics of the heart, referred to as myocardial strain analysis, gives diagnostic
hints. The movement of the heart and each chamber can be tracked using cardiac imaging. There are
three possible directions for the motion: radial, longitudinal and circumferential, which can be seen
as intrinsic dynamics. For instance in case of acute myocarditis the intrinsic dynamics differ significantly
from those of healthy patients. As the left ventricle is the largest of the four heart chambers –
dysfunctions might have the highest impact on its function and therefore on its dynamics [2, 3].
To perform a cardiac functional analysis or to be precise a strain analysis of the left ventricle noninvasive
imaging modalities are used. Cardiac magnetic resonance and echocardiography are commonly
used and build the state of the art methods [4, 5]. Up to now computed tomography is barely used for
assessing the cardiac function but more for identifying diseases as coronary atherosclerosis [6]. The
myocardium tends to have a low contrast in computed tomography images of the heart. Therefore
tracking of endo- and epicardium becomes more complicated. Due to this, the complexity of performing
a cardiac functional analysis increases. But aspects like high resolution, time- and cost efficiency
motivate to overcome these difficulties.
Within this project it will be investigate whether computed tomography is capable to perform a
cardiac functional analysis. The goal is to automate the strain analysis of the left ventricle using
computed tomography.
Given several 3D CT images of a heartbeat cycle the strain analysis will be performed in the following
way:
1. The myocardium of the left ventricle has to be localized and segmented using an active shape
model.
2. For the motion tracking different registration algorithms will be evaluated. The range of approaches
is broad. Possible methods for landmark based registration would be thin-plate splines [7]. As the
complexity increases with the dimension 2D intensity based registration could be performed on sliced
images [8]. Using the whole 3D image there are more classic algorithms as well as deep learning based
approaches are conceivable [9, 10].
3. The intrinsic dynamics, namely longitudinal, radial and circumferential strain, are calculated by
projecting the transformation vectors provided by the registration algorithm onto the main axes of an
intrinsic coordinate system.
4. The results will be visualized as color coded strain magnitude within the CT image or in a polar
map [3].
5. The results of different registration algorithms are qualitatively and quantitatively compared using
the visualizations as well as different metrics, possibly such as the correlation coefficient [11] and the
Hausdorff distance [12].

References
[1] Gregory A Roth et al. Global burden of cardiovascular diseases and risk factors, 1990–2019:
update from the gbd 2019 study. Journal of the American College of Cardiology, 76(25):2982–
3021, 2020.
[2] Otto A Smiseth, Hans Torp, Anders Opdahl, Kristina H Haugaa, and Stig Urheim. Myocardial
strain imaging: how useful is it in clinical decision making? European Heart Journal, 37(15):1196–
1207, 2016.
[3] Aldostefano Porcari et al. Strain analysis reveals subtle systolic dysfunction in confirmed and
suspected myocarditis with normal lvef. a cardiac magnetic resonance study. Clinical Research
in Cardiology, 109(7):869–880, 2020.
[4] Dagmar F Hernandez-Suarez and Angel L´opez-Candales. Strain imaging echocardiography: what
imaging cardiologists should know. Current Cardiology Reviews, 13(2):118–129, 2017.
[5] Alessandra Scatteia, Anna Baritussio, and Chiara Bucciarelli-Ducci. Strain imaging using cardiac
magnetic resonance. Heart Failure Reviews, 22(4):465–476, 2017.
[6] Marc R Dweck, Michelle C Williams, Alastair J Moss, David E Newby, and Zahi A Fayad.
Computed tomography and cardiac magnetic resonance in ischemic heart disease. Journal of the
American College of Cardiology, 68(20):2201–2216, 2016.
[7] Rainer Sprengel, Karl Rohr, and H Siegfried Stiehl. Thin-plate spline approximation for image
registration. In Proceedings of 18th annual international conference of the IEEE Engineering in
Medicine and Biology Society, volume 3, pages 1190–1191. IEEE, 1996.
[8] Christoph Guetter, Hui Xue, Christophe Chefd’Hotel, and Jens Guehring. Efficient symmetric
and inverse-consistent deformable registration through interleaved optimization. In 2011 IEEE
international symposium on biomedical imaging: from nano to macro, pages 590–593. IEEE, 2011.
[9] Huajun Song and Peihua Qiu. Intensity-based 3d local image registration. Pattern Recognition
Letters, 94:15–21, 2017.
[10] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Voxelmorph:
a learning framework for deformable medical image registration. IEEE Transactions on Medical
Imaging, 38(8):1788–1800, 2019.
[11] Richard Taylor. Interpretation of the correlation coefficient: a basic review. Journal of Diagnostic
Medical Sonography, 6(1):35–39, 1990.
[12] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge. Comparing images using the hausdorff
distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850–863, 1993.

Thrombus detection using nnDetection

Efficient Methods for Post Myocardial Infarction Ventricular Tachycardia Modeling: from Image Processing to Electrophysiological Simulation

Cardiovascular diseases are the leading cause of death worldwide with an estimate of 17.9 million in
2019, as reported by theWorld Health Organization (WHO) [1]. Among the numerous diseases included
in this category, ischemic heart disease is one of the most frequent, with an estimate of 8.8 million deaths
in 2019 [1]. A common result of ischemic heart disease is myocardial infarction (MI), which occurs when
the blood flow to an area of the heart is blocked. This damages the heart tissue resulting in necrosis
[2]. Approximately 10% of the patients that survive a previous MI event have an increased risk of
death in the following months or years after hospital discharge. In these cases, up to 50% of the deaths
can be secondary to a sustained Ventricular Tachycardia (VT) or Ventricular Fibrillation (VF) event [2].
Ventricular Tachycardia is a heart arrhythmia that occurs when a fast and abnormal heart rate originates
in the ventricles. A well-known mechanism for VT is an action potential wave re-entry caused by
a unidirectional conduction block in slow conductive areas of the myocardium [2]. These areas contain
a complex mixture of scar (i.e. infarcted tissue) and surviving myocytes that is often referred to as
heterogeneous or border zone tissue [3]. For these cases, ablation therapy is the preferred surgical approach.
Approximately 50% of the patients that undergo ablation therapy show VT recurrence before
five years after surgery [4]. In order to improve the outcome, a critical part of the procedure is to
properly localize and ablate the arrhythmic substrate [4].
In this context, it is hypothesized that a combination of scar lesion imaging, cardiac electrophysiology
modeling, and artificial intelligence, can improve the localization of VT ablation targets, the detection
of incomplete ablation procedures, and the selection of the ablation strategy. Previous studies have
already investigated the usage of digital twin technologies for VT therapy planning [3, 5, 6]. However,
the VT modeling pipeline is extensive and still presents several challenges.
First, the manual segmentation of the heart chambers from Late Gadolinium Enhanced (LGE) MRI
images is a tedious procedure that is prone to inter- and intra-operator variability [7]. The segmentation
of the heart chambers is a necessary step for the generation of anatomical 3D models. Furthermore,
it allows to analyze, locate, and quantify myocardial scar, which can be used to guide the ablation
procedure [8]. Lastly, simulating VT in-silico is very dependant on the selected model and simulation
parameters. Previous studies have addressed how different parameter combinations affect the
inducibility of VT re-entrant activity [5, 6]. These studies usually rely on finite element method (FEM)
simulations on very detailed geometries, which can require up to several hours of run-time per simulation.
This inevitably constrains the number of parameter combinations that can be studied.
With these challenges in mind, the main contributions of this study will be:
• Literature review of the state-of-the-art methods in Late Gadolinium Enhanced (LGE) image
processing and segmentation.
• Literature review of the state-of-the-art methods for electrophysiology modeling and simulation
of virtual VT inducibility.
• Evaluation of a deep learning method for automatic myocardium segmentation from LGE images,
with a possible extension to automatically locate and quantify scar tissue. This automatic
segmentation method will focus on the potential advantages compared to manual segmentation
approaches in terms of reproducibility and time savings [7].
• Study of virtual VT inducibility in a set of porcine models after MI with focus on optimal selection
of model parameters. This task will be carried out using the Lattice-Boltzmann method, a monodomain
solver which allows to perform electrophysiology simulations of VT re-entrant activity,
with the advantage of being faster than other FEM approaches [9].
References
[1] World Health Statistics 2021: Monitoring Health for the SDGs: Sustainable Development Goals. Geneva, Switzerland:
World Health Organization, 2021. Licence: CC BY-NC-SA 3.0 IGO.
[2] J. Bhar-Amato, W. Davies, and S. Agarwal, “Ventricular arrhythmia after acute myocardial infarction: ”the perfect
storm”,” Arrhythmia & Electrophysiology Review, vol. 6, no. 3, p. 134, 2017.
[3] H. Ashikaga, H. Arevalo, F. Vadakkumpadan, R. C. Blake, J. D. Bayer, S. Nazarian, M. Muz Zviman, H. Tandri,
R. D. Berger, H. Calkins, D. A. Herzka, N. A. Trayanova, and H. R. Halperin, “Feasibility of image-based simulation
to estimate ablation target in human ventricular arrhythmia,” Heart Rhythm, vol. 10, pp. 1109–1116, Aug. 2013.
[4] M. Wolf, F. Sacher, H. Cochet, and T. Kitamura, “Long-term outcome of substrate modification in ablation of
postˆamyocardial infarction ventricular tachycardia,” Circulation: Arrhythmia and Electrophysiology, vol. 11, Feb.
2018.
[5] F. O. Campos, J. Whitaker, R. Neji, S. Roujol, M. OˆaNeill, G. Plank, and M. J. Bishop, “Factors promoting conduction
slowing as substrates for block and reentry in infarcted hearts,” Biophysical Journal, vol. 117, pp. 2361–2374, Dec.
2019.
[6] A. Lopez-Perez, R. Sebastian, M. Izquierdo, R. Ruiz, M. Bishop, and J. M. Ferrero, “Personalized cardiac computational
models: From clinical data to simulation of infarct-related ventricular tachycardia,” Frontiers in Physiology,
vol. 10, p. 580, May 2019.
[7] Y. Wu, Z. Tang, B. Li, D. Firmin, and G. Yang, “Recent advances in fibrosis and scar segmentation from cardiac mri:
A state-of-the-art review and future perspectives,” Frontiers in Physiology, vol. 12, p. 709230, Aug. 2021.
[8] S. Toupin, T. Pezel, A. Bustin, and H. Cochet, “Whole-heart high-resolution late gadolinium enhancement: Techniques
and clinical applications,” Journal of Magnetic Resonance Imaging, p. jmri.27732, June 2021.
[9] S. Rapaka, T. Mansi, B. Georgescu, M. Pop, G. A.Wright, A. Kamen, and D. Comaniciu, “Lbm-ep: Lattice-boltzmann
method for fast cardiac electrophysiology simulation from 3d images,” in Medical Image Computing and Computer-
Assisted Intervention – MICCAI 2012, vol. 7511, pp. 33–40, Berlin, Heidelberg: Springer Berlin Heidelberg, 2012.

Fruit Terminator – Annotation of Lung Fluid Cells via Gamification

Deep learning method for emotion recognition through the Fusion of body and context features

Emotion recognition currently has a wide range of applications in various fields. For example, doctors can use emotion recognition to take better care of patients, or teachers can use emotion recognition to judge the concentration level of students, etc. Previous visual emotion recognition researches have mainly focused on facial expression. Still, it has not achieved good results in unconstrained scenarios. This thesis presents a network for emotion recognition based on posture, body features, and context information; the aim is to improve emotion recognition accuracy in different unconstrained scenarios.

The network has been designed in a three-branch architecture, including three feature streams: body, skeleton and context streams. The extraction of human key points from images explored three different networks: Openpose[1], Alphapose[2] and Mediapipe[3]. This thesis uses GCN to extract features from key points. Body and context feature extraction using Transfomer Vision and Resnet50. The three streams are fused to predict dimensional emotion representation, valence, arousal, and dominance. The experiment is trained on public datasets (EMOTIC datasets[4]), and the verification of the model is on the CAER-S datasets[5]. Experimental results show that the proposed method effectively integrates emotional information expressed by body and context and has good generalization ability and applicability. Considering body poses to recognize people’s emotions provides a new benchmark for emotion recognition in visual poses.

References:

[1] Cao, Zhe, et al. “Realtime multi-person 2d pose estimation using part affinity fields.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

[2]  Fang, Hao-Shu, et al. “Rmpe: Regional multi-person pose estimation.” Proceedings of the IEEE international conference on computer vision. 2017.

[3] Lugaresi, Camillo, et al. “Mediapipe: A framework for building perception pipelines.” arXiv preprint arXiv:1906.08172 (2019).

• Kosti, Ronak, et al. “Context based emotion recognition using emotic dataset.” IEEE transactions on pattern analysis and machine intelligence11 (2019): 2755-2766.

[5]  Lee, Jiyoung, et al. “Context-aware emotion recognition networks.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.

Einfluss der Feldstärke (B0) auf den Intravoxel Incoherent Motion (IVIM) Effekt in der Diffusions-MRT der Wade

Die Struktur und Funktion der Muskulatur im menschlichen Körper kann heutzutage
anhand verschiedener nichtinvasiver Magnetresonanztomographie-Methoden (MRT)
untersucht werden. [1]
Seit der erstmaligen Erwähnung durch Denis Le Bihan im Jahre 1986 [2], hat die
Intravoxel Incoherent Motion (IVIM) MRT hierbei im Laufe der Jahre zunehmend an
Bedeutung gewonnen. Sie ermöglicht eine gleichzeitige Beurteilung von Komponenten
des Blutflusses und der Gewebediffusion. [3]
IVIM wird hauptsächlich zur Untersuchung von Geweben mit hoher Gefäßdichte
eingesetzt, beispielsweise bei Organen wie der Leber, der Niere und dem Gehirn.
Hierbei kann der Blutfluss Rückschlüsse auf die Organgesundheit und auf den
Zusammenhang zwischen metabolischen und vaskulären Reaktionen auf Stimuli
zulassen. [4]
In der Skelettmuskulatur ist der Blutfluss abhängig vom Aktivitätszustand des Muskels
und damit für die Erfassung und Interpretation von IVIM-Daten von besonderem
Interesse. [5]
IVIM ist eine fortgeschrittene Technik der diffusionsgewichteten
Magnetresonanztomographie (engl. diffusion weighted imaging, DWI).
Die DWI misst die Diffusionsbewegung im Gewebe, welche gemäß Brown‘scher
Molekularbewegung als Zufallsbewegung von Wassermolekülen beschrieben werden
kann. Im Gegensatz zu den meisten anderen MRT-Modalitäten handelt es sich bei der
DWI um eine quantitative Methode. [6]
Da die Diffusionsbewegung im Gewebe abhängig von Zellrestriktionen ist, ermöglichen
Diffusionsmessungen Aussagen über den Aufbau des Gewebes. Ein wichtiger
Parameter der DWI ist der hierbei zeitabhängige Diffusionskoeffizient D(t). [7]
Im diffusionsgewichteten Signal spiegelt sich die inkohärente Blutbewegung im
Kapillarbett ähnlich wider wie die Diffusion selbst. Somit kann dem kapillaren Blutfluss
ein Pseudodiffusionskoeffizient D* zugeordnet werden, welcher etwa einen Faktor 10
größer ist als der Diffusionskoeffizient D(t). [8]
Daher bietet IVIM sowohl einen Einblick in die Bewegung in intravaskulären Räumen
als gleichzeitig auch in extravaskulären Räumen.
Ziel dieser Masterarbeit ist es, den Einfluss der magnetischen Feldstärke auf den IVIM
Effekt an der Wade zu untersuchen. Hierzu werden bis zu 10 Probanden und
Probandinnen rekrutiert. Die Datenaufnahme findet hierbei an einem 7T MRT Scanner
(MAGNETOM Terra) und an einem 0,55T MRT Scanner (MAGNETOM Free.Max)
statt. Anschließend sollen die generierten Daten hinsichtlich der IVIM Parameter
miteinander verglichen werden.
[1] Englund, E.K., Reiter, D.A., Shahidi, B. and Sigmund, E.E. (2021), Intravoxel
Incoherent Motion Magnetic Resonance Imaging in Skeletal Muscle: Review and
Future Directions. J Magn Reson Imaging.
https://doi.org/10.1002/jmri.27875
[2] Le Bihan, Denis , Breton, Eric , Lallemand, Denis , Aubin, ML , Vignaud, Jean ,
Laval-Jeantet, M: Separation of diffusion and perfusion in intravoxel incoherent
motion MR imaging. In: Radiology 168 (1988), Nr. 2, S. 497–505
[3] Le Bihan, Denis: What can we see in IVIM MRI?. In: NeuroImage, Volume 187
(2019), Pages 56-67
[4] Denis Le Bihan et al.: Intravoxel Incoherent Motion (IVIM) MRI: Principles and
Applications. 1. Aufl. Pan Stanford Publishing, 2019. isbn: 978-981-4800-19-8,
S.117-145
[5] Jungmann PM, Pfirrmann C, Federau C. Characterization of lower limb muscle
activation patterns during walking and running with Intravoxel Incoherent Motion
(IVIM) MR perfusion imaging. Magn Reson Imaging. 2019 Nov;63:12-20. doi:
10.1016/j.mri.2019.07.016. Epub 2019 Jul 26. PMID: 31356861.
[6] Stieltjes et al.: Diffusion Tensor Imaging. 1. Aufl. Springer Verlag, 2013. isbn: 978-
3-642-20456-2.
[7] Laun, F., Fritzsche, K., Kuder, T. et al. Einführung in die Grundlagen und
Techniken der Diffusionsbildgebung. Radiologe 51, 170–179 (2011).
https://doi.org/10.1007/s00117-010-2057-y
[8] Le Bihan, Denis , Breton, Eric , Lallemand, Denis , Aubin, ML ; Vignaud, Jean ,
Laval-Jeantet, M: Separation of diffusion and perfusion in intravoxel incoherent
motion MR imaging. In: Radiology 168 (1988), Nr. 2, S. 497–505

Letter Inpainting and Detection of Mathematical Diagrams in Multi-Lingual Manuscripts using a Deep Neural Network approach

Determining the Influence of Papyrus Characteristics on Fragments Retrieval with Deep Metric Learning

Multi-Task Learning for Glacier Segmentation and Calving Front Detection with the nnU-Net Framework

With the nnU-Net, Fabian Isensee et al. [1] provide a training framework for the widely applied
segmentation network U-Net [2]. The framework automates the tedious adjustment of hyperparameters
(e.g. number of layers, learning rate, batch size, etc.) that is needed to optimize the performance of
the U-Net. The framework takes a fingerprint of the given dataset and adjusts the hyperparameters
accordingly. This approach achieved stunning results on multiple independent datasets in the medical
domain without manual adjustments [1].

This thesis evaluates the performance of nnU-Net on the segmentation of synthetic aperture radar
(SAR) images of glaciers taken by satellites. Additionally to the segmentation of the different regions
(glacier, ocean, rock outcrop, SAR shadow), precise detection of the calving front position provides
important information for the glacier surveillance [3]. The relatedness of these two problems suggests
the application of multi-task learning (MTL) [4]. MTL with a single input image and multiple output
labels can be divided into late branching and early branching networks [5].

• Late branching: The network is trained with multiple output channels for each task. This approach is widely used, because of its straightforward implementation [6, 7]. Dot et al. already uses the nnU-Net framework for segmentation of craniomaxillofacial structures in CT scans [8].
• Early branching: Two separate Decoders are trained for each task with a common Encoder. This approach was used by Amyar et al. [9] to jointly segment lesion from lung CT scans and identify COVID-19 patients.

The nnU-Net framework will be extended for the application of MTL. Both approaches will be implemented
and evaluated on a dataset of glacier images. The dataset contains SAR images of seven
glaciers, their corresponding segmentation masks, and calving front positions. To emphasize the effect
of MTL on the performance of the nnU-Net, additional tasks like reconstruction of the input image will
be integrated into the training. The resulting models will be compared quantitatively and qualitatively
with the single-task networks and state-of-the-art in calving front detection.

[1] Fabian Isensee et al. “nnU-Net: a self-configuring method for deep learning-based biomedical
image segmentation”. In: Nature Methods 18.2 (Feb. 2021), pp. 203–211.

[2] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for
biomedical image segmentation”. In: International Conference on Medical image computing and
computer-assisted intervention. Springer. 2015, pp. 234–241.

[3] Celia A. Baumhoer et al. “Environmental drivers of circum-Antarctic glacier and ice shelf front
retreat over the last two decades”. In: The Cryosphere 15.5 (May 2021), pp. 2357–2381.

[4] Konrad Heidler et al. “HED-UNet: Combined Segmentation and Edge Detection for Monitoring
the Antarctic Coastline”. In: IEEE Transactions on Geoscience and Remote Sensing (Mar. 2021),
pp. 1–14.

[5] Kelei He et al. “HF-UNet: Learning Hierarchically Inter-Task Relevance in Multi-Task U-Net for
Accurate Prostate Segmentation in CT Images”. In: IEEE transactions on medical imaging 40.8
(Aug. 2021), pp. 2118–2128.

[6] Ava Assadi Abolvardi, Len Hamey, and Kevin Ho-Shon. “UNET-Based Multi-Task Architecture
for Brain Lesion Segmentation”. In: Digital Image Computing: Techniques and Applications
(DICTA). Melbourne, Australia: IEEE, Nov. 2020, pp. 1–7.

[7] Soumyabrata Dev et al. “Multi-label Cloud Segmentation Using a Deep Network”. In: USNCURSI
Radio Science Meeting (Joint with AP-S Symposium). July 2019, pp. 113–114.

[8] Gauthier Dot et al. “Fully automatic segmentation of craniomaxillofacial CT scans for computerassisted
orthognathic surgery planning using the nnU-Net framework”. In: medRxiv (2021).

[9] Amine Amyar et al. “Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia:
Classification and segmentation”. In: Computers in Biology and Medicine 126 (Nov. 2020),
p. 104037.

Virtual Dynamic Contrast Enhanced Image Prediction of Breast MRI using Deep Learning Architectures

Test