Index

Cone-Beam CT X-Ray Image Simulation for the Generation of Training Data

Contextual Meta Knowledge integrated into a Fully-Automated Multi-Stage Deep Learning Framework for Killer Whale Individual Classification

1. Problem
Killer whales (also known as Orcas) are the topmost predators of marine life, and the availability of right information is crucial in studying the population. Photo-identification, a research method, is used to identify individual orca based on the photographs of their natural markings [4]. But the identification task is extremely laborious and time consuming. Additionally, it requires high expertise in the field. As opposed to this, a numerous computer-based algorithm is created to simplify the task of detection and classification of the Orca population [2]. However, any computer-based algorithm experiences a struggle to perform in a situation caused by various reasons including, low resolution, poor luminosity, orientation of Orca, dynamic backgrounds, bad weather condition. Hence, the scope for improvement cannot be ignored and a better generalized solution addressing these challenges needs to be presented.
2. Motivation
With the advancement in machine learning techniques, Marine Researchers are accurately predicting and consolidating the Killer whale data with availability of the limited information [1].
Meta information related to Orca individuals such as age, gender, shape of the dorsal fin in combination with saddle patch, family relationship to one another, movement pattern etc., plays a significant role in studying Orca individuals. This information can be employed when the task of detection and classification seem challenging.
The statistical approach to Bayesian theory has found its wide range of applications in the field of machine learning [3]. The idea is to use Bayesian approach to improve accuracy of the existing framework.
3. Proposition
The objective of this Master thesis is to obtain ultimate result in terms of accuracy in Orca detection and classification. This can be achieved by building a statistical correlation between Orca occurrences and their behavioural pattern with one another using the available information (also known as ‘meta’).
With meta-information such as image labels and day, time, and location of the capture a matrix can be built which contains relative information of Orca to one another. This matrix can further be utilized to improve recognition and classification accuracy.
Since the algorithm is data-driven and the matrix is built from input data, it is important to provide unbiased evaluation and not over or underfit. For this purpose, 2011 to 2017, except 2018, data is utilized. Furthermore, the data has been split into training, testing and validation bit out of which test split is excluded.
References:
1. Ahsan Jalala, Ahmad Salmana, Ajmal Mian, bMark Shortisc, Faisal Shafait., “Fish detection and species classification in underwater environments using deep learning with temporal information”, DOI: 10.1016/j.ecoinf.2020.101088
2. Alexander Gebhard, 2020, “Orca Individual Identification based on Image Classification Using Deep Learning”
3. Kersten D, Mamassian P, Yuille A. Object perception as Bayesian inference. Annu Rev Psychol. 2004; 55:271-304. doi: 10.1146/annurev.psych.55.090902.142005. PMID: 14744217.
4. J. Towers, G. Sutton, T. Shaw, M. Malleson, D. Matkin, B. Gisborne, J. Forde, D. Ellifrit, G. Ellis, J. Ford, et al. Photo-identification catalogue, population status, and distribution of bigg’s killer whales known from coastal waters of british columbia, canada. Fisheries and Oceans Canada, Pacific Biological Station, Nanaimo, BC, 2019.

Cardiac Functional Analysis – Automated Strain Analysis of the left Ventricle using Computed Tomography

The heart is one of the most important organs within the human body. It maintains the blood circulation
which is crucial to transport substances through the body to e.g. oxygenate the brain or muscles.
Dysfunctions can lead to death. Cardiovascular and circulatory diseases are one of the most common
causes of mortality [1]. To indicate such a disease a cardiac functional analysis is often performed. A
cardiac functional analysis assesses the vitality of the heart based on objective criteria such as ejection
fraction and motion evaluation. This analysis helps to identify, localize and treat dysfunctions. Especially
tracing the dynamics of the heart, referred to as myocardial strain analysis, gives diagnostic
hints. The movement of the heart and each chamber can be tracked using cardiac imaging. There are
three possible directions for the motion: radial, longitudinal and circumferential, which can be seen
as intrinsic dynamics. For instance in case of acute myocarditis the intrinsic dynamics differ significantly
from those of healthy patients. As the left ventricle is the largest of the four heart chambers –
dysfunctions might have the highest impact on its function and therefore on its dynamics [2, 3].
To perform a cardiac functional analysis or to be precise a strain analysis of the left ventricle noninvasive
imaging modalities are used. Cardiac magnetic resonance and echocardiography are commonly
used and build the state of the art methods [4, 5]. Up to now computed tomography is barely used for
assessing the cardiac function but more for identifying diseases as coronary atherosclerosis [6]. The
myocardium tends to have a low contrast in computed tomography images of the heart. Therefore
tracking of endo- and epicardium becomes more complicated. Due to this, the complexity of performing
a cardiac functional analysis increases. But aspects like high resolution, time- and cost efficiency
motivate to overcome these difficulties.
Within this project it will be investigate whether computed tomography is capable to perform a
cardiac functional analysis. The goal is to automate the strain analysis of the left ventricle using
computed tomography.
Given several 3D CT images of a heartbeat cycle the strain analysis will be performed in the following
way:
1. The myocardium of the left ventricle has to be localized and segmented using an active shape
model.
2. For the motion tracking different registration algorithms will be evaluated. The range of approaches
is broad. Possible methods for landmark based registration would be thin-plate splines [7]. As the
complexity increases with the dimension 2D intensity based registration could be performed on sliced
images [8]. Using the whole 3D image there are more classic algorithms as well as deep learning based
approaches are conceivable [9, 10].
3. The intrinsic dynamics, namely longitudinal, radial and circumferential strain, are calculated by
projecting the transformation vectors provided by the registration algorithm onto the main axes of an
intrinsic coordinate system.
4. The results will be visualized as color coded strain magnitude within the CT image or in a polar
map [3].
5. The results of different registration algorithms are qualitatively and quantitatively compared using
the visualizations as well as different metrics, possibly such as the correlation coefficient [11] and the
Hausdorff distance [12].

References
[1] Gregory A Roth et al. Global burden of cardiovascular diseases and risk factors, 1990–2019:
update from the gbd 2019 study. Journal of the American College of Cardiology, 76(25):2982–
3021, 2020.
[2] Otto A Smiseth, Hans Torp, Anders Opdahl, Kristina H Haugaa, and Stig Urheim. Myocardial
strain imaging: how useful is it in clinical decision making? European Heart Journal, 37(15):1196–
1207, 2016.
[3] Aldostefano Porcari et al. Strain analysis reveals subtle systolic dysfunction in confirmed and
suspected myocarditis with normal lvef. a cardiac magnetic resonance study. Clinical Research
in Cardiology, 109(7):869–880, 2020.
[4] Dagmar F Hernandez-Suarez and Angel L´opez-Candales. Strain imaging echocardiography: what
imaging cardiologists should know. Current Cardiology Reviews, 13(2):118–129, 2017.
[5] Alessandra Scatteia, Anna Baritussio, and Chiara Bucciarelli-Ducci. Strain imaging using cardiac
magnetic resonance. Heart Failure Reviews, 22(4):465–476, 2017.
[6] Marc R Dweck, Michelle C Williams, Alastair J Moss, David E Newby, and Zahi A Fayad.
Computed tomography and cardiac magnetic resonance in ischemic heart disease. Journal of the
American College of Cardiology, 68(20):2201–2216, 2016.
[7] Rainer Sprengel, Karl Rohr, and H Siegfried Stiehl. Thin-plate spline approximation for image
registration. In Proceedings of 18th annual international conference of the IEEE Engineering in
Medicine and Biology Society, volume 3, pages 1190–1191. IEEE, 1996.
[8] Christoph Guetter, Hui Xue, Christophe Chefd’Hotel, and Jens Guehring. Efficient symmetric
and inverse-consistent deformable registration through interleaved optimization. In 2011 IEEE
international symposium on biomedical imaging: from nano to macro, pages 590–593. IEEE, 2011.
[9] Huajun Song and Peihua Qiu. Intensity-based 3d local image registration. Pattern Recognition
Letters, 94:15–21, 2017.
[10] Guha Balakrishnan, Amy Zhao, Mert R Sabuncu, John Guttag, and Adrian V Dalca. Voxelmorph:
a learning framework for deformable medical image registration. IEEE Transactions on Medical
Imaging, 38(8):1788–1800, 2019.
[11] Richard Taylor. Interpretation of the correlation coefficient: a basic review. Journal of Diagnostic
Medical Sonography, 6(1):35–39, 1990.
[12] D.P. Huttenlocher, G.A. Klanderman, and W.J. Rucklidge. Comparing images using the hausdorff
distance. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(9):850–863, 1993.

Estimation of Forming Limit Curves Pattern using Deep Learning Methods

Thrombus detection using nnDetection

Efficient Methods for Post Myocardial Infarction Ventricular Tachycardia Modeling: from Image Processing to Electrophysiological Simulation

Cardiovascular diseases are the leading cause of death worldwide with an estimate of 17.9 million in
2019, as reported by theWorld Health Organization (WHO) [1]. Among the numerous diseases included
in this category, ischemic heart disease is one of the most frequent, with an estimate of 8.8 million deaths
in 2019 [1]. A common result of ischemic heart disease is myocardial infarction (MI), which occurs when
the blood flow to an area of the heart is blocked. This damages the heart tissue resulting in necrosis
[2]. Approximately 10% of the patients that survive a previous MI event have an increased risk of
death in the following months or years after hospital discharge. In these cases, up to 50% of the deaths
can be secondary to a sustained Ventricular Tachycardia (VT) or Ventricular Fibrillation (VF) event [2].
Ventricular Tachycardia is a heart arrhythmia that occurs when a fast and abnormal heart rate originates
in the ventricles. A well-known mechanism for VT is an action potential wave re-entry caused by
a unidirectional conduction block in slow conductive areas of the myocardium [2]. These areas contain
a complex mixture of scar (i.e. infarcted tissue) and surviving myocytes that is often referred to as
heterogeneous or border zone tissue [3]. For these cases, ablation therapy is the preferred surgical approach.
Approximately 50% of the patients that undergo ablation therapy show VT recurrence before
five years after surgery [4]. In order to improve the outcome, a critical part of the procedure is to
properly localize and ablate the arrhythmic substrate [4].
In this context, it is hypothesized that a combination of scar lesion imaging, cardiac electrophysiology
modeling, and artificial intelligence, can improve the localization of VT ablation targets, the detection
of incomplete ablation procedures, and the selection of the ablation strategy. Previous studies have
already investigated the usage of digital twin technologies for VT therapy planning [3, 5, 6]. However,
the VT modeling pipeline is extensive and still presents several challenges.
First, the manual segmentation of the heart chambers from Late Gadolinium Enhanced (LGE) MRI
images is a tedious procedure that is prone to inter- and intra-operator variability [7]. The segmentation
of the heart chambers is a necessary step for the generation of anatomical 3D models. Furthermore,
it allows to analyze, locate, and quantify myocardial scar, which can be used to guide the ablation
procedure [8]. Lastly, simulating VT in-silico is very dependant on the selected model and simulation
parameters. Previous studies have addressed how different parameter combinations affect the
inducibility of VT re-entrant activity [5, 6]. These studies usually rely on finite element method (FEM)
simulations on very detailed geometries, which can require up to several hours of run-time per simulation.
This inevitably constrains the number of parameter combinations that can be studied.
With these challenges in mind, the main contributions of this study will be:
• Literature review of the state-of-the-art methods in Late Gadolinium Enhanced (LGE) image
processing and segmentation.
• Literature review of the state-of-the-art methods for electrophysiology modeling and simulation
of virtual VT inducibility.
• Evaluation of a deep learning method for automatic myocardium segmentation from LGE images,
with a possible extension to automatically locate and quantify scar tissue. This automatic
segmentation method will focus on the potential advantages compared to manual segmentation
approaches in terms of reproducibility and time savings [7].
• Study of virtual VT inducibility in a set of porcine models after MI with focus on optimal selection
of model parameters. This task will be carried out using the Lattice-Boltzmann method, a monodomain
solver which allows to perform electrophysiology simulations of VT re-entrant activity,
with the advantage of being faster than other FEM approaches [9].
References
[1] World Health Statistics 2021: Monitoring Health for the SDGs: Sustainable Development Goals. Geneva, Switzerland:
World Health Organization, 2021. Licence: CC BY-NC-SA 3.0 IGO.
[2] J. Bhar-Amato, W. Davies, and S. Agarwal, “Ventricular arrhythmia after acute myocardial infarction: ”the perfect
storm”,” Arrhythmia & Electrophysiology Review, vol. 6, no. 3, p. 134, 2017.
[3] H. Ashikaga, H. Arevalo, F. Vadakkumpadan, R. C. Blake, J. D. Bayer, S. Nazarian, M. Muz Zviman, H. Tandri,
R. D. Berger, H. Calkins, D. A. Herzka, N. A. Trayanova, and H. R. Halperin, “Feasibility of image-based simulation
to estimate ablation target in human ventricular arrhythmia,” Heart Rhythm, vol. 10, pp. 1109–1116, Aug. 2013.
[4] M. Wolf, F. Sacher, H. Cochet, and T. Kitamura, “Long-term outcome of substrate modification in ablation of
postˆamyocardial infarction ventricular tachycardia,” Circulation: Arrhythmia and Electrophysiology, vol. 11, Feb.
2018.
[5] F. O. Campos, J. Whitaker, R. Neji, S. Roujol, M. OˆaNeill, G. Plank, and M. J. Bishop, “Factors promoting conduction
slowing as substrates for block and reentry in infarcted hearts,” Biophysical Journal, vol. 117, pp. 2361–2374, Dec.
2019.
[6] A. Lopez-Perez, R. Sebastian, M. Izquierdo, R. Ruiz, M. Bishop, and J. M. Ferrero, “Personalized cardiac computational
models: From clinical data to simulation of infarct-related ventricular tachycardia,” Frontiers in Physiology,
vol. 10, p. 580, May 2019.
[7] Y. Wu, Z. Tang, B. Li, D. Firmin, and G. Yang, “Recent advances in fibrosis and scar segmentation from cardiac mri:
A state-of-the-art review and future perspectives,” Frontiers in Physiology, vol. 12, p. 709230, Aug. 2021.
[8] S. Toupin, T. Pezel, A. Bustin, and H. Cochet, “Whole-heart high-resolution late gadolinium enhancement: Techniques
and clinical applications,” Journal of Magnetic Resonance Imaging, p. jmri.27732, June 2021.
[9] S. Rapaka, T. Mansi, B. Georgescu, M. Pop, G. A.Wright, A. Kamen, and D. Comaniciu, “Lbm-ep: Lattice-boltzmann
method for fast cardiac electrophysiology simulation from 3d images,” in Medical Image Computing and Computer-
Assisted Intervention – MICCAI 2012, vol. 7511, pp. 33–40, Berlin, Heidelberg: Springer Berlin Heidelberg, 2012.

Fruit Terminator – Annotation of Lung Fluid Cells via Gamification

Deep learning method for emotion recognition through the Fusion of body and context features

Emotion recognition currently has a wide range of applications in various fields. For example, doctors can use emotion recognition to take better care of patients, or teachers can use emotion recognition to judge the concentration level of students, etc. Previous visual emotion recognition researches have mainly focused on facial expression. Still, it has not achieved good results in unconstrained scenarios. This thesis presents a network for emotion recognition based on posture, body features, and context information; the aim is to improve emotion recognition accuracy in different unconstrained scenarios.

The network has been designed in a three-branch architecture, including three feature streams: body, skeleton and context streams. The extraction of human key points from images explored three different networks: Openpose[1], Alphapose[2] and Mediapipe[3]. This thesis uses GCN to extract features from key points. Body and context feature extraction using Transfomer Vision and Resnet50. The three streams are fused to predict dimensional emotion representation, valence, arousal, and dominance. The experiment is trained on public datasets (EMOTIC datasets[4]), and the verification of the model is on the CAER-S datasets[5]. Experimental results show that the proposed method effectively integrates emotional information expressed by body and context and has good generalization ability and applicability. Considering body poses to recognize people’s emotions provides a new benchmark for emotion recognition in visual poses.

References:

[1] Cao, Zhe, et al. “Realtime multi-person 2d pose estimation using part affinity fields.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.

[2]  Fang, Hao-Shu, et al. “Rmpe: Regional multi-person pose estimation.” Proceedings of the IEEE international conference on computer vision. 2017.

[3] Lugaresi, Camillo, et al. “Mediapipe: A framework for building perception pipelines.” arXiv preprint arXiv:1906.08172 (2019).

• Kosti, Ronak, et al. “Context based emotion recognition using emotic dataset.” IEEE transactions on pattern analysis and machine intelligence11 (2019): 2755-2766.

[5]  Lee, Jiyoung, et al. “Context-aware emotion recognition networks.” Proceedings of the IEEE/CVF International Conference on Computer Vision. 2019.

Einfluss der Feldstärke (B0) auf den Intravoxel Incoherent Motion (IVIM) Effekt in der Diffusions-MRT der Wade

Die Struktur und Funktion der Muskulatur im menschlichen Körper kann heutzutage
anhand verschiedener nichtinvasiver Magnetresonanztomographie-Methoden (MRT)
untersucht werden. [1]
Seit der erstmaligen Erwähnung durch Denis Le Bihan im Jahre 1986 [2], hat die
Intravoxel Incoherent Motion (IVIM) MRT hierbei im Laufe der Jahre zunehmend an
Bedeutung gewonnen. Sie ermöglicht eine gleichzeitige Beurteilung von Komponenten
des Blutflusses und der Gewebediffusion. [3]
IVIM wird hauptsächlich zur Untersuchung von Geweben mit hoher Gefäßdichte
eingesetzt, beispielsweise bei Organen wie der Leber, der Niere und dem Gehirn.
Hierbei kann der Blutfluss Rückschlüsse auf die Organgesundheit und auf den
Zusammenhang zwischen metabolischen und vaskulären Reaktionen auf Stimuli
zulassen. [4]
In der Skelettmuskulatur ist der Blutfluss abhängig vom Aktivitätszustand des Muskels
und damit für die Erfassung und Interpretation von IVIM-Daten von besonderem
Interesse. [5]
IVIM ist eine fortgeschrittene Technik der diffusionsgewichteten
Magnetresonanztomographie (engl. diffusion weighted imaging, DWI).
Die DWI misst die Diffusionsbewegung im Gewebe, welche gemäß Brown‘scher
Molekularbewegung als Zufallsbewegung von Wassermolekülen beschrieben werden
kann. Im Gegensatz zu den meisten anderen MRT-Modalitäten handelt es sich bei der
DWI um eine quantitative Methode. [6]
Da die Diffusionsbewegung im Gewebe abhängig von Zellrestriktionen ist, ermöglichen
Diffusionsmessungen Aussagen über den Aufbau des Gewebes. Ein wichtiger
Parameter der DWI ist der hierbei zeitabhängige Diffusionskoeffizient D(t). [7]
Im diffusionsgewichteten Signal spiegelt sich die inkohärente Blutbewegung im
Kapillarbett ähnlich wider wie die Diffusion selbst. Somit kann dem kapillaren Blutfluss
ein Pseudodiffusionskoeffizient D* zugeordnet werden, welcher etwa einen Faktor 10
größer ist als der Diffusionskoeffizient D(t). [8]
Daher bietet IVIM sowohl einen Einblick in die Bewegung in intravaskulären Räumen
als gleichzeitig auch in extravaskulären Räumen.
Ziel dieser Masterarbeit ist es, den Einfluss der magnetischen Feldstärke auf den IVIM
Effekt an der Wade zu untersuchen. Hierzu werden bis zu 10 Probanden und
Probandinnen rekrutiert. Die Datenaufnahme findet hierbei an einem 7T MRT Scanner
(MAGNETOM Terra) und an einem 0,55T MRT Scanner (MAGNETOM Free.Max)
statt. Anschließend sollen die generierten Daten hinsichtlich der IVIM Parameter
miteinander verglichen werden.
[1] Englund, E.K., Reiter, D.A., Shahidi, B. and Sigmund, E.E. (2021), Intravoxel
Incoherent Motion Magnetic Resonance Imaging in Skeletal Muscle: Review and
Future Directions. J Magn Reson Imaging.
https://doi.org/10.1002/jmri.27875
[2] Le Bihan, Denis , Breton, Eric , Lallemand, Denis , Aubin, ML , Vignaud, Jean ,
Laval-Jeantet, M: Separation of diffusion and perfusion in intravoxel incoherent
motion MR imaging. In: Radiology 168 (1988), Nr. 2, S. 497–505
[3] Le Bihan, Denis: What can we see in IVIM MRI?. In: NeuroImage, Volume 187
(2019), Pages 56-67
[4] Denis Le Bihan et al.: Intravoxel Incoherent Motion (IVIM) MRI: Principles and
Applications. 1. Aufl. Pan Stanford Publishing, 2019. isbn: 978-981-4800-19-8,
S.117-145
[5] Jungmann PM, Pfirrmann C, Federau C. Characterization of lower limb muscle
activation patterns during walking and running with Intravoxel Incoherent Motion
(IVIM) MR perfusion imaging. Magn Reson Imaging. 2019 Nov;63:12-20. doi:
10.1016/j.mri.2019.07.016. Epub 2019 Jul 26. PMID: 31356861.
[6] Stieltjes et al.: Diffusion Tensor Imaging. 1. Aufl. Springer Verlag, 2013. isbn: 978-
3-642-20456-2.
[7] Laun, F., Fritzsche, K., Kuder, T. et al. Einführung in die Grundlagen und
Techniken der Diffusionsbildgebung. Radiologe 51, 170–179 (2011).
https://doi.org/10.1007/s00117-010-2057-y
[8] Le Bihan, Denis , Breton, Eric , Lallemand, Denis , Aubin, ML ; Vignaud, Jean ,
Laval-Jeantet, M: Separation of diffusion and perfusion in intravoxel incoherent
motion MR imaging. In: Radiology 168 (1988), Nr. 2, S. 497–505

Letter Inpainting and Detection of Mathematical Diagrams in Multi-Lingual Manuscripts using a Deep Neural Network approach