Index

Digitization of Handwritten Rey Osterrieth Complex Figure Test Score Sheets

The Rey Osterrieth Complex Figure Test (ROCF) is a neuropsychological test to detect cognitive
impairments.
As the scoring is mostly implemented by hand from experts the goal is to automate the ROCF by
means of machine learning.
The whole project consists of four milestones:
1. State-of-the-art literature research
2. Development of an OCR-based algorithm to digitize the handwritten score sheet into machine
readable structured format for training an automatic algorithm
3. Development of a deep learning algorithm for automatic scoring ROFCs based on the 36-point
scoring system
4. Evaluation of the algorithm based on the data and publication of the results
This thesis will mainly examine the first two steps.
The used scoring sheets consist of an identical structure and just the score itself is handwritten.
Therefore only digits have to be recognized.
The idea is to use networks already trained on the MNIST database (e.g. [1], [2], [3]) and to gain the
best outcome performance for the described issue.
Therefore some preprocessing of the scanned scoring sheets such as detecting areas of interest, binari-
zation or rotation will be necessary to match the requirements for input data of the specific algorithms
as well as for improving performance.
Other options for preprocessing could be template matching or taking advantage of the HU-moments
[4]. Hereby text detection, i.e. finding areas of interests, is one of the typically performed steps in any
text processing pipeline [5].
Furthermore modifying algorithms and weights will be used to achieve different outcomes which than
can be compared in relation to their performances.
The implementation should be done in Python.

References
[1] Gargi Jha. Mnist handwritten digit recognition using neural network, Sep 2020.
[2] Muhammad Ardi. Simple neural network on mnist handwritten digit dataset, Sep 2020.
[3] Dan Claudiu Ciresan, Ueli Meier, Luca Maria Gambardella, and Jürgen Schmidhuber. Deep big simple
neural nets excel on handwritten digit recognition. CoRR, abs/1003.0358, 2010.
[4] Zengshi Chen, Emmanuel Lopez-Neri, Snezana Zekovich, and Milan Tuba. Hu moments based handwritten
digits recognition algorithm. In Recent advances in knowledge engineering and systems science: Proceedings
of the 12TH international conference on artificial intelligence, knowledge engineering and data bases, page
98–104. WSEAS Press, 2013.
[5] Simon Hofmann, Martin Gropp, David Bernecker, Christopher Pollin, Andreas Maier, and Vincent Christlein.
Vesselness for text detection in historical document images. In 2016 IEEE International Conference on
Image Processing (ICIP), pages 3259–3263, 2016.

Glioma Growth Prediction Using Reaction-Diffusion Modelling and Machine Learning

Fully Automated Classification of Anatomical Variants of the Coronary Arteries from Cardiac Computed Tomography Angiography

Radiomics, Delta- and Dose-Radiomics in brain metastases

Writer Identification using Transformer-based Deep Neural Networks

Deep learning-based respiratory navigation for abdominal MRI

In Magnetic Resonance Imaging (MRI) of the abdomen, breathing motion
and cardiac motion are the main confounding factors introducing
artifacts and causing diminished image quality. Different strategies to
minimize the susceptibility to (breathing) motion-related artifacts have
been developed over the last decades, the most routinely used ones being
breath-held acquisitions, retrospective gating, and prospective
triggering. Breath-held techniques are sampling efficient but may not be
applicable in seriously ill patients and pediatric patients. In
addition, MRI techniques such as 3D high-resolution Magnetic Resonance
Cholangiopancreatography (MRCP) require parameter sets making it extremely
difficult to perform the exam in a single breath-hold or multi
breath-hold fashion. Under the assumption that breathing patterns are
stable and regular, triggered acquisition schemes aim to acquire data in
certain states/sectors of the breathing cycle, typically during the
relatively stable end-expiratory phase. This technique is less sampling
efficient and has additional challenges in irregular breathers.
The aim of this thesis is to analyze existing techniques for breathing
trigger point detection and breathing pattern analysis and to explore if
neural networks are suitable to derive optimal trigger points, adapt
triggering schemes to changing patient conditions, and investigate
whether breathing irregularities can be predicted from previous
breathing cycles.

AI-based classification of diffuse liver disease

Cerebral Vessel Tree Estimation from Non-Contrast CT using Deep Learning Methods

Thesis Description

Globally seen, the WHO (World Health Organization) classifies stroke as the second leading cause of death and the third leading cause of disability [1, 2]. In the United States, on average, every 40 seconds someone has a stroke as statistics from the AHA (American Heart Association) demonstrate. 87% of these strokes are ischemic, the rest is of hemorrhagic nature [3].

The primary first-line neuroimaging technique that is applied in case of a suspected stroke is a non-contrast CT (NCCT) scan. Based on this scan one can differentiate between an ischemic and a hemorrhagic stroke [4]. In case of an acute ischemic stroke a reperfusion can be accomplished either by intravenous thrombolytic drug treatment or with endovascular mechanical thrombectomy. Whereas for thrombolysis the short treatment window and the risk of symptomatic intracranial hemorrhage are limitations, endovascular treatment by using stent retrievers is only securely feasible for large proximal vessel occlusions. Given that, thrombectomy is the preferred method for eligible patients [5, 4]. The decision for or against recanalization by thrombectomy requires the localization of the thrombus on artery-level, which is done using further angiographic imaging [6].

In practice computed tomography angiography (CTA) and magnetic resonance angiography (MRA) are the most important modalities for cerebral angiography. Even though both are in principle suitable for the task, long acquisition time and high operational cost are major drawbacks when it comes to practical application of MRA technique [7]. However, for CTA the patient is exposed to X-rays and intravenous contrast. This contrast agent bears the risk of allergic reactions, contrast-induced nephropathy and thyrotoxicosis [8]. While an angiographic scan is still necessary for thrombus localization, it would be beneficial to also gather as much additional information as possible from the previously acquired NCCT scan. Providing a estimation of the cerebral vessel tree, which is the goal of this thesis, could, for instance, be useful when developing methods for the automatic detection of hyperdense artery signs that can indicate a clot. Such insights can be used to improve decision-making in examination and treatment and serve as a verification for CTA or MRA results. Due to the lack of contrast agent, the vasculature is typically barely visible in NCCT scans, which renders the diagnosis of cerebral ischemia a challenging task [9].

Since the rise of deep learning in medical image analysis, its applications for image segmentation have been a prominent field of research. Especially the U-Net architecture that is designed for fast training on small datasets has gathered huge attention [10]. Previous work addressing cerebral angiography segmentation from NCCT was presented by Klimont et al. [11]. Their approach to generate cerebral angiographies from NCCT scans suffers from several limitations: 1) The segmentation algorithm used to generate the target samples from CTA scans is seen as subpar. 2) The use of 3D-U-Net architecture was not possible due to limited computational resources. Therefore the U-Net is only trained on slices of the scan, which results in a lack of context for the axial dimension. 3) A CycleGAN, which is a deep learning method for image-to-image translation, achieved unsatisfactory results for generating realistic CTA scans. They assume that using an adequate loss function will produce better results [11].

To tackle their first limitation, an already developed, enhanced segmentation algorithm on CTA is applied to provide a better ground truth. Furthermore partitioning the data into patches to enable training with volumetric data on 3D-U-Net is targeted in the first step of this thesis [12]. Then a corresponding CTA should be added as an auxiliary target, which is optimized by extending the previous architecture with a discriminator. This should lead to a more realistic cerebral vessel tree segmentation by the U-Net [13]. A similar architecture has already successfully been applied to a task where synthetic non-contrast images have been generated from CTA scans [14]. Time permitting, a further, optional goal is to train the model to predict separate masks for specific brain vessels (instance segmentation). The evaluation is done on a dataset of 150 patients. For each patient there is a NCCT scan, a CTA scan and a segmentation mask, that is generated out of the given CTA.

The thesis will comprise the following work items:

1. Literature research on related work
2. Design, implementation and parametrization of the segmentation model
   1. Cerebral vessel tree segmentation with 3D-U-Net architecture
   2. Addition of CTA as auxiliary target and extension of U-Net with Discriminator
   3. Possibly: Multi-class prediction to predict individual vessel masks
3. Quantitative evaluation of the implemented system on real-world data (150 samples)

References

[1] Global Health Estimates 2019: Deaths by Cause, Age, Sex, by Country and by Region, 2000-2019. Technical report, World Health Organization, Geneva, 2020.

[2] Global Health Estimates 2019: Disease burden by Cause, Age, Sex, by Country and by Region, 2000-2019. Technical report, World Health Organization, Geneva, 2020.

[3] Salim S. Virani, Alvaro Alonso, Emelia J. Benjamin, Marcio S. Bittencourt, Clifton W. Callaway, April P.
Carson, Alanna M. Chamberlain, Alexander R. Chang, Susan Cheng, Francesca N. Delling, Luc Djousse,
Mitchell S.V. Elkind, Jane F. Ferguson, Myriam Fornage, Sadiya S. Khan, Brett M. Kissela, Kristen L.
Knutson, Tak W. Kwan, Daniel T. Lackland, Ten´e T. Lewis, Judith H. Lichtman, Chris T. Longenecker, Matthew Shane Loop, Pamela L. Lutsey, Seth S. Martin, Kunihiro Matsushita, Andrew E. Moran,
Michael E. Mussolino, Amanda Marma Perak, Wayne D. Rosamond, Gregory A. Roth, Uchechukwu K.A.
Sampson, Gary M. Satou, Emily B. Schroeder, Svati H. Shah, Christina M. Shay, Nicole L. Spartano,
Andrew Stokes, David L. Tirschwell, Lisa B. VanWagner, Connie W. Tsao, and null null. Heart Disease and Stroke Statistics—2020 Update: A Report From the American Heart Association. Circulation,
141(9):e139–e596, March 2020.

[4] C. Zerna, Z. Assis, C. D. d’Esterre, B. K. Menon, and M. Goyal. Imaging, Intervention, and Workflow in
Acute Ischemic Stroke: The Calgary Approach. American Journal of Neuroradiology, 37(6):978–984, June
2016.

[5] Salwa El Tawil and Keith W Muir. Thrombolysis and thrombectomy for acute ischaemic stroke. Clinical
Medicine, 17(2):161–165, April 2017.

[6] Murugan Palaniswami and Bernard Yan. Mechanical Thrombectomy Is Now the Gold Standard for Acute Ischemic Stroke: Implications for Routine Clinical Practice. Interventional Neurology, 4(1-2):18–29, 2015.

[7] D. A. Katz, M. P. Marks, S. A. Napel, P. M. Bracci, and S. L. Roberts. Circle of Willis: evaluation with
spiral CT angiography, MR angiography, and conventional angiography. Radiology, 195(2):445–449, May
1995.

[8] James V. Rawson and Allen L. Pelletier. When to Order a Contrast-Enhanced CT. American Family
Physician, 88(5):312–316, September 2013.

[9] Tom van Seeters, Geert Jan Biessels, Joris M. Niesten, Irene C. van der Schaaf, Jan Willem Dankbaar,
Alexander D. Horsch, Willem P. T. M. Mali, L. Jaap Kappelle, Yolanda van der Graaf, Birgitta K. Velthuis,
and on behalf of the Dust Investigators. Reliability of Visual Assessment of Non-Contrast CT, CT Angiography Source Images and CT Perfusion in Patients with Suspected Ischemic Stroke. PLOS ONE,
8(10):e75615, August 2013.

[10] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional Networks for Biomedical
Image Segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi,
editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Lecture Notes in
Computer Science, pages 234–241, Cham, 2015. Springer International Publishing.

[11] Micha l Klimont, Agnieszka Oronowicz-Ja´skowiak, Mateusz Flieger, Jacek Rzeszutek, Robert Juszkat, and Katarzyna Jo´nczyk-Potoczna. Deep learning for cerebral angiography segmentation from non-contrast computed tomography. PLOS ONE, page 15, July 2020.

[12] Ozg¨un C¸ i¸cek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox, and Olaf Ronneberger. 3D U-Net: ¨Learning Dense Volumetric Segmentation from Sparse Annotation. In Sebastien Ourselin, Leo Joskowicz, Mert R. Sabuncu, Gozde Unal, and William Wells, editors, Medical Image Computing and ComputerAssisted Intervention – MICCAI 2016, Lecture Notes in Computer Science, pages 424–432, Cham, 2016. Springer International Publishing.

[13] Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron
Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11):139–
144, October 2020.

[14] Florian Thamm, Oliver Taubmann, Felix Denzinger, Markus J¨urgens, Hendrik Ditt, and Andreas Maier.
SyNCCT: Synthetic Non-Contrast Images of the Brain from Single-Energy Computed Tomography Angiography. volume 12907 of Lecture Notes in Computer Science. Springer, Cham, September 2021.

Entwicklung von Prozessabläufen für die Forschungszusammenarbeit in datengetriebenen und institutionsübergreifenden Forschungsprojekten

Der vorliegende Artikel adressiert die Entwicklung von Prozessabläufen in institutionsübergreifenden und datengetriebenen Forschungsprojekten. Hierbei wird die Frage behandelt, ob eine Standardisierung der Prozessketten möglich ist, inwiefern Governance-Strukturen der Medizininformatikinitiative (MII) für datengetriebene Forschungsvorhaben inkludiert werden können und abschließend, ob somit die Handlungssicherheit für Beteiligte erhöht werden kann. Hierfür wurden Ist-Abläufe innerhalb durchgeführter Kooperationen mit empfohlenen Standards der MII abgeglichen und mithilfe des Knowhows beteiligter Mitarbeiter in Prozessketten überführt. Es konnten so Prozessabläufe entwickelt werden, die durch kaskadierende Prozessketten, Erläuterungen und Checklisten eine standardisierte Handreichung für Kooperationsprojekte bilden. Ebenfalls können durch die Dokumente zukünftig Fehler innerhalb der einzelnen Prozesselemente vermieden werden und Kooperationsprojekte einfacher, zielorientierte und übersichtlicher durchgeführt werden.

Detection of Pulmonary Embolisms in NCCT Data using Deep Learning Methods

Thesis Description

Pulmonary embolism (PE) is the third most common case of cardiovascular disease, after myocardial infection and stroke [1]. According to the report of the American Heart Association in 2020, there were 370,000 cases of PE in the US in 2016 [2]. PE most commonly originate from a deep venous thrombosis in the legs, which breaks free and migrates towards the heart and lodges in the pulmonary arteries [3]. The 30-day mortality rate for PE in the US was 9.1% in 2020 [2]. Hereby, the majority of cases of preventable deaths occur due to missed diagnosis, but not due to failure of therapy [4]. Therefore, a rapid and correct diagnosis can reduce the fatality rate and is crucial for the patient’s prognosis [5, 6].

In the context of diagnostic tools for PE, computed tomographic pulmonary angiography (CTPA) has become the gold standard imaging test, as the pulmonary filling defect can be visualized with the help of contrast agent [5]. However, CTPA does not provide functional information on lung perfusion and clots in more distal branches of the lung arteries are hard to detect [7]. A rather new method, dual energy CT (DECT), counteracts these problems, as it is possible to visualize the iodine distribution in the lung parenchyma with the help of iodine maps for functional assessment of lung perfusion [8]. The use of a dual-source CT scanner allows for simultaneous image acquisition at two different energy enabling a decomposition into iodine and non-iodine, which is not possible in plain CTPA [7, 9]. The decomposition helps to understand hemodynamics in the parenchyma of interest, here the pulmonary vascular tree. However, there are drawbacks of DECT scans for PE diagnosis. A common reason for misdiagnosis of PE is the occurrence of heterogeneous perfusion due to beam-hardening artifacts from high concentration of contrast agent [7, 9]. Another problem is the accessibility of special scanners with two X-ray sources, that can execute the DECT.

In general, the performance of a CT scan with contrast agent is a time-consuming and cumbersome procedure that might require special scanners, which may not be available in every part of the world. Additionally, patients are asked to hold their breath during scan time, which can be infeasible considering children or patients that are not fully conscious [10, 11]. A non-contrast CT scan (NCCT) can be performed within seconds and is a less difficult procedure. The detection of PE in a NCCT scan is difficult but possible. Several studies showed that especially central PE could be identified in unenhanced CT scans [12, 13]. Yet, the evaluation of NCCT scans by radiologists is neither sensitive nor specific enough to reliably detect PE [14].

To guide and speed up the physicians diagnosis of PE, machine learning algorithms have been used to automatically detect PE in current research [15, 16]. These methods were developed with the intent to identify clinically important PE and prioritize worklists, considering the increasing number of e.g. CTPA scans [15]. Weikert et al. evaluated the performance of a two-staged AI-powered algorithm detecting PE from CTPA scans [15]. The algorithm includes a 3D deep convolutional neural network based on a ResNet architecture. Evaluation of the algorithm with the institution’s test set (n=1499) resulted in a sensitivity of 92.7% and specificity of 95.5% for detecting PE. Another model, PENet, was developed by Hunag et al., a 77-layer 3D convolutional neural network (CNN), evaluated on data from 2 different institutions [16]. Detecting PE, it achieved an AUROC of 0.84 on the internal test set and 0.85 on the external dataset, using the entire volumetric CTPA imaging data.

Under the aspect of availability and feasibility of contrast-enhanced CT scans, the automatic detection of PE using a NCCT scan is desirable. Additionally, modern deep learning methods could possibly exceed human performance in the detection of PE in unenhanced CT scans. Other work has already proven the success of AIbased detection of PE in CTPA data [17], whereas it is now to be investigated if the automated detection based on unenhanced CT scans is possible. Therefore, the main goal of this work is to detect PE in an NCCT scan. This thesis explores a recently published method for medical object detection, nnDetection, which is supposed to adapt itself to arbitrary detection problems [18]. Furthermore, depending on its performance on the given task, this method is extended, or other deep learning architectures are exploited alongside nnDetection. These consider the topology of the lung tree, in order to tackle the goal of PE detection in NCCT images. In addition, this work includes a detailed analysis of the models’ performances accomplishing the task of PE detection. In general, the thesis covers the following topics:

1. Systematic literature research
2. Detection of PE with Deep Learning
   1. nnDetection as baseline
   2. Extension of the baseline, by further improvements using prior knowledge of data and/or
   3. Development of other suitable Deep Learning methods
3. Analysis of the Deep Learning model
   1. Evaluation of the results
   2. Comparison with baseline model

References

[1] Meredith Turetz, Andrew T. Sideris, Oren A. Friedman, Nidhi Triphathi, and James M. Horowitz. Epidemiology, Pathophysiology, and Natural History of Pulmonary Embolism. Seminars in Interventional
Radiology, 35(2):92–98, 2018.

[2] Salim S. Virani, Alvaro Alonso, Emelia J. Benjamin, Marcio S. Bittencourt, Clifton W. Callaway, April P.  Carson, Alanna M. Chamberlain, Alexander R. Chang, Susan Cheng, Francesca N. Delling, Luc Djousse,
and et al. Heart disease and stroke statistics – 2020 update: A report from the american heart association. Circulation, 141(9):e139–e596, 2020.

[3] S. Takach Lapner and C. Kearon. Diagnosis and management of pulmonary embolism. BMJ (Online), 346(7896):1–9, 2013.

[4] Peter F. Fedullo and Victor F. Tapson. The Evaluation of Suspected Pulmonary Embolism. New England Journal of Medicine, 349(13):1247–1256, 2003.

[5] Waleed Abdellatif, Mahmoud Ahmed Ebada, Souad Alkanj, Ahmed Negida, Nicolas Murray, Faisal Khosa, and Savvas Nicolaou. Diagnostic Accuracy of Dual-Energy CT in Detection of Acute Pulmonary Embolism: A Systematic Review and Meta-Analysis. Canadian Association of Radiologists Journal, 72(2):285–292, 2021.

[6] Cecilia Becattini, Maria Cristina Vedovati, and Giancarlo Agnelli. Diagnosis and prognosis of acute pulmonary embolism: Focus on serum troponins. Expert Review of Molecular Diagnostics, 8(3):339–349,
2008.

[7] Long Jiang Zhang, Chang Sheng Zhou, U. Joseph Schoepf, Hui Xue Sheng, Sheng Yong Wu, Aleksander W. Krazinski, Justin R. Silverman, Felix G. Meinel, Yan E. Zhao, Zong Jun Zhang, and Guang Ming Lu. Dualenergy CT lung ventilation/perfusion imaging for diagnosing pulmonary embolism. European Radiology, 23(10):2666–2675, 2013.

[8] Dong Jin Im, Jin Hur, Kyung Hwa Han, Hye Jeong Lee, Young Jin Kim, Woocheol Kwon, and Byoung Wook Choi. Acute pulmonary embolism: Retrospective cohort study of the predictive value of perfusion defect volume measured with dual-energy CT. American Journal of Roentgenology, 209(5):1015–1022, 2017.

[9] Guang Ming Lu, S. Y. Wu, B. M. Yeh, and L. J. Zhang. Dual-energy computed tomography in pulmonary embolism. British Journal of Radiology, 83(992):707–718, 2010.

[10] Marc Rodger and Philip S. Wells. Diagnosis of pulmonary embolism. Thrombosis Research, 103(6):225–238, 2001.

[11] Konstantin Nikolaou, Sven Thieme, Wieland Sommer, Thorsten Johnson, and Maximilian F. Reiser. Diagnosing pulmonary embolism: New computed tomography applications. Journal of Thoracic Imaging,
25(2):151–160, 2010.

[12] Christopher Thom and Nathan Lewis. Never say never: Identification of acute pulmonary embolism on non-contrast computed tomography imaging. American Journal of Emergency Medicine, 35(10):1584.e1– 1584.e3, 2017.

[13] Rocco Cobelli, Maurizio Zompatori, Giovanni De Luca, Gianfranco Chiari, Paolo Bresciani, and Carla Marcato. Clinical usefulness of computed tomography study without contrast injection in the evaluation
of acute pulmonary embolism. Journal of Computer Assisted Tomography, 29(1):6–12, 2005.

[14] Simon Sun, Alexandre Semionov, Xuanqian Xie, John Kosiuk, and Benoˆıt Mesurolle. Detection of central pulmonary embolism on non-contrast computed tomography: A case control study. International Journal of Cardiovascular Imaging, 30(3):639–646, 2014.

[15] Thomas Weikert, David J. Winkel, Jens Bremerich, Bram Stieltjes, Victor Parmar, Alexander W. Sauter, and Gregor Sommer. Automated detection of pulmonary embolism in CT pulmonary angiograms using an AI-powered algorithm. European Radiology, 30(12):6545–6553, 2020.

[16] Shih Cheng Huang, Tanay Kothari, Imon Banerjee, Chris Chute, Robyn L. Ball, Norah Borus, Andrew Huang, Bhavik N. Patel, Pranav Rajpurkar, Jeremy Irvin, Jared Dunnmon, Joseph Bledsoe, Katie Shpanskaya, Abhay Dhaliwal, Roham Zamanian, Andrew Y. Ng, and Matthew P. Lungren. PENet—a scalable deep-learning model for automated diagnosis of pulmonary embolism using volumetric CT imaging. npj Digital Medicine, 3(1):1–9, 2020.

[17] Shelly Soffer, Eyal Klang, Orit Shimon, Yiftach Barash, Noa Cahan, Hayit Greenspana, and Eli Konen. Deep learning for pulmonary embolism detection on computed tomography pulmonary angiogram: a systematic review and meta-analysis. Scientific Reports, 11(1):1–8, 2021.

[18] Michael Baumgartner, Paul F. J¨ager, Fabian Isensee, and Klaus H. Maier-Hein. nnDetection: A Selfconfiguring Method for Medical Object Detection. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12905 LNCS:530–539, 2021.