Index

Deep Learning based Model Observers for Multi-modal Imaging

Task based measurements are needed to measure the quality of medical images. Common task based measures include the use of model observers. Model observers are used to measure the confidence that an object (eg. A tumor or another structure) is present in a particular image.  Common model observers in medical imaging include the Chanellised Hotelling Observer for CT image quality, or the Non Pre Whitening Filter for SPECT image quality.

Current implementations of model observers for task based measurements are executed on phantoms. The use of phantoms makes the use of task based measurements an SKE/BKE task. However, this means that the use of task based measurements cannot be directly moved into a clinical task without prior knowledge. Moreover, multiple noise realisations of a single phantom are needed to get meaningful results from a model observer.

Deep learning has been used to replicate the behaviour of model observers. However, there is no work done on a general model observer which can work across imaging modalities. In this project, we would be interested in investigating the following:

1. The possibility of using deep learning to create a ‘general’ model observer
2. Cross modality performance of a deep learned model observer
3. Training this model observer with zero-, one-, or few- shot learning for greater future generalisation.

We would look for someone who could support us with the following skills:

1. Knowledge in Python/C++ programming
2. Some knowledge on image processing. Medical image processing and DICOM standards are a plus.
3. Knowledge of relevant libraries like NumPy, OpenCV, PyTorch, TensorFlow
4. Experience with model observers is a plus (not strictly necessary)

Augmentation of CT Images by Variation of Non-Rigid Deformation Vector Field Amplitudes

End-to-End Gaze Estimation Network for Driver Monitoring

Automated analysis of Parkinson’s Disease on the basis of evaluation of handwriting

In this thesis current state-of-the-art methods of automatic analysis of Parkinson’s disease (PD) are tested along with new ideas of signal processing. Since there is currently no cure for PD, it is important to introduce methods for automatic monitoring and analysis. Therefore handwriting-samples of 49 healthy subjects and 75 PD patients acquired with a graphic tablet are used. Those subjects performed different drawing tasks. With a kinematic analysis
accuracies of up 77% are achieved when using one task alone and accuracies up to 86% are achieved when combining different tasks. A newly developed spectral analysis resulted in scores of up to 96% for an individual task. Combining the spectral features of a standalone task with features from different tasks or a different analysis did not lead to better results. Making predictions about the severity of the disease based on the features acquired for the bi-class problem failed. An attempt was made modeling the velocity profile of strokes with lognormal distributions and using the thereby obtained parameters for classification. Because of difficulties with the modeling of strokes with different lengths, a classification failed.

Modelling of Speech Aspects in Parkinson’s Disease by Multitask Deep Learning

Parkinson’s disease is a progressive neurodegenerative disorder with a variety of motor and non-
motor symptoms. Although also other factors are influenced by the disease, the current evaluation
process relies mostly on motor aspects and is often subjective. While speech deficits can be
found in a majority of patients, its analysis is still underrepresented in the clinical assessment. To
increase objectivity and enable long-term monitoring of the patient’s status, several computational
methods have been been proposed in the literature. Along with the success of deep learning,
multitask techniques received more and more attention in recent years. Hence, this Master’s thesis
proposes the use of a multitask neural network-based approach in order to assess multiple aspects
of Parkinsonian speech. The data set included various recordings in numerous sessions obtained
from 94 Parkinson patients and 87 healthy controls. A defined set of statistical features was
extracted for each utterance to be used as input to the model. The multitask setting was defined
with three tasks regarding the distinction between diseased and healthy, as well as, two common
Parkinson rating scales, namely the Movement Disorder Society – Unified Parkinson’s Disease
Rating Scale and the modified Frenchay Dysarthria Assessment. These tasks were supposed to
be optimized together compared to individual networks. In order to get a deeper understanding
with regard to the influence of each task and the specific recording settings, several experiments
with different focuses were conducted. Additionally, the multitask setting was expanded with four
additional tasks to exploit the variability of this method. The experimental results demonstrate the
classification capabilities with accuracy values of 81.73%, 52.45% and 43.56% for the respective
three tasks based on a per session evaluation. These results improve the outcome of individually
trained networks for values between 3 and 16 percent points. Further comparison against an
Adaboost baseline does not show a clear improvement, however, the proposed model delivers
competitive results, especially with focus on other neural network approaches. Thus, this work
gives new insights to the application of multitask deep learning to Parkinsonian speech and builds
the basis for further research in the field

Deep-learning-based MR image denoising considering noise maps as supplementary input

Introduction
In magnetic resonance (MR) images, noise is a common issue which can lead to degraded image
quality and reduced clinical value. The signal-to-noise ratio (SNR) of an image is directly proportional
to specific factors such as the magnetic field strength or the scan acquisition time but increasing those
makes the examination more expensive. Therefore, especially for low-field MR imaging, denoising
techniques can be used to improve the SNR and thus increase the diagnostic value of the resulting
images. The aim of this thesis is to implement a deep-learning-based denoising approach which
operates on reconstructed MR images using corresponding noise maps as supplementary input.
Methods and data
The data for this work is based on internal sources of Siemens Healthineers. There are around 10,000
2-D slice images of 862 studies available which were acquired with 1.5 T or 3 T MRI scanners.
Corresponding noise maps, i.e. spatially resolved image maps showing the standard deviation of
the underlying image noise, were calculated from the image data. To simulate lower field strengths,
synthetic noise will be added to the available image data.
In general, supervised deep learning methods are more straight forward than unsupervised methods, but
good ground truth data (i.e., noise-free images) is often hard to obtain for medical imaging applications.
Metzler et al. [1] proposed using Stein’s unbiased risk estimator (SURE) to train convolutional neural
networks for image denoising without any ground truth data. They have shown that SURE can be
applied to compute the mean-squared- error loss associated with an estimate of the noiseless ground
truth image under the assumption that the noise is normally distributed. Zhussip et al. [2] applied
SURE for unsupervised training of image recovery and simultaneous denoising with undersampled
compressed sensing measurements.
The goal of this thesis is to adapt a neural network for denoising using SURE loss and investigate the
benefits of including the noise map of an MR image as supplementary input. This approach will be
compared with standard supervised and unsupervised methods, such as Noise2Void [3] and Noise2Self
[4], which require nothing but the noisy data as input. For the supervised approach, the original 3 T
images might be used as ground truth and the images with added, synthetic noise to simulate lower
field strengths as input data.
Evaluation
The following aspects will be evaluated:
 Different neural network architectures will be implemented and compared w.r.t their denoising
performance.
 The SURE-based approach will be compared to other proposed unsupervised or supervised deep
learning methods (e.g., Noise2Void, Noise2Self) as well as conventional denoising algorithms.
 An extended evaluation of the network’s performance will be conducted on unseen image data.

References
[1] C. Metzler, A. Mousavi, R. Heckel, and R.G. Baraniuk. Unsupervised learning with stein’s unbiased risk
estimator. arXiv:1805.10531, 2020.
[2] M. Zhussip, S. Soltanayev, and S.Y. Chun. Training deep learning based image denoisers from undersampled
measurements without ground truth and without image prior. CVPR, pages 10255–10264, 2019.
[3] A. Krull, T.O. Buchholz, and F. Jug. Noise2void – learning denoising from single noisy images. CVPR,
pages 2129–2137, 2019.
[4] J. Batson and L. Royer. Noise2self: Blind denoising by self-supervision. PLMR, 97:524–533, 2019.

Mobile 3D-Shape Estimation in Telemedical Dermatologic Diagnosis and Documentation

Abstract: Imaging techniques for dermatology have been explored from mobile two-dimensional (2D)
RGB images to professional clinical imaging systems measuring three-dimensional (3D) information,
even beyond the epidermis, using techniques like ‘optical coherence tomography’ or laser-based
scanners. Nevertheless, to accomplish structural topographic depth measurements, technologies
lack to provide a mobile and precise 3D imaging system using only the hardware of a smartphone
or tablet. Previously a lot of approaches using ‘structure from motion’ or ‘structured light’ have been
explored for mobile wound documentation and measurement. However, to perform more
comprehensive 3D scanning of not only large chronic wounds, I want to investigate the use of mobile
‘phase-measuring deflectometry’ for dermatologic evaluation of skin to find potential biomarkers to
accompany a treatment or diagnosis based on a mobile app for both medical personnel and
untrained patients. During my master thesis I want to work on potential hardware adjustments for
dermatologic use, perform experiments for data acquisition, and design algorithms to process and
possibly classify the obtained data. Finally, I want to develop a fully functional application to create
and process 3D images, build up a database and enable a communication platform between doctor
and patient. My master thesis will jointly be conducted by my home university, Friedrich-Alexander-
Universität Erlangen-Nürnberg (FAU), and Northwestern University (NU).

Thrombus Detection in Non-Contrast Head CT using Graph Deep Learning

Thesis Description
Stroke is a severe cerebrovascular disease and one of the major causes of death and disability worldwide [1].
For patients suering from acute stroke, rapid diagnosis and immediate execution of therapeutic measures are
crucial for a successful recovery. In clinical routine, Non-Contrast Computed Tomography (NCCT) is typically
acquired as a rst-line imaging tool to identify the type of the stroke. In case of an acute ischemic infarct,
appropriate therapy planning requires an accurate detection and localization of the occluding blood clot. An
automated detection system would decrease the probability to miss an obstruction, save time and improve the
overall clinical outcome.
Several methods have been proposed to detect large vessel occlusion (LVO) using enhanced CT data like CT
angiography (CTA) [2, 3, 4]. CTA is mainly used in addition to NCCT and enables accurate evaluation of the
occlusion [5]. Nevertheless, studies have shown that the thrombus which causes the occlusion can be detected in
NCCT images due to its abnormal high density structure [6]. Classication from NCCT data can be achieved
by using Convolutional Neural Networks (CNNs) [7]. However, LVOs account for only 24% to 46% of acute
ischemic strokes [8]. Recent approaches for automated intracranial thrombus detection in NCCT are based on
Random Forest classication or CNNs [9, 10]. The results are promising, but further improvement is required
to ensure utility in clinical routine.
This thesis aims to achieve higher reliability in detecting the thrombus on NCCT data, assuming clot localization
in the entire cerebrovascular system. More specically, the goal is to build and improve upon an
existing detection model which applies a 2D U-Net to the slices of a volumetric dataset, consisting of multiple
channels that had been extracted from the raw CT dataset. The locations of the 15 local maxima with the
highest probability in the resulting prediction map are used as potential candidates for the nal prediction
of the thrombus location. The model to be developed shall classify each candidate (as clot / no clot) while
comprehensively considering all candidates found in the patient as well as corresponding regions on the opposite
hemisphere, as this is considered crucial context for the decision. To this end, a region of interest is extracted
around each candidate position and its opposite position obtained by mirroring at the brain mid plane. Each
such region is considered a node and connected with others to form a graph that describes all regions of interest
in a patient. As such, the problem is formulated as a (partial) node classication and graph neural network
models will be investigated to solve it.
In summary, this thesis will comprise the following work items:
ˆ Literature research of state-of-the-art methods for automated thrombus detection
ˆ Extraction of suitable regions of interest based on previously detected clot candidates
ˆ Design and implementation of a (graph) neural network architecture for joint classication of all clot
candidates in a patient
ˆ Investigation of multiple graph structures and model architectures
Master Thesis Antonia Popp
ˆ Optimization and evaluation of the deep learning model
References
[1] Walter Johnson, Oyere Onuma, Mayowa Owolabi, and Sonal Sachdev. Stroke: a global response is needed.
Bulletin of the World Health Organization, pages 94:634{634A, 2016.
[2] Sunil A. Sheth, Victor Lopez-Rivera, Arko Barman, James C. Grotta, Albert J. Yoo, Songmi Lee,
Mehmet E. Inam, Sean I. Savitz, and Luca Giancardo. Machine learning-enabled automated determination
of acute ischemic core from computed tomography angiography. Stroke, 50(11):3093{3100, 2019.
[3] Matthew T. Stib, Justin Vasquez, Mary P. Dong, Yun Ho Kim, Sumera S. Subzwari, Harold J. Triedman,
Amy Wang, Hsin-Lei Charlene Wang, Anthony D. Yao, Mahesh Jayaraman, Jerrold L. Boxerman, Carsten
Eickho, Ugur Cetintemel, Grayson L. Baird, and Ryan A. McTaggart. Detecting large vessel occlusion at
multiphase CT angiography by using a deep convolutional neural network. Radiology, page 200334, 2020.
[4] Midas Meijs, Frederick J. A. Meijer, Mathias Prokop, Bram van Ginneken, and Rashindra Manniesing.
Image-level detection of arterial occlusions in 4D-CTA of acute stroke patients using deep learning. Medical
image analysis, 66:101810, 2020.
[5] Michael Knauth, Rudiger von Kummer, Olav Jansen, Stefan Hahnel, Arnd Dor
er, and Klaus Sartor.
Potential of CT angiography in acute ischemic stroke. American journal of neuroradiology, 18(6):1001{
1010, 1997.
[6] G. Gacs, A. J. Fox, H. J. Barnett, and F. Vinuela. CT visualization of intracranial arterial thromboembolism.
Stroke, 14(5):756{762, 1983.
[7] Manon L. Tolhuisen, Elena Ponomareva, Anne M. M. Boers, Ivo G. H. Jansen, Miou S. Koopman, Renan
Sales Barros, Olvert A. Berkhemer, Wim H. van Zwam, Aad van der Lugt, Charles B. L. M. Majoie,
and Henk A. Marquering. A convolutional neural network for anterior intra-arterial thrombus detection
and segmentation on non-contrast computed tomography of patients with acute ischemic stroke. Applied
Sciences, 10(14):4861, 2020.
[8] Robert C. Rennert, Arvin R. Wali, Jerey A. Steinberg, David R. Santiago-Dieppa, Scott E. Olson, J. Scott
Pannell, and Alexander A. Khalessi. Epidemiology, natural history, and clinical presentation of large vessel
ischemic stroke. Neurosurgery, 85(suppl 1):S4{S8, 2019.
[9] Patrick Lober, Bernhard Stimpel, Christopher Syben, Andreas Maier, Hendrik Ditt, Peter Schramm, Boy
Raczkowski, and Andre Kemmling. Automatic thrombus detection in non-enhanced computed tomography
images in patients with acute ischemic stroke. Visual Computing for Biology and Medicine, 2017.
[10] Aneta Lisowska, Erin Beveridge, Keith Muir, and Ian Poole. Thrombus detection in (ct brain scans using
a convolutional neural network. In Margarida Silveira, Ana Fred, Hugo Gamboa, and Mario Vaz, editors,
Bioimaging, BIOSTEC 2017, pages 24{33. SCITEPRESS – Science and Technology Publications Lda,
Setubal, 2017.

Deep Learning-based motion correction of free-breathing diffusion-weighted imaging in the abdomen

Since diffusion is particularly disturbed in tissues with high cell densities such as tumors, diffusion-weighted imaging (DWI) constitutes an essential tool for the detection and characterization of lesions in modern MRI-based diagnostics. However, despite the great influence and frequent use of DWI, the image quality obtained is still variable, which can lead to false diagnoses or costly follow-up examinations.

A common way to increase the signal-to-noise ratio (SNR) in MR imaging is to repeat the acquisition several times, i.e. use multiple number of excitations (NEX). The final image is then calculated by ordinary averaging. While the single images are relatively unaffected by bulk motion due to the short acquisition time, relative motion between the excitations and subsequent averaging will lead to motion blurring in the final image. One way to mitigate this is to perform prospective gating (also known as triggering) using a respiratory signal. However, triggered acquisitions come at the cost of significantly increased scan time. Retrospective gating (also known as binning) constitutes an alternative approach in which data is acquired continuously and subsequently assigned to discrete motion states. The drawback of this approach is that there is no guarantee that data is collected for a given slice within the target motion state. In previous works, mapping of the images from other motion states onto the target motion state was achieved by using a motion model given by an additional navigator acquisition.

In recent years, deep learning has shown great potential in the field of MRI in a wide variety of applications. The goal of this thesis is the development of a deep learning-based algorithm which performs navigator-free registration of DW images given a respiratory signal only. Missing data for certain motion states as well as the inherently low SNR of DW images constitute the main challenges of this work. Successful completion of this work promises significant improvements in image quality for diffusion-weighted imaging in motion-sensitive body regions such as the abdomen.

Deep Learning-based Pitch Estimation and Comb Filter Construction

Typically a clean speech consists of two components, a locally periodic component and a stochastic component. If a speech signal only has a stochastic component, the difference between the enhanced signal applied with the corresponding ideal ratio mask and the clean speech signal is barely perceivable. However, if a speech has a perfect periodic component, then the enhanced signal applied with the corresponding ideal ratio mask is affected by the inter-harmonic noise.
A comb filter based on the speech signal’s pitch period is able to attenuate noise between the pitch harmonics. Thus, a robust pitch estimate is of fundamental importance. In this work, a deep learning-based method for robust pitch estimation in noisy environments will be investigated.