Index

Letter Inpainting and Detection of Mathematical Diagrams in Multi-Lingual Manuscripts using a Deep Neural Network approach

Determining the Influence of Papyrus Characteristics on Fragments Retrieval with Deep Metric Learning

Multi-Task Learning for Glacier Segmentation and Calving Front Detection with the nnU-Net Framework

With the nnU-Net, Fabian Isensee et al. [1] provide a training framework for the widely applied
segmentation network U-Net [2]. The framework automates the tedious adjustment of hyperparameters
(e.g. number of layers, learning rate, batch size, etc.) that is needed to optimize the performance of
the U-Net. The framework takes a fingerprint of the given dataset and adjusts the hyperparameters
accordingly. This approach achieved stunning results on multiple independent datasets in the medical
domain without manual adjustments [1].

This thesis evaluates the performance of nnU-Net on the segmentation of synthetic aperture radar
(SAR) images of glaciers taken by satellites. Additionally to the segmentation of the different regions
(glacier, ocean, rock outcrop, SAR shadow), precise detection of the calving front position provides
important information for the glacier surveillance [3]. The relatedness of these two problems suggests
the application of multi-task learning (MTL) [4]. MTL with a single input image and multiple output
labels can be divided into late branching and early branching networks [5].

Late branching: The network is trained with multiple output channels for each task. This approach is widely used, because of its straightforward implementation [6, 7]. Dot et al. already uses the nnU-Net framework for segmentation of craniomaxillofacial structures in CT scans [8].
Early branching: Two separate Decoders are trained for each task with a common Encoder. This approach was used by Amyar et al. [9] to jointly segment lesion from lung CT scans and identify COVID-19 patients.

The nnU-Net framework will be extended for the application of MTL. Both approaches will be implemented
and evaluated on a dataset of glacier images. The dataset contains SAR images of seven
glaciers, their corresponding segmentation masks, and calving front positions. To emphasize the effect
of MTL on the performance of the nnU-Net, additional tasks like reconstruction of the input image will
be integrated into the training. The resulting models will be compared quantitatively and qualitatively
with the single-task networks and state-of-the-art in calving front detection.

[1] Fabian Isensee et al. “nnU-Net: a self-configuring method for deep learning-based biomedical
image segmentation”. In: Nature Methods 18.2 (Feb. 2021), pp. 203–211.

[2] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for
biomedical image segmentation”. In: International Conference on Medical image computing and
computer-assisted intervention. Springer. 2015, pp. 234–241.

[3] Celia A. Baumhoer et al. “Environmental drivers of circum-Antarctic glacier and ice shelf front
retreat over the last two decades”. In: The Cryosphere 15.5 (May 2021), pp. 2357–2381.

[4] Konrad Heidler et al. “HED-UNet: Combined Segmentation and Edge Detection for Monitoring
the Antarctic Coastline”. In: IEEE Transactions on Geoscience and Remote Sensing (Mar. 2021),
pp. 1–14.

[5] Kelei He et al. “HF-UNet: Learning Hierarchically Inter-Task Relevance in Multi-Task U-Net for
Accurate Prostate Segmentation in CT Images”. In: IEEE transactions on medical imaging 40.8
(Aug. 2021), pp. 2118–2128.

[6] Ava Assadi Abolvardi, Len Hamey, and Kevin Ho-Shon. “UNET-Based Multi-Task Architecture
for Brain Lesion Segmentation”. In: Digital Image Computing: Techniques and Applications
(DICTA). Melbourne, Australia: IEEE, Nov. 2020, pp. 1–7.

[7] Soumyabrata Dev et al. “Multi-label Cloud Segmentation Using a Deep Network”. In: USNCURSI
Radio Science Meeting (Joint with AP-S Symposium). July 2019, pp. 113–114.

[8] Gauthier Dot et al. “Fully automatic segmentation of craniomaxillofacial CT scans for computerassisted
orthognathic surgery planning using the nnU-Net framework”. In: medRxiv (2021).

[9] Amine Amyar et al. “Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia:
Classification and segmentation”. In: Computers in Biology and Medicine 126 (Nov. 2020),
p. 104037.

Virtual Dynamic Contrast Enhanced Image Prediction of Breast MRI using Deep Learning Architectures

Test

The UKER BrainMet Dataset: A brain metastasis dataset from University Hospital Erlangen

Brain Metastasis Synthesis Using Deep Learning in MRI Images

GAN-based Synthetic Chest X-ray Generation for Training Lung Disease Classification Systems

Project description

With the rise and ever-growing potential of Deep Learning (DL) techniques in recent years, completely new opportunities have emerged in the field of medical image processing, in particular in fundamental application areas such as image detection and recognition, image segmentation, image registration, and computer-aided diagnosis. However, DL techniques are known to require very large amounts of data to train the neural networks (NN), which can sometimes be a problem due to limited data availability. In recent years, the public release of medical image data has increased and has led to significant advances in the scientific community. For instance, publicly availabe large-scale chest X-ray datasets enabled the development of novel systems for automated lung abnormality classification [1, 2]. In recent work, however, it has been shown that DL techniques can also be used maliciously, e. g., for linkage attacks on public chest X-ray datasets [3]. This constitutes a tremendous issue in terms of data security and patient privacy, as a potential attacker may leak available information (e. g. age, gender, diseases, and more) about a specific patient present in a public dataset. To alleviate privacy concerns, the question now arises whether the exlusive use of synthetically generated images can represent a serious alternative for the development of diagnostic algorithms in the medical field.

In this work, we investigate whether synthetically generated chest X-ray images can be used to train a reliable classification system for lung diseases. Therefore, we will use different approaches, e. g., [4–6], to synthesize realistic looking chest X-ray scans from a real data distribution. In doing so, we will focus on ensuring that characteristic disease patterns will be preserved in the generated images. For our experiments, we will employ the NIH ChestX-ray14 [7] dataset, a collection of 112,120 frontal-view chest X-ray images from 30,805 unique patients with the text-mined fourteen disease image labels.

The Master’s thesis covers the following aspects:

Overview of the current state-of-the-art in DL for the generation of synthetic medical image data.
Building one or multiple GAN-based image generation networks which includes:
- Hyper-parameter tuning
- Analyzing the performance of the networks
- Analyzing the realism of the generated images
Evaluating the feasibility of using synthetically generated chest X-ray images for training a lung disease classification system.
Outlining strategies and research directions to enhance the preservation of patient privacy in public datasets (optional).

All DL implementations will be implemented using PyTorch [8].

References

[1] S. Gündel, S. Grbic, B. Georgescu, S. Liu, A. Maier, and D. Comaniciu, “Learning to Recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networks,” in Iberoamerican Congress on Pattern Recognition, pp. 757–765, Springer, 2018.

[2] S. Gündel, A. A. Setio, F. C. Ghesu, S. Grbic, B. Georgescu, A. Maier, and D. Comaniciu, “Robust classification from noisy labels: Integrating additional knowledge for chest radiography abnormality assessment,” Medical Image Analysis, vol. 72, p. 102087, 2021.

[3] K. Packhäuser, S. Gündel, N. Münster, C. Syben, V. Christlein, and A. Maier, “Is Medical Chest X-ray Data Anonymous?,” arXiv preprint arXiv:2103.08562, 2021.

[4] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative Adversarial Nets,” Advances in Neural Information Processing Systems, vol. 27, 2014.

[5] M. Mirza and S. Osindero, “Conditional Generative Adversarial Nets,” arXiv preprint arXiv:1411.1784, 2014.

[6] A. Odena, C. Olah, and J. Shlens, “Conditional Image Synthesis with Auxiliary Classifier GANs,” in International Conference on Machine Learning, pp. 2642–2651, PMLR, 2017.

[7] X. Wang, Y. Peng, L. Lu, Z. Lu, M. Bagheri, and R. M. Summers, “ChestX-ray8: Hospital-Scale Chest X-Ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.

[8] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al., “PyTorch: An Imperative Style, High-Performance Deep Learning Library,” Advances in Neural Information Processing Systems, vol. 32, pp. 8026–8037, 2019.