With the nnU-Net, Fabian Isensee et al. [1] provide a training framework for the widely applied
segmentation network U-Net [2]. The framework automates the tedious adjustment of hyperparameters
(e.g. number of layers, learning rate, batch size, etc.) that is needed to optimize the performance of
the U-Net. The framework takes a fingerprint of the given dataset and adjusts the hyperparameters
accordingly. This approach achieved stunning results on multiple independent datasets in the medical
domain without manual adjustments [1].
This thesis evaluates the performance of nnU-Net on the segmentation of synthetic aperture radar
(SAR) images of glaciers taken by satellites. Additionally to the segmentation of the different regions
(glacier, ocean, rock outcrop, SAR shadow), precise detection of the calving front position provides
important information for the glacier surveillance [3]. The relatedness of these two problems suggests
the application of multi-task learning (MTL) [4]. MTL with a single input image and multiple output
labels can be divided into late branching and early branching networks [5].
- Late branching: The network is trained with multiple output channels for each task. This approach is widely used, because of its straightforward implementation [6, 7]. Dot et al. already uses the nnU-Net framework for segmentation of craniomaxillofacial structures in CT scans [8].
- Early branching: Two separate Decoders are trained for each task with a common Encoder. This approach was used by Amyar et al. [9] to jointly segment lesion from lung CT scans and identify COVID-19 patients.
The nnU-Net framework will be extended for the application of MTL. Both approaches will be implemented
and evaluated on a dataset of glacier images. The dataset contains SAR images of seven
glaciers, their corresponding segmentation masks, and calving front positions. To emphasize the effect
of MTL on the performance of the nnU-Net, additional tasks like reconstruction of the input image will
be integrated into the training. The resulting models will be compared quantitatively and qualitatively
with the single-task networks and state-of-the-art in calving front detection.
[1] Fabian Isensee et al. “nnU-Net: a self-configuring method for deep learning-based biomedical
image segmentation”. In: Nature Methods 18.2 (Feb. 2021), pp. 203–211.
[2] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. “U-net: Convolutional networks for
biomedical image segmentation”. In: International Conference on Medical image computing and
computer-assisted intervention. Springer. 2015, pp. 234–241.
[3] Celia A. Baumhoer et al. “Environmental drivers of circum-Antarctic glacier and ice shelf front
retreat over the last two decades”. In: The Cryosphere 15.5 (May 2021), pp. 2357–2381.
[4] Konrad Heidler et al. “HED-UNet: Combined Segmentation and Edge Detection for Monitoring
the Antarctic Coastline”. In: IEEE Transactions on Geoscience and Remote Sensing (Mar. 2021),
pp. 1–14.
[5] Kelei He et al. “HF-UNet: Learning Hierarchically Inter-Task Relevance in Multi-Task U-Net for
Accurate Prostate Segmentation in CT Images”. In: IEEE transactions on medical imaging 40.8
(Aug. 2021), pp. 2118–2128.
[6] Ava Assadi Abolvardi, Len Hamey, and Kevin Ho-Shon. “UNET-Based Multi-Task Architecture
for Brain Lesion Segmentation”. In: Digital Image Computing: Techniques and Applications
(DICTA). Melbourne, Australia: IEEE, Nov. 2020, pp. 1–7.
[7] Soumyabrata Dev et al. “Multi-label Cloud Segmentation Using a Deep Network”. In: USNCURSI
Radio Science Meeting (Joint with AP-S Symposium). July 2019, pp. 113–114.
[8] Gauthier Dot et al. “Fully automatic segmentation of craniomaxillofacial CT scans for computerassisted
orthognathic surgery planning using the nnU-Net framework”. In: medRxiv (2021).
[9] Amine Amyar et al. “Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia:
Classification and segmentation”. In: Computers in Biology and Medicine 126 (Nov. 2020),
p. 104037.