This thesis aims at integrating time series information into the static segmentation of glaciers and their
calving fronts in synthetic aperture radar (SAR) image sequences. U-Nets have recently been shown
to provide promising results for glacier (front) segmentation using synthetic aperture radar (SAR)
imagery [1]. However, this approach only incorporates the spatial information in a single image. The
temporal information of complete image sequences, each showing one glacier at different time points,
has not been addressed thus far. To fill this gap two approaches shall be worked on:
- approach 1; using Long Short-Term Memory (LSTM) layers in the U-Net architecture:
Recurrent Neural Networks like LSTMs are designed such that information from previous
inputs in a sequence can be stored in a memory and used to ameliorate the prediction for
the current input. The combination of structured LSTMs and Fully Convolutional Networks
(FCNs) showed promising results for joint 4D segmentation of longitudinal MRI [2]. In [3], a
U-Net was successfully combined with a bi-directional convolutional LSTM for aortic image
sequence segmentation outperforming a simple U-Net in segmentation accuracy. In this thesis,
the combination of LSTMs and U-Nets will be tested for glacier segmentation and calving
front detection in SAR image sequences. Moreover, the use of Recurrent layers (RNN), Gated
Recurrent Units (GRU) and bi-directional LSTMS instead of simple LSTMs shall be investigated
as well. - approach 2; Multi-Task Learning (MLT): As the region to be segmented for calving front
detection is a small part of the image, this task shows a severe class-imbalance. To improve its
performance, an MLT approach shall be implemented jointly training glacier segmentation and
calving front detection. Performance enhancement of U-Nets have been observed using stacking
[4] and shared encoding networks [5, 6]. In this thesis, both MLT techniques shall be tested
using U-Nets in combination with LSTMs (see point 1).
The resulting models will be compared quantitatively and qualitatively with the state-of-the-art and
shall be implemented in Keras.
[1] Zhang et al. “Automatically delineating the calving front of Jakobshavn Isbræ from multitemporal
TerraSAR-X images: a deep learning approach.” The Cryosphere 13, no. 6 (2019): 1729-1741.
[2] Gao et al. “Fully convolutional structured LSTM networks for joint 4D medical image segmentation.”
In: IEEE 15th International Symposium on Biomedical Imaging, Washington, DC, 2018, IEEE, pp.
1104-1108.
[3] Bai et al. “Recurrent Neural Networks for Aortic Image Sequence Segmentation with Sparse Annotations.”
In Alejandro F. Frangi, Julia A. Schnabel, Christos Davatzikos, Carlos Alberola-L´opez, Gabor Fichtinger
(Eds.): Medical Image Computing and Computer Assisted Intervention – MICCAI, 2018, pp. 586-594.
[4] Sun et al. “Stacked U-Nets with Multi-Output for Road Extraction.” In: CVPR Workshops, Salt Lake
City, 2018, pp. 202-206.
[5] Ke et al. “Learning to segment microscopy images with lazy labels.” In: ECCV Workshop on BioImage
Computing, 2020.
[6] Lee et al. “Multi-Task Learning U-Net for Single-Channel Speech Enhancement and Mask-Based Voice
Activity Detection.” Applied Sciences 10, no. 9 (2020): p. 3230.