Deep Learning-Based Limited Data Glacier Segmentation using Bayesian U-Nets and GANs-based Data Augmentation

The main application that this thesis is focusing on is the segmentation of Glaciers and their calving front in synthetic aperture radar (SAR) images. Accurate pixel-wise ground truth for remote sensing images, including the SAR images, are scarce and very expensive to generate. On the other hand, depending on the application, the regions of interest that we want to segment may be a small part of the image and thus, introduce a severe class-imbalance problem to the segmentation pipeline. It is a universally acknowledged fact that supervised learning-based algorithms suffer from the limited training data and class imbalance, with the main drawback being the overfitting and the model’s high uncertainty. In this work, we want to address this issue in a two-fold approach:
1. Data Augmentation: Data augmentation is a natural approach to tackle the limited and imbalanced data problem in the supervised learning-based systems [1]. Generative adversarial networks came a long way and have shown great potential in generating natural looking images. Recently, they have been to augment and populate the training set [2, 3, 4, 5, 6]. In this thesis, we are interested in conducting a thorough study on the effect of different GANs variants for data augmentation and synthetically populating the limited training data, similar to [3, 7, 8].
2. Bayesian U-net: As already mentioned, the limited training data is a bottleneck for the supervised learning-based algorithms. Moreover, the tedious task of manual labeling of the images for generating the ground truth may cause inaccuracy in this process. Both the aforementioned problems introduce uncertainty to the model. If we can measure this uncertainty, we can use it in an active learning process to improve the learning process. Bayesian algorithms provide a quantitative value for this uncertainty. In the second part of this thesis, we adapt the Bayesian U-Net [10] and/or Bayesian Seg-Net [11] to our SAR Glacier segmentation dataset and measure the uncertainty maps for the images.
Finally, we compare our results from the sections above (1 and 2) with the state-of-the-art, both quantitatively and qualitatively.
References: [1] Davari, AmirAbbas, et al. “Fast and Efficient Limited Data Hyperspectral Remote Sensing Image Classification via GMM-Based Synthetic Samples,” in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 12, no. 7, pp. 2107-2120, July 2019, doi: 10.1109/JSTARS.2019.2916495. [2] Nejati Hatamian, Faezeh, et al. “The Effect of Data Augmentation on Classification of Atrial Fibrillation in Short Single-Lead ECG Signals Using Deep Neural Networks,” ICASSP 2020 – 2020 IEEE
International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, pp. 1264-1268, doi: 10.1109/ICASSP40776.2020.9053800. [3] Neff, Thomas, et al. “Generative adversarial network based synthesis for supervised medical image segmentation.” Proc. OAGM and ARW Joint Workshop. 2017. [4] Neff, Thomas, et al. “Generative Adversarial Networks to Synthetically Augment Data for Deep Learning based Image Segmentation.” Proceedings of the OAGM Workshop 2018: Medical Image Analysis. Verlag der Technischen Universität Graz, 2018. [5] Caballo, Marco, et al. “Deep learning-based segmentation of breast masses in dedicated breast CT imaging: Radiomic feature stability between radiologists and artificial intelligence.” Computers in Biology and Medicine 118 (2020): 103629. [6] Qasim, Ahmad B., et al. “Red-GAN: Attacking class imbalance via conditioned generation. Yet another medical imaging perspective.” arXiv preprint arXiv:2004.10734 (2020). [7] Bailo, Oleksandr, DongShik Ham, and Young Min Shin. “Red blood cell image generation for data augmentation using conditional generative adversarial networks.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 2019. [8] Pollastri, Federico, et al. “Augmenting data with GANs to segment melanoma skin lesions.” Multimedia Tools and Applications (2019): 1-18. [9] T. Qin, Z. Wang, K. He, Y. Shi, Y. Gao and D. Shen, “Automatic Data Augmentation Via Deep Reinforcement Learning for Effective Kidney Tumor Segmentation,” ICASSP 2020 – 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain, 2020, pp. 1419-1423, doi: 10.1109/ICASSP40776.2020.9053403. [10] Hiasa, Yuta, et al. “Automated muscle segmentation from clinical CT using Bayesian U-net for personalized musculoskeletal Modeling.” IEEE Transactions on Medical Imaging (2019). [11] Kendall, Alex, Vijay Badrinarayanan, and Roberto Cipolla. “Bayesian segnet: Model uncertainty in deep convolutional encoder-decoder architectures for scene understanding.” arXiv preprint arXiv:1511.02680 (2015).