Evaluation of imperfect segmentation labels and the influence on deep learning models

Type: BA thesis

Status: finished

Date: December 1, 2022 - April 28, 2023

Supervisors: Chang Liu, Andreas Maier

Multi-organ segmentation in CT is of great clinical and research value [1], which can benefit the development of automatic computer-aided diagnosis tools and the accuracy of some interventional therapies, such as the treatment planning of radiation therapy. With the development of the deep learning (DL), the performance of the DL-based models has dramatically improved, compared with traditional segmentation methods [2].
For training a DL model for segmentation task, a paired segmentation dataset is needed. A paired segmentation dataset here indicates the accurate annotation of all voxels in all CT volumes, which is tedious and time-consuming. For this reason, the large-scale segmentation datasets for multiple organs in large body region are rarely published and mostly contain annotation errors. Several researches have been done to study the influence of imperfect segmentation labels on the training of the segmentation network, but to the best of our knowledge, none is done for the multi-organ segmentation task in CT. [3, 4]
In this thesis, our research problem is how the segmentation network will be influenced by the typical annotation errors. To achieve this, several typical annotation errors will be simulated on a public multiorgan segmentation dataset, CT-ORG, [5] and the influence will be analysed both quantitatively and qualitatively.

The thesis will comprise the following work items:

  • Literature overview of related analysis of imperfect segmentation labels.
  • Simulate some typical annotation errors on a segmentation dataset.
  • Train the baseline model on the perfect dataset and the models on the imperfect datasets.
  • Evaluate the influence of the errors with the baseline.
  • Record the result in the thesis

[1] Andreas Maier, Christopher Syben, Tobias Lasser, and Christian Riess. A gentle introduction to deep learning in medical image processing. Zeitschrift f¨ur Medizinische Physik, 29(2):86–101, 2019.
[2] Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He, and Paul Kennedy. Deep learning techniques for medical image segmentation: achievements and challenges. Journal of digital imaging, 32:582–596, 2019.
[3] Eugene Vorontsov and Samuel Kadoury. Label noise in segmentation networks: mitigation must deal with bias. In DGM4MICCAI 2021 and DALI 2021, pages 251–258. Springer, 2021.
[4] Nicholas Heller, Joshua Dean, and Nikolaos Papanikolopoulos. Imperfect segmentation labels: How much do they matter? In CVII-STENT 2018 and LABELS 2018, pages 112–120. Springer, 2018.
[5] Blaine Rister, Darvin Yi, Kaushik Shivakumar, Tomomi Nobashi, and Daniel L Rubin. Ct-org, a new dataset for multiple organ segmentation in computed tomography. Scientific Data, 7(1):381, 2020