Index

Statistical Evaluation of Orca Vocalization Activity

Evaluation of imperfect segmentation labels and the influence on deep learning models

Multi-organ segmentation in CT is of great clinical and research value [1], which can benefit the development of automatic computer-aided diagnosis tools and the accuracy of some interventional therapies, such as the treatment planning of radiation therapy. With the development of the deep learning (DL), the performance of the DL-based models has dramatically improved, compared with traditional segmentation methods [2].
For training a DL model for segmentation task, a paired segmentation dataset is needed. A paired segmentation dataset here indicates the accurate annotation of all voxels in all CT volumes, which is tedious and time-consuming. For this reason, the large-scale segmentation datasets for multiple organs in large body region are rarely published and mostly contain annotation errors. Several researches have been done to study the influence of imperfect segmentation labels on the training of the segmentation network, but to the best of our knowledge, none is done for the multi-organ segmentation task in CT. [3, 4]
In this thesis, our research problem is how the segmentation network will be influenced by the typical annotation errors. To achieve this, several typical annotation errors will be simulated on a public multiorgan segmentation dataset, CT-ORG, [5] and the influence will be analysed both quantitatively and qualitatively.

The thesis will comprise the following work items:

Literature overview of related analysis of imperfect segmentation labels.
Simulate some typical annotation errors on a segmentation dataset.
Train the baseline model on the perfect dataset and the models on the imperfect datasets.
Evaluate the influence of the errors with the baseline.
Record the result in the thesis

[1] Andreas Maier, Christopher Syben, Tobias Lasser, and Christian Riess. A gentle introduction to deep learning in medical image processing. Zeitschrift f¨ur Medizinische Physik, 29(2):86–101, 2019.
[2] Mohammad Hesam Hesamian, Wenjing Jia, Xiangjian He, and Paul Kennedy. Deep learning techniques for medical image segmentation: achievements and challenges. Journal of digital imaging, 32:582–596, 2019.
[3] Eugene Vorontsov and Samuel Kadoury. Label noise in segmentation networks: mitigation must deal with bias. In DGM4MICCAI 2021 and DALI 2021, pages 251–258. Springer, 2021.
[4] Nicholas Heller, Joshua Dean, and Nikolaos Papanikolopoulos. Imperfect segmentation labels: How much do they matter? In CVII-STENT 2018 and LABELS 2018, pages 112–120. Springer, 2018.
[5] Blaine Rister, Darvin Yi, Kaushik Shivakumar, Tomomi Nobashi, and Daniel L Rubin. Ct-org, a new dataset for multiple organ segmentation in computed tomography. Scientific Data, 7(1):381, 2020

Fetal Re-Identification: Deep Learning on Pregnancy Ultrasound Images

Project description

Accurate analysis of ultrasound images during pregnancy is important for monitoring fetal development and detecting abnormalities. For better accuracy and time convenience, the help of artificial intelligence or accordingly deep learning is useful [1]. However, currently there is less research in deep learning in the field of ultrasound imaging compared to MRI or CT images [2].

Considering its non-invasive nature, lower cost, and lower risk to patients compared to other modalities such as MRI or CT, ultrasound imaging is the most commonly used method to assess fetal development and maternal health [3]. Overall, the correct acquisition of fetal ultrasound data is difficult and time-consuming. Deep Learning can help to reduce examiner dependence, and improve analysis as well as maternal-fetal medicine in general [1].

Although literature on fetal ultrasound imaging in conjunction with deep learning exists [4-6], little previous work investigated fetal re-identification. In multiple pregnancies with fetuses of the same sex or early in pregnancy, the fetuses cannot be distinguished. Therefore, the fetuses are assigned an order at physician descretion (usually based on position in the mother’s womb), although it is not clear whether this order preserves during subsequent examinations. However, this information is important because the risk of fetal abnormalities is greater in multiple pregnancies than in singleton pregnancies [7]. In addition, a good representation of the fetus is also eminent for the emotional connect of the parents [8].

Consequently, the aim of this thesis is an early feasibility investigation of re-identification approaches in fetal ultrasound.

References

[1] J. Weichert, A. Rody, and M. Gembicki. Zukünftige Bildanalyse mit Hilfe automatisierter Algorithmen. Springer Medizin Verlag GmbH, 2020.

[2] Xavier P. Burgos-Artizzu, David Coronado-Guiérrez, Brenda Valenzuela-Alcaraz, Elisenda Bonet-Carne, Elisenda Eixarch, Fatima Crispi, and Eduard Gratacos. Evaluation of deep convolutional neural networks for automatic classification of common maternal fetal ultrasound planes. Nature Scientific Reports, 2020.

[3] D. Selvathi and R. Chandralekha. Fetal biometric based abnormality detection during prenatal development using deep learning techniques. Springer, 2021.

[4] Jan Weichert, Amrei Welp, Jann Lennard Scharf, Christoph Dracopoulos, Achim Rody, and Michael Gembicki. Künstliche Intelligenz in der pränatalen kardialen Diagnostik. page 10, 2021.

[5] Christian F. Baumgartner, Konstantinos Kamnitsas, Jacqueline Matthew, Tara P. Fletcher, Sandra Smith, Lisa M. Koch, Bernhard Kainz, and Daniel Rueckert. SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound. page 12, 2017.

[6] Juan C. Prieto, Hina Shah, Alan J. Rosenbaum, Xiaoning Jiang, Patrick Musonda, Joan T. Price, Elizabeth M. Stringer, Bellington Vwalika, David M. Stamilio, and Jeffrey S. A. Stringer. An automated framework for image classification and segmentation of fetal ultrasound images for gestational age estimation. page 11, 2021.

[7] R Townsend and A Khalil. Ultrasound surveillance in twin pregnancy: An update for practitioners. Ultrasound, 2018.

[8] Tejal Singh, Srinivas Rao Kudavelly, and Venkata Suryanarayana. Deep Learning Based Fetal Face Detection And Visualization In Prenatal Ultrasound. 2021.

Projection Domain Metal Segmentation with Epipolar Consistency using Known Operator Learning

Evaluation of a Pixel-wise Regression Model Solving a Segmentation Task and a Deep Learning Model with the Matthew’s Correlation Coefficient as an Early Stopping Criterion

With global sea level rising and mass loss of polar ice sheets as the main cause, it becomes increasingly important to enchance ice dynamics modeling. A very fundamental information for this is the calving front position (CFP) of glaciers. Traditionally the delineating of the CFP has been done manually, which is a very subjective, tedious and expensive task. Since then, there has been a lot of development in automating this process. Gourmelon et al. [1] introduce the first publicly available benchmark dataset for calving front delineation on synthetic aperture radar (SAR) imagery dubbed CaFFe. The dataset consists of the SAR imagery and two corresponding labels: one showing the calving front vs the background and the other showing different landscape regions. However, for this paper we will only look at methods using the former. As there are many different approaches to calving front delineation the question of what method provides the best performance arises. Subsequently, the aim of this thesis is to evaluate the codes of the following two papers [2],[3] on the CaFFe benchmark dataset and compare their performance with the baselines provided by Gourmelon et al. [1].

paper 1: Davari et al. [2] reformulates the segmentation problem into a pixel-wise regression task by using a Convolutional Neural Network (CNN) that gets optimized to predict a distance map containing a distance value for each pixel of the image to extract the glacier calving front line with the help of a second U-net.

paper 2: Davari et al. [3] proposes a deep learning model with the Mathew Correlation Coefficient as an early stopping criterion to counter the extreme class imbalance of this problem. Moreover, a distance map based binary cross-entropy (BCE) loss function gets introduced to add context about the important regions for segmentation. To make a fair and reasonable comparison, the hyperparameters of each model will be optimized on the CaFFe benchmark dataset and the model weights will be re-trained on CaFFe’s train set. The evaluation will be conducted on the provided test set and the metrics introduced in Gourmelon et al. [1] will be used for the comparison.

References

[1] Gourmelon, N.; Seehaus, T.; Braun, M.; Maier, A.; and Christlein, V.: Calving Fronts and Where to Find Them: A Benchmark Dataset and Methodology for Automatic Glacier Calving Front Extraction from SAR Imagery, Earth Syst. Sci. Data Discuss. [preprint]. 2022, https://doi.org/10.5194/essd-2022-139, in review.

[2] A. Davari, C. Baller, T. Seehaus, M. Braun, A. Maier and V. Christlein, “Pixelwise Distance Regression for Glacier Calving Front Detection and Segmentation,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-10, 2022, Art no. 5224610, doi: 10.1109/TGRS.2022.3158591.

[3] A. Davari et al., “On Mathews Correlation Coefficient and Improved Distance Map Loss for Automatic Glacier Calving Front Segmentation in SAR Imagery,” in IEEE Transactions on Geoscience and Remote Sensing, vol. 60, pp. 1-12, 2022, Art no. 5213212, doi: 10.1109/TGRS.2021.3115883.

Tomographic Projection Selection with Quantum Annealing

Introduction

The object of interest in computed tomography (CT) is exposed to X-rays from multiple angles. The radiation intensity measured by the detector opposite the radiation source then depends on the object’s density. Volumetric information about the object can be reconstructed using many such projection images. Another way to obtain projection data for reconstruction is single-photon emission computed tomography (SPECT), where a radioisotope is injected into the object, and the gamma rays emitted by radioactive decay are measured. The two methods have extensive applications in radiology but are restricted due to the harmful radiation emitted, which can damage cells in the human body [1].

There is a strong interest in performing the reconstruction task with a small number of projection images to limit the patient’s radiation exposure. Determining an optimal set of angles for projection data acquisition is referred to as projection selection. This set shall contain as few angles as possible while allowing a satisfactory reconstruction of the original object. Suppose some a priori information is available, e.g., in discrete tomography, where the object is known to consist of only a few materials with known densities. In that case, this can be used to improve projection selection algorithms. Some of these algorithms are compared in [2]. In particular, simulated annealing (SA) was proposed as a possible method for projection selection.

SA is a minimization method that allows a worsening of the current solution with some probability based on the slowly decreasing temperature of the system. The annealing process mimics the cooling of a material, which terminates in its lowest-energy state. Since a worsening of the current solution is accepted, the solution can not be “trapped” in a sub-optimal local minimum, as can happen with gradient-descent methods. A realistic annealing technique based on superconducting qubits is quantum annealing (QA). Quantum annealing is a quantum computing technique where quantum effects like superposition, entanglement, and tunneling can help traverse the barriers between local minima.

Methods

Starting from the SA formulation of the projection selection problem proposed in [2], a mathematical formulation as a quadratic unconstrained binary optimization (QUBO) problem will be given. The QUBO formulation can then be used to develop a program for the D-Wave quantum annealer, which will be run using simulation software. The discrete algebraic reconstruction technique (DART) will reconstruct the image from the selected projections. Using the images reconstructed by DART, the projection selection with QA can be compared to other projection selection algorithms.

Expected Results

Various iterative reconstruction methods are reviewed. In particular, a python implementation of the DART algorithm is provided, as it can perform an accurate reconstruction even from a small number of projections [3]. Furthermore, the projection selection problem in discrete tomography is formulated as a QUBO problem. This formulation will evaluate the possibility of running the projection selection problem using simulation software and a D-Wave quantum annealer.

[1] A. Maier, S. Steidl, V. Christlein, and J. Hornegger, “Medical imaging systems: An introductory guide,” 2018.
[2] L. Varga, P. Bal ́azs, and A. Nagy, “Projection selection algorithms for discrete tomography,” in Advanced Concepts
for Intelligent Vision Systems (J. Blanc-Talon, D. Bone, W. Philips, D. Popescu, and P. Scheunders, eds.), (Berlin,
Heidelberg), pp. 390–401, Springer Berlin Heidelberg, 2010.
[3] K. J. Batenburg and J. Sijbers, “Dart: A practical reconstruction algorithm for discrete tomography,” IEEE
Transactions on Image Processing, vol. 20, no. 9, pp. 2542–2553, 2011.

Convolutional LSTM for Multi-organ Segmentation on CT and MR Images in Abdominal Region

Automatic Pathological Speech Intelligibility Assessment Using Speech Disentanglement Without Bottleneck Information

Optimizing the Preprocessing Pipeline for “virtual Dynamic Contrast Enhancement” in Breast MRI

Semi-supervised learning for multi-modal bone segmentation

Since AlexNet won the ImageNet Challenge by a wide margin in 2012, the popularity of deep learning has been steadily increasing. In the last years, a technique that has been especially popular is semantic segmentation, as it is used in self-driving cars and medical image analysis. A big challenge that arises when training neural networks (NN) for this task is the acquisition of adequate segmentation masks, because the labeling has often times to be performed by industry experts and is very time consuming. Resulting from that, solutions circumventing this problem had to be found. A popular solution for this task is semi-supervised learning, where only a certain amount of the data is annotated. This approach has the obvious advantage of reducing the time needed for the data acquisition process, but NNs trained this way still have a worse performance compared to ones that were trained fully-supervised.

A common disease affecting one in three women and one in twelve men is osteoporosis. It’s symptoms include low bone mass and a deterioration of bone tissue, leading to an increased fracture risk. The malady affects especially elderly people and for their protection, providing diagnostic tools and suitable treatments is important [1]. Structures that can be found in the bone include lacunae containing osteocytes and trans-cortical vessels (TCV). Murine and human tibia consists of two parts; the inner trabecular bone and the outer cortical bone, where TVCs can be found. To study them and their importance for the development of osteoporosis, we are trying to automatically segment the cortical bone from the surrounding tissue. Additionally, we will attempt to build a NN for the detection of TVCs and lacunae.

We want to achieve this using a model based on convolutional neural networks (CNN) for semantic segmentation. Similar tasks have already been performed [2], but our approach differs as we try to use as few labels as possible for the training process. Methods we want to incorporate are pre-training and the use of image transformations to make the most out of a limited amount of segmentation masks. If those approaches do not yield the desired results, we will also try to incorporate techniques of weakly- and self-supervised learning.

In detail, the thesis will consist of the following parts:

• implementation of multiple CNN-based architectures [3][4] to find a suitable model for our task,

• optimization of this model using different approaches,

• evaluation of the usefulness of pre-training and different semi-supervised learning techniques,

• integration of different techniques to increase the accuracy

References
[1] S P Tuck and R M Francis. Osteoporosis. Postgraduate Medical Journal, 78(923):526–532, 2002.
[2] Oliver Aust, Mareike Thies, DanielaWeidner, FabianWagner, Sabrina Pechmann, Leonid Mill, Darja Andreev, Ippei Miyagawa, Gerhard Kronke, Silke Christiansen, Stefan Uderhardt, Andreas Maier, and Anika Gruneboom. Tibia cortical bone segmentation in micro-ct and x-ray microscopy data using a single neural network. In Klaus Maier-Hein, Thomas M. Deserno, Heinz Handels, Andreas Maier, Christoph Palm, and Thomas Tolxdorff, editors, Bildverarbeitung fur die Medizin 2022 , pages 333–338, Wiesbaden, 2022. Springer Fachmedien Wiesbaden.
[3] Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. CoRR, abs/1505.04597, 2015.
[4] Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. CoRR, abs/1411.4038, 2014.
[5] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Kopf, Edward Yang, Zachary DeVito, Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: An imperative style, high-performance deep learning library. In H. Wallach, H. Larochelle, A. Beygelzimer, F. d’Alche-Buc, E. Fox, and R. Garnett, editors, ´ Advances in Neural Information Processing Systems 32, pages 8024–8035. Curran Associates, Inc., 2019