Index

Improving Deep Learning Target Volume Autosegmentation in Head and Neck Cancer for MR-guided Radiotherapy Treatment Planning

Automated Body Composition Analysis in Pancreatic Cancer CT Datasets using Deep Learning Auto-Segmentation

Development of a 3D CNN for Classifying Distortion Correction Status in Brain MRI Images for Radiotherapy Treatment Planning

Identifying Killer Whale Vocalizations using Graph Neural Networks

Spectral Plaque Analysis from Photon Counting CT

Generation of Artificial Vessel Trees for X-ray Image Analysis

Description

For the training of modern deep-learning-based image analysis algorithms, large scale datasets are required. For example, the segment anything model was trained 11 million images [1]. In the field of interventional X-ray imaging, this scale of data is unattainable, since not that many interventional procedures are performed, the images are not routinely stored, and are acquired as little as possible due to dose concerns. Furthermore, for some image analysis problems manual annotation of ground truth is infeasible even for optical computer vision. In such cases, synthetic data is used frequently, like the flying chairs dataset for optical flow [2].
 
For vessels synthesis, simple procedural approaches exist like the Sapling Tree Gen sampling algorithm [3] and space-filling [8]. For liver vessel analysis applications, an algorithm for generation of hepatic arterial trees is available [4]. In [5], hepatic vascular structure is generated using a physiological model of vessel growth. Generative learning-based methods have also been developed [6,9].
 
In this work, a pipeline for training and evaluating 2d X-ray image analysis methods is developed. The starting point is a procedural 3d vessel tree synthesis method [4,8]. As an exemplary organ, liver is used. The next step in the pipeline is forward projection to a 2d image according to the geometry of a real C-arm. As potential labels for the 2d images, depth, vessel segment IDs, vessel centerline IDs and overlap information are created. Finally, to demonstrate the applicability of the vessel tree for image analysis tasks, a vessel registration approach is evaluated on the resulting 2d vessel maps [2,7,10].

Work Packages

  • Literature research on simulation of vessel tree structures
  • Implementation in prototype environment (Python, Matlab, C++, …)
  • Quantitative and qualitative evaluation of registration using synthetic ground truth
  • Discussions of results & comparison to state of the art

 

Public software libraries should be used where possible. The complete source code including documentation must be provided.

 

Literature

[1] A. Kirillov et al., “Segment Anything,” in IEEE International Conference on Computer Vision (ICCV), 2023

[2] A. Dosovitskiy et al., “FlowNet: Learning Optical Flow with Convolutional Networks,” in IEEE International Conference on Computer Vision (ICCV), 2015

[3] N. Maul et al., “Transient Hemodynamics Prediction Using an Efficient Octree-Based Deep Learning Model,” in International Conference on Information Processing in Medical Imaging, 2023

[4] J. F. Whitehead, P. F. Laeseke, S. Periyasamy, M. A. Speidel, and M. G. Wagner, “In silico simulation of hepatic arteries: An open-source algorithm for efficient synthetic data generation,” Medical Physics, 2023

[5] M. Kretowski, Y. Rolland, J. Bézy-Wendling, and J.-L. Coatrieux, “Physiologically based modeling of 3-D vascular networks and CT scan angiography,” IEEE Transactions on Medical Imaging, 2003

[6] T. P. Kuipers, P. R. Konduri, H. Marquering, and E. J. Bekkers, “Generating Cerebral Vessel Trees of Acute Ischemic Stroke Patients using Conditional Set-Diffusion,” in Medical Imaging with Deep Learning, 2024

[7] G. Balakrishnan, A. Zhao, M. R. Sabuncu, J. Guttag, und A. V. Dalca, „Voxelmorph: a learning framework for deformable medical image registration“, IEEE Transactions on Medical Imaging, 2019

[8] N. Rauch and M. Harders, „Interactive Synthesis of 3D Geometries of Blood Vessels.“, in Eurographics (Short Papers), 2021

[9] C. Prabhakar et al., „3D Vessel Graph Generation Using Denoising Diffusion“, in International Conference on Medical Image Computing and Computer-Assisted Intervention, 2024

[10] S. Klein, M. Staring, K. Murphy, M. A. Viergever, and J. P. Pluim, “Elastix: a toolbox for intensity-based medical image registration,” IEEE Transactions on Medical Imaging, 2009

Alzheimer’s Classification from MRI

Frequency Domain Hierarchical Vision Transformer-based Perceptual Loss

This project focuses on improving image processing tasks, such as super-resolution or image restoration, by employing a novel feature comparison method. It leverages a Hierarchical Vision Transformer to extract multi-scale feature representations from images. These features capture both local and global information at various levels of abstraction. Crucially, these extracted features are then transformed into the frequency domain, likely via a Fast Fourier Transform (FFT) or similar method. The comparison between the generated image and the target image occurs in this frequency space. By analyzing differences in magnitude and/or phase across different frequency bands, the model can better understand and rectify discrepancies in texture, detail, and overall structure. This approach aims to produce perceptually superior results by guiding the model to reconstruct images that are more aligned with the frequency characteristics of the target, leading to improved visual quality, especially in terms of sharpness and fine-grained details.

Generative Modeling for Glottal Signals Synthesis

Improved Deep Learning Dose Prediction for Automated Head & Neck Radiotherapy Treatment Planning

Radiation therapy is one of the most important local cancer treatment modalities, enabling the non-invasive delivery of a spatially varying dose distribution within a patient’s body with high precision. Radiotherapy treatment planning aims to identify the optimal treatment plan that maximally spares surrounding normal tissue structures while delivering a high dose to the target volume. Deep Learning dose prediction is a novel technique that can automate the treatment planning process, improve standardization and reduce the treatment planning time to enable novel and improved therapies like same-day treatment and adaptive radiotherapy [1]. Deep learning dose prediction uses deep learning models like 3D U-Nets [2] to predict spatial dose distributions based on planning CT datasets, target volume, and organs at risk (OAR) segmentations [1, 3] (Figure 1). Subsequently, dose mimicking algorithms are used to translate these 3D dose predictions into deliverable treatment plans [1, 3–5]. An additional resource for dose prediction research is provided by the Open Knowledge-Based Planning Challenge (OpenKBP) [6], which offers a publicly available dataset and a standardized evaluation framework. The aim of this master thesis project is to improve an existing 3D
U-Net dose prediction model and evaluate its performance using a cohort of more than 450 Head & Neck cancer  datasets from the Department of Radiation Oncology, University Hospital Erlangen.

The thesis will include the following points:
• Literature review on deep learning techniques for radiotherapy dose prediction and automated treatment planning.
• Data preprocessing for the private Head & Neck treatment plan dataset including conversion of DICOM (CT, RT DOSE, RT STRUCT) files into CSV files and label maps. Automatic cleaning and splitting of the datasets into subgroups according to plan parameters using custom Python scripts (Pydicom library).
• Training, Inference and Testing of a pre-existing 3D U-net (Figure 2) dose prediction model (Keras, PyTorch) on the cohort of 450 Head & Neck cancer datasets.
• Reframe of the pre-existing dose prediction model to enable multi-GPU training with PyTorch. Enable increased resolution of the predicted voxel grid using multi-GPU training, splitting into subvolumes and/or further strategies to decrease memory consumption (e.g., Automatic Mixed Precision). Optimization of training time and GPU usage by shifting pre-processing steps to before the training phase.
• Extension of the pre-existing dose prediction model by implementing an on-line augmentation pipeline tailored to the task of dose prediction. Hyperparameter optimization on the validation dataset.
• Optional: Exploration of additional deep learning architectures for radiotherapy dose prediction including novel techniques like diffusion models [7, 8] and transformer architectures [9, 10].
• Optional: Adaption of the developed dose prediction pipeline to lung tumors with a data set of 130 patient plans.
• Detailed evaluation of the deep learning dose prediction performance on the test dataset comparing automated treatment plans to manually created treatment plans. Comparison of the different deep learning approaches in regard to accuracy and inference time. Five training and inference repeats to enable statistical analysis.

If you are interested in the project, please send your request to: johann.brand@uk-erlangen.de
Having prior experience with building neural networks in Python, especially using frameworks such as PyTorch or TensorFlow, will greatly help to develop the project.