Index

Deep Learning-Based Classification of Skin Diseases: A Comparative Analysis of CNN and Transformer Architectures

Influence of Age in Neural Embeddings to Analyze Voice Disorders of Parkinson’s Disease Patients

Establishment of a Fully Automated Treatment Planning Pipeline for Prostate Cancer Brachytherapy (Radiotherapy)

Radiation therapy, specifically brachytherapy, is one of the most important local cancer treatment modalities for prostate cancer, enabling the delivery of a spatially varying dose distribution within a patient’s body with very high precision [1]. Brachytherapy treatment planning aims to identify the optimal treatment plan that maximally spares surrounding normal tissue structures while delivering a high dose to the target volume. For this process, a sufficient delineation of target volume, organs at risk, and identification of implanted brachytherapy needles in ultra-sound images is crucial. Deep Learning segmentation for brachytherapy treatment planning in ultrasound images is still a challenging problem, that could automate and smoothen the treatment planning process, improve standardization and reduce the treatment planning time to enable novel and improved therapies like adaptive brachytherapy [2]. Auto-segmentation of brachytherapy ultra-sound images is challenging, as ultra-sound images are noisy, have high slice-thickness and contain artifacts (e.g. from implanted needles). Based on deep learning auto-segmentations, dose calculation and optimization algorithms can be used to automatically create deliverable treatment plans [3]. The aim of this master thesis project is to create a deep learning auto-segmentation pipeline for brachytherapy ultrasound images to automate treatment planning on a cohort of more than 600 prostate cancer datasets from the Department of Radiation Oncology, University Hospital Erlangen.

Figure 1: Overview of target volume and organ segmentation as well as needle reconstruction as performed during clinical routine (left). Structure delineation on acquired ultrasound images enables planning of the treatment dose distribution (right). Image from Karius et al. [4].

The thesis will include the following points:

Literature review on deep learning techniques for target volume and organs-at-risk segmentation, as well needle detection on prostate Literature review on novel techniques to improve auto-segmentation perfor- mance on noisy ultra-sound images with artifacts including cardiac and fetal ultra-sound that could be trans- ferred to brachytherapy ultrasound auto-segmentation with a particular focus on integrating prior knowledge on structure shape.
Data preprocessing for the private ultrasound treatment plan dataset including conversion of DICOM (CT, RT STRUCT) files into label maps. Automatic cleaning and splitting of the datasets into subgroups according to clinical treatment concepts using custom Python scripts (Pydicom library).
Conversion of available needle point coordinates into binary segmentations by mapping of the known needle shape to the point coordinates.

Training, Inference, and Testing of an nnU-net 2D and 3D fullres model on the cohort of 600 prostate can- cer ultrasound datasets to establish a baseline for comparison. Comparison of the ultrasound segmentation performance for the 2D and 3D model variant.
Improve segmentation performance by integrating shape priors into ultrasound segmentation networks. Inte- gration of the novel method of shape prior modules (SPMs [5]) into a Unet-based segmentation Com- parative evaluation of SPM with the conventional U-net model.
Fallback alternative / optional: Evaluate a Vision Transformer (e.g. Nnformer [6]) or a CNN-Transformer- Hybrid for brachytherapy ultrasound auto-segmentation, which could provide improved ultrasound segmenta- tion performance by improved modeling of global image context.
Establish a pipeline for processing clinical brachytherapy ultra-sound images in DICOM format, auto-segmenting the images and exporting the auto-segmentations into DICOM RTStruct format for import into routine treatment planning programs. Evaluation of the pipeline processing time.
Detailed evaluation of the developed brachytherapy ultra-sound auto-segmentation solution in reference to clin- ical ground truths. Perform five training and inference repeats to enable statistical analysis.

If you are interested in the project, please send your request to: andre.karius@uk-erlangen.de

Having prior experience with building neural networks in Python, especially using frameworks such as PyTorch or TensorFlow, will greatly help to develop the project.

Category-Level Segmentation of industrial Parts Using SAM2 Memory System

The Segment Anything Model (SAM), developed by Meta AI in 2023, introduced a powerful zero-shot segmentation approach, allowing object segmentation without additional training. In 2024, Meta released SAM2, an upgraded version designed for video object tracking using a hierarchical vision transformer and a cross-attention mechanism with memory integration. Despite its advancements, SAM2 lacks category-specific segmentation, limiting its ability to distinguish objects based on contextual understanding.

This research aims to enhance SAM2 by leveraging its automatic mask generator (AMG) and memory system to improve object recognition. The proposed method involves generating segmentation candidates using AMG and employing cross-attention mechanisms to compare feature similarities, enabling consistent identification of objects across frames. Additionally, to address SAM2’s challenges with light reflection and over-segmentation, small-scale fine-tuning of the image encoder and mask decoder will be implemented.

The study will include an analysis of SAM2’s architecture, the development of an embedding-based object identification method, and performance benchmarking using the ARMBench dataset and a custom industrial dataset. The findings will contribute to improving SAM2’s capability in complex environments, particularly in robotic applications requiring precise object classification and tracking.

Local Fine-Tuning of LLMs for Post-Treatment Physician Letter Prediction in Radiation Oncology Based on the Pre-Treatment Letter as well as Treatment Information from the OIS

Video-based pose and distance estimation of an excavator bucket

CADGLM – Integrating Graph Neural Networks and Large Language Models to Predict Machining Information from Graph Representations of 3D Models

Vishal_Thesis_Proposal