Index
Deep Learning based Vascular Contouring in Photon-Counting Computed Tomography
Thesis Description
Heart diseases, particularly ischemic strokes, are a leading global cause of mortality and morbidity. Atherosclerotic
plaque formation thickens blood vessels walls, serving as a risk indicator for future ischemic stroke
occurrences [1] [2]. Automatic estimation of the vessel wall thickness would offer new potential for screening
patients with respect to high-risk artherosclerotic plaques. The vessel wall thickness can be obtained by segmenting
the vessel wall in cross-section images along the vessels’ centerlines from medical imaging modalities
such as Magnetic Resonance Imaging (MRI) and Computer Tomography (CT). Classical vessel segmentation
methods such as Region Growing [3] and Adaptive Frangi Filtering [4] pose significant challenges as they need to
be manually tuned and do not scale with an increase in data. These issues are adressed by deep learning based
approaches as they do not need manual configuration during inference and can benefit from large amounts of
data. However, popular deep learning based segmentation algorithms such as the U-net pose challenges when
predicting contour points of vessels as these algorithms predict discrete segmentation maps which require further
conversion into the continous domain [5]. Thus, they are dependant on the input resolution and not suitabe to
predict contours on a submillimeter scale. A more suitable way to predict vessel contour points is to perform
deep learning based radius regression on polar unfolded cross-section images as proposed by Ablas et al. [6] or
Chen et al. [7] on black-blood MRI images. The aim of this thesis is to translate the approach to Computer
Tomography data from Photon-Counting CT scanners (PCCT). Furthermore, the thesis aims to exploit the
potential of PCCT scanners and investigate the impact on the prediction of Photon-Counting-based spectral
image information to be able to implicitly suppress non-vessel-like structures. For this purpose, different architectures
(e.g. attention mechanisms [8]) to perform the radius regression are tested and compared against a state
of the art segmentation baseline algorithm on standard inputs. The models are trained and tested on manually
annotated real world CT data. To capture the details of both inner and outer vessel contours, conventional loss
functions and metrics for segmentation, like Intersection-over-Union (IoU), fall short in accommodating subtle
variations in border regions, such as small plaques in the vessel wall. To evaluate the models with respect to
the final application, a suitable distance-based metric should be found that not only accounts for these small
variations but also addresses the multiscale characteristics of the different vessels in the dataset. To ensure
and improve data quality, part of the work will be the iterative training of the model to find errorneous annotations
for manual corrections. Finally, the individual cross-section predictions are converted back to 3D meshes.
Summary:
1. State of the art research
2. Development of a deep learning based algorithm for inner and outer vessel wall contouring on Photon-
Counting CT data
(a) Investigation of the impact of Photon-Counting-based spectral image information
(b) Investigation and optimization of different architectures for vessel contour regression
(c) Investigation of different loss functions and metrics that address the multiscale characteristics and
infrequent occurences of anomalities in the training set
3. Comparison against various baseline algorithms
4. Reconstruction of 3D mesh from contour predictions on 2D cross-sections
5. Evaluation of the developed pipeline and its impact with respect to the final application
References
[1] L. E. Chambless, A. R. Folsom, L. X. Clegg, et al. Carotid wall thickness is predictive of incident clinical
stroke: The atherosclerosis risk in communities (aric) study. American Journal of Epidemiology, 151:478–
487, 3 2000.
[2] Gregory L. Burke, Gregory W. Evans, Ward A. Riley, et al. Arterial wall thickness is associated with
prevalent cardiovascular disease in middle-aged adults. Stroke, 26:386–391, 3 1995.
[3] S.A. Hojjatoleslami and J. Kittler. Region growing: a new approach. IEEE Transactions on Image Processing,
7:1079–1084, 7 1998.
[4] Alejandro Frangi, W J Niessen, Koen Vincken, and Max Viergever. Multiscale vessel enhancement filtering.
Med. Image Comput. Comput. Assist. Interv., 1496, 10 2000.
[5] Florian Thamm, Felix Denzinger, Leonhard Rist, Celia Martin Vicario, Florian Kordon, and Andreas Maier.
Segmentation of the carotid lumen and vessel wall using deep learning and location priors. 1 2022.
[6] Dieuwertje Alblas, Christoph Brune, and Jelmer M. Wolterink. Deep learning-based carotid artery vessel
wall segmentation in black-blood mri using anatomical priors. 12 2021.
[7] Li Chen, Jie Sun, Gador Canton, et al. Automated artery localization and vessel wall segmentation using
tracklet refinement and polar conversion. IEEE Access, 8:217603–217614, 2020.
[8] Wentao Liu, Huihua Yang, Tong Tian, Xipeng Pan, andWeijin Xu. Multiscale attention aggregation network
for 2d vessel segmentation. pages 1436–1440. IEEE, 5 2022.
Development of an Oriented Bone Detection Algorithm on X-Ray Images
Mobile C-arms are a tool commonly used in trauma and orthopedic surgery. They have many different applications, including spine, knee, and wrist surgery. With the possibility of intra-operative imaging, mobile C-arms are a great enrichment for checking the progress of the surgery or for providing guidance during minimally invasive procedures. However, there is one drawback to using them, and that is the relatively increased radiation exposure of the patient and the surgical staff [1]. Collimation is an option for reducing the radiation exposure. By focusing the x-ray beams only on the region of interest, it becomes possible to reduce the irradiated area and consequently lower radiation doses. Another effect of choosing the field-of-view is enhanced contrast and improved image quality. Enhanced image quality is consistently sought after in surgical procedures, as it can significantly benefit the surgeon and contribute to
better and more efficient surgical outcomes [2]. Therefore, the accurate adjustment of the collimators has a great impact on the outcome of the surgery, the patient and the medical staff involved. Nevertheless, this crucial adjustment is frequently overlooked due to lack of time and insufficient training of the staff. Software that automatically finds the region of interest can
help to properly adjust the collimation without additional effort for the medical staff. Various hardware- and software-based approaches have previously been employed to address this issue. Yap et al.[3] proposed a deep-learning based method to detect the region of interest. They have used a Faster R-CNN to predict axis-aligned boxes covering the region of interest. The experiments focus on the detection of breast lesion and only consider breast ultrasound data. The aim of this master thesis is to use Transformer networks for finding bounding boxes of bones from various anatomical regions to automate the collimation which leads to a reduction of radiation exposure and increase in image quality.
References
[1] Yang-Sub Lee, Hae-Kag Lee, Jae-Hwan Cho, and Ham-Gyum Kim. Analysis of radiation risk to patients
from intra-operative use of the mobile x-ray system (c-arm). J. Res. Med. Sci., 20(1):7–12, January 2015.
[2] Thomas Werncke, Christian von Falck, Matthias Luepke, Georg Stamm, Frank K Wacker, and Bernhard
Christian Meyer. Collimation and image quality of c-arm computed tomography. Invest. Radiol.,
50(8):514–521, August 2015.
[3] Moi Hoon Yap, Manu Goyal, Fatima Osman, Robert Mart´ı, Erika Denton, Arne Juette, and Reyer Zwiggelaar.
Breast ultrasound region of interest detection and lesion localisation. Artificial Intelligence in Medicine,
107:101880, 2020.
Geometric Domain Adaptation for CBCT Segmentation
In this project, we perform a computational domain transfer to introduce cone-beam artifacts to the training data. We evaluate its impact on the results of supervised training for the segmentation of the lungs. For this, already labeled CT volumes are reconstructed to artificial CBCT volumes without a complex deep learning-based method, like introduced by Jia X et al.,5 but rather by computational reconstruction. The purpose is to have a network for stable segmentation on real CBCT volumes. A major advantage of our approach is that the artificial
CBCT volumes can not only be computed easily from thoracic CT volumes but also the pixel-wise segmentation can be re-used without putting in the great effort of labeling. This allows for supervised training.
Realistic Simulation of Collimated X-Ray images for Collimator Edge Segmentation using Deep Learning
Collimator detection in X-ray systems has long posed a challenge, particularly when information about the detector’s position relative to the source is either unreliable or completely unavailable. In this paper [1], we introduce a physically motivated image processing pipeline designed to simulate the intricate characteristics of collimator shadows in X-ray images. The primary objective of this pipeline is to address the scarcity of training data for deep neural networks, which are increasingly promising for collimator detection. By applying the pipeline to deep networks initially limited by small datasets, our approach equips them with the necessary information to learn and generalize effectively.
Our pipeline is a comprehensive solution that leverages several key components to
generate realistic collimator images. Employing randomized labels to describe collimator shapes and their respective locations ensures diversity and representativeness. In addition, we integrate a convolution kernel based scattered radiation simulation mechanism, which is a crucial factor in real-world X-ray imaging. To complete the simulation process, we introduce Poisson noise to replicate the inherent characteristics of collimator shadows in X-ray images.
Comparing the simulated data with real collimator shadows demonstrates the authenticity of our approach and its potential to bridge the gap between synthetic and real-world data. Moreover, incorporating simulated data into our deep learning framework not only serves as a valid substitute for real collimators but also significantly improves generalization in real-world applications, holding great promise for the field of collimator detection.
This work was presented at the DALI workshop at the MICCAI conference in Vancouver, Canada and was published in the proceedings:
1. El-Zein B, Eckert D, Weber T, Rohleder M, Ritschl L, Kappler S et al. A Realistic Collimated
X-Ray Image Simulation Pipeline. Data Augmentation, Labelling, and Imperfections – Third
MICCAIWorkshop, DALI 2023, Held in Conjunction with MICCAI 2023, Vancouver, October
12, 2023, Proceedings. Springer Nature. 2023, (in press)
Unsupervised Domain Adaptation Using Contrastive Learning for Multi-modal Cardiac MR Segmentation
Deep Learning Computed Tomography based on the Defrise and Clack Algorithm for Specific CBCT Orbits
The RoboCT system enables the exploration of questions that are not feasible with traditional rotating table or gantry setups. By placing the source and detector freely around the object, non-standard trajectories (e.g., circle, helix) can be obtained, which is essential for various reasons such as complex objects, ROI reconstructions, limited angle imaging, etc. However, reconstructing data acquired with these trajectories using an FBP-based algorithm like FDK is not straightforward. Currently, deviations from the circular trajectory are reconstructed using an algebraic reconstruction technique (ART). The parameterization of ART significantly affects the reconstruction quality, especially under challenging conditions such as limited angle, sparse sampling, or truncation artefacts. ART also has drawbacks, including longer computation time compared to FBP and the need to start the iterative process after data acquisition.
Theoretical descriptions exist for FBP-based reconstruction for general trajectories [1, 2]. However, the filtering step cannot be performed with shift-invariant filter kernels like in FDK. Instead, trajectory-specific filter kernels must be derived and determined, making the process complex and time-consuming.
This master’s thesis aims to investigate the possibility of learning invariant or variant filters specific to trajectories using Known Operator Learning [3]. Building on previous work in filter learning [4,5], the implementation will be carried out using the Pyronn framework [6]. The study will also explore whether these filters can be learned purely from simulated data using a specially created phantom and their generalization to real data with different objects under the specified trajectory, based on previous research [5].
[1] Defrise, Michel, and Rolf Clack. “A cone-beam reconstruction algorithm using shift-variant filtering and cone-beam backprojection.” IEEE transactions on medical imaging 13.1 (1994): 186-195.
[2] Oeckl, Steven. “Rekonstruktionsverfahren mit der approximativen Inversen und einer neuen Formel zur Inversion der Röntgen-Transformation.” (2014).
[3] Maier, Andreas K., et al. “Learning with known operators reduces maximum error bounds.” Nature machine intelligence 1.8 (2019): 373-380.
[4] Syben, Christopher, et al. “Precision learning: reconstruction filter kernel discretization.” arXiv preprint arXiv:1710.06287 (2017).
[5] Syben, Christopher, et al. “Known operator learning enables constrained projection geometry conversion: Parallel to cone-beam for hybrid MR/X-ray imaging.” IEEE Transactions on Medical Imaging 39.11 (2020): 3488-3498.
[6] Syben, Christopher, et al. “PYRO-NN: Python reconstruction operators in neural networks.” Medical physics 46.11 (2019): 5110-5115.
Robot Movement Planning for Obstacle Avoidance using Reinforcement Learning
Obstacle avoidance for robotic arms is an important issue in robot control. Limited by factors such as equipment, cost, and labor, some application scenarios require the robot to have the ability to plan its own movement to reach the goal position or state. 
In real working environments, where obstacles’ properties are various, the traditional search algorithms are difficult to adapt to large-scale space and continuous action requirements. Artificial potential field (APF) method is a widely used obstacle avoidance path planning method, but it alsohas some shortcomings and will fall into local optimality in some special situations. Based on APF method, reinforcement learning (RL) can theoretically achieve optimization in continuous space. Combined with some modifications to traditional APF method, we define states, and actions and design the reward function, which is regarded as an important part of reinforcement learning, to form a motion planning agent in the
3D world, so that the robot end-effector is able to reach the goal position, emphasizing the avoidance of collision between obstacles and the whole robot arm.
Sinogram Analysis Using Attention U-Net: A Methodological Approach to Defect Detection and Localization in Parallel Beam Computed Tomography
The emergence of deep learning has ushered in a transformative era within the realm of image processing, notably in the context of Computed Tomography (CT). Nevertheless, it is noteworthy that a majority of image processing algorithms traditionally rely on processed or reconstructed images, often overlooking the raw sensor data. This thesis, however, shifts its focus toward the utilization of unprocessed computed tomography data, which we refer to as sinogram. Within this framework, we present a comprehensive three-step deep learning algorithm, leveraging a UNet-based architecture, designed to identify and analyze defects within objects without resorting to image reconstruction. The initial phase entails sinogram segmentation, facilitating the extraction of defect masks within the sinogram. Subsequently, instance segmentation is employed to effectively segregate these masks, resulting in their individualization. Lastly, the isolated masks are subjected to thorough defect analysis. Our research endeavors encompass comprehensive experimentation, conducted on both simulated datasets and real-world data.