Index

Image Segmentation via Transformers

The recent outburst of Transformers has started after having outperformed previously known stateof-
the-art approaches like long short-term memory and gated recurrent neural networks in sequence
modelling and transduction problems such as language modelling and machine translation. Transformers
avoid recurrence and instead rely entirely on an attention mechanism to draw global dependencies
between input and output [1]. Furthermore, Transformers are now being incorporated and tested out
in domains of computer vision tasks like classification [2], detection [3], segmentation [4] and as
generative adversarial networks (GANs) [5] by considering image-patches to have a sequence-potential.
Transformer Architecture was successfully used to perform object detection which helped drop away
many hand-designed components like a non-maximum suppression procedure or anchor generation
that explicitly encodes our prior knowledge about the task. Subsequently, it was extended for panoptic
segmentation. Although, Transformers used for segmentation did not only exploit the sequencepotential
but typically still used some form of Convolutional Neural Networks (CNNs) along with it.
However, Jiang et al. has proposed a pure Transformer based model in GAN environment (TransGAN)
for image generation ensuring the possibility of dropping CNNs in GANs [5].
In this work, the idea of using image patches as a sequence input into a Transformer model without
CNNs is carried out for segmentation tasks.
The thesis consists of the following milestones:

• Modifying TransGAN discriminator and generator as encoder and decoder respectively for
  segmentation
• Evaluating performance on the Cityscapes dataset [6].
• Further experiments and improvements regarding learning and network architecture.
  The implementation should be done in PyTorch Lightning.

[1] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser,
and Illia Polosukhin. Attention is all you need. In Proceedings of the 31st International Conference on
Neural Information Processing Systems, pages 6000–6010, 2017.

[2] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner,
Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16×16
words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020.

[3] Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey
Zagoruyko. End-to-end object detection with transformers. In European Conference on Computer Vision,
pages 213–229. Springer, 2020.

[4] Sixiao Zheng, Jiachen Lu, Hengshuang Zhao, Xiatian Zhu, Zekun Luo, Yabiao Wang, Yanwei Fu, Jianfeng
Feng, Tao Xiang, Philip HS Torr, et al. Rethinking semantic segmentation from a sequence-to-sequence
perspective with transformers. arXiv preprint arXiv:2012.15840, 2020.

[5] Yifan Jiang, Shiyu Chang, and Zhangyang Wang. Transgan: Two transformers can make one strong gan.
arXiv preprint arXiv:2102.07074, 2021.

[6] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson,
Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding.
In Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.

Manifold Forests

Random Forests for Manifold Learning

Description: There are many different methods for manifold learning, such as Locally Linear Embedding, MDS, ISOMAP or Laplacian Eigenmaps. All of them use a type of local neighborhood that tries to approximate the relationship of the data locally, and then try to find a lower dimensional representation which preserves this local relationship. One method to learn a partitioning of the feature space is by training a density forest on the data [1]. In this project the goal is to implement a Manifold Forest algorithm that finds a 1-D signal of length N in a series of N input images by learning a density forest on the data and afterwards applying Laplacian Eigenmaps on the data. For this, existing frameworks, like [2], [3], or [4] can be used as forest implementation. The Laplacian Eigenmaps algorithm is already implemented and can be integrated.

The concept of Manifold Forests is also introduced in the FAU lecture Pattern Analysis by Christian Riess, which makes candidates who have already heard this lecture preferred.

This project is intended for students wanting to do a 5 ECTS sized module like a research internship, starting now or asap. The project will be implemented in Python.

References:

[1]: Criminisi, A., Shotton, J., & Konukoglu, E. (2012). Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning. Foundations and Trends® in Computer Graphics and Vision, 7(2–3), 81–227. ; https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/CriminisiForests_FoundTrends_2011.pdf

[2]: https://github.com/CyrilWendl/SIE-Master

[3]: https://github.com/ksanjeevan/randomforest-density-python

[4]: https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomTreesEmbedding.html#sklearn.ensemble.RandomTreesEmbedding

Learning Multi-Catheter Reconstructions for Interstitial Breast Brachytherapy

Thesis Description

Female breast cancer accounts for 355.000 new cases among all types of cancer in EU-27 countries in 2020. In Germany alone, approximately 69,000 new cases are diagnosed each year [1]. During the past four decades, breast conserving surgery (BCS) after lumpectomy in combination with radiotherapy (RT) has been most widely accepted as this treatment technique reduces both a patient’s emotional as well as psychological traumata due to superior aesthetic outcome [2]. The standard technique of giving RT after BCS is whole breast irradiation (WBI) where a patient’s entire breast is irradiated up to a total dose of 40 to 50 Gray (Gy). BCS with adjuvant WBI yields evident equivalence in terms of local tumor control compared to mastectomy where the entire breast is amputated. However, approximately 50 % of early breast cancer patients still undergo mastectomy in order to omit either RT at all or 5 to 7 weeks of treatment time [3]. In contrast to external breast irradiation, accelerated partial breast irradiation (APBI) is an emerging standalone post-operative alternative treatment option in brachytherapy [4]. One valid strategy of applying APBI is multi-catheter interstitial brachytherapy (iBT). Thereby, up to 30 highly flexible plastic catheters are implanted into a patient’s breast in order to precisely and locally damage the tumor by guiding a radioactive source through the tissue. In BCT, the radioactive dose is delivered by utilizing a high dose rate (HDR) technique where the prescribed dose is administered with a rate of 12 Gy/h by single source within minutes [5]. This is performed by an afterloading system connected to the catheters via transfer tubes [4, 6]. Sole APBI is not only intended to drastically reduce treatment times to only 4 to 5 days but also to decrease the amount of radiation exposure of adjacent organs at risk (OAR) such as the lung, the skin and, in particular, the heart [7]. After implantation, catheter traces are manually reconstructed based on an acquired computed tomography (CT) image for treatment planning and determining the implant geometry. Then, in the acquired CT of the patient’s breast, physicians precisely define the target volume depending on a tumor’s size and location [6]. While treatment planning, implanted plastic catheters are manually reconstructed slice by slice which takes approximately 45% of the whole treatment time [8]. Along each catheter trajectory dwell positions (DPs) connecting the points in the slices as well as dwell times (DTs) are defined. DPs determine positions where the radioactive source stops for a certain DT, thus irradiation surrounding tumor tissue. Active DPs and DTs are defined at the location of the target volume to optimally deliver prescribed radioactive dose [9]. As treatment plan dosimetry and DP positioning are directly related, accurate and fast catheter trace reconstructions are crucial [4].

However, the manual reconstruction of up to 30 catheter tubes is a time-consuming process. Kallis et al. state that manual reconstructions on average take up to 139 ± 47 seconds(s) per catheter. They also observed an interobserver variability of 0.6 ± 0.35 millimeter (mm) in terms of mean Euclidean distance between two experienced medical physicists and the autoreconstruction approach proposed by [8], thus, yielding reproducible and reliable reconstructions [6]. Similar findings were proven by Milickovic et al. in 2001 [10]. The insufficient amount of ground truth catheter trace positions as well as blurry CT imaging quality make it hard to reliably and accurately reconstruct DPs. Hence, this suggests further research to conducting automated reconstruction approaches [10].

In the last 20 years, mainly two different catheter auto-reconstruction approaches were proposed. Both techniques aim to minimize the error of implant geometries, thus, improve optimal dose coverage as well as drastically reduce reconstruction times. Milickovic et al. developed an automated catheter reconstruction algorithm based on analyzing post-implant CT data [8, 10]. However as stated by Kallis et al., CT based treatment planning in multi-catheter iBT highly depends on image quality. Due to patient movements, artifacts, as well as acquisition noise, automatically extracted DPs have to be corrected by manual intervention which increases reconstruction times [6]. As introduced by Zhou et al. in 2013, electromagnetic tracking (EMT) became a promising alternative compared to CT based auto-reconstruction [11]. Further analysis has proven that EMT is applicable to iBT as this technique of localizing dwell positions in iBT offers sparse, precise, and sufficiently accurate dose calculations [12]. Reducing uncertainties including measurement noise is investigated by postprocessing of sensor data by particle filters. In their work, a mean error of 2.3 mm between clinically approved plan and reconstructed DPs has been reported [13]. Although tracking multi-catheter positions in iBT based on EMT offers imaging artifact independent and fast results, the performance of EMT systems depends heavily on system configurations, e.g. the distance between CT table and patient bed. The error drastically increases from approximately 1 to 4 mm when decreasing the table/bed distance [12].

In recent years, deep learning (DL) has shown to be a powerful technique tackling a variety of computer vision tasks such as medical image analysis. DL based approaches offer highly competitive results in terms of accuracy and efficiency [14, 15]. Deep neural network (DNN) model architectures are able to represent high dimensional non-linear spaces, thus are well suited for the task of automatically reconstructing multi-catheter traces in iBT. Built upon an elegant way of designing DNN architectures – so called Fully Convolutional Networks (FCN) [16] – the UNet architecture has proven to be well suited for image based segmentation tasks as this specific model structure’s output has the same shape as the input [17]. C¸i¸cek et al. developed an extended version of the UNet where all 2D operations are replaces with corresponding 3D ones. This topological modification enables volumetric semantic segmentations [18]. In this Master’s thesis a deep learning based multi-catheter reconstruction method for iBT is presented, investigated, and evaluated using real world breast cancer data from the radiation clinic in Erlangen, Germany. To the best of our knowledge this is the first approach where we introduce artificial intelligence based multi-catheter reconstruction algorithm in breast brachytherapy.

References

1. Jacques Ferlay et al. Global cancer observatory: Cancer today. https://gco.iarc.fr/today. Accessed: 2021-03-22.
2. Csaba Polg´ar et al. High-dose-rate brachytherapy alone versus whole breast radiotherapy with or without tumor bed boost after breast-conserving surgery: Seven-year results of a comparative study. International journal of radiation oncology, biology, physics, 60:1173–81, 12 2004.
3. Vratislav Stranad et al. 5-year results of accelerated partial breast irradiation using sole interstitial multicatheter brachytherapy versus whole-breast irradiation with boost after breast-conserving surgery for low-risk invasive and in-situ carcinoma of the female breast: a randomised, phase 3, non-inferiority trial. The Lancet, 387(10015):229–238, 2016.
4. Vratislav Strnad, R. P¨otter, G. Kov´acs, and T. Block. Practical Handbook of Brachytherapy. UNI-MED Science. UNI-MED-Verlag, 2010.
5. Daniela Kauer-Dorner and Daniel Berger. The role of brachytherapy in the treatment of breast cancer. Breast Care, 13, 05 2018.
6. Karoline Kallis et al. Impact of inter- and intra-observer variabilities of catheter reconstruction on multicatheter interstitial brachytherapy of breast cancer patients. Radiotherapy and Oncology, 135:25–32, 06 2019.
7. Vratislav Strnad et al. Estro-acrop guideline: Interstitial multi-catheter breast brachytherapy as accelerated partial breast irradiation alone or as boost – gec-estro breast cancer working group practical recommendations. Radiotherapy and Oncology, 128, 04 2018.
8. Milickovic et al. Catheter autoreconstruction in computed tomography based brachytherapy treatment planning. Medical Physics, 27(5):1047–1057, 2000.
9. Cheng B. Saw, Leroy J. Korb, Brenda Darnell, K.V. Krishna, and Dennis Ulewicz. Independent technique of verifying high-dose rate (hdr) brachytherapy treatment plans. International Journal of Radiation Oncology*Biology*Physics, 40(3):747–750, 1998.
10. Natasa Milickovic, Dimos Baltas, and Nikolaos Zamboglou. Automatic reconstruction of catheters in ct based bracytherapy treatment planning. In ISPA 2001. Proceedings of the 2nd International Symposium on Image and Signal Processing and Analysis. In conjunction with 23rd International Conference on Information Technology Interfaces (IEEE Cat., pages 202–206, 2001.
11. Jun Zhou, Evelyn Sebastian, Victor Mangona, and Di Yan. Real-time catheter tracking for high-dose-rate prostate brachytherapy using an electromagnetic 3d-guidance device: A preliminary performance study. Medical Physics, 40(2):021716, 2013.
12. Markus Kellermeier, Jens Herbolzheimer, Stephan Kreppner, Michael Lotter, Vratislav Strnad, and Christoph Bert. Electromagnetic tracking (emt) technology for improved treatment quality assurance in interstitial brachytherapy. Journal of Applied Clinical Medical Physics, 18:211–222, 01 2017.
13. Theresa Ida G¨otz et al. A tool to automatically analyze electromagnetic tracking data from high dose rate brachytherapy of breast cancer patients. PLOS ONE, 12(9):1–31, 09 2017.
14. Florian Kordon et al. Multi-task localization and segmentation for x-ray guided planning in knee surgery. In Dinggang Shen, Tianming Liu, Terry M. Peters, Lawrence H. Staib, Caroline Essert, Sean Zhou, Pew-Thian Yap, and Ali Khan, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, pages 622–630, Cham, 2019. Springer International Publishing.
15. Florian Kordon, Ruxandra Lasowski, Benedict Swartman, Jochen Franke, Peter Fischer, and Holger Kunze. Improved x-ray bone segmentation by normalization and augmentation strategies. In Heinz Handels, Thomas M. Deserno, Andreas Maier, Klaus Hermann Maier-Hein, Christoph Palm, and Thomas Tolxdorff, editors, Bildverarbeitung fu¨r die Medizin 2019, pages 104–109, Wiesbaden, 2019. Springer Fachmedien Wiesbaden.
16. Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. CoRR, abs/1411.4038, 2014.
17. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi, editors, Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, pages 234–241, Cham, 2015. Springer International Publishing.
18. Ozgu¨n C¸i¸cek, Ahmed Abdulkadir, Soeren S. Lienkamp, Thomas Brox, and Olaf Ronneberger. 3d u-net:¨ Learning dense volumetric segmentation from sparse annotation. CoRR, abs/1606.06650, 2016.

Abnormality detection on musculoskeletal radiographs

Thesis Description

The primary objective of this thesis project is to develop an algorithm that can determine whether a musculoskeletal X-ray study is normal or abnormal. For this purpose, we only consider X-rays of the upper extremities including the shoulder, humerus, elbow, forearm, wrist, hand, and finger. By abnormalities we consider fractures, hardware, degenerative joint diseases, lesions, subluxations, and other deviations from the standard structural composition and morphology. Given an X-ray image as an input, the devised algorithm should output a labeled image which indicates the presence or absence of an abnormality.  Such a system could be used to enhance the confidence of the radiologist or prioritize subsequent analysis and treatment options.

The task to determine abnormality on musculoskeletal radiographs is particularly critical since more than 1.7 billion people around the globe are affected by musculoskeletal conditions [12]. Since a radiograph is the cheapest, best available and usually the first measure to detect musculoskeletal abnormalities, automatic detection and localization of such potential abnormalities enables a faster initial diagnosis, saves valuable time for physicians, and reduces the number of subsequent diagnostic treatments required on the patient.  This will also reduce the work pressure and fatigue of radiologists [10] which is caused by overwhelming number of X-ray studies they have to diagnose every day [11].

In this project we will use a large public data set called ‘MURA-v1.1’ published by Stanford Machine Learning Group of Stanford University [1]. The data set consists of 14,863 studies from 12,173 patients with a total of 40,561 multi-view radiographic images. Board-certified radiologists from Stanford Hospital manually labeled the radiographs as normal or abnormal. Out of 14,863 studies 9,045 are normal and 5,818 are abnormal.

The project is structured into three parts. First, a learning-based classification algorithm is used to predict whether a radiograph is normal or abnormal [1,2]. Second, anatomical information derived from the dataset’s annotation is incorporated to additionally predict the anatomical origin of the radiograph [3,4,6,7,8]. In a last step, the abnormality is localized and visualized by incorporating the results from the previous steps in combination with targeted feature space analysis. All components should then be combined to a framework capable to predict, localize and visualize musculoskeletal abnormality. Algorithmic development is based on recent advances in deep learning techniques building upon the DenseNet [9] and ResNet [13] neural network architecture. A main aspect of the work is the conception and implementation of an integration strategy of additional anatomical information. It shall also be analyzed to what extent this information can support and improve the classification of abnormal and normal radiographs. Prior work of multi-task/multi-label optimization is investigated and examined for applicability to this project’s task [3,4,5,6,7].  The project is fixed to a six-month period timeline and will be concluded by a detailed project report. Technical implementation of the prototype will be performed within the PyTorch environment for the Python programming language.

References

1. Rajpurkar P., Irvin J., Bagul A., Ding D., Duan T., Methta H., Yang B., Zhu K., Laird D., Ball R., Langlotz C., Shpanskaya K., Lungren M., Ng A. , “MURA: Large Dataset for Abnormality Detection in Musculoskeletal Radiographs” 1st Conference on Medical Imaging with Deep Learning (MIDL 2018)
2. Guendel S., Grbic S., Gerogescu B., Zhou K., Ludwig R., Meier A., “Learning to recognize abnormalities in chest x-rays with location aware dense networks.” arxiv preprint arXiv:1803.04565 ,2018
3. Guendel S., Ghesu F., Grbic S., Gibson E., Gerogescu B., Maier A.,“ Multi-task Learning for Chest X-ray Abnormality Classification on Noisy Labels “ arxiv preprint arXiv:1905.06362 ,2019
4. Yang, X., Zeng, Z., Yeo, S.Y., Tan, C., Tey, H.L., Su, Y., “A novel multi-task deep learning model for skin lesion segmentation and classification.” arxiv preprint arXiv:1703.01025 ,2017
5. Vesal S., Ravikumar N., Maier A., ‘‘A Multi-task Framework for Skin Lesion Detection and Segmentation’’ arxiv preprint arXiv:1808.01676 ,2018
6. Kendall A., Gal Y., Cipolla R.,”Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics” arxiv preprint arXiv:1705.07115 ,2017
7. Vandenhende S., Brandere B., Gool Luc., “Branched Multi-Task Networks: Deciding What Layers To Share“ arxiv preprint arXiv:1904.02920 ,2019
8. Berlin L., ”Liability of interpreting too many radiographs.” American Journal of Roentgenology, 175(1):17–22, 2000
9. Huang G., Liu Z., Weinberger K.Q., and van der Maaten, Laurens, “Densely connected convolutional networks.” arXiv preprint arXiv:1608.06993, 2016.
10. Lu Y., Zhao S., Chu P.W., and Arenson R.L., “An update survey of academic radiologists’ clinical productivity.” Journal of the American College of Radiology, 5(7):817–826, 2008.
11. Nakajima Y., Yamada K., Imamura K., and Kobayashi K.. ,”Radiologist supply and workload: international comparison.” Radiation medicine, 26(8):455–465, 2008.
12. URL http://www.boneandjointburden.org/2014-report.
13. He K., Zhang X., Ren S., Sun J., “Deep Residual Learning for Image Recognition” arxiv preprint arXiv:1512.03385

Localization and Standard Plane Regression of Vertebral Bodies in Intra-Operative CBCT Volumes

Automatic detection of standard planes in surgical FD-CT volumes

Thesis Description

Intra-articular fractures are commonly treated by open reduction and internal fixxation (ORIF). This procedure comprises first, reorient the bone fracture into the normal position and secondly fix it using metal screws, plates or rods. Malreduction of the fracture, intra-articular position of the screws, remaining gaps, or steps offs may lead to malunion or post-traumatic osteoarthritis. The use of mobile C-arms to acquire 3D images during intervention has become a standard since it enables the evaluation of fractures of complex anatomical regions. Two dimensional images often lack information about fracture reduction and implant position in non-planar joints [1]. This includes fractures of the tibial head as well as the calcaneus, ankle injuries involving the syndesmosis or spinal injuries among others. After fracture treatment, if the surgeon is not satisfied with the result, a correction can be made within the frame of the intervention and can avoid the patient a revision surgery in the future. Several studies show intraoperative revision rates depending on the anatomical region up to 40% [2].

Acquisition of standard planes that contains key anatomical structures is decisive for the assessment of intervention results. Multiplanar reconstruction (MPR) is the standard method for reconstruction of the 3D image which allows the generation of slices from arbitrary viewpoint and orientation. Absence of information about position between patient and the C-arm device results in the need for adjustment of the standard planes at a workstation in the operating room. Till now, surgeons must manually find standard planes orientation and position which takes from 46 to 55 second depending
on the experience level of the surgeon and can thus be considered a time-consuming and complicated task [3].

No methods for the fully automatic adjustment of standard planes of mobile C-arms have been described. However, it is possible to find several works in other modalities as ultrasound. In [4] a CNN is used to detect transventricular and transcerebrall standard planes in fetal brain ultrasound. The network learns the mapping between a 2D plane, and the transformation required to move the plane towards the standard plane in the volume. Another approach used in [5] is based on reinforcement learning approach to automatically localize transthalamic and transcerebellar standard planes in 3D fetal ultrasound.

This thesis aims to design a framework for the automatic adjustment of standard planes in different anatomical joint regions using deep learning algorithms. The thesis will comprise the following work items:

• Literature overview of state-of-the-art automatic standard plane adjustment
• Characterization of standard planes for different anatomical regions
• Design and formalization of the to be developed method
• Overview and explanation of the algorithms used
• Implementation of the plane detection framework
• Evaluation of results

References

[1] Paul Alfred Grützner. Rontgenhelfer 3D: Handbuch intraoperative 3D-Bildgebung mit mobilen C-Bögen. Bengelsdorf & Schimmel, 2004.

[2] Jochen Franke, Klaus Wendl, Arnold J Suda, Thomas Giese, Paul Alfred Grützner, and Jan von Recum. Intraoperative three-dimensional imaging in the treatment of calcaneal fractures. JBJS, 96(9):e72, 2014.

[3] Michael Brehler, Joseph Gorres, Jochen Franke, Karl Barth, Sven Y Vetter, Paul A Grützner, Hans-Peter Meinzer, Ivo Wolf, and Diana Nabers. Intra-operative adjustment of standard planes in C-arm CT image data. International journal of computer assisted radiology and surgery, 11(3):495-
504, 2016.

[4] Yuanwei Li, Bishesh Khanal, Benjamin Hou, Amir Alansary, Juan J Cerrolaza, Matthew Sinclair, Jacqueline Matthew, Chandni Gupta, Caroline Knight, Bernhard Kainz, et al. Standard plane detection in 3D fetal ultrasound using an iterative transformation network. In International Conference on Medical Image Computing and Computer-Assisted Intervention, p. 392-400. Springer, 2018.

[5] Haoran Dou, Xin Yang, Jikuan Qian, Wufeng Xue, Hao Qin, Xu Wang, Lequan Yu, Shujun Wang, Yi Xiong, Pheng-Ann Heng, et al. Agent with warm start and active termination for plane localization in 3D ultrasound, 2019.

Micro CT Denoising Using Low Parameter Models

Computed Tomography (CT) is widely used as a diagnostic tool due to its ability to acquire three-dimensional structures while preserving great bone-soft tissue contrast. Motivated by these contrast properties, it is instructive to use high-resolution CT imaging (Micro CT) in preclinical osteoporosis research to resolve bone structures in mice. Especially in vivo Micro CT imaging of mouse tibia bones is interesting for understanding osteoporosis and developing a medication [1]. However, radiation dose and image quality are strongly connected. A significant amount of radiation must be deposited in the imaged object to acquire a desired contrast. When scanning a living animal, the deposited energy will harm the tissue and increase the risk of cancer and other diseases. Therefore, minimizing the dose is crucial, which is usually connected to degraded image quality.
Using denoising algorithms can leverage image quality. Here, iterative reconstruction algorithms have been successfully applied in the past. While their algorithms are usually based on reasonable statistical assumptions, these methods are computationally costly and limited in their denoising performance. In recent years, deep learning approaches have shown promising results in terms of image quality.

The goal of this master thesis is to use the deep learning-based joint bilateral filtering (JBFnet) [2] to denoise Micro CT data of mouse tibia bones. The JBFnet is a promising approach for denoising Micro CT data as it requires only a few trainable parameters while achieving state-of-the-art denoising performance. Hence, the integrity of the denoised structures can be claimed which is crucial considering the tiny bone features that shall be restored. After achieving reasonable denoising results, multiple modifications of the JBFnet are planned to adapt the filtering better to the respective noise characteristics of the data. In the last part of the thesis, an extensive performance evaluation of the network and its modifications will be performed.

[1] A. Grüneboom, L. Kling, S. Christiansen, L. Mill, A. Maier, K. Engelke, H. H. Quick, G. Schett, and M. Gunzer, “Next-generation imaging of the skeletal system and its blood supply,” Nature Reviews Rheumatology, vol. 15, no. 9, pp. 533–549, 2019.

[2] M. Patwari, R. Gutjahr, R. Raupach, and A. Maier, “Jbfnet-low dose ct denoising by trainable joint bilateral filtering,” in International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 506–515, Springer, 2020.

Multi-scale Tissue Segmentation on Canine Cutaneous Tumors

Cutaneous tumors, i.e., tumors originating from skin cells, are one of the most common tumor types in dogs [1]. As cost efficiency is an important driver in animal care, it would be strongly beneficial to support veterinary pathologists in the diagnosis of those tumors and their respective subtypes. Besides the use in a decision-support system, computerized segmentation and classification of tumors can potentially increase precision for therapeutic options and – through quantitative evaluation – provide new insights into tumor development with inter-species relevance, which includes humans.

This thesis aims to perform tissue segmentation from a data set comprising of the nine most common canine cutaneous tumor types of whole slide images. A particularly challenging component is the combination of predictions performed for differing magnifications: while some tissue types can be spotted with higher accuracy using lower magnifications, for others human experts will utilize higher magnifications, especially for fine-grained differentiation versus neighboring tissue segments.

The thesis comprises the following items:

• Literature review concerning detection on multi-scale images
• Training of state-of-the-art segmentation networks on multiple scales of microscopy images
• Analysis of detection results on single scales
• Development of a multi-scale fusion system, achieving high precision at multiple image scales
• Documentation and presentation of the findings, documentation of code

[1] Murphy, S. (2006). Skin neoplasia in small animals 3. Common canine tumours. In practice, 28(7), 398-402.

Real-Time Prospective Respiratory Triggering for Free-Breathing Lung Computed Tomography

Thesis Description

Respiratory diseases are among the leading causes of death, according to the World Health Organization. With more than 3 million deaths in 2016, chronic obstructive pulmonary disease is the third leading cause of death worldwide [1]. An important tool for diagnosing respiratory diseases is computed tomography of the lungs [2]. The current state of the art approach for this is breath-hold CT, which requires patients to follow breathing cues and hold their breath on command [3]. This procedure is not appropriate for certain groups of patients who are unable to follow instructions. Some of these patients are unconscious, mentally impaired, or are infants and young children [4]. In these cases, medication must be administered to stop the breathing so that sufficient scans can be acquired. This not only carries risks for the patient, but also requires additional clinical staff [4, 5, 6].

To improve these issues, free-breathing computed tomography has been proposed. In this approach, scans are performed while the patient continues to breathe [4, 5, 7]. Because the lungs move during the scan, this method produces images with more artifacts, compared to the breath-hold approach. To optimize image quality and for comparability between scans, it would be beneficial to scan at times with little lung movement, such as during inhalation and exhalation. One way to achieve comparability is retrospective respiratory triggering, which uses the respiratory waveform to select the correct phase after CT images are acquired. However, this is not ideal for clinical use because of the high radiation dose involved. To achieve lower radiation exposure, prospective respiratory triggering utilizes a shorter duration scan triggered by a respiratory gating device. [8, 9]

To address the challenges associated with the continuous lung movement, two recording modes are further investigated.

• Sequential mode acquires images without CT couch motion, with image size limited by collimator width.  Multiple images are stacked to cover a larger area. This is expected to work better for children, who tend to have a high respiratory rate and a small lung area to scan [10].
• The flash spiral mode utilizes a technique previously used for cardiac imaging with a high-pitch spiral CT [11]. This makes it possible to scan the entire lung in one pass in less than one second. The advantages of this high pitch mode for lung imaging have already been demonstrated [4, 5, 8]. Promising results were achieved with regularly breathing patients, although the limitations of triggering with respect to irregular breathing were noted by Goo et al. [8]

For both of these approaches, triggering algorithms expand on phase space based respiratory triggering as presented by Werner et al.. A set of criteria is used to define target regions in the phase space representation of the respiratory signal data in order to emit a trigger depending on amplitude and velocity [12]. The goal is to achieve the best possible robustness on patient data with a focus on especially challenging breathing patterns.

This thesis deals with the following work items:

1. Customization of respiratory triggering algorithms to both acquisition modes
   1. Sequential respiratory triggering
   2. Flash spiral respiratory triggering
2. Evaluation of robustness against other approaches
   1. Commercial respiratory gaiting
   2. Possibly: Reinforcement learning based triggering
3. Image based validation of the reconstruction result
   1. With simulated breathing signals
   2. With breathing signals from real patients

References

[1] World Health Organization (WHO) et al. Global health estimates 2016: estimated deaths by age, sex and cause. Geneva: WHO, 2018.

[2] Darel E Heitkamp, Matthias M Albin, Jonathan H Chung, Traves P Crabtree, Mark D Iannettoni, Geoffrey B Johnson, Clinton Jokerst, Barbara L McComb, Anthony G Saleh, Rakesh D Shah, et al. Acr appropriateness criteria® acute respiratory illness in immunocompromised patients. Journal of thoracic imaging, 30(3):W2–W5, 2015.

[3] Toshizo Katsuda, Shigeru Eiho, Chikazumi Kuroda, and Tsutomu Hashimoto. Analysis of breath holding for lung ct imaging. Radiography, 11(4):235–241, 2005.

[4] Michael M Lell, Michael Scharf, Achim Eller, Wolfgang Wuest, Thomas Allmendinger, Florian Fuchs, Stephan Achenbach, Michael Uder, and Matthias S May. Feasibility of respiratory-gated high-pitch spiral
ct:: Free-breathing inspiratory image quality. Academic radiology, 23(4):406–412, 2016.

[5] Ilias Tsiflikas, Christoph Thomas, Dominik Ketelsen, Guido Seitz, Steven Warmann, Claus Claussen, and Juergen Schaefer. High-pitch computed tomography of the lung in pediatric patients: an intraindividual comparison of image quality and radiation dose to conventional 64-mdct. RoFo, 186(6):585–590, 2014.

[6] Mike Sury, Ian Bullock, Silvia Rabar, and Kathleen DeMott. Sedation for diagnostic and therapeutic procedures in children and young people: summary of nice guidance. Bmj, 341, 2010.

[7] Hyun Woo Goo. Combined prospectively electrocardiography-and respiratory-triggered sequential cardiac computed tomography in free-breathing children: success rate and image quality. Pediatric radiology, 48(7):923–931, 2018.

[8] Hyun Woo Goo and Thomas Allmendinger. Combined electrocardiography-and respiratory-triggered ct of the lung to reduce respiratory misregistration artifacts between imaging slabs in free-breathing children: initial experience. Korean journal of radiology, 18(5):860, 2017

[9] Cyrus Behzadi, Michael Groth, Frank Oliver Henes, Dorothee Schwarz, Andr´e Deibele, Philipp GC Begemann, Gerhard Adam, and Marc Regier. Intraindividual comparison of image quality using retrospective
and prospective respiratory gating for the acquisition of thin sliced four dimensional multidetector ct of the thorax in a porcine model. Experimental lung research, 41(9):489–498, 2015.

[10] Edmond A Hooker, Daniel F Danzl, Mary Brueggmeyer, and Edith Harper. Respiratory rates in pediatric emergency patients. The Journal of emergency medicine, 10(4):407–410, 1992.

[11] Stephan Achenbach, Mohamed Marwan, Tiziano Schepis, Tobias Pflederer, Herbert Bruder, Thomas Allmendinger, Martin Petersilka, Katharina Anders, Michael Lell, Axel Kuettner, et al. High-pitch spiral
acquisition: a new scan mode for coronary ct angiography. Journal of cardiovascular computed tomography, 3(2):117–121, 2009.[12] Ren´e Werner, Thilo Sentker, Frederic Madesta, Tobias Gauer, and Christian Hofmann. Intelligent 4d ct sequence scanning (i4dct): concept and performance evaluation. Medical physics, 46(8):3462–3474, 2019.

Deep-learning-based behaviour prediction of rear-end road users when changing lane as a system design reference for highly automated driving

Motivation

Until fully automated vehicles reach full market saturation, a mixed operation between human-driven and highly automated vehicles will characterize traffic activities. Mutual understanding of driving intentions is therefore crucial for harmonizing road traffic.

The prediction of driving intentions of other road users is a subject of numerous scientific researches and is the link between environmental perception and maneuver planning. The driving environment is determined by kinematic vehicle parameters and their temporal history and as well as the context of the traffic situation. Based on these, predictions about future trajectories of the surrounding vehicles are made possible and one’s own target behavior can be derived in the form of a target trajectory.

However, the influence of one’s own driving behaviour on other road users is only part of very few investigations yet. For the acceptance of highly automated driving functions, it is not only essential that the driving behaviour is perceived safe and comfortable by passengers of a highly automated vehicle, but also predictable by other road users.

Approach

The thesis aims to gain specific insights on interactions between road users. The core target of this thesis is to train a model, which describes how a lane change influences the behaviour of rear-end road users on highways. The highD dataset will be used to build a ‘Deep Learning’ model, which learns the dependencies between lane changes and the reactions caused by it.

The goals of this thesis are:

• Defining the scenario
• Identifying the relevant input and output parameters for the deep learning module
• Creating and training of a suitable model
• Using the developed model to provide a reference for mutual interactions between road users and to derive possible behavioural patterns
• Assessing the impact on rear road users when changing lane for highly automated driving