Index

Knowledge distillation for landmark segmentation in medical image analytics

Motivation: One major challenge that comes with deep learning approaches in medical image processing is the issue of high cost of expert annotated data. Therefore semi-supervised learning approaches are of high interest in the research community.

Methods: The application of the work concerns landmark segmentation by heatmap re-gression in thorax diagnostics. To keep the need of annotated data low an approach of knowledge distillation with application of a teacher-student concept will be persued. The essence of this method is to transfer knowledge from a pre-trained model or an ensemble ofmodels to a new model. Originally this technique was introduced to reduce the capacity of huge good-performing networks while keeping the accuracy [1]. But it has also already been applied for semi-supervised learning purposes [2], what will be further investigated in this work. Aim of this master theses is to examine the benefit of student-teacher approaches in semi-supervised learning. For this purpose different variants of this method will be considered, implemented and compared to each other. This will be done in cooperation and with provision of data and infrastructure by Siemens Healthineers.

The Master’s thesis covers the following aspects:

1. Literature research on state-of-the-art methods

2. Set up infrastructure and data used for the project

3. Implementation of algorithms in the framework

4. Training and tuning of hyper-parameters

5. Performance comparison of the different variants

6. Evaluation of developed algorithms

[1] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015.

[2] S. Sedai, B. Antony, R. Rai, K. Jones, H. Ishikawa, J. Schuman, W. Gadi, and R. Garnavi, “Uncertainty guided semi-supervised segmentation of retinal layers in oct images,”in International Conference on Medical Image Computing and Computer-Assisted Inter-vention, pp. 282–290, Springer, 2019.

Deep Learning Classification and Optimization of Manufacturing Process Parameters

At Mercedes-Benz AG, “Die Casting” is an integral manufacturing process in which car parts are manufactured by forcing molten metal under high pressure and speed into a die cavity. The casting machines produce with a high number of process parameters, which are currently being tweaked by the engineers solely based on their experience to ensure that the process is as robust as possible. Hence, few defective parts are produced initially until the number of bad quality parts increases again and new adjustment is needed. Improvement of this procedure will lead to higher efficiency in this casting process. Owing to this, the thesis aims to build up a relationship between the process parameters and the quality measurements through data-driven modeling, and then later on use the learned mapping for finding an optimal set of parameters which maximizes the probability for producing a correct part.
From the last few decades, Artificial Neural Networks have been quite successful in capturing the complex, often non-linear, relationship which exists between the process parameters and the process output [1]. Therefore, the first goal in this thesis is fitting the given data with different neural network architectures such as Multi-Layer Perceptron [2], Residual Network [3], fine-tuning them and then selecting the best performing architecture out of them in terms of correctly mapping the process parameters to the ground-truth class labels.
Two major challenges in the first objective of the thesis are:
1. Pre-Processing of raw data which consists of a combination of categorical and numerical variables
2. Class Imbalance which poses a big challenge for unbiased training of neural networks [4]
Furthermore, the final step in the thesis would be to use our trained neural network for optimizing the input parameters in such a way that the probability of producing a good part is maximized by those parameters. One possible proposal to achieve this is by generating adversarial attacks on the neural network [5] where the input parameters are iteratively modified in the negative direction of the gradient of the loss function between the predicted class and the desired output class while the weights of the network are kept constant. Hence, instead of doing backpropagation onto the weights, backpropagation is performed onto the input parameters. Another popular approach in the literature that has shown satisfactory results in the manufacturing industry is to use the learned neural network mapping as the fitness function for evaluating candidate solutions of the optimal parameter set in a genetic algorithm [6].
In summary, the thesis will include the following points:
1. Training and Evaluation of different neural network architectures over the given data
2. Optimization of Process Parameters through the best-performing trained model
Supervisors: Prof. Dr. Andreas Maier, Olga Moreva, Lyubka Rund, Stephan Schwarz
Student: Saad Munir
Date: June 3, 2021 -November 30, 2021
References [1] Sukthomya, W., Tannock, J. The training of neural networks to model manufacturing processes. J Intell Manuf 16, 39–51 (2005). [2] Marius, Popescu & Balas, Valentina & Perescu-Popescu, Liliana & Mastorakis, Nikos. (2009). Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems. 8. [3] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. [4] Ling C.X., Sheng V.S. (2011) Class Imbalance Problem. In: Sammut C., Webb G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. [5] Qiu, Shilin; Liu, Qihe; Zhou, Shijie; Wu, Chunjiang. 2019. “Review of Artificial Intelligence Adversarial Attack and Defense Technologies” Appl. Sci. 9, no. 5: 909.
[6] Julius Pfrommer, Clemens Zimmerling, Jinzhao Liu, Luise Kärger, Frank Henning, Jürgen Beyerer, Optimisation of manufacturing process parameters using deep neural networks as surrogate models, Procedia CIRP, Volume 72, 2018, Pages 426-431, ISSN 2212-8271.

Prediction of Steam Turbine Blade Vibration Amplitudes using Machine Learning Methods

Thesis Description

During the typical start-up process of the steam turbine, after the nominal speed has been reached, the massfow is continuously increased until the setpoint is reached. Due to the very low ow at the beginning of the start-up process, instationary ow phenomena can occur which can cause elevated blade vibrations [1]. These flow phenomena can also occur in low-load operations. It is important to simulate and assess these instationary flow phenomena and the efects on the blade vibrational behaviour of the blades [2]. For this purpose, both experimental model turbine investigations and transient 3D Computational Fluid Dynamics (CFD) simulations are carried out, which provide a variety of time-dependent data. As a result, the data includes the operating status such as mass ow, pressures, and temperatures, the excitation forces such as pressure changes and frequencies, and vibration data such as amplitudes and modes.
First of all, the existing experimental data will be analyzed. The aim is to investigate whether a powerful method for predicting vibration amplitudes of the blades can be formulated. If necessary, additional data must be generated with the help of CFD simulations to enable an assessment of the Machine Learning (ML) [3] processes for the entire design space. Based on the data analysis, suitable ML algorithms should be identified and tested. Here, for example, Multi-layer Perceptron Regressor (MLP Regressor) [4, 5], Random Forests [6], Decision Tree Regressor (DTR) [7], Support Vector Regressor (SVR) [8], etc. can be used and tested. In particular, ML algorithms are to be tested that are suitable for mapping time series, such as Recurrent Neural Networks (RNN) [9]. The predictive quality of the individual processes should be assessed especially with respect to the entire design space.

In summary, the thesis deals with the following points:

1.  Analysis of the existing experimental data with respect to:
   (a) Coverage of the design space (sampling)
   (b) Dimensionality refinement of the design space
2. Application of suitable ML algorithms:
   (a) Various regression models
   (b) Deep Learning for mapping time series
3.  Analysis of the generated CFD data by Machine Learning Models:
   (a) Examining on how a further improvement of the prediction quality can be achieved by generating data by CFD analysis and correlating with the measured data points

References
[1] Neville Rieger. Progress with the solution of vibration problems of steam turbine blades. 05 2021.
[2] R.S. Mohan, A. Sarkar, and A.S. Sekhar. Vibration analysis of a steam turbine blade. INTERNOISE 2014
– 43rd International Congress on Noise Control Engineering: Improving the World Through Noise Control,
01 2014.
[3] Alfredo Rodriguez, Youness El Hamzaoui, J A Perez, Juan Garcia, Jose Flores-Chan, and A.L. Tejeda. The
use of artificial neural network (ann) for modeling the useful life of the failure assessment in blades of steam
turbines. Engineering Failure Analysis, 35:562{575, 06 2013.
[4] Rishabh Bhardwaj, Navonil Majumder, and Soujanya Poria. Investigating gender bias in bert, 09 2020.
[5] Tejas Subramanya, Davit Harutyunyan, and Roberto Riggio. Machine learning-driven service function chain
placement and scaling in mec-enabled 5g networks. Computer Networks, 166:106980, 11 2019.
[6] Andy Liaw and Matthew Wiener. Classification and regression by randomforest. Forest, 23, 11 2001.
[7] Wei-Yin Loh. Classification and regression trees. Wiley Interdisciplinary Reviews: Data Mining and Knowl-
edge Discovery, 1:14 { 23, 01 2011.
[8] Mariette Awad and Rahul Khanna. Support Vector Regression, pages 67{80. 01 2015.
[9] Alex Sherstinsky. Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm)
network. Physica D: Nonlinear Phenomena, 404:132306, 03 2020.

Terahertz Image Reconstruction for Historical Document Analysis

Scene Evolution on Polarimetric Radar Data in Automated Driving Scenarios

Description

The task of autonomous driving demands vehicles to create a precise understanding of their surroundings. Currently, camera, LiDAR, and radar sensors are used to perceive the vehicle’s environment. While conventional automotive radar sensors have the advantage of instantly detecting velocities and are less prone to disturbances by difficult weather conditions than the aforementioned alternatives, they also offer a lower resolution and therefore less information for classification and tracking tasks. [1]

To compensate for this weakness, the usage of newly available polarimetric radar sensors for the automotive domain is researched. Polarimetric radar sensors emit and detect radar waves with different polarizations. The analysis of polarization changes adds additional information about reflection patterns, allowing, inter alia, estimation of vehicle orientation and extend in traffic scenarios. [2]

Prior research also indicates improvements in classification tasks using polarimetric radar data of stationary targets in test areas [3] and stationary and moving traffic participants in real-world urban scenarios [4]. In this work, polarimetric radar data is used to track and predict the evolution of the vehicle’s environment over time.

The thesis consists of the following milestones:

• Implement and compare optical flow methods to create correlation images from polarimetric and conventional radar imagery
• Implement and compare track generation methods using the correlation images
• Implement and compare track prediction methods using the generated tracks
• Evaluate advantages of polarimetric radar information

References

[3] Tristan Visentin.Polarimetric Radar for Automotive Applications, volume 90 ofKarlsruher Forschungs-berichte aus dem Institut f ̈ur Hochfrequenztechnik und Elektronik. KIT Scientific Publishing, Karlsruhe,Baden, 2019.
[4] J. F. Tilly, F. Weishaupt, O. Schumann, J. Dickmann, and G. Wanielik. Road user classification with polarimetric radars. In2020 17th European Radar Conference (EuRAD), pages 112–115, 2021.

Detection of In-plane Rotation of Extremities on X-ray Images

Thesis Description

A core goal in a medical imaging pipeline is to optimize the workflow for improving the patient throughput at a radiography system. By limiting manual tasks in a workflow, we can efficiently optimize the pipeline [1]. A common manual task in a X-ray radiography workflow is to manually rotate digital X-ray images to a canonical orientation preferred by radiologists. This has an impact on the number of patients that can be analyzed in a given period of time. A Deep Learning based detection system for the in-plane rotation of body parts in X-ray images can solve this problem, but it is still an open research topic. We identified three major challenges that such automatic systems need to address: First, in clinical routine there are up to 23 different examinations, consisting of 13 anatomies (e.g. hand, chest, etc) and 4 projection types (e.g. posterior-anterior (PA), anterior-posterior (AP), lateral, and oblique), makes this task very diverse. Second, computation time must be as small as possible and third, a high alignment accuracy with respect to the canonical orientation is needed [2]. A simulation estimates that technologists at a medium to large sized hospital spend nearly 20 hours, or 3 working days a year, doing 70,000+ manual clicks to rotate chest images on portable x-ray machines. With an Artificial Intelligence (AI) algorithm being 99.4% accurate, it is estimated that the 19.59 hours of manual ”clicks” would be reduced to 7 minutes a year, and the 70,512 clicks to 423 clicks respectively [3]. This shows that a deep learning based AI system has the potential to significantly improve the overall workflow in X-ray radiography.

To the best of our knowledge, this is the first work that shows to detect the in-plane rotation of the extremities of the body in x-ray images. Several methods were published on automatic orientation detection of a single anatomy, for e.g. chest [3, 4, 5]. However, most of these approaches focus on orienting the x-ray images into 4 sectors e.g. 0^◦, 90^◦, 180^◦, 270^◦and not a precise orientation prediction for the full angular range of 0^◦– 360^◦. Baltruschat et al. proposed a transfer learning approach with ResNet architecture for precise orientation regression in hand radiographs, achieving state-of-the-art performance with a mean absolute angle error of 2.79^◦ [2]. Luo et al. addressed the orientation correction for radiographs in PACS environment by using well-defined low-level visual features from the anatomical region with a SVM classifier, achieving 96.1% accuracy [6]. The idea of estimating the hand orientation in probability density form by Kondo et al., solves the cyclicity problem in direct angular representation and uses multiple predictions based on different features [7]. Kausch et al. proposed a Convolutional Neural Network (CNN) regression model that predicts 5^◦of freedom pose updates directly from an initial X-ray image [8]. Here, they used a two-step approach (coarse CNN regressor and fine CNN regressor) to detect the orientation of the anatomy.

This thesis aims to develop a framework for the detection of the in-plane rotation of the extremities of the human body in a single 2D X-ray image using deep learning algorithms. Based on this information, the image shall be subsequently rotated to a predefined orientation based on the anatomy instead of the detector orientation (with respect to the X-ray source). This is especially important with portable Wireless Fidelity (WiFi) detectors, where the original orientation of the anatomy w.r.t. the detector plane can theoretically take on any angular value. In this work, we will initially focus on hands and fingers (also partially visible hands), but other extremities can also be taken into account at later point in time. In detail, the thesis will comprise the following work items:

• Literature overview of the state-of-the-art regression models for the detection of the body part orientation
• Survey for the optimal canonical orientation of each projection of the X-ray image
• Implementation of a deep learning based method with direct learning of the orientation Comparing and evaluating the performance of the deep learning models based on specific projection vs. combined projections and specific anatomy vs. combined anatomies.
• Visualizing the features learned by the model in each approach
• Quantitative evaluation on real-world data

References

[1] Paolo Russo. Handbook of X-ray imaging: physics and technology. CRC press, 2017.

[2] Ivo M Baltruschat, Axel Saalbach, Mattias P Heinrich, Hannes Nickisch, and Sascha Jockel. Orientation regression in hand radiographs: a transfer learning approach. In Medical Imaging 2018: Image Processing, volume 10574, page 105741W. International Society for Optics and Photonics, 2018.

[3] Khaled Younis, Min Zhang, Najib Akram, German Vera, Katelyn Nye, Gireesha Rao, Gopal Avinash, and John M. Sabol. Leveraging deep learning artificial intelligence in detecting the orientation of chest x-ray images. 09 2019.

[4] Ewa Pietka and HK Huang. Orientation correction for chest images. Journal of Digital Imaging, 5(3):185– 189, 1992.

[5] Hideo Nose, Yasushi Unno, Masayuki Koike, and Junji Shiraishi. A simple method for identifying image orientation of chest radiographs by use of the center of gravity of the image. Radiological physics and technology, 5(2):207–212, 2012.

[6] Hui Luo and Jiebo Luo. Robust online orientation correction for radiographs in pacs environments. IEEE transactions on medical imaging, 25(10):1370–1379, 2006.

[7] Kazuaki Kondo, Daisuke Deguchi, and Atsushi Shimada. Hand orientation estimation in probability density form. arXiv preprint arXiv:1906.04952, 2019.

[8] Lisa Kausch, Sarina Thomas, Holger Kunze, Maxim Privalov, Sven Vetter, Jochen Franke, Andreas H Mahnken, Lena Maier-Hein, and Klaus Maier-Hein. Toward automatic c-arm positioning for standard projections in orthopedic surgery. International Journal of Computer Assisted Radiology and Surgery, 15(7):1095–1105, 2020.

Analysis of Deep Learning Methods for Re-identification on Chest Radiographs

Two-Dimensional-Dwell-Time Analysis of Ion-Channel Kinetics using Deep Learning

In this project, we want to explore the capability of neural networks to infer kinetics of electrophysiological time series with Markov models. Patch-Clamp recordings of single ion channels provide a wealth of information on the functional properties of proteins that by far outperform macroscopic measurements. However, modelling of single-channel data is still a major challenge not only because it is very time-consuming. Likely, the most sophisticated way to relate kinetics and protein function is to utilize hidden Markov models (Huth et al., 2008). Scientists have developed several different methods for this purpose (Sakmann and Neher, 1995). All these methods share at least some of the following disadvantages. They require specific assumptions, specific corrections dependent on the time series, are sensitive to noise (Huth et al., 2006), are limited to the bandwidth of the recording system (Huth et al., 2006; Qin, 2014) or do not provide statistics to estimate how well they have approximated the data.

We have developed a 2D-Fit algorithm with simulations and have improved it over the years (Huth et al., 2006). The algorithm is based on the idealization of time series and the generation of two-dimensional-dwell-time-distribution from neighboring events. To a certain extent, it does not share afore mentioned limitations and it has some unique features that makes it superior to other tools. It does capture gating kinetics with a high background of noise and can extract rate constants even beyond the recording bandwidth. That could make the 2D-Fit exceptional valuable for relating electrophysiology kinetics with data from simulations of single protein molecules. In addition, 2D-distributions preserve the coherency of connected states. Thereby the algorithm can extract the full complexity of underlying models and distinguish different Markov models. However, the computational requirements are enormous, repeatedly for each time series that is analyzed. Neural networks have the reverse approach. Once datasets are generated and the networks are trained, time series could be analyzed in real time during experiments. It has to be determined whether deep networks are capable of outperforming the powerful simulation approach.

The basic aim of this thesis is to analyze two-dimensional-dwell-time-histograms with neuronal networks, a task of image analysis, to extract the underlying kinetics of Markov models. In parallel, another master student will explore the direct analysis of time series. Both approaches have to our knowledge not yet been investigated for Patch-Clamp data. It will be very interesting to compare the results of both approaches.

In the first part of the project, the objective is to generate training datasets with the 2D-Fit algorithm (already implemented) and to deploy networks capable to analyze simple Markov models (preliminary results are available for a 3-state model). The master student will evaluate the capabilities of the networks related to bandwidth and noise of time series. The next step will be to find strategies to increase the number of states of the underlying models that the network is able to distinguish. Finally, and not directly related, the capabilities of networks to distinguish different Markov models will be explored.  We expect that networks could really excel in this task of pattern recognition.

Sources

Huth T, Schmidtmayer J, Alzheimer C, Hansen UP (2008) Four-mode gating model of fast inactivation of sodium channel Nav1.2a. Pflugers Archiv European Journal of Physiology 457:103–119.

Huth T, Schroeder I, Hansen U-P (2006) The power of two-dimensional dwell-time analysis for model discrimination, temporal resolution, multichannel analysis and level detection. Journal of Membrane Biology 214:19–32.

Qin F (2014) Principles of single-channel kinetic analysis. Methods in Molecular Biology 1183:371–399.

Sakmann B, Neher E (1995) Single-Channel Recordings (Sakmann B, Neher E, eds)., 2nd ed. New York and Lodon: Plenum Press.

3D Segmentation of metal objects based on Cone-Beam CT Projection Images for Metal Artefact Removal

Computer Tomography (CT) imaging from intraoperative mobile C-Arms is commonly used to validate tool and implant placement during surgery. As a majority of tools and implants are composed of metal, physical effects such as beam hardening, photon scattering, and high absorption induce artefacts in the volume domain. These Metal Artefacts arise from a loss of signal in the projection images which is not accounted for in standard reconstruction algorithms. Metal Artefact Reduction (MAR) techniques rely on an accurate segmentation of the metal volume.[1], [2] This first segmentation step is commonly based on thresholding the volume domain which makes it prone to errors induced by metal artefacts. This thesis investigates an end-to-end trainable segmentation model which produces 3D-metal masks from 2D projection data of a 3D Cone Beam Scan of a Cios Spin System. The robustness against metal artefacts shall be evaluated and compared to common volume-domain metal segmentation approaches.

Low Dose Helical CBCT denoising by domain filtering with deep reinforcement learning improved by Neural Ordinary Differential Equations approach

In previous research, we have developed a method, based on reinforcement learning, to denoise cone-beam CT. This method involved the use of denoisers in both the sinogram and the reconstructed image domain. The denoisers are bilateral filters with the sigma parameters tuned by a convolutional agent.

Recent research has shown the use of neural ODEs to improve the speed of convergence of neural network training. Neural ODEs have been applied to tasks which can be modelled by differential equations, such as fluid mechanics. They have also been expanded to cover classical deep learning tasks, such as image segmentation.

In this thesis we aim to complete the following tasks:

1. Experiment with different recon kernels (B40, B70 etc.) to observe the effect of sharpness dependent noise.
2. Implement neural ODE to speed up reinforcement learning convergence, and also reduce parameter count.
3. Implement data consistent reward to ensure correct reconstruction and data consistent denoising.
4. Experiment with deep learned quality metrics as additional reward functions for parameter tuning

As a dataset, we will use the Mayo Clinic TCIA dataset for testing the quality of our denoising algorithms. Quality can be compared with standard dose images using PSNR and SSIM, and can be calculated reference-free using the IRQM. If time permits, we can use deep model observers to assess low contrast preservation.

Requirements:

• Knowledge of CT reconstruction techniques. Knowledge of the ASTRA toolbox is a plus.
• Understanding of reinforcement learning
• Experience with PyTorch for developing neural networks
• Experience with image processing