Index
Knowledge distillation for landmark segmentation in medical image analytics
Motivation: One major challenge that comes with deep learning approaches in medical image processing is the issue of high cost of expert annotated data. Therefore semi-supervised learning approaches are of high interest in the research community.
Methods: The application of the work concerns landmark segmentation by heatmap re-gression in thorax diagnostics. To keep the need of annotated data low an approach of knowledge distillation with application of a teacher-student concept will be persued. The essence of this method is to transfer knowledge from a pre-trained model or an ensemble ofmodels to a new model. Originally this technique was introduced to reduce the capacity of huge good-performing networks while keeping the accuracy [1]. But it has also already been applied for semi-supervised learning purposes [2], what will be further investigated in this work. Aim of this master theses is to examine the benefit of student-teacher approaches in semi-supervised learning. For this purpose different variants of this method will be considered, implemented and compared to each other. This will be done in cooperation and with provision of data and infrastructure by Siemens Healthineers.
The Master’s thesis covers the following aspects:
1. Literature research on state-of-the-art methods
2. Set up infrastructure and data used for the project
3. Implementation of algorithms in the framework
4. Training and tuning of hyper-parameters
5. Performance comparison of the different variants
6. Evaluation of developed algorithms
Project_Description_Viktoria[1] G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,”arXiv preprint arXiv:1503.02531, 2015.
[2] S. Sedai, B. Antony, R. Rai, K. Jones, H. Ishikawa, J. Schuman, W. Gadi, and R. Garnavi, “Uncertainty guided semi-supervised segmentation of retinal layers in oct images,”in International Conference on Medical Image Computing and Computer-Assisted Inter-vention, pp. 282–290, Springer, 2019.
Enhanced Generative Learning Methods for Real-World Super-Resolution Problems in Smartphone Images
The goal of this bachelor thesis is to extend the work of Lugmayr et al. [1] in order to improve the generative network by using a learned image down sampler motivated from CAR network [2] instead of bicubic down sampling. The aim is to achieve a better image quality or a more robust SR network for images of Real-World data distribution.
[1] Lugmayr, Andreas, Martin Danelljan, and Radu Timofte. “Unsupervised learning for real-world super-resolution.” 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW). IEEE, 2019.
[2] Sun, Wanjie, and Zhenzhong Chen. “Learned image downscaling for upscaling using content adaptive resampler.” IEEE Transactions on Image Processing 29 (2020): 4027-4040.
Deep Learning Classification and Optimization of Manufacturing Process Parameters
At Mercedes-Benz AG, “Die Casting” is an integral manufacturing process in which car parts are manufactured by forcing molten metal under high pressure and speed into a die cavity. The casting machines produce with a high number of process parameters, which are currently being tweaked by the engineers solely based on their experience to ensure that the process is as robust as possible. Hence, few defective parts are produced initially until the number of bad quality parts increases again and new adjustment is needed. Improvement of this procedure will lead to higher efficiency in this casting process. Owing to this, the thesis aims to build up a relationship between the process parameters and the quality measurements through data-driven modeling, and then later on use the learned mapping for finding an optimal set of parameters which maximizes the probability for producing a correct part.
From the last few decades, Artificial Neural Networks have been quite successful in capturing the complex, often non-linear, relationship which exists between the process parameters and the process output [1]. Therefore, the first goal in this thesis is fitting the given data with different neural network architectures such as Multi-Layer Perceptron [2], Residual Network [3], fine-tuning them and then selecting the best performing architecture out of them in terms of correctly mapping the process parameters to the ground-truth class labels.
Two major challenges in the first objective of the thesis are:
1. Pre-Processing of raw data which consists of a combination of categorical and numerical variables
2. Class Imbalance which poses a big challenge for unbiased training of neural networks [4]
Furthermore, the final step in the thesis would be to use our trained neural network for optimizing the input parameters in such a way that the probability of producing a good part is maximized by those parameters. One possible proposal to achieve this is by generating adversarial attacks on the neural network [5] where the input parameters are iteratively modified in the negative direction of the gradient of the loss function between the predicted class and the desired output class while the weights of the network are kept constant. Hence, instead of doing backpropagation onto the weights, backpropagation is performed onto the input parameters. Another popular approach in the literature that has shown satisfactory results in the manufacturing industry is to use the learned neural network mapping as the fitness function for evaluating candidate solutions of the optimal parameter set in a genetic algorithm [6].
In summary, the thesis will include the following points:
1. Training and Evaluation of different neural network architectures over the given data
2. Optimization of Process Parameters through the best-performing trained model
Supervisors: Prof. Dr. Andreas Maier, Olga Moreva, Lyubka Rund, Stephan Schwarz
Student: Saad Munir
Date: June 3, 2021 -November 30, 2021
References [1] Sukthomya, W., Tannock, J. The training of neural networks to model manufacturing processes. J Intell Manuf 16, 39–51 (2005). [2] Marius, Popescu & Balas, Valentina & Perescu-Popescu, Liliana & Mastorakis, Nikos. (2009). Multilayer perceptron and neural networks. WSEAS Transactions on Circuits and Systems. 8. [3] K. He, X. Zhang, S. Ren and J. Sun, “Deep Residual Learning for Image Recognition,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. [4] Ling C.X., Sheng V.S. (2011) Class Imbalance Problem. In: Sammut C., Webb G.I. (eds) Encyclopedia of Machine Learning. Springer, Boston, MA. [5] Qiu, Shilin; Liu, Qihe; Zhou, Shijie; Wu, Chunjiang. 2019. “Review of Artificial Intelligence Adversarial Attack and Defense Technologies” Appl. Sci. 9, no. 5: 909.
[6] Julius Pfrommer, Clemens Zimmerling, Jinzhao Liu, Luise Kärger, Frank Henning, Jürgen Beyerer, Optimisation of manufacturing process parameters using deep neural networks as surrogate models, Procedia CIRP, Volume 72, 2018, Pages 426-431, ISSN 2212-8271.
Development of Automated Hardware Requirement Checks for Medical Android Applications
Prediction of Steam Turbine Blade Vibration Amplitudes using Machine Learning Methods
Thesis Description
During the typical start-up process of the steam turbine, after the nominal speed has been reached, the massfow is continuously increased until the setpoint is reached. Due to the very low ow at the beginning of the start-up process, instationary ow phenomena can occur which can cause elevated blade vibrations [1]. These flow phenomena can also occur in low-load operations. It is important to simulate and assess these instationary flow phenomena and the efects on the blade vibrational behaviour of the blades [2]. For this purpose, both experimental model turbine investigations and transient 3D Computational Fluid Dynamics (CFD) simulations are carried out, which provide a variety of time-dependent data. As a result, the data includes the operating status such as mass ow, pressures, and temperatures, the excitation forces such as pressure changes and frequencies, and vibration data such as amplitudes and modes.
First of all, the existing experimental data will be analyzed. The aim is to investigate whether a powerful method for predicting vibration amplitudes of the blades can be formulated. If necessary, additional data must be generated with the help of CFD simulations to enable an assessment of the Machine Learning (ML) [3] processes for the entire design space. Based on the data analysis, suitable ML algorithms should be identified and tested. Here, for example, Multi-layer Perceptron Regressor (MLP Regressor) [4, 5], Random Forests [6], Decision Tree Regressor (DTR) [7], Support Vector Regressor (SVR) [8], etc. can be used and tested. In particular, ML algorithms are to be tested that are suitable for mapping time series, such as Recurrent Neural Networks (RNN) [9]. The predictive quality of the individual processes should be assessed especially with respect to the entire design space.
In summary, the thesis deals with the following points:
- Analysis of the existing experimental data with respect to:
(a) Coverage of the design space (sampling)
(b) Dimensionality refinement of the design space - Application of suitable ML algorithms:
(a) Various regression models
(b) Deep Learning for mapping time series - Analysis of the generated CFD data by Machine Learning Models:
(a) Examining on how a further improvement of the prediction quality can be achieved by generating data by CFD analysis and correlating with the measured data points
References
[1] Neville Rieger. Progress with the solution of vibration problems of steam turbine blades. 05 2021.
[2] R.S. Mohan, A. Sarkar, and A.S. Sekhar. Vibration analysis of a steam turbine blade. INTERNOISE 2014
– 43rd International Congress on Noise Control Engineering: Improving the World Through Noise Control,
01 2014.
[3] Alfredo Rodriguez, Youness El Hamzaoui, J A Perez, Juan Garcia, Jose Flores-Chan, and A.L. Tejeda. The
use of artificial neural network (ann) for modeling the useful life of the failure assessment in blades of steam
turbines. Engineering Failure Analysis, 35:562{575, 06 2013.
[4] Rishabh Bhardwaj, Navonil Majumder, and Soujanya Poria. Investigating gender bias in bert, 09 2020.
[5] Tejas Subramanya, Davit Harutyunyan, and Roberto Riggio. Machine learning-driven service function chain
placement and scaling in mec-enabled 5g networks. Computer Networks, 166:106980, 11 2019.
[6] Andy Liaw and Matthew Wiener. Classification and regression by randomforest. Forest, 23, 11 2001.
[7] Wei-Yin Loh. Classification and regression trees. Wiley Interdisciplinary Reviews: Data Mining and Knowl-
edge Discovery, 1:14 { 23, 01 2011.
[8] Mariette Awad and Rahul Khanna. Support Vector Regression, pages 67{80. 01 2015.
[9] Alex Sherstinsky. Fundamentals of recurrent neural network (rnn) and long short-term memory (lstm)
network. Physica D: Nonlinear Phenomena, 404:132306, 03 2020.
Terahertz Image Reconstruction for Historical Document Analysis
Network Deconvolution as Sparse Representations for Medical Image Analysis
Scene Evolution on Polarimetric Radar Data in Automated Driving Scenarios
Description
The task of autonomous driving demands vehicles to create a precise understanding of their surroundings. Currently, camera, LiDAR, and radar sensors are used to perceive the vehicle’s environment. While conventional automotive radar sensors have the advantage of instantly detecting velocities and are less prone to disturbances by difficult weather conditions than the aforementioned alternatives, they also offer a lower resolution and therefore less information for classification and tracking tasks. [1]
To compensate for this weakness, the usage of newly available polarimetric radar sensors for the automotive domain is researched. Polarimetric radar sensors emit and detect radar waves with different polarizations. The analysis of polarization changes adds additional information about reflection patterns, allowing, inter alia, estimation of vehicle orientation and extend in traffic scenarios. [2]
Prior research also indicates improvements in classification tasks using polarimetric radar data of stationary targets in test areas [3] and stationary and moving traffic participants in real-world urban scenarios [4]. In this work, polarimetric radar data is used to track and predict the evolution of the vehicle’s environment over time.
The thesis consists of the following milestones:
- Implement and compare optical flow methods to create correlation images from polarimetric and conventional radar imagery
- Implement and compare track generation methods using the correlation images
- Implement and compare track prediction methods using the generated tracks
- Evaluate advantages of polarimetric radar information
References
[3] Tristan Visentin.Polarimetric Radar for Automotive Applications, volume 90 ofKarlsruher Forschungs-berichte aus dem Institut f ̈ur Hochfrequenztechnik und Elektronik. KIT Scientific Publishing, Karlsruhe,Baden, 2019.
[4] J. F. Tilly, F. Weishaupt, O. Schumann, J. Dickmann, and G. Wanielik. Road user classification with polarimetric radars. In2020 17th European Radar Conference (EuRAD), pages 112–115, 2021.
Detection of In-plane Rotation of Extremities on X-ray Images
Thesis Description
A core goal in a medical imaging pipeline is to optimize the workflow for improving the patient throughput at a radiography system. By limiting manual tasks in a workflow, we can efficiently optimize the pipeline [1]. A common manual task in a X-ray radiography workflow is to manually rotate digital X-ray images to a canonical orientation preferred by radiologists. This has an impact on the number of patients that can be analyzed in a given period of time. A Deep Learning based detection system for the in-plane rotation of body parts in X-ray images can solve this problem, but it is still an open research topic. We identified three major challenges that such automatic systems need to address: First, in clinical routine there are up to 23 different examinations, consisting of 13 anatomies (e.g. hand, chest, etc) and 4 projection types (e.g. posterior-anterior (PA), anterior-posterior (AP), lateral, and oblique), makes this task very diverse. Second, computation time must be as small as possible and third, a high alignment accuracy with respect to the canonical orientation is needed [2]. A simulation estimates that technologists at a medium to large sized hospital spend nearly 20 hours, or 3 working days a year, doing 70,000+ manual clicks to rotate chest images on portable x-ray machines. With an Artificial Intelligence (AI) algorithm being 99.4% accurate, it is estimated that the 19.59 hours of manual ”clicks” would be reduced to 7 minutes a year, and the 70,512 clicks to 423 clicks respectively [3]. This shows that a deep learning based AI system has the potential to significantly improve the overall workflow in X-ray radiography.
To the best of our knowledge, this is the first work that shows to detect the in-plane rotation of the extremities of the body in x-ray images. Several methods were published on automatic orientation detection of a single anatomy, for e.g. chest [3, 4, 5]. However, most of these approaches focus on orienting the x-ray images into 4 sectors e.g. 0◦, 90◦, 180◦, 270◦and not a precise orientation prediction for the full angular range of 0◦– 360◦. Baltruschat et al. proposed a transfer learning approach with ResNet architecture for precise orientation regression in hand radiographs, achieving state-of-the-art performance with a mean absolute angle error of 2.79◦ [2]. Luo et al. addressed the orientation correction for radiographs in PACS environment by using well-defined low-level visual features from the anatomical region with a SVM classifier, achieving 96.1% accuracy [6]. The idea of estimating the hand orientation in probability density form by Kondo et al., solves the cyclicity problem in direct angular representation and uses multiple predictions based on different features [7]. Kausch et al. proposed a Convolutional Neural Network (CNN) regression model that predicts 5◦of freedom pose updates directly from an initial X-ray image [8]. Here, they used a two-step approach (coarse CNN regressor and fine CNN regressor) to detect the orientation of the anatomy.
This thesis aims to develop a framework for the detection of the in-plane rotation of the extremities of the human body in a single 2D X-ray image using deep learning algorithms. Based on this information, the image shall be subsequently rotated to a predefined orientation based on the anatomy instead of the detector orientation (with respect to the X-ray source). This is especially important with portable Wireless Fidelity (WiFi) detectors, where the original orientation of the anatomy w.r.t. the detector plane can theoretically take on any angular value. In this work, we will initially focus on hands and fingers (also partially visible hands), but other extremities can also be taken into account at later point in time. In detail, the thesis will comprise the following work items:
- Literature overview of the state-of-the-art regression models for the detection of the body part orientation
- Survey for the optimal canonical orientation of each projection of the X-ray image
- Implementation of a deep learning based method with direct learning of the orientation Comparing and evaluating the performance of the deep learning models based on specific projection vs. combined projections and specific anatomy vs. combined anatomies.
- Visualizing the features learned by the model in each approach
- Quantitative evaluation on real-world data
References
[1] Paolo Russo. Handbook of X-ray imaging: physics and technology. CRC press, 2017.
[2] Ivo M Baltruschat, Axel Saalbach, Mattias P Heinrich, Hannes Nickisch, and Sascha Jockel. Orientation regression in hand radiographs: a transfer learning approach. In Medical Imaging 2018: Image Processing, volume 10574, page 105741W. International Society for Optics and Photonics, 2018.
[3] Khaled Younis, Min Zhang, Najib Akram, German Vera, Katelyn Nye, Gireesha Rao, Gopal Avinash, and John M. Sabol. Leveraging deep learning artificial intelligence in detecting the orientation of chest x-ray images. 09 2019.
[4] Ewa Pietka and HK Huang. Orientation correction for chest images. Journal of Digital Imaging, 5(3):185– 189, 1992.
[5] Hideo Nose, Yasushi Unno, Masayuki Koike, and Junji Shiraishi. A simple method for identifying image orientation of chest radiographs by use of the center of gravity of the image. Radiological physics and technology, 5(2):207–212, 2012.
[6] Hui Luo and Jiebo Luo. Robust online orientation correction for radiographs in pacs environments. IEEE transactions on medical imaging, 25(10):1370–1379, 2006.
[7] Kazuaki Kondo, Daisuke Deguchi, and Atsushi Shimada. Hand orientation estimation in probability density form. arXiv preprint arXiv:1906.04952, 2019.
[8] Lisa Kausch, Sarina Thomas, Holger Kunze, Maxim Privalov, Sven Vetter, Jochen Franke, Andreas H Mahnken, Lena Maier-Hein, and Klaus Maier-Hein. Toward automatic c-arm positioning for standard projections in orthopedic surgery. International Journal of Computer Assisted Radiology and Surgery, 15(7):1095–1105, 2020.