Index

Homogenization of Mammograms using a GAN-based Approach to Improve Breast Lesion Diagnosis

Despite many medical breakthroughs, breast cancer is still one of the main causes of death caused by malignant disease among women. It accounts for one out of ten cancer diagnoses each year [1]. Due to its silent evolution, breast cancer diagnosis requires regular screening. The gold standard for this procedure is X-ray mammography, which is particularly used in older women. As with most cancers, survival rate increases with early diagnosis significantly.

Commonly, diagnosing breast cancer requires a trained professional radiologist, who examines the individual mammography images of a patient by adjusting different image properties, such as brightness or contrast, for better visualization of anatomical or pathological structures. This form of manual diagnosing is time- and resource consuming. Furthermore, the manual diagnostic process is associated with a high risk of false positives and false negatives [2] as the diagnosis, to a certain extend, is subject to the radiologist’s interpretation. Therefore, the demand of accelerating and supporting the diagnostic process has increased in recent years. Additionally, the rapid advancement of machine learning has led to the rise of new research focusing on classifying malignant structures in medical imaging, especially in tasks like mammography, facilitated by deep learning.

Early detection of malignant structures in mammography images with the help of deep learning is a challenging task for various reasons. Most publicly available databases lack annotations, preventing deep learning models from unfolding their true potential of discovering the desired malignant region of interest. Furthermore image properties, such as the overall brightness and the contrast, may differ, because of different acquisition protocols or acquisition models. Large variations of image properties can introduce further noise, which can not be addressed by simply adjusting the window-width and the window-level of the displayed image. This inhomogeneity can cause the Machine Learning model performance to worsen.

For this work, Full-Field Digital Mammography (FFDM) images from 283 patients, provided by the Women’s Hospital of the University Hospital in Erlangen, are inspected and processed. About 15% of the acquired data shows large variations with regards to brightness and contrast, introducing inhomogeneity to the data and preventing a deep learning model from accurately detecting breast lesions. This problem can be addressed by removing inhomogeneous training samples. While such an approach would improve the classification performance compared to training with the total dataset, it also keeps the model from leveraging all available information, as the amount of training samples is reduced.
This thesis aims at analyzing and solving this challenge by transforming inhomogeneous images into homogeneous ones. Thereby, increasing the amount of available training samples while simultaneously reducing the influence of inhomogeneous data. The transformation is done with the help of generative learning, on the described dataset. The proposed method would approximate the joint probability P(x,y) of an original (inhomogeneous) image x and a generated (homogeneous) image y using a generator. The mentioned method builds up on Armanious et al. work in which medical images were translated into various domains with the help of generative learning. Their frameworks, MedGAN [3] and Cycle-MedGAN [4] utilize conditional Generative Adversarial Networks (cGANs) to learn a mapping between the original source domain and the synthetic target domain in an unsupervised manner.

The thesis consists of the following milestones:

• First, analyzing the performance of a baseline model for the detection of breast lesions using a reduced (homogeneous) portion of the dataset.
• Second, building and optimizing the GAN-models for the homogenization of mammograms.
• Finally, evaluating the lesion detection performance when including the homogenized mammograms on the training process and comparing its performance with the baseline model.
• Additionally, if time allows it: retraining and evaluating the models on a publicly available dataset.

[1]Fadi M. Alkabban and Troy Ferguson. Breast cancer. InStatPearls (Internet). StatPearls Publishing, 2019.
[2]Li Shen, Laurie R Margolies, Joseph H Rothstein, Eugene Fluder, Russell B McBride, and Weiva Sieh. Deep learning to improve breast cancer early detection on screening mammography.arXiv preprintarXiv:1708.09427, 2017.
[3]Karim Armanious, Chenming Yang, Marc Fischer, Thomas Küstner, Konstantin Nikolaou, Sergios Gatidis,and Bin Yang. Medgan: Medical image translation using gans. CoRR, abs/1806.06397, 2018.
[4]Karim Armanious, Chenming Jiang, Sherif Abdulatif, Thomas Küstner, Sergios Gatidis, and Bin Yang. Unsupervised medical image translation using cycle-medgan.CoRR, abs/1903.03374, 2019.

Machine Learning-Based Feature Classification and Position Detection of Spherical Markers in CT Volumes

Deep Learning-based image correction for Diffusion Weighted Imaging sequences

Diffusion weighted echo planar imaging pulse sequences are commonly used for clinical routine Magnetic Resonance Imaging, e.g., for stroke or tumor assessment. These sequences, especially if acquired in a segmented approach, are prone to artefacts such as ghosting, geometric distortions, blurring and mesh artefacts. These can be caused due to patient physiology, hardware imperfections or mismatches between acquired segments. There exists a variety of software-based correction approaches to counter these artefacts which however fail in various scenarios. The goal of this thesis is to evaluate deep learning based correction approaches to improve the image quality compared to conventionally implemented correction approaches. An example for this is shown below.

The figure above shows Partial Fourier reconstruction for a readout segmented EPI acquisition (RESOLVE). The Zero filled reconstruction is blurry and the POCS reconstruction suffers from mesh/stripe artefacts. The goal is to train a neural network to perform Partial Fourier reconstruction to overcome the downsides of the conventional approaches.

Bewertung verschiedener Verfahren zur Lungenregistrierung und möglicher Verbesserungen dieser im Bereich der CT Bildgebung

Die Strahlentherapie ist häug essentieller Bestandteil der Behandlung von verschiedenen Tumorerkrankungen.
Um bestmögliche Ergebnisse zu erzielen, wird im Voraus für jeden Patienten eine individuelle
Bestrahlungsplanung durchgeführt. Bei Bestrahlungszielen im Bereich des Brustkorbs, wie z.B.
Brustkrebs oder Lungenkrebs, ist die Planung aufgrund der atmungsbedingten zeitlichen Lageänderung
komplex. [1] Zusätzlich besteht noch die Möglichkeit, den physiologischen Zustand einzelner Lungenbereiche
in die Planung miteinzubeziehen. So soll die Bestrahlung bestenfalls durch bereits geschädigte
anstatt gesunder Lungenareale appliziert werden, um die Lungenfunktion zu erhalten.
Für diesen Zweck kann ein Ventilations CT durchgeführt werden. Hierbei werden CT Aufnahmen von
verschiedenen Atemzuständen angefertigt. Aus Unterschieden in den aufeinander registrierten Atemzust
änden können Lungenbereiche identiziert werden, die vergleichsweise schwach durchlüftet und somit
wenig relevant für die Lungenleistung sind.
Aufgrund der Elastizität der Lunge gestaltet sich die Registrierung jedoch als schwierig. Auÿerdem
existiert noch kein Goldstandard für die Evaluierung verschiedener Lungenregistrierungsmethoden. [2]
Ziel dieser Arbeit ist es, systematische Unterschiede verschiedener Registrierer herauszuarbeiten. Dazu
sind folgende Aufgaben vorgesehen:
ˆ Literaturrecherche über verschiedene Verfahren der Lungenregistrierung und Stand der Technik
ˆ Sammlung und Strukturierung von Patientenscans anhand diverser Eigenschaften wie z.B. Pathologien
ˆ Entwicklung von geeigneten Qualitätskriterien
ˆ Durchführung der Registrierung mittels unterschiedlicher Verfahren
ˆ Analyse der Ergebnisse
ˆ (Evaluierung methodischer Verbesserung von Registrierern)
Literatur
[1] Sean Brown, Kathryn Banll, Marianne C. Aznar, Philip Whitehurst, and Corinne Faivre Finn.
The evolving role of radiotherapy in non-small cell lung cancer. The British Journal of Radiology,
92(1104):20190524, December 2019.
[2] K. Murphy, B. van Ginneken, J. M. Reinhardt, S. Kabus, Kai Ding, Xiang Deng, Kunlin Cao,
Kaifang Du, G. E. Christensen, V. Garcia, T. Vercauteren, N. Ayache, O. Commowick, G. Malandain,
B. Glocker, N. Paragios, N. Navab, V. Gorbunova, J. Sporring, M. de Bruijne, Xiao Han,
M. P. Heinrich, J. A. Schnabel, M. Jenkinson, C. Lorenz, M. Modat, J. R. McClelland, S. Ourselin,
S. E. A. Muenzing, M. A. Viergever, Dante De Nigris, D. L. Collins, T. Arbel, M. Peroni, Rui Li,
G. C. Sharp, A. Schmidt-Richberg, J. Ehrhardt, R. Werner, D. Smeets, D. Loeckx, Gang Song,
N. Tustison, B. Avants, J. C. Gee, M. Staring, S. Klein, B. C. Stoel, M. Urschler, M. Werlberger,
J. Vandemeulebroucke, S. Rit, D. Sarrut, and J. P. W. Pluim. Evaluation of Registration
Methods on Thoracic CT: The EMPIRE10 Challenge. IEEE Transactions on Medical Imaging,
30(11):19011920, November 2011.

Synthetic X-rays from CT volumes for deep learning

X-rays are a standard imaging modality in clinical care and various artificial intelligence (AI) applications have been proposed to support clinical work with X-ray images. AI-based applications employing deep learning requires a great number of training data that must be structured and annotated with respect to the anatomical regions of interest. However, acquiring this training data is challenging due to the time intensive, error prone and expensive nature of annotating and labelling image data.  As an alternative, Computed Tomography (CT) data along with annotations generated from existing AI-software can be used to generate synthetic X-ray images with the corresponding transformed annotations [1][2].

In this master’s thesis, the use of synthetic X-rays generated from CT volumes for deep learning shall be investigated. Synthetic X-rays are a simulation of radiographic images produced through a perspective projection of the three-dimensional (CT) image volume onto a two-dimensional image plane. The application focuses mainly on orthopedic imaging, in particular spine imaging. A deep neural network is trained to identify anatomical landmarks of the vertebrae (e.g. corners or centers) using only the generated synthetic X-ray data [3][4]. This trained network is then extensively tested on unseen datasets of real X-ray images. The hypothesis is that the synthetic 2D data from CT volumes (image, annotations) can improve training a Deep Neural Network for X-ray applications. The results should be able to demonstrate if generated images can effectively be used in place of real data for training.

The thesis consists of the following milestones:

1: Create a landmark detector model (vertebral corners or center) from real spine X-ray data

2: Generate synthetic X-ray images and corresponding annotations from available CT data

3: Train the landmark detector model using only the synthetic X-rays

4: Evaluate the results generated from the two trained models

References:

[1] B. Bier, F. Goldmann, J. Zaech, J. Fatouhi, R. Hageman, R. Grupp, M. Armand, G. Osgood, N. Navab, A. Maier & M. Unberath, “Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views”, International Journal of Computer Assisted Radiology and Surgery 14, 1463-1473 (2019)

[2] M. Unberath, J. Zaech, S.C. Lee, B. Bier, J. Fatouhi, M. Armand & N. Navab, “Deep DRR – A catalyst for machine learning in fluoroscopy-guided procedures” (2018) arXiv:1803.08606 [physics.med-ph]

[3] Khanal B., Dahal L., Adhikari P., Khanal B. (2020) Automatic Cobb Angle Detection Using Vertebra Detector and Vertebra Corners Regression. In: Cai Y., Wang L., Audette M., Zheng G., Li S. (eds) Computational Methods and Clinical Applications for Spine Imaging. CSI 2019. Lecture Notes in Computer Science, vol 11963. Springer, Cham.

[4] J. Yi, P. Wu, Q. Huang, H. Qu, D.N. Metaxas, “Vertebra-focused landmark detection for scoliosis assessment” (2020) arXiv:2001.03187 [eess.IV]

Automatic characterization of nanoparticles using deep learning techniques

Nanotechnology has been bringing numerous advances in all its applications fields, ranging from electronics to
medicine. Nanomedicine, as it is called the emerging field of the meeting of pharmaceutical, biomedical sciences
and nanotechnology, investigates the potentials of nanoparticles to improve diagnostics and therapy in healthcare
[1, 2]. Interactions of these particles with the biological environment are dependent on some key factors, as particle
size, shape and distribution. These aspects impact the particles efficacy, safety, and toxicological profiles [1–4].
Therefore, it is important to develop an accurate method to measure particle size, distribution, and characterize them
to assess their quality and safety [2].
To assist in this task, an automatic yet reliable method would be desirable to eliminate human subjectivity [5]. Recently,
deep learning is emerging as a powerful tool and will continue to attract considerable interests in microscopy
image analysis, as object detection and segmentation, extraction of regions of interest (ROIs), image classification,
etc. [6].
In this thesis, we will employ a well-established deep neural network to automatically detect, segment, and classify
nanoparticles in microscopy images. Additionally, we will extend the method to measure the size of our nanoparticles,
which also requires annotation of the particles’ measurements beforehand. Finally, we will evaluate our approach and
analyze our outcomes.
The thesis will include the following points:
• Getting familiar with the nanoparticle characterization problem and tools applied in this work.
• Extend the dataset’s annotations with the nanoparticles measurements.
• Modify the chosen network to predict the nanoparticles’ size.
• Employ the modified network to detect, segment, and classify nanoparticles and predict their size.
• Evaluate the results according to appropriate metrics for the task.
• Elaboration of further improvements for the proposed method.
Academic advisors:

References
[1] D. Bobo, K. J. Robinson, J. Islam, K. J. Thurecht, and S. R. Corrie, “Nanoparticle-based medicines: a review
of fda-approved materials and clinical trials to date,” Pharmaceutical research, vol. 33, no. 10, pp. 2373–2387,
2016.
[2] F. Caputo, J. Clogston, L. Calzolai, M. R¨osslein, and A. Prina-Mello, “Measuring particle size distribution of
nanoparticle enabled medicinal products, the joint view of euncl and nci-ncl. a step by step approach combining
orthogonal measurements with increasing complexity,” Journal of Controlled Release, vol. 299, pp. 31–43, 2019.
[3] V. Mohanraj and Y. Chen, “Nanoparticles-a review,” Tropical journal of pharmaceutical research, vol. 5, no. 1,
pp. 561–573, 2006.
[4] A. G. Roca, L. Guti´errez, H. Gavil´an, M. E. F. Brollo, S. Veintemillas-Verdaguer, and M. del Puerto Morales, “Design
strategies for shape-controlled magnetic iron oxide nanoparticles,” Advanced drug delivery reviews, vol. 138,
pp. 68–104, 2019.
[5] B. Sun and A. S. Barnard, “Texture based image classification for nanoparticle surface characterisation and machine
learning,” Journal of Physics: Materials, vol. 1, no. 1, p. 016001, 2018.
[6] L. Lu, Y. Zheng, G. Carneiro, and L. Yang, “Deep learning and convolutional neural networks for medical image
computing,” Advances in Computer Vision and Pattern Recognition; Springer: New York, NY, USA, 2017.

Weakly supervised localization of defects in electroluminescence images of solar cells

With the recent rise of renewable energy, usage of solar energy has also grown rapidly. Detecting faulty panels inproduction and on-site therefore has become more important. Prior works focus on fault detection using the e.g. the current, voltage and temperature of solar modules as inputs [6, 1], but the localization of defects using imaging and machine learning has only recently gained attention [5, 4].

This work studies the detection of defects in electroluminescence (EL) images of solar cells using state of the art computer vision techniques with a focus on crack detection. Previously, in order to train a model to predict pixel classifications, exhaustive labelling of every pixel in an image of the dataset was required. State of the art training methods allow models to predict coarse segmentations using only image-wise classification labels by means of weakly supervised training. Recently, it has been shown that these methods can be applied to perform a coarse segmentation of cracks on EL images of solar cells as well [5].

This thesis aims to improve upon the existing method. To this end, weakly supervised learning methods like guided backpropagation, grad-cam, score-cam and adversarial learning [5, 9, 2, 7, 8, 3] will be implemented to train a model that reliably and accurately localizes cracks in a dataset of about 40k image-wise annotated EL images of solar cells. Finally, a thorough evaluation will show, if these methods can improve over the state of the art.

References

[1] Ali, Mohamed Hassan, et al. “Real time fault detection in photovoltaic systems.” Energy Procedia 111 (2017): 914-923.
[2] Chattopadhay, Aditya, et al. “Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.” 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018.
[3] Choe, Junsuk, and Hyunjung Shim. “Attention-based dropout layer for weakly supervised object localization.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[4] Deitsch, Sergiu, et al. “Automatic classification of defective photovoltaic module cells in electroluminescence images.” Solar Energy 185 (2019): 455-468.
[5] Mayr, Martin, et al. “Weakly Supervised Segmentation of Cracks on Solar Cells Using Normalized L p Norm.” 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019.
[6] Triki-Lahiani, Asma, Afef Bennani-Ben Abdelghani, and Ilhem Slama-Belkhodja. “Fault detection and monitoring systems for photovoltaic installations: A review.” Renewable and Sustainable Energy Reviews 82 (2018): 2680-2692.
[7] Wang, Haofan, et al. “Score-CAM: Score-weighted visual explanations for convolutional neural networks.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020.
[8] Zhang, Xiaolin, et al. “Adversarial complementary learning for weakly supervised object localization.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[9] Zhou, Bolei, et al. “Learning deep features for discriminative localization.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

Detection of Label Noise in Solar Cell Datasets

On-site inspection of solar panels is a time-consuming and difficult process, as the solar panels are often difficult to reach. Furthermore, identifying defects can be hard, especially for small cracks. Electroluminescence (EL) imaging enables the detection of small cracks, for example using a convolutional neural network (CNN) [1,2]. Hence, it can be used to identify such cracks before they propagate and result in a measurable impact on the efficiency of a solar panel [3]. This way costly inspection and replacement of solar panels can be avoided.

To train a CNN for the detection of cracks, a comprehensive dataset of labeled solar cells is required. Unfortunately, assessing, if a certain structure on a polycrystalline solar cell corresponds to a crack or not, is a hard task, even for human experts. As a result, setting up a consistently labeled dataset is nearly impossible. That is why EL datasets of solar cells favor a significant amount of label noise.

It has been shown that CNNs are robust against small amounts of label noise, but there may be drastic influence on the performance starting at 5%-10% of label noise [4]. This thesis will

(1) analyze the given dataset with respect to label noise and
(2) attempts to minimize the negative impact on the performance of the trained network caused by label noise.

Recently, Ding et. al. proposed to identify label noise by clustering of the features learned by the CNN [4]. As part of this thesis, the proposed method will be applied to a dataset consisting of more than 40k labeled samples of solar cells, which is known to contain a significant amount of label noise. As a result, it will be investigated, if the method can be used to identify noisy samples. Furthermore, it will be evaluated, if abstaining from noisy samples improves the performance of the resulting model. To this end, a subset of the dataset will be labeled by at least three experts to obtain a cleaned subset. Finally, an extension of the method will be developed. Here, it shall be evaluated, if the clustering can be omitted, since this proved instable in prior experiments using the same data.

[1] Deitsch, Sergiu, et al. “Automatic classification of defective photovoltaic module cells in electroluminescence images.” Solar Energy 185 (2019): 455-468.
[2] Mayr, Martin, et al. “Weakly Supervised Segmentation of Cracks on Solar Cells Using Normalized L p Norm.” 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019.
[3] Köntges, Marc, et al. “Impact of transportation on silicon wafer‐based photovoltaic modules.” Progress in Photovoltaics: research and applications 24.8 (2016): 1085-1095.
[4] Ding, Guiguang, et al. “DECODE: Deep confidence network for robust image classification.” IEEE Transactions on Image Processing 28.8 (2019): 3752-3765.

Comparison of different text attention techniques for writer identification

Distillation Learning for Speech Enhancement

Noise suppression has remained a field of interest for more than five decades now, and a number of techniques have been employed to extract clean and/or noise free data. Continuous audio and video signals offer greater challenges when it comes to noise reduction, and deep neural network (DNN) techniques have been designed to enhance those signals (Valin, 2018). While the DNNs are efficient, they are computationally expensive and demand adequate memory resources. The aim of the proposed thesis will remain on addressing these constraints when working limited memory and computational power, without compromising much on the model efficiency.

A Neural Network (NN) can easily be overfitted with the training data, owing to the large number of parameters and training sessions for which the network was trained on the given data (Dakwale & Monz, 2019). One solution to this is to use ensemble (combination) of models trained on the same data to achieve generalization. The limitation of this solution comes with hardware constraints and when the network needs to be used on a hardware with limited memory and computational power, such as mobile phones. This resource limitation seeds the idea of distillation learning, in which the knowledge from a complex or ensembled network is transferred to a relatively simpler and computationally less expensive model.

Following the framework of distillation learning, a Teacher-Student network will be designed, with an existing trained Teacher network. The teacher network has been trained on audio data with hard labels, using a dense parameter matrix. The high number of parameters dictates the complexity of the neural network and also the efficiency to identify and suppress signal noise (Hinton, et al., 2015). The proposed method is to design a student network, which tries to imitate the output of the teacher, i.e., the probability distribution, without the need to be trained with the same number of parameters. By transferring the learning of the teacher to the student network, a simpler model can be designed, with a reduced set of parameters, which would be more suited for hardware with lower memory and computational power.