Index

Predictive Maintenance for SINAMICs Frequency Converter

Thesis Description

 

Maintaining machines causes large expenses in the modern industrial world. For example, in the product manufactoring sector, maintenance makes up to 15%-60% of their costs [1]. To reduce these costs, more efficient methods of maintenance were invented. In general, Maintenance can be divided into three different categories [2]:

  1. Corrective: The machine’s failure has already occured.
  2. Preventive: The maintanence is done on a regular basis to decrease the likelihood of a failure.
  3. Predictive: Keep track of the machine’s condition and prediction of its failure occurance.

Predictive Maintenance uses industrial data modeling and analysis to perform equipment fault diagnosis through real-time monitoring. By direct application of domain knowledge onto given problems, it is possible to predict that status and its trend with respect to possible failures. With this, maintenance plans become more accurate. This reduces unplanned downtime as well as cost of operation [1]. One common problem in the field of predictive maintenance is the detection of anomalies. As the name implies, the classifier’s task is to detect anomalies within data by comparing it to expected data. These anomalies are used as indicators for problems and basis for the failure prediction [2]. Anomalies can be found in any kind of data. In this thesis the focus is on machines with a SINAMICS Low
Voltage Converter [3]. In general, a converter adapts the constant AC to AC with various frequency and voltage [4]. The SINAMICS Low Voltage Converter produces sensor data in the high frequency spectrum based on a low voltage [3]. The data is gathered and computed on an Industrial Edge [5]. Edge Computing is defined as any computing done at the edge of the network that produced the data. Compared to Cloud Computing which bottleneck is the massive data transfer, Edge Computing saves a lot of bandwith since the data is already processed and only the relevant information is sent to the cloud [6].
Anomaly detection is already used in different security aspects like fraud, intrusion detection but also regarding medical health anomalies. As a result, there exist various methods for different kind of data (continous, discret, labeled or unlabeled etc.). Since in this thesis not all anomalies are known, the given data is unlabeled and there is a time element, the focus in this thesis will be on methods for unsupervised time series data, for example, Hidden Markov Models, Clustering, Neural Network and Nearest neighbour, etc. [7, 8, 9]. The goal of this thesis is to apply different methods for the detection and classification of signal anomalies resulting in a comparison between them regarding resource usage (e.g. RAM), runtime and accuracy for a specific type of converter. To be able to accurately predict any failures at the machine, the anomalies will be ranked regarding their severness in respect to the machine. This thesis aims to find a robust, exible and unsupervised model for anomaly detection in high frequency, low voltage sensor data as a basis for Predictive Maintenance. The data is computed directly on an Industrial Edge device (227E, 427E) attached to the SINAMICS Low Voltage Converter. In summary, the thesis deals with the following points:

  1.  Define anomalies and their consequences/meanings for the SINAMICS Low Voltage Converter:
    (a) Detection and identification of anomalies
    (b) Ranking of found anomalies based on their inuence
  2. Development of different models for anomaly detection and classification
  3. Evaluation based on the different models considering the following points:
    (a) Accuracy
    (b) Flexibility
    (c) Resource Usage
    (d) Robustness
    (e) Runtime

References
[1] Bei-ke Zhang Cheng Cheng and Dong Gao. A predictive maintenance solution for bearing production line
based on edge-cloud cooperation. In 2019 Chinese Automation Congress (CAC), pages 5885{5889, 2019.
[2] Masaru Kurihara Pushe Zhao, Shigeyoshi Chikuma Junichi Tanaka, Tojiro Noda, and Tadashi Suzuki. Advanced
correlation-based anomaly detection method for predictive maintenance. In 2017 IEEE International
Conference on Prognostics and Health Management (ICPHM), pages 78{83. IEEE, 2017.
[3] Siemens Global Website. Siemens Digital Industries. https://new.siemens.com/global/en/products/
drives/sinamics/low-voltage-converter.html, 1996-2021. [Online; accessed 23-02-2021].
[4] Pawel Szczesniak Zbigniew Fedyczak and Marius Klytta. Ac drives with buck-boost voltage switched frequency
converters without dc storage. In 2018 International Symposium on Power Electronics, Electrical
Drives, Automation and Motion (SPEEDAM), pages 507{512, 2018.
[5] Siemens Global Website. Industrial Edge. https://new.siemens.com/global/en/products/automation/
topic-areas/industrial-edge/simatic-edge.html, 1996-2021. [Online; accessed 18-02-2021].
[6] Jie Cao Weisong Shi, Youhuizi Li Quan Zhang, and Lanyu Xu. Edge computing: Vision and challenges.
IEEE Internet of Things Journal, 3(5):637{646, 2016.
[7] Charu C. Aggarwal Manish Gupta, Jing Gao and Jiawei Han. Outlier detection for temporal data: A survey.
IEEE Transactions on Knowledge and Data Engineering, 26(9):2250{2267, 2014.
[8] Lei Clifton Marco A.F. Pimentel, David A. Clifton and Lionel Tarassenko. A review of novelty detection.
Signal Processing, 99:215{249, 2014.
[9] Arindam Banerjee Varun Chandola and Vipin Kumar. Anomaly detection: A survey. ACM Computing
Surveys, 41:1{58, 2009.

Optimization of the fat-water separation for muscle imaging at 7 T with application in quantitative Na23/K39 MRI

Magnetic Resonance Imaging (MRI) is a noninvasive imaging method, without ionizing radiation, which
has the ability to particularly resolve soft tissue such as muscle, fat, and connective tissue. Numerous
studies have shown the ability of MRI to detect alterations in skeletal muscle structure and composition,
for example, patients with Duchenne Muscular Dystrophy (DMD) [1]. The muscle destruction observed
in muscle dystrophy patients is associated with ion homeostasis dysregulation and chronic in
ammation,
which leads ultimately to bro-fatty replacement of muscle tissue [2]. An increase in total sodium
concentration (TSC) has been observed in these patients in addition to elevated intra-cellular weighted
sodium signal (ICwS) based on an inversion recovery (IR) method. [3] Na23/K39 MRI detects a muscular
Na+ overload in these patients and thus it could depict early changes in the ion homeostasis in skeletal
muscle tissue of these patients. [2]
In this thesis, we aim to optimize the Na23/K39 quanti cation in muscle tissue by combining Na23/K39
MRI data with fat fractions (the proportion of the acquired signal derived from fat protons) obtained
by H1 MRI. As sodium and particularly potassium concentrations are strongly reduced in fat tissue
compared to muscle tissue, the Na23/K39 quanti cation in fat in ltrated muscles is generally distorted.
Thus, a fat correction needs to be applied to obtain the real ion concentrations of muscle tissue. This
can be achieved by applying the Multiple point Dixon technique for Water/Fat decomposition on the
H1 MRI Images acquired at 7T. The original fat quanti cation is based on the fact that fat and water
possess di erent resonance frequencies in MRI, which is called the Chemical Shift. Using a gradient
echo-based Dixon acquisition with very closely spaced echo times, the fat and water from these images
can be successfully separated using Dixon’s fat water separation algorithm [4]. It is based on the
principle that the signal in the image acquired when the water and fat have the same phase, interfere
constructively whereas the images acquired when water and fat are in opposed phase they interfere
destructively. Thus fat in ltration in the muscle tissue of a muscle dystrophy patient could be studied
using the Fat fraction [5].
The goal of the thesis is to create an image processing protocol at 7T correctly maps the fat and
water in the calf muscles and generate a fat-fraction which can further be used on the Na23 images where
both the 1H and Na23 images are acquired at a 7T MRI. Currently, no such protocols exist at 7T, and
adapting them from 3T comes with various challenges. This thesis aims to e ectively facilitate easy and
early diagnosis for various muscular dystrophies.
1
The following points will be covered in this thesis:
1. Design and production of a measurement phantom
(a) Prerequisites: Multiple compartments (e.g. cylinders) containing di erent fat-water fractions,
size tting into 1H knee coil @ 7T (and other typical 1H coils, e.g. @3T)
(b) Research on typical fat-fractions in muscle tissue/materials suitable for achieving di erent
fat fractions/phantoms used in other publications on fat-water imaging; and thereby design
the phantom.
2. Optimization of measurement protocol for fat-water imaging at 7T
(a) Compare and evaluate existing H1 MRI acquisition protocols for fat-water separation at 3T,
choose the most suitable .rotocol to be translated to 7T.
(b) Optimize multiple-echo acquisition protocols by calculating optimized TEs for fat-water separation
at 7T.
3. Optimization of post-processing for fat-water imaging
(a) Comparison of di erent algorithms for fat-water separation using the ISMRM fat-water toolbox.
(b) Study e ects of B0 inhomogeneities using di erent B0 shimming protocols to resolve phase
wrapping artifacts.
4. Application of fat-water imaging in vivo
Application of optimized fat-water imaging protocol to healthy subjects (approx. 5); Quanti cation
of fat fraction in healthy muscle tissue and comparison of results to literature; Maybe (towards
the end of thesis): application to patients with muscular dystrophies
5. Comparison of fat-water imaging at di erent eld strengths
Optimize protocol (or use existing protocols from literature) for di erent eld strengths, e.g.3T
and 7T; Compare resulting fat fraction values in phantom and in vivo measurements (healthy
muscle)

References
[1] Erika L. Finanger, Barry Russman, Sean C. Forbes, William D. Rooney, Glenn A.Walter, and Krista
Vandenborne. Use of skeletal muscle mri in diagnosis and monitoring disease progression in duchenne
muscular dystrophy. Physical medicine and rehabilitation clinics of North America, 23(1):1{ix, 2012.
[2] Teresa Gerhalter, Lena V. Gast, Benjamin Marty, Jan Martin, Regina Trollmann, Stephanie
Schussler, Frank Roemer, Frederik B. Laun, Michael Uder, Rolf Schroder, Pierre G. Carlier, and
Armin M. Nagel. 23na mri depicts early changes in ion homeostasis in skeletal muscle tissue of patients
with duchenne muscular dystrophy. Journal of Magnetic Resonance Imaging, 50(4):1103{1113,
2019.
[3] Armin M. Nagel, Marc-Andre Weber, Arijitt Borthakur, and Ravinder Reddy. Skeletal Muscle MR
Imaging Beyond Protons: With a Focus on Sodium MRI in Musculoskeletal Applications, pages
115{133. Springer Berlin Heidelberg, Berlin, Heidelberg, 2014.
[4] W. T. Dixon. Simple proton spectroscopic imaging. Radiology, 153(1):189{94, 1984.
[5] T. J. Bray, M. D. Chouhan, S. Punwani, A. Bainbridge, and M. A. Hall-Craggs. Fat fraction mapping
using magnetic resonance imaging: insight into pathophysiology. Br J Radiol, 91(1089):20170344,
2018.
3

Incorporating GAN-Translated Tomosynthesis Images for improved automatic Lesion Detection in Mammography Images

Computer Aided Diagnosis (CAD) systems assist physicians in the interpretation of medical images. One of the most common application for CAD systems is in the field of mammography where they help in the detection and analysis of lesions e.g. masses, microcalcifications and tumors. X-ray mammography is the most used method for breast cancer screening and CAD systems for this modality are well established. CAD systems do also exist for other modalities like ultrasound and magnetic resonance imaging. A further modality is digital breast tomosynthesis (DBT) which uses image recordings from different angles to produce a 3D-representation of the breast. These 3D representations can overcome some shortcomings of conventional X-ray mammography like overlapping breast tissue due to the necessary breast compression during an examination. Using digital breast tomosynthesis has shown to increase accuracy and leading to a reduced false positive rate in breast cancer screening.

CAD systems are usually trained for a single modality and therefore often not applicable to other modalities. They rely upon state of the art machine learning approaches like deep learning. In general deep learning needs high amounts of data for training which for specific problems is often not available or hard to come by. Especially labelling and annotation of images is a labour-intensive and expensive task. Being able to come by the issue of limited modality usage and the problem of few data by combining different data sets would be beneficial for an automatic lesion detection method. However, combining data from different sources usually makes further model adaption necessary.

For this thesis two data sets of labeled mammography images are available. The first data set contains X-ray mammograms of 237 patients. The second consists of digital breast tomosynthesis images of 42 patients. The task of this thesis is to incorporate tomosynthesis images into a method for lesion detection in mammograms. This incorporation should improve the methods ability to detect lesions in mammograms and expand the method’s usability to a second modality.

This task can be considered as a problem of Domain Adaption [1] with two source and one target distribution. The source distributions are mammograms and tomosynthesis images whereas the target distribution are mammograms. One method of Domain Adaption appropriate for this task is adversarial-based Domain Adaptation with generative models. In the medical context Generative Adversarial Networks (GANs) have been applied for e.g. the translation of images between modalities, denoising of images and artifact correction [2]. In this thesis GANs will be used to generate synthetic mammograms out of tomosynthesis data which will be used to enhance the data set used for the training of a deep convolutional neural network (CNN). The hypothesis is that using such an enhanced data set for model training leads to an improved performance in lesion detection compared to training with a smaller data set solely consisting of genuine mammograms.

The master thesis contains several milestones:

  1. Creation of a baseline deep CNN model trained and evaluated for lesion detection in mammograms.
  2. Development and optimization of a Cycle-GAN [3] to translate tomosynthesis images into mammograms.
  3. Evaluation of the classification performance for lesion detection of a deep CNN model trained on mammograms and Cycle-GAN generated mammograms.
  4. Application of the model obtained in 2 to verify the hypothesis on another publicly available mammogram data set.

[1] Wang M. and Deng W. (2018). Deep visual domain adaptation: A survey . Neurocomputing, 312,135–153. https://doi.org/10.1016/j.neucom.2018.05.083
[2] Armanious K., Jiang C., Fischer M., Küstner T., Hepp T., Nikolaou K., Yang B. (2020). MedGAN: Medical image translation using GANs. Computerized Medical Imaging and Graphics,79, 1–16. https://doi.org/10.1016/j.compmedimag.2019.101684
[3] Zhu J. Y., Park T., Isola P. and Efros A. A. (2017). Unpaired Image-to-Image Translation Us-ing Cycle-Consistent Adversarial Networks. Proceedings of the IEEE International Conference on Computer Vision, 2017-October, 2242–2251. https://doi.org/10.1109/ICCV.2017.244

Homogenization of Mammograms using a GAN-based Approach to Improve Breast Lesion Diagnosis

Despite many medical breakthroughs, breast cancer is still one of the main causes of death caused by malignant disease among women. It accounts for one out of ten cancer diagnoses each year [1]. Due to its silent evolution, breast cancer diagnosis requires regular screening. The gold standard for this procedure is X-ray mammography, which is particularly used in older women. As with most cancers, survival rate increases with early diagnosis significantly.

Commonly, diagnosing breast cancer requires a trained professional radiologist, who examines the individual mammography images of a patient by adjusting different image properties, such as brightness or contrast, for better visualization of anatomical or pathological structures. This form of manual diagnosing is time- and resource consuming. Furthermore, the manual diagnostic process is associated with a high risk of false positives and false negatives [2] as the diagnosis, to a certain extend, is subject to the radiologist’s interpretation. Therefore, the demand of accelerating and supporting the diagnostic process has increased in recent years. Additionally, the rapid advancement of machine learning has led to the rise of new research focusing on classifying malignant structures in medical imaging, especially in tasks like mammography, facilitated by deep learning.

Early detection of malignant structures in mammography images with the help of deep learning is a challenging task for various reasons. Most publicly available databases lack annotations, preventing deep learning models from unfolding their true potential of discovering the desired malignant region of interest. Furthermore image properties, such as the overall brightness and the contrast, may differ, because of different acquisition protocols or acquisition models. Large variations of image properties can introduce further noise, which can not be addressed by simply adjusting the window-width and the window-level of the displayed image. This inhomogeneity can cause the Machine Learning model performance to worsen.

For this work, Full-Field Digital Mammography (FFDM) images from 283 patients, provided by the Women’s Hospital of the University Hospital in Erlangen, are inspected and processed. About 15% of the acquired data shows large variations with regards to brightness and contrast, introducing inhomogeneity to the data and preventing a deep learning model from accurately detecting breast lesions. This problem can be addressed by removing inhomogeneous training samples. While such an approach would improve the classification performance compared to training with the total dataset, it also keeps the model from leveraging all available information, as the amount of training samples is reduced.
This thesis aims at analyzing and solving this challenge by transforming inhomogeneous images into homogeneous ones. Thereby, increasing the amount of available training samples while simultaneously reducing the influence of inhomogeneous data. The transformation is done with the help of generative learning, on the described dataset. The proposed method would approximate the joint probability P(x,y) of an original (inhomogeneous) image x and a generated (homogeneous) image y using a generator. The mentioned method builds up on Armanious et al. work in which medical images were translated into various domains with the help of generative learning. Their frameworks, MedGAN [3] and Cycle-MedGAN [4] utilize conditional Generative Adversarial Networks (cGANs) to learn a mapping between the original source domain and the synthetic target domain in an unsupervised manner.

The thesis consists of the following milestones:

  • First, analyzing the performance of a baseline model for the detection of breast lesions using a reduced (homogeneous) portion of the dataset.
  • Second, building and optimizing the GAN-models for the homogenization of mammograms.
  • Finally, evaluating the lesion detection performance when including the homogenized mammograms on the training process and comparing its performance with the baseline model.
  • Additionally, if time allows it: retraining and evaluating the models on a publicly available dataset.

[1]Fadi M. Alkabban and Troy Ferguson. Breast cancer. InStatPearls (Internet). StatPearls Publishing, 2019.
[2]Li Shen, Laurie R Margolies, Joseph H Rothstein, Eugene Fluder, Russell B McBride, and Weiva Sieh. Deep learning to improve breast cancer early detection on screening mammography.arXiv preprintarXiv:1708.09427, 2017.
[3]Karim Armanious, Chenming Yang, Marc Fischer, Thomas Küstner, Konstantin Nikolaou, Sergios Gatidis,and Bin Yang. Medgan: Medical image translation using gans. CoRR, abs/1806.06397, 2018.
[4]Karim Armanious, Chenming Jiang, Sherif Abdulatif, Thomas Küstner, Sergios Gatidis, and Bin Yang. Unsupervised medical image translation using cycle-medgan.CoRR, abs/1903.03374, 2019.

Machine Learning-Based Feature Classification and Position Detection of Spherical Markers in CT Volumes

Deep Learning-based image correction for Diffusion Weighted Imaging sequences

Diffusion weighted echo planar imaging pulse sequences are commonly used for clinical routine Magnetic Resonance Imaging, e.g., for stroke or tumor assessment. These sequences, especially if acquired in a segmented approach, are prone to artefacts such as ghosting, geometric distortions, blurring and mesh artefacts. These can be caused due to patient physiology, hardware imperfections or mismatches between acquired segments. There exists a variety of software-based correction approaches to counter these artefacts which however fail in various scenarios. The goal of this thesis is to evaluate deep learning based correction approaches to improve the image quality compared to conventionally implemented correction approaches. An example for this is shown below.

1) Zero filled Fourier Transform, 2) POCS Recon, 3) Target

The figure above shows Partial Fourier reconstruction for a readout segmented EPI acquisition (RESOLVE). The Zero filled reconstruction is blurry and the POCS reconstruction suffers from mesh/stripe artefacts. The goal is to train a neural network to perform Partial Fourier reconstruction to overcome the downsides of the conventional approaches.

Bewertung verschiedener Verfahren zur Lungenregistrierung und möglicher Verbesserungen dieser im Bereich der CT Bildgebung

Die Strahlentherapie ist häug essentieller Bestandteil der Behandlung von verschiedenen Tumorerkrankungen.
Um bestmögliche Ergebnisse zu erzielen, wird im Voraus für jeden Patienten eine individuelle
Bestrahlungsplanung durchgeführt. Bei Bestrahlungszielen im Bereich des Brustkorbs, wie z.B.
Brustkrebs oder Lungenkrebs, ist die Planung aufgrund der atmungsbedingten zeitlichen Lageänderung
komplex. [1] Zusätzlich besteht noch die Möglichkeit, den physiologischen Zustand einzelner Lungenbereiche
in die Planung miteinzubeziehen. So soll die Bestrahlung bestenfalls durch bereits geschädigte
anstatt gesunder Lungenareale appliziert werden, um die Lungenfunktion zu erhalten.
Für diesen Zweck kann ein Ventilations CT durchgeführt werden. Hierbei werden CT Aufnahmen von
verschiedenen Atemzuständen angefertigt. Aus Unterschieden in den aufeinander registrierten Atemzust
änden können Lungenbereiche identiziert werden, die vergleichsweise schwach durchlüftet und somit
wenig relevant für die Lungenleistung sind.
Aufgrund der Elastizität der Lunge gestaltet sich die Registrierung jedoch als schwierig. Auÿerdem
existiert noch kein Goldstandard für die Evaluierung verschiedener Lungenregistrierungsmethoden. [2]
Ziel dieser Arbeit ist es, systematische Unterschiede verschiedener Registrierer herauszuarbeiten. Dazu
sind folgende Aufgaben vorgesehen:
ˆ Literaturrecherche über verschiedene Verfahren der Lungenregistrierung und Stand der Technik
ˆ Sammlung und Strukturierung von Patientenscans anhand diverser Eigenschaften wie z.B. Pathologien
ˆ Entwicklung von geeigneten Qualitätskriterien
ˆ Durchführung der Registrierung mittels unterschiedlicher Verfahren
ˆ Analyse der Ergebnisse
ˆ (Evaluierung methodischer Verbesserung von Registrierern)
Literatur
[1] Sean Brown, Kathryn Banll, Marianne C. Aznar, Philip Whitehurst, and Corinne Faivre Finn.
The evolving role of radiotherapy in non-small cell lung cancer. The British Journal of Radiology,
92(1104):20190524, December 2019.
[2] K. Murphy, B. van Ginneken, J. M. Reinhardt, S. Kabus, Kai Ding, Xiang Deng, Kunlin Cao,
Kaifang Du, G. E. Christensen, V. Garcia, T. Vercauteren, N. Ayache, O. Commowick, G. Malandain,
B. Glocker, N. Paragios, N. Navab, V. Gorbunova, J. Sporring, M. de Bruijne, Xiao Han,
M. P. Heinrich, J. A. Schnabel, M. Jenkinson, C. Lorenz, M. Modat, J. R. McClelland, S. Ourselin,
S. E. A. Muenzing, M. A. Viergever, Dante De Nigris, D. L. Collins, T. Arbel, M. Peroni, Rui Li,
G. C. Sharp, A. Schmidt-Richberg, J. Ehrhardt, R. Werner, D. Smeets, D. Loeckx, Gang Song,
N. Tustison, B. Avants, J. C. Gee, M. Staring, S. Klein, B. C. Stoel, M. Urschler, M. Werlberger,
J. Vandemeulebroucke, S. Rit, D. Sarrut, and J. P. W. Pluim. Evaluation of Registration
Methods on Thoracic CT: The EMPIRE10 Challenge. IEEE Transactions on Medical Imaging,
30(11):19011920, November 2011.

Synthetic X-rays from CT volumes for deep learning

X-rays are a standard imaging modality in clinical care and various artificial intelligence (AI) applications have been proposed to support clinical work with X-ray images. AI-based applications employing deep learning requires a great number of training data that must be structured and annotated with respect to the anatomical regions of interest. However, acquiring this training data is challenging due to the time intensive, error prone and expensive nature of annotating and labelling image data.  As an alternative, Computed Tomography (CT) data along with annotations generated from existing AI-software can be used to generate synthetic X-ray images with the corresponding transformed annotations [1][2].

In this master’s thesis, the use of synthetic X-rays generated from CT volumes for deep learning shall be investigated. Synthetic X-rays are a simulation of radiographic images produced through a perspective projection of the three-dimensional (CT) image volume onto a two-dimensional image plane. The application focuses mainly on orthopedic imaging, in particular spine imaging. A deep neural network is trained to identify anatomical landmarks of the vertebrae (e.g. corners or centers) using only the generated synthetic X-ray data [3][4]. This trained network is then extensively tested on unseen datasets of real X-ray images. The hypothesis is that the synthetic 2D data from CT volumes (image, annotations) can improve training a Deep Neural Network for X-ray applications. The results should be able to demonstrate if generated images can effectively be used in place of real data for training.

 

The thesis consists of the following milestones:

1: Create a landmark detector model (vertebral corners or center) from real spine X-ray data

2: Generate synthetic X-ray images and corresponding annotations from available CT data

3: Train the landmark detector model using only the synthetic X-rays

4: Evaluate the results generated from the two trained models

References:

[1] B. Bier, F. Goldmann, J. Zaech, J. Fatouhi, R. Hageman, R. Grupp, M. Armand, G. Osgood, N. Navab, A. Maier & M. Unberath, “Learning to detect anatomical landmarks of the pelvis in X-rays from arbitrary views”, International Journal of Computer Assisted Radiology and Surgery 14, 1463-1473 (2019)

[2] M. Unberath, J. Zaech, S.C. Lee, B. Bier, J. Fatouhi, M. Armand & N. Navab, “Deep DRR – A catalyst for machine learning in fluoroscopy-guided procedures” (2018) arXiv:1803.08606 [physics.med-ph]

[3] Khanal B., Dahal L., Adhikari P., Khanal B. (2020) Automatic Cobb Angle Detection Using Vertebra Detector and Vertebra Corners Regression. In: Cai Y., Wang L., Audette M., Zheng G., Li S. (eds) Computational Methods and Clinical Applications for Spine Imaging. CSI 2019. Lecture Notes in Computer Science, vol 11963. Springer, Cham.

[4] J. Yi, P. Wu, Q. Huang, H. Qu, D.N. Metaxas, “Vertebra-focused landmark detection for scoliosis assessment” (2020) arXiv:2001.03187 [eess.IV]

 

Automatic characterization of nanoparticles using deep learning techniques

Nanotechnology has been bringing numerous advances in all its applications fields, ranging from electronics to
medicine. Nanomedicine, as it is called the emerging field of the meeting of pharmaceutical, biomedical sciences
and nanotechnology, investigates the potentials of nanoparticles to improve diagnostics and therapy in healthcare
[1, 2]. Interactions of these particles with the biological environment are dependent on some key factors, as particle
size, shape and distribution. These aspects impact the particles efficacy, safety, and toxicological profiles [1–4].
Therefore, it is important to develop an accurate method to measure particle size, distribution, and characterize them
to assess their quality and safety [2].
To assist in this task, an automatic yet reliable method would be desirable to eliminate human subjectivity [5]. Recently,
deep learning is emerging as a powerful tool and will continue to attract considerable interests in microscopy
image analysis, as object detection and segmentation, extraction of regions of interest (ROIs), image classification,
etc. [6].
In this thesis, we will employ a well-established deep neural network to automatically detect, segment, and classify
nanoparticles in microscopy images. Additionally, we will extend the method to measure the size of our nanoparticles,
which also requires annotation of the particles’ measurements beforehand. Finally, we will evaluate our approach and
analyze our outcomes.
The thesis will include the following points:
• Getting familiar with the nanoparticle characterization problem and tools applied in this work.
• Extend the dataset’s annotations with the nanoparticles measurements.
• Modify the chosen network to predict the nanoparticles’ size.
• Employ the modified network to detect, segment, and classify nanoparticles and predict their size.
• Evaluate the results according to appropriate metrics for the task.
• Elaboration of further improvements for the proposed method.
Academic advisors:

References
[1] D. Bobo, K. J. Robinson, J. Islam, K. J. Thurecht, and S. R. Corrie, “Nanoparticle-based medicines: a review
of fda-approved materials and clinical trials to date,” Pharmaceutical research, vol. 33, no. 10, pp. 2373–2387,
2016.
[2] F. Caputo, J. Clogston, L. Calzolai, M. R¨osslein, and A. Prina-Mello, “Measuring particle size distribution of
nanoparticle enabled medicinal products, the joint view of euncl and nci-ncl. a step by step approach combining
orthogonal measurements with increasing complexity,” Journal of Controlled Release, vol. 299, pp. 31–43, 2019.
[3] V. Mohanraj and Y. Chen, “Nanoparticles-a review,” Tropical journal of pharmaceutical research, vol. 5, no. 1,
pp. 561–573, 2006.
[4] A. G. Roca, L. Guti´errez, H. Gavil´an, M. E. F. Brollo, S. Veintemillas-Verdaguer, and M. del Puerto Morales, “Design
strategies for shape-controlled magnetic iron oxide nanoparticles,” Advanced drug delivery reviews, vol. 138,
pp. 68–104, 2019.
[5] B. Sun and A. S. Barnard, “Texture based image classification for nanoparticle surface characterisation and machine
learning,” Journal of Physics: Materials, vol. 1, no. 1, p. 016001, 2018.
[6] L. Lu, Y. Zheng, G. Carneiro, and L. Yang, “Deep learning and convolutional neural networks for medical image
computing,” Advances in Computer Vision and Pattern Recognition; Springer: New York, NY, USA, 2017.

Weakly supervised localization of defects in electroluminescence images of solar cells

With the recent rise of renewable energy, usage of solar energy has also grown rapidly. Detecting faulty panels inproduction and on-site therefore has become more important. Prior works focus on fault detection using the e.g. the current, voltage and temperature of solar modules as inputs [6, 1], but the localization of defects using imaging and machine learning has only recently gained attention [5, 4].

This work studies the detection of defects in electroluminescence (EL) images of solar cells using state of the art computer vision techniques with a focus on crack detection. Previously, in order to train a model to predict pixel classifications, exhaustive labelling of every pixel in an image of the dataset was required. State of the art training methods allow models to predict coarse segmentations using only image-wise classification labels by means of weakly supervised training. Recently, it has been shown that these methods can be applied to perform a coarse segmentation of cracks on EL images of solar cells as well [5].

This thesis aims to improve upon the existing method. To this end, weakly supervised learning methods like guided backpropagation, grad-cam, score-cam and adversarial learning [5, 9, 2, 7, 8, 3] will be implemented to train a model that reliably and accurately localizes cracks in a dataset of about 40k image-wise annotated EL images of solar cells. Finally, a thorough evaluation will show, if these methods can improve over the state of the art.

References

[1] Ali, Mohamed Hassan, et al. “Real time fault detection in photovoltaic systems.” Energy Procedia 111 (2017): 914-923.
[2] Chattopadhay, Aditya, et al. “Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks.” 2018 IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, 2018.
[3] Choe, Junsuk, and Hyunjung Shim. “Attention-based dropout layer for weakly supervised object localization.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2019.
[4] Deitsch, Sergiu, et al. “Automatic classification of defective photovoltaic module cells in electroluminescence images.” Solar Energy 185 (2019): 455-468.
[5] Mayr, Martin, et al. “Weakly Supervised Segmentation of Cracks on Solar Cells Using Normalized L p Norm.” 2019 IEEE International Conference on Image Processing (ICIP). IEEE, 2019.
[6] Triki-Lahiani, Asma, Afef Bennani-Ben Abdelghani, and Ilhem Slama-Belkhodja. “Fault detection and monitoring systems for photovoltaic installations: A review.” Renewable and Sustainable Energy Reviews 82 (2018): 2680-2692.
[7] Wang, Haofan, et al. “Score-CAM: Score-weighted visual explanations for convolutional neural networks.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 2020.
[8] Zhang, Xiaolin, et al. “Adversarial complementary learning for weakly supervised object localization.” Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2018.
[9] Zhou, Bolei, et al. “Learning deep features for discriminative localization.” Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.