Index

Towards Efficient Incremental Extreme Value Theory Algorithms for Open World Recognition

Classification problems that evolve over time require classifiers that can adapt to previously unseen
classes and also to a change in the class over time. This is called “open world classification“ and is an
already well established research topic [1, 2, 3, 4, 5]. One good example for this is face recognition. A
classifier for this task should recognize previously unseen persons and classify them in another class.
But also the face of a person might change over time, that is where incremental updates would be
better suited instead of retraining the complete models. While there are many classifiers performing
this static task, it is not straightforward to transfer those into an efficient incremental update algorithm
[6]. The Extreme Value Machine (EVM) [6] is an open set classifier, which can perform incremental
learning and is based on a strong theoretical foundation. It uses extreme value theory based calibrations
to fit Weibull-distributions around each sample. Compared to a classification via a similarity function
and thresholds, the EVM leads to naturally softer and more robust bounds. It is also less sensitive to
noise and performs well, especially in open space [7].
However the Extreme Value Machine can be used incrementally, there is no efficient update mechanism
provided. Transferring an algorithm to incremental updates can have several direct problems like
finding an efficient update function, or a limitation in space. One cannot save all previously seen
samples. Therefore a model reduction algorithm, which approximates the set cover algorithm, is
proposed in the EVM. Yet there are also several indirect problems, like a concept drift, that is when
the samples for a class change, either gradually or abruptly. The model is able to adapt to it, but with
it comes another challenge the so called “stability-plasticity dilemma“ [8]. This means the weigh up
between a rather fast or slow adaption to change in one class. A fast adaption can result in an unwanted
adaption to noise. Yet a slow adaption can lead to missing cyclical or constantly gradual change in the
data. Also the model reduction used in the EVM can lead to unbalanced classes and in the extreme
case to a nearly complete removal of some classes. This is called “catastrophic forgetting“ [8]. These
problems are not directly solved by the EVM and should also be assessed in this work.
In this thesis, the EVM will be extended by incremental update mechanisms. Starting with an exact
naive approach, later approximative algorithms will be tested for both efficiency and accuracy. One of
the main problems in terms of efficiency is that when adding new samples to the EVM, all previously
trained points need to be updated according to the naive EVM algorithm. Also when updating an
existing sample it needs to be compared to all the samples of all other classes. The first optimizations
are based on these two properties. First not all previously trained samples have to be updated, finding
the ones, on sample and class base, is the challenge. Second not all feature-points have to be compared
to while (re-)training another feature-point. Later those variants will be evaluated on different datasets.
One important one will be a face dataset for the above stated, given challenges.
The thesis consists of the following milestones:
• Implementation of the exact incremental update algorithm.

• Evaluating performance on MNIST, ImageNet, Face Dataset.

• Expanding of approximative update algorithms.

• Further experiments regarding learning and forgetting procedure.

The implementation should be done in C++.

References
[1] Walter Scheirer, Anderson Rocha, Archana Sapkota, and Terrance Boult. Toward Open Set Recognition.
IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:1757–72, 07 2013.
[2] Abhijit Bendale and Terrance E. Boult. Towards Open Set Deep Networks. CoRR, abs/1511.06233, 2015.
[3] Pedro Ribeiro Mendes Júnior, Jacques Wainer, and Anderson Rocha. Specialized Support Vector Machines
for Open-Set Recognition. CoRR, abs/1606.03802, 2016.
[4] Ziwei Liu, Zhongqi Miao, Xiaohang Zhan, Jiayun Wang, Boqing Gong, and Stella X. Yu. Large-Scale Long-
Tailed Recognition in an Open World. In IEEE Conference on Computer Vision and Pattern Recognition
(CVPR), 2019.
[5] Zongyuan Ge and Sergey Demyanov and Zetao Chen and Rahil Garnavi. Generative OpenMax for Multi-
Class Open Set Classification. In British Machine Vision Conference Proceedings 2017.
[6] Ethan M. Rudd, Lalit P. Jain, Walter J. Scheirer, and Terrance E. Boult. The Extreme Value Machine. CoRR,
abs/1506.06112, 2015.
[7] Manuel Günther, Steve Cruz, Ethan M. Rudd, and Terrance E. Boult. Toward Open-Set Face Recognition.
CoRR, abs/1705.01567, 2017.
[8] Alexander Gepperth and Barbara Hammer. Incremental Learning Algorithms and Applications. In European
Symposium on Artificial Neural Networks (ESANN), 2016.

Deep Learning-based Matching of Chest X-Ray Scans

The use of human identification has become an increasingly important factor over the past years, with
facial recognition being potentially the most common form used in daily life. But the face is not the
only biometric identifier that can be used as a feature for identification. In this work, we will investigate
chest X-rays as biometric identifiers. If they were proven to be viable, it would for example allow
identification post mortem, where common techniques currently have shortcomings [1]. Also, a success
in such a way of identification may have far-reaching consequences and implications concerning data
protection and anonymity in the medical field.
In pattern recognition, the use of deep learning has proven to be successful in improving or even
replacing classical methods entirely. To test the limits of what is currently possible, a neural network
will be created that takes in two different x-ray scans as inputs and outputs a score measuring their
similarity.
To increase the chances of success, a registration step will be incorporated in the preprocessing step. It
will be be implemented as a neural network layer, as this has proven to be effective in the past [2].
The thesis consists of the following milestones:
• Testing out the capabilities of different network architectures concerning the task of finding
matches in chest X-Ray scans
• Further enhancing the functionality by incorporating a layer into the network that is capable of
affine registrations, e. g. by means of a spatial transformer network [3]
The implementation should be done in Python.

References
[1] Ryudo Ishigami, Thi Thi Zin, Norihiro Shinkawa, and Ryuichi Nishii. Human identification using x-ray
image matching. In Proceedings of The International MultiConference of Engineers and Computer Scientists
2017, volume 1, pages 415–418, 2017.
[2] Grant Haskins, Uwe Kruger, and Pingkun Yan. Deep learning in medical image registration: a survey.
Machine Vision and Applications, 31(1–2), Jan 2020.
[3] Max Jaderberg, Karen Simonyan, Andrew Zisserman, and Koray Kavukcuoglu. Spatial transformer networks.
In Advances in Neural Information Processing Systems 28, pages 2017–2025. Curran Associates, Inc., 2015.

Analysis of NVIDIA Optix Engine for Ray Tracing in SPECT

Looking for a student for the project: Analysis of NVIDIA Optix as a Ray Tracing platform for SPECT forward projection

Topic motivation

Ray tracing is massively used in videogames to determine what object within the scene should be shown in the viewpoint of the observer.
Furthermore Ray Tracing is also used to determine the shadows, lights and reflections to be portrayed in the screen.
Optix is an extremely powerful API designed by NVIDIA due to its modularity and its flexibility. In 2015, Optix was used to model a SPECT system, achieving a significant speed up over other simulation frameworks for the same task [1]

Project description

The project would consist of five parts:

Part I: Set up Optix as a ray tracing framework for nuclear imaging, without physics
Part II: Run a simulation with a simple SPECT parallel hole collimator
Part III: Set up Optix as a ray tracing framework for nuclear imaging, with physics
Part IV: Validation of the tool with simulated data from SIMIND (data provided)
Part V: Validation of the tool with data acquired from a system (data provided)

Success measurements:

Project would be considered successful after step II
At step IV, it could become a conference paper.

Other information:

Topic can be 5 or 10 ECTS Research/Master Project. Can also be extended to a thesis.
Contact: maximilian.reymann@fau.de
Applicants ideally have experience with C++ or GPU programming, or are looking to gain expertise in these areas.

GAN Generated Model Observer for one Class Detection in SPECT Imaging

Early stage inﬂammatory musculoskeletal diseases classiﬁcation with deep learning

The Pattern Recognition Lab together with the medical clinic 3 (rheumatology and immunology) is offering the following master thesis:

„Early stage inﬂammatory musculoskeletal diseases classiﬁcation with deep learning“

Overview

Close collaboration with the clinic
Development of deep learning-based classification networks
Development of a neural network approach that combines clinical data and MRI images of the patients

Requirements

Strong background in implementing DL methods (Python)
Knowledge of MRI physics

For more information reach out to lukas.folle@fau.de

Start, follow, read, stop: Incorporating new steps into end-to-end full-page handwriting recognition method

In this work, new steps are incorporated into a known offline recognition method [1] as an attempt to
improve the transcription of degraded and poor-quality historical documents. The previously proposed
model consists of three components:
1. Start-of-line (SOL)
This network predicts the starting points of lines, together with an indication of the size and
direction of the handwriting.
2. Line-follower (LF)
Given a starting point, the LF network follows the handwriting line in incremental steps and
outputs a dewarped line image that is suitable for text recognition purposes.
3. Handwriting recognition (HWR)
After having the LF network produce several normalized line images, these can then be fed to a
CNN-LSTM HWR network [2] to produce transcriptions of the detected lines.
The method performed well on warped lines and has the advantage of outputting polygonal regions
instead of bounding boxes [3], but it still has several shortcomings, specially when considering
documents where unrelated pieces of information are frequently horizontally adjacent to one another.
It cannot detect and adapt to changes in handwriting size either, relying solely on the initial prediction
made by the SOL network to extract lines.
Modifications are to be made to the network architecture of the model in order to address these
shortcomings, and the thesis would then consist of the following milestones:
• Extending the SOL network architecture in order to include End-of-Line (EOL) detection.
• Modifying the LF network architecture to capture variations in handwriting size.
• Applying the LF network backwards from EOL predictions and finding an effective way of
merging both line information.
• Evaluating performance on historical full page datasets.
• Further experiments regarding procedure and network architecture.

The implementation should be done in Python.

References
[1] Davis B. Barrett W. Price B. Cohen S. Wigington C., Tensmeyer C. Start, follow, read: End-to-end full-page
handwriting recognition. Computer Vision – European Conference on Computer Vision 2018 (ECCV) pages
372-388, 2018.
[2] Stewart S. Davis B. Barrett W. Price B. Cohen S. Wigington, C. Data augmentation for recognition of
handwritten words and lines using a cnn-lstm network. 14th International Conference on Document Analysis
and Recognition (ICDAR) pp. 639–645, 2017.
[3] Wolf C. Moysset B., Kermorvant C. Full-page text recognition: Learning where to start and when to stop.
14th IAPR International Conference on Document Analysis and Recognition (ICDAR), 2017.

AI based Localization of Ischemic Heart diseases using Magnetocardiography signals

Description – Magnetocardiography (MCG) is a functional imaging system that is used for medical heart diagnosis. The so-called heart current induces a magnetic field that can be measured by an MCG system.This current results from the excitation of the heart muscle cells. Thus, the signal of the magnetic field exactly maps the electrophysiological activity of the heart. Due to this passive measuring technique, an MCG does not need any electromagnetic radiation. In comparison to an electrocardiography system (ECG), the heart signal is not evaluated by different ECG lead measurements. The MCG signal is acquired in the proximity of the heart. The signal can be detected almost without inhomogeneous influences or other restrictions. In particular, the MCG is more sensitive to tangential current as an ECG. Furthermore, an MCG is additionally sensitive to the vortex current (e.g. perpendicular to the tangent space) that cannot be detected by an ECG system. The MCG reconstruction task is based on the Biot-Savart law and leads to an inverse problem. Hence, its solution is also known as pseudo current. Biomagnetik Park GmbH (bmp) provides a high data quality with its MCG technology.The Superconducting Quantum Interference Devices (SQUIDs) can detect the MCG signal in the femto Tesla range. The medical sensitivity can be achieved currently up to approximately 97% (Park et al.1 Dobutamine stress magnetocardiography for the detection of significant coronary artery stenoses – A prospective study in comparison with simultaneous 12-lead electrocardiography).The article (Tao et al. Magnetocardiography based Ischemic Heart Disease Detection and Localization using Maschine Learning Methods) demonstrates that the potential of magnetocardiography is not exhausted analytically. Since the early detection of ischemic heart diseases based on MCG imaging is already ensured, e.g. refer to (Park and Jung Qualitative and quantitative description of myocardial ischemia by means of magnetocardiography), the functionality shall be now extended to a localization feature. i.e. it should be focused, where the stenosis is placed. The lack of morphological information in the signal results in an inverse problem that complexity increases proportionally to the accuracy of the model

Thesis objectives – To analyze and evaluate state-of-the-art deep learning methods, e.g. shown in (Maier et al. A gentle introduction to deep learning in medical image processing), in order to enable the MCG based localization of ischemic heart disease. Therefore, the publications Tao et al. should be used as a conceptual guideline. The objective of the master thesis consists in reconstructing at least the results of this article. In particular, the results shall be enhanced by the application of deep learning optimization methods and the sensitive bmp MCG data.

Rigid Registration of Bones for Freely Deforming Follow-Up CT Scans

Estimation and evaluation of a CT image based on electromagnetic tracking data for adaptive interstitial multi-catheter breast brachytherapy

The second leading cause of death worldwide is cancer. In 2018 9.6 million people died of cancer [1]. Additionally, in 2018 24.2% of the cancer incidences in women were breast cancer [2]. Common therapies for breast cancer are chemotherapy, surgery, and radiation therapy [3]. There are two options for radiation therapy: whole breast irradiation and the accelerated partial breast irradiation [4]. One well-established possibility to apply accelerated partial breast irradiation is the interstitial multi-catheter brachytherapy (iBT), a treatment technique that uses γ-radiation from enclosed radioactive sources positioned very close to the tissue to be irradiated [5]. In order to guide the radiation source there, multiple plastic catheters are inserted surgically and remain within the body throughout the course of the treatment (typically five days). A CT scan, the so-called planning CT (PCT), serves as a basis for treatment planning.
At the University Hospital Erlangen, the standard protocol for iBT consists of nine treatment fractions of 3.8 Gy per fraction, administered within five consecutive days. To date, the treatment plan remains unchanged throughout this period and hence does not account for interfractional changes. However, in a study assessing the need for treatment adaptation in the course of the treatment it was found that 4% of the patients would have benefitted from replanning [6]. In our workflow, the only measure to ensure the correct position of the catheter implant is a follow-up CT (FCT) after the fourth fraction. However, this exposes the patient to additional dose and therefore cannot be performed prior to every fraction.
The aim of this Master’s thesis is to develop a CT estimation (estCT) based on the data from the PCT and a dose-free electromagnetic tracking system. The first step is calculating the deformation vector field from the EMT data acquired immediately after the PCT (without moving the patient in between) and the EMT data acquired immediately after the FCT. In a second step, this deformation field is used to warp the PCT to yield the estCT. The estCT can then be compared to the FCT as a ground truth.
If the estCT could accurately “reproduce” the FCT, this method would allow describing interfractional changes of the catheter geometry without additional dose exposure. Since EMT measurements are easy and fast to conduct at any time point in the course of the treatment, estCTs could be acquired prior each treatment fraction. Finally, these estCTs can be evaluated dosimetrically to trigger patient-specific changes to the treatment plan, which would mean an essential step towards adaptive brachytherapy.
Master’s Thesis Description
References
[1] World Health Organization: “Cancer”, 2018, https://www.who.int/news-room/fact-sheets/detail/cancer, (Date last accessed 2018-06-19)
[2] BRAY, Freddie, et al. Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA: a cancer journal for clinicians, 2018, 68. Jg., Nr. 6, S. 394-424.
[3] WAKS, Adrienne G.; WINER, Eric P. Breast cancer treatment: a review. Jama, 2019, 321. Jg., Nr. 3, S. 288-300.
[4] STRNAD, Vratislav, et al. Practical handbook of brachytherapy. UNI-MED Verlag, Bremen-London-Boston, 2014, S. 166-183.
[5] NJEH, Christopher F.; SAUNDERS, Mark W.; LANGTON, Christian M. Accelerated partial breast irradiation (APBI): a review of available techniques. Radiation Oncology, 2010, 5. Jg., Nr. 1, S. 90.
[6] KALLIS, Karoline, et al. Is adaptive treatment planning in multi-catheter interstitial breast brachytherapy necessary?. Radiotherapy and Oncology, 2019, 141. Jg., S. 304-311