Index

Evaluation of SHViT for volumetric Semantic Segmentation in Industrial CT Scans

Industrial computed tomography (iCT) is a widely applied tool in non-destructive testing, material analysis, quality control, and metrology. Semantic segmentation of industrial CT data plays a central role in these applications by enabling quality inspection, material differentiation and part separation [1]. While convolutional neural networks (CNNs) have traditionally performed well in segmentation tasks by capturing local structures, their limited ability to model long-range dependencies poses challenges in complex 3D datasets.

Transformer-based models have recently emerged as promising alternatives. By dividing the input into patches and using self-attention mechanisms, transformers can model global dependencies. However, early vision transformers had difficulties capturing spatial structure and learning from limited data. The Swin Transformer was one of the first models to address these issues by introducing a hierarchical structure and shifted windows, combined with an inductive bias that improves generalization on small datasets [2].

Despite these advances, transformers remain resource intensive. New models such as the Shifted-window Hierarchical Vision Transformer (SHViT) aim to reduce computational costs while maintaining performance. SHViT extends the Swin architecture and improves spatial modeling and efficiency through a refined hierarchical structure with shifted windows [3].

This thesis focuses on the implementation and evaluation of a volumetric SHViT model for 3D semantic segmentation. The model is tested on a real-world dataset of industrial CT scans of boxed shoes, which includes several segmentation tasks: separating the shoes from their surroundings and identifying individual components such as the insole, outsole, and upper [4]. Typically for industrial CT data, the dataset is limited in size. Yet, its structural variability makes it an interesting benchmark for assessing model generalization. As evaluation metric for the class imbalanced segmentation dataset primarily the F₁-score is used. The network is also evaluated in terms of memory and computational resource use.

The SHViT model will be compared to a CNN-based baseline, evaluating accuracy, robustness, and computational efficiency in the context of 3D industrial segmentation. While the study aims to inform the selection of neural architectures for iCT applications, its conclusions are limited using a single dataset. Nonetheless, SHViT shows potential for broader use in iCT, as it could enable the efficient application of transformer-based models to volumetric segmentation across diverse industrial datasets.

Literature

Automated calibration of the scan trajectory in dedicated breast CT with circle-spiral trajectory

Video-based pose and distance estimation of an excavator bucket

Automatic Speech Recognition at Phoneme and Word-Level To Analyze Parkinson’s Disease

A Comparative Study of Transformer-Based Models and CNNs for Semantic Segmentation in Industrial CT Scans

Industrial computed tomography (iCT) is extensively utilized for non-destructive testing, material analysis, quality control, and metrology. In these applications, semantic segmentation is crucial, particularly for material analysis [1]. In recent years, Convolutional Neural Networks (CNNs) have been employed successfully for material segmentation, handling low-quality reconstructions, and performing complex segmentations where local context is vital. However, CNNs often struggle to capture long-range dependencies due to their localized nature.

Recently, transformer architectures have shown superior performance in various segmentation tasks. Unlike CNNs, which rely on filter banks to extract local features, transformers encode image patches into visual tokens and utilize self-attention mechanisms to process the entire input in parallel. This allows transformers to effectively capture long-distance relationships, though they may be less efficient when it comes to preserving fine local details [2]. In the context of three-dimensional (3D) data segmentation, both CNNs and transformers face challenges due to the increased memory requirements and the higher complexity of patterns that arise in 3D space.

One promising model for addressing these challenges is the Swin Transformer, which incorporates a hierarchical structure with shifted windows, enabling it to capture both local and global dependencies more efficiently [3]. To tackle the limitations of CNNs and transformers individually, hybrid models combining both architectures have been proposed. For instance, Cai et al. introduced a model that combines the Swin Transformer with CNNs for 3D segmentation tasks, taking advantage of each architecture’s strengths [2]. Both CNNs and Swin Transformers are known for their ability to generalize well on smaller datasets, thanks to their inductive bias [3] [4].

This thesis focuses on applying a hybrid approach combining the Swin Transformer and CNNs to a complex dataset of iCT scans of shoes, where the objective is to segment the shoes into their individual components, as demonstrated in previous work by Leipert et al. [5]. Despite the dataset’s small size, its high intrinsic variability and the relevance of both local and global dependencies make it an ideal candidate for evaluating segmentation methods. The segmentation will be performed in 3D, highlighting the challenges and opportunities of using these advanced models.

Through a comparative analysis of CNNs, the Swin Transformer, and their combined approaches, this thesis aims to provide insights into the strengths and limitations of each method in the context of 3D semantic segmentation on complex industrial CT datasets. The findings will contribute to improving segmentation techniques for iCT applications, potentially enhancing the accuracy and efficiency of material analysis in industrial contexts. The main limitation of the study is the application to a single dataset.

Literaturverzeichnis

EcoScapes: LLM-powered advice for crafting sustainable cities

EcoScapes: LLM-powered advice for crafting sustainable cities

Climate adaptation is vital for the sustainability and sometimes the mere survival of our urban
areas [1, chapters TS.C.8 and TS.D.1]. However, small cities often struggle with limited personnel
resources and integrating vast amounts of data from multiple sources for a comprehensive analysis
[1, chapter TS.D.1.3]. Moreover, the complexity of the topic can overwhelm administrative staff and
local politicians alike. To overcome these challenges, this thesis proposes a multi-layered system
combining specialized Large Language Models (LLMs), satellite imagery and a knowledge base to aid
in developing effective climate adaptation strategies.
Initially, the system uses provided location information to request relevant satellite imagery, which can
be used by all subsequent components.
The architecture’s modular core encompasses several LLMs and expert systems that examine the
satellite data to offer insights on different climate adaptation aspects. Examples of potential functions
might include, but are not limited to, the identification of heat islands, areas threatened by flooding, or
the assessment of vegetation cover.
In the last step, the system consolidates the findings from the preceding modules to generate a
comprehensive report on the existing situation and recommend potential adaptation strategies.
In order to assess the system’s performance, we will compare the generated outputs with those of
unaltered LLMs and a model inspired by ChatClimate [2].

References

[1] P¨ortner, H.-O., D.C. Roberts, H. Adams. et al. 2022: Technical Summary. [H.-O. P¨ortner, D.C. Roberts,
E.S. Poloczanska, K. Mintenbeck, M. Tignor, A. Alegr´ıa, M. Craig, S. Langsdorf, S. L¨oschke, V. M¨oller,
A. Okem (eds.)]. In: Climate Change 2022: Impacts, Adaptation, and Vulnerability. Contribution of
Working Group II to the Sixth Assessment Report of the Intergovernmental Panel on Climate Change
[H.-O. P¨ortner, D.C. Roberts, M. Tignor, E.S. Poloczanska, K. Mintenbeck, A. Alegr´ıa, M. Craig, S.
Langsdorf, S. L¨oschke, V. M¨oller, A. Okem, B. Rama (eds.)]. Cambridge University Press, Cambridge,
UK and New York, NY, USA, pp. 37-118, doi:10.1017/9781009325844.002.

[2] Vaghefi, S.A., Stammbach, D., Muccione, V. et al. ChatClimate: Grounding conversational AI in climate
science. Commun Earth Environ 4, 480 (2023). https://doi.org/10.1038/s43247-023-01084-x

Verification of deep learning classifications in test systems for industrial productions

Analyzing the influence of writer-dependent features in writer identification using Convolutional Neural Networks

Definition und Implementierung einer prototypischen Smart Home Schnittstelle für ein cloudbasiertes Energiemanagementsystem

Transformers vs. Convolutional Networks for 3D segmentation in industrial CT data

The current state of the art for segmentation in industrial CT are oftentimes CNNs.
Transformer based models are sparsely used.
Therefore, this project wants to compare the semantic segmentation performance of transformers (that include global context into segmentation), pure convolutional neural networks (that use local context) and combined methods (like this one: https://doi.org/10.1186/s12911-023-02129-z) on an industrial CT dataset of shoes like in this study: https://doi.org/10.58286/27736 .

Only available as Bachelors thesis / Research Project