In the domain of predictive maintenance within manufacturing industries, the introduction of transformer-based machine learning models marks a significant leap towards more sophisticated anomaly detection mechanisms. These models can discern complex patterns in time series data, predicting potential equipment malfunctions before they lead to costly downtimes. The methodology of this thesis builds on two promising approaches: the first leverages a masked autoencoder framework designed for transformers to predict obscured parts of the input data, thereby learning the normal operational patterns of the machinery [1]. The second approach utilizes the reconstruction loss of a Variational Autoencoder (VAE) to signal deviations from the norm, which may indicate anomalies [2]. Both approaches are integral to this research’s objective of enhancing predictive maintenance strategies.
Furthering the innovation in this field, the thesis will incorporate cutting-edge transformer models such as TranAD and AnomalyBERT, which have shown exceptional results in quickly and accurately identifying anomalies in multivariate time series data [3][4]. TranAD’s focus score-based self-conditioning and adversarial training, along with AnomalyBERT’s data degradation scheme for self-supervised learning, position these models at the forefront of the predictive maintenance revolution.
The anticipated outcomes of this research encompass the development and validation of a robust framework for industrial anomaly detection. This will be achieved by adapting and optimizing transformer networks that are proficient in handling the high volatility and label scarcity characteristic of industrial datasets.
The implications of this study are profound, offering not only a technological edge to predictive maintenance but also a significant academic contribution to the application of AI in manufacturing. The models and methodologies derived from this thesis could serve as benchmarks for future research and applications in the AI and industrial maintenance landscape.
References
[1] Tang, Peiwang & Zhang, Xianchao. (2022). MTSMAE: Masked Autoencoders for Multivariate Time-Series Forecasting.
[2] Niu, Zijian, Ke Yu, and Xiaofei Wu. (2020). “LSTM-Based VAE-GAN for Time-Series Anomaly Detection.” Sensors 20, no. 13: 3738. https://doi.org/10.3390/s20133738.
[3] Shreshth Tuli, Giuliano Casale, Nicholas R. Jennings. “TranAD: deep transformer networks for anomaly detection in multivariate time series data.” Proceedings of the VLDB Endowment 15.6 (2022): 1201-1214. https://doi.org/10.14778/3514061.3514067.
[4] Yungi Jeong, Eunseok Yang, Jung Hyun Ryu, Imseong Park, Myungjoo Kang. “ANOMALYBERT: SELF-SUPERVISED TRANSFORMER FOR TIME SERIES ANOMALY DETECTION USING DATA DEGRADATION SCHEME.” Presented at the ICLR 2023 workshop on Machine Learning for IoT.