Deep learning based information retrieval from technical drawings

Type: MA thesis

Status: running

Supervisors: Siming Bayer, (FAU FAPS), (Firma Baumüller), Andreas Maier

According to the Competitive Industrial Performance Report 2020 [1], Germany is recognized as the most competitive country in terms of industrial performance, largely due to the quality and competitiveness of its products. Achieving precise and efficient automation can keep costs low and minimize human errors, enabling companies to produce competitive products even with full customization that leads to a wide variety of products and small batch sizes in today’s flexible world.
Baumüller Nürnberg GmbH has been designing motors for over 30 years, resulting in more than 10,000 different models. To automate the verification process of the designed assemblies, a research project [2] has been initiated by FAU, Baumüller Nürnberg GmbH, and SIMON IBV GmbH. It aims to utilize deep learning techniques for visual verification. To achieve this, a comparison between the desired and measured parameters is necessary. To fully automate this verification process, the desired parameters must be extracted precisely from technical documentation, such as technical drawings, without manual intervention. This master’s thesis focuses on the extraction process of the desired parameters, which faces two major challenges. The first challenge is the significant changes in the documentation over a period of more than 30 years, not only in terms of structure but also in storage (digital or handwritten). The second challenge is the handling of multiple implementations since each documentation describes a different assembly.
Cutting-edge techniques for extracting technical information from technical documentation have been discussed in recent research studies. For example, in Paper [3] and at the International Conference on Flexible Automation and Intelligent Manufacturing 2022 [4], state-of-the-art approaches combine Optical Character Recognition (OCR) with object detection to accurately extract technical information from technical drawings.

The master’s thesis comprises three main components:
– a comprehensive review of the state-of-the-art techniques for extracting data from technical documentation
– an algorithm is tailored to the specific situation that involves a high variety of models
– the implemented algorithm is evaluated and tested using real-world data

[1] N. Correa and V. Todorov, Competitive Industrial Performance Report 2020, United Nations Industrial Development Organization, 2021.
[2] “MuViS – Hybride KI zur lernfähigen, dateneffizienten und erklärbaren Multi-View-Sichtprüfung von variantenreichen Montagebaugruppen“ FAU FAPS. (accessed April, 12, 2023).
[3] T. H. Nguyen, L. Van Pham, C. N. Nguyen, and V. Van Nguyen, Object Detection and Text Recognition in Large-scale Technical Drawings. 2021. doi: 10.5220/0010314406120619.
[4] C. Haar, H. Kim, and L. Koberg, “AI-Based Engineering and Production Drawing Information Extraction,” in Lecture notes in mechanical engineering, Springer, Singapore, 2022, pp. 374–382. doi: 10.1007/978-3-031-18326-3_36.