Thesis Description
An electrical circuit diagram (ECD) is a graphical representation of an electrical circuit. ECDs consist of electrical circuit components (ECC), where for each ECC an unique symbol is defined in the international standard [1]. The ECCs are connected with lines, which correspond to wires in the real world. Furthermore, ECCs are further specified by an annotation next to their symbol, which consists of a digit followed by a unit. For instance a resistor can be denoted as ”100 mΩ” (Milliohm). Voltage sources and current sources are ECCs, which provide either a voltage (U) or a current (I) through the circuit. While U and I provided by sources are given, U and I with respect to certain ECCs have to be obtained through calculations. For small circuits this can be done by hand, however the calculation complexity grows with the size of the circuit and even more when alternating U/I sources are used, since certain component calculations become dependent on the frequency of the used source. Therefore, often a circuit simulation software (CSS) is used, where complex simulations can easily be performed in an automated way. Before a circuit can be simulated in a CSS, it first has to be modeled in the application. Refaat et al. [2] compared the drawing speed of structured diagrams by hand and with the diagram drawing tool Microsoft Visio. Their experiments have shown that drawing by hand was around 90% faster than drawing with Microsoft Visio. Since ECDs are also structured diagrams it seems that a hand drawn approach could be done more efficient than an application based drawing approach. Hence, an automated method to convert an image of a hand drawn ECD into a digital format processable by a CSS, would ease the use of CSS.
So far various researches have been conducted on the segmentation, recognition and the tracing of connections between ECCs, which will be briefly described in the following. The proposed approaches, can be structured as follows: 1) classification of ECCs [3, 4, 5], 2) segmentation and classification of ECCs [6, 7], 3) segmentation and classification of ECCs and ECD topology acquisition [8], 4) object detection of ECCs and ECD topology acquisition [9]. Moetesum et al. [6] used computer vision methods to segment ECCs from an ECD, where for different ECC types different strategies were used to obtain a segmentation mask. For instance sources were segmented by filling the region inside the source symbol, followed by a bounding box drawn around the segmentation mask. A Histogram of Oriented Gradients was applied on the region inside the bounding box, to obtain a feature vector for a following Support Vector Machine classifier. While this approach yielded good classification results, it is only partially extendable. For ECCs which have a similar shape to components which are already covered by a segmentation strategy, the existing strategy can probably be reused, but for completely new shapes, a new strategy has to be introduced. The aim of the proposed method by Dhanushika et al. [9] was to extract a boolean expression from an ECD made out of logical gate components (AND, OR, NOT, etc.). The ECS classification was modeled here by using the object detection algorithm YOLO (You Only Look Once) [10], which localizes and classifies an object in a single step. The ECD topology was recognized, by removing the bounding boxes from the image and applying a Hough Transform on the remaining connections. Hough Lines and bounding box intersections were now used to form the ECD topology, from which the final boolean expression was generated.
All of the above mentioned methods were restricted to drawings on white paper only. As it is quite common to also draw on gridded paper, this might become too restrictive for the use in real world scenarios. Furthermore, no method has been proposed so far, which aims to cover the full conversion, beginning with the image to the simulation based on a CSS formatted file.
Thus, this thesis aims to cover the development of a full processing pipeline able to convert images of hand drawn ECDs into an intermediate format, which reflects the topologies of the ECDs. Extensibility should be ensured by using an object detection deep neural network architecture, which is due to the nature of neural networks, simply extended by providing new data and labels for the training step. The pipeline should also be invariant to image quality (paper type, lighting conditions, background etc.), at least considering white and grid paper. Furthermore, the pipeline should contain the recognition of component annotations e.g. component values and voltage/current flow symbols. The conversion into a CSS format should be realized on the example of LTspice. Additionally, the used methods should be chosen such that the pipeline could be executed on mobile hardware, thus the computational effort for the whole pipeline must be kept as low as possible.
The thesis will comprise of the following work items:
- Collection of a suitable dataset
- Object detection of ECCs and annotations in images of a hand drawn ECDs
- Segmentation of the ECD from the drawing
- Identification of the ECD topology
- Postprocessing
- Building the ECD topology
- Assigning annotations to corresponding ECCs
- Embedding gathered information into a LTspice file
- Optional: Mobile demo application
References
[1] IEC-60617. https://webstore.iec.ch/publication/2723. Accessed: 21-12-2020.
[2] K. Refaat, W. Helmy, A. Ali, M. AbdelGhany, and A. Atiya. A new approach for context-independent
handwritten offline diagram recognition using support vector machines. In 2008 IEEE International Joint
Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pages 177–182,
2008.
[3] M. Rabbani, R. Khoshkangini, H.S. Nagendraswamy, and M. Conti. Hand drawn optical circuit recognition.
Procedia Computer Science, 84:41 – 48, 2016. Proceeding of the Seventh International Conference on
Intelligent Human Computer Interaction (IHCI 2015).
[4] M. G¨unay, M. K¨oseo˘glu, and O. Yıldırım. Classification of hand-drawn basic circuit components using con- ¨
volutional neural networks. In 2020 International Congress on Human-Computer Interaction, Optimization
and Robotic Applications (HORA), pages 1–5, 2020.
[5] S. Roy, A. Bhattacharya, and N. Sarkar et al. Offline hand-drawn circuit component recognition using
texture and shape-based features. Springer Science+Business Media, August 2020.
[6] M. Moetesum, S. Waqar Younus, M. Ali Warsi, and I. Siddiqi. Segmentation and recognition of electronic
components in hand-drawn circuit diagrams. EAI Endorsed Transactions on Scalable Information Systems,
5(16), 4 2018.
[7] M. D. Patare and M. Joshi. Hand-drawn digital logic circuit component recognition using svm. International
Journal of Computer Applications, 143:24–28, 2016.
[8] B. Edwards and V. Chandran. Machine recognition of hand-drawn circuit diagrams. In 2000 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.00CH37100),
volume 6, pages 3618–3621 vol.6, 2000.
[9] T. Dhanushika and L. Ranathunga. Fine-tuned line connection accompanied boolean expression generation
for hand-drawn logic circuits. In 2019 14th Conference on Industrial and Information Systems (ICIIS),
pages 436–441, 2019.
[10] J. Redmon, S. K. Divvala, R. B. Girshick, and A. Farhadi. You only look once: Unified, real-time object
detection. CoRR, abs/1506.02640, 2015.