In today’s automated robotics industry, there is a growing need for efficient and automated methods for generating text descriptions for actuator & sensor variables, functions, and properties, which are collectively referred to as symbolic descriptions hereby. This is because symbolic descriptions are often used to document the functionality of robotic systems, model the functionality of the robots, and communicate the functionality of robotic systems to human operators. The current manual process of generating text descriptions for symbolic descriptions is time-consuming, labor-intensive, and inconsistent.
This research proposes to develop an automated text generation system for symbolic descriptions in robotics using open-source pre-trained language models like GPT-2 [1], LLaMA [2], MPT etc. The decision to finalize the model will be based on factors including, but not limited to, performance, suitability for the downstream task, cost of training and inference, and interpretability of the results produced by the models. The implemented model will be able to generate text descriptions in English and German and preserve the structure of the original text descriptions. The system will be implemented using a fine-tuning approach and trained on a dataset of symbolic descriptions and their corresponding text descriptions.
The expected outcomes of this research are:
- Fine-tune a language model for automated text generation for symbolic descriptions in robotics.
- Demonstrating its efficiency in generating accurate and contextually relevant text descriptions. Evaluating and analyzing the model’s performance using perplexity, ROUGE, and BLEU scores.
- A conceptual lifecycle consideration of the training pipeline, highlighting scalability, maintenance, adaptability, and reusability aspects.
- One of the constraints of this research is to interpret the reasoning behind the model. This is an active research field, with some established techniques like attention visualization [3], feature attribution [4], and potentially, Weights and Bias could be used for this purpose.
The proposed system is still under development, and several future work directions could be explored. These include:
- Expanding the system to support other languages.
- Improving the system’s ability to generate text descriptions that are consistent with the original text descriptions.
This research has the potential to significantly contribute to the field of automated text generation for robotics. The proposed system has the potential to be used in a variety of applications, such as documentation, modeling functionality, and communication. The research results will be of interest to the research community and practitioners in these domains.
References:
[1] Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., & Sutskever, I. (2019). Language Models are Unsupervised Multitask Learners.
[2] Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., Rodriguez, A., Joulin, A., Grave, E., & Lample, G. (2023). LLaMA: Open and Efficient Foundation Language Models. ArXiv, abs/2302.13971.
[3] Yeh, C., Chen, Y., Wu, A., Chen, C., Vi’egas, F., & Wattenberg, M. (2023). AttentionViz: A Global View of Transformer Attention. ArXiv, abs/2305.03210.
[4] Zhou, Y., Booth, S., Ribeiro, M., & Shah, J.A. (2021). Do Feature Attribution Methods Correctly Attribute Features? ArXiv, abs/2104.14403.