This thesis aims at designing and evaluating two open source and representative machine learning applications for on-board data processing in space systems, as part of the OBPMark-ML benchmarking suite. Recently there is an increased interest for the adoption of machine learning and artificial intelligence methods in on-board processing, as demonstrated in the European Space Agency’s (ESA) Phi-Sat-1 In-Orbit-Demonstration (IOD) mission launched in 2020 [1,2]. In addition, future missions, are expected to rely on machine learning and deep learning methods to offer increased autonomy.
However, it is not clear which hardware architectures should be employed in such future systems. Currently, space systems use simple processors specifically designed for space, which cannot provide the required performance. Therefore, several alternatives are currently investigated as future candidates for space, such as embedded GPUs, FPGAs, custom AI accelerators etc.
However, different devices have significantly different properties, e.g. in terms of number of computations per second, numerical format (integer or floating point), data width (1, 8, 16, 32 or 64 bits), memory requirements etc. which make the comparison and trade-offs of their computation performance and accuracy difficult.
In addition, there is a lack of representative application cases that can be used to perform such comparison. MLPerf, the de facto benchmarking suite for machine learning is not representative of the type of processing required in space. The only available space-related software is the open source benchmarking suite GPU4S\_Bench [3,4] (also known as OBPMark Kernels) and its evolution OBPMark (On-Board Data Processing Benchmarks) Applications, developed at the Barcelona Supercomputing Center (BSC) as part of the ESA-funded project GPU4S (GPU for Space)[5]. However, GPU4S\_Bench provides software kernels solely on algorithmic building blocks such as matrix multiplication and convolution used in deep learning, as well as a simple inference chain targeting CIFAR-10, but without evaluating performance and accuracy trade offs, nor using real space data sets for training and/or evaluation. In contrast to this, the OBPMark suite implements a set of computational performance benchmarks developed specifically for spacecraft on-board data processing applications, like radar processing and data compression [6]. OBPMark-ML is going to be a third variant of OBPMark, which will include realistic space applications covering multiple types of machine learning and deep learning processing.
Two of these applications are going to be designed and implemented in this Master’s Thesis, which will be performed during a research visit at the Barcelona Supercomputing Center, within the GPU4S ESA project. The applications will cover two different types of imaging tasks and will be trained with real space data. The design and implementation will cover both the training and the implementation parts of the applications in standard machine learning frameworks, both in Python and C. It will also include the production of the trained models in various formats, as well as the necessary material for the reproduction of the training for potentially new architectures which are not covered by the pre-trained models which will be generated. In particular:
- Instance Segmentation: Cloud Screening: The first task uses instance segmentation for cloud detection on an open source data set called Cloud95 [7]. U-Nets have become a standard approach for segmentation tasks and were also shown to be effective on cloud screening tasks [8]. Constraints for the use in on board processing are the enormous amount of parameters and the high computational cost, though. A good trade-off between computational complexity/memory footprint and prediction accuracy has to be found. For this the number of parameter needs to be scaled down. This can be achieved through a reduction of the depth and the number of filters in convolution layers. Another method is using techniques from the MobileNetv2 architecture and adapt them to the U-Net architecture [9]. Both methods will be evaluated.
- Object Detection: Ship Detection: The second application will be an object detection task, like ship detection on satellite pictures [10]. The same restrictions like in the segmentation case apply here. State-of-the-art architectures for object detection are single shot detectors (SSD) [11] and “You only look once” (YOLO) networks [12]. These architectures can use different backbones models. To reduce the number of parameters and the computational complexity, a MobileNet will be used in comparison to a heavy network like ResNet [13].
The models will be trained on Tensorflow/Keras and PyTorch, as well as converted to the ONNX format for portability and reproducibility. As different hardware like i.e. FPGAs require fixed point arithmetic, all models will be trained and provided in different precisions ranging from (double/full/half) floating point to integer (int8/int16). BSC will provide access to supercomputing resources for the training process. Accuracy-wise, they are to be compared qualitatively with literature results. The computational performance will be tested on some processor of interest, like i.e. the Myriad VPU or a Xilinx FPGA leveraging Vitis AI or embedded GPUs which have been identified as candidates for use in GPU4S. This depends of the accessibility of these development boards. The thesis is developed together with the Barcelona Computer Center (BSC) in the GPU4S program co-funded by the European Space Agency and all code and models will be published open source on Github under the current ESA-PL license.
[1] Jan-Gerd Meß, Frank Dannemann and Fabian Greif. Techniques of Artificial Intelligence for Space Applications – A Survey. In European Workshop on On-Board Data Processing (OBDP), 2019.
[2] https://www.esa.int/Applications/Observing_the_Earth/Ph-sat
[3] Ivan Rodriguez, Leonidas Kosmidis, Jerome Lachaize, Olivier Notebaert, David Steenari, GPU4S Bench: Design and Implementation of an Open GPU Benchmarking Suite for Space On-board Processing, Technical Report UPC-DAC-RR-CAP-2019-1, [online] Available: https://www.ac.upc.edu/app/research-reports/public/html/research_center_index-CAP-2019,en.html
[4] Leonidas Kosmidis, Iván Rodriguez, Alvaro Jover-Alvarez, Sergi Alcaide, Jérôme Lachaize, Olivier Notebaert, Antoine Certain, David Steenari. GPU4S: Major Project Outcomes, Lessons Learnt and Way Forward. Design Automation and Test in Europe Conference (DATE) 2021
[5] David Steenari, Leonidas Kosmidis, Ivan Rodriquez, Alvaro Jover, and Kyra Förster. OBPMark (On-Board Processing Benchmarks) – Open Source Computational Performance Benchmarks for Space Applications. In European Workshop on On-Board Data Processing (OBDP), 2021. [online], Available: https://zenodo.org/record/5638577
[6] OBPMark and GPU4S\_Bench open source repositories. [online] Available: https://obpmark.github.io
[7] https://www.kaggle.com/sorour/95cloud-cloud-segmentation-on-satellite-images (accessed on 1.2.2022)
[8] Johannes Drönner, Nikolaus Korfhage, Sebastian Egli, Markus Mühling, Boris Thies, Jörg Bendix, Bernd Freisleben and Bernhard Seeger. Fast Cloud Segmentation Using Convolutional Neural Networks. Remote Sens. 2018
[9] Junfeng Jing, Zhen Wang, Matthias Rätsch and Huanhuan Zhang. Mobile-Unet: An efficient convolutional neural network for fabric defect detection. Textile Research Journal, 2020.
[10]https://www.kaggle.com/c/airbus-ship-detection/overview
[11] Liu W. et al. SSD: Single Shot MultiBox Detector. Computer Vision – ECCV 2016, 2016.
[12] Joseph Redmon, Santosh Kumar Divvala, Ross B. Girshick, Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. CoRR 2015, 2015.
[13] MobileNetV2: The Next Generation of On-Device Computer Vision Networks, [online] Available: https://ai.googleblog.com/2018/04/mobilenetv2-next-generation-of-on.html.