Contextual Meta Knowledge integrated into a Fully-Automated Multi-Stage Deep Learning Framework for Killer Whale Individual Classification

1. Problem
Killer whales (also known as Orcas) are the topmost predators of marine life, and the availability of right information is crucial in studying the population. Photo-identification, a research method, is used to identify individual orca based on the photographs of their natural markings [4]. But the identification task is extremely laborious and time consuming. Additionally, it requires high expertise in the field. As opposed to this, a numerous computer-based algorithm is created to simplify the task of detection and classification of the Orca population [2]. However, any computer-based algorithm experiences a struggle to perform in a situation caused by various reasons including, low resolution, poor luminosity, orientation of Orca, dynamic backgrounds, bad weather condition. Hence, the scope for improvement cannot be ignored and a better generalized solution addressing these challenges needs to be presented.
2. Motivation
With the advancement in machine learning techniques, Marine Researchers are accurately predicting and consolidating the Killer whale data with availability of the limited information [1].
Meta information related to Orca individuals such as age, gender, shape of the dorsal fin in combination with saddle patch, family relationship to one another, movement pattern etc., plays a significant role in studying Orca individuals. This information can be employed when the task of detection and classification seem challenging.
The statistical approach to Bayesian theory has found its wide range of applications in the field of machine learning [3]. The idea is to use Bayesian approach to improve accuracy of the existing framework.
3. Proposition
The objective of this Master thesis is to obtain ultimate result in terms of accuracy in Orca detection and classification. This can be achieved by building a statistical correlation between Orca occurrences and their behavioural pattern with one another using the available information (also known as ‘meta’).
With meta-information such as image labels and day, time, and location of the capture a matrix can be built which contains relative information of Orca to one another. This matrix can further be utilized to improve recognition and classification accuracy.
Since the algorithm is data-driven and the matrix is built from input data, it is important to provide unbiased evaluation and not over or underfit. For this purpose, 2011 to 2017, except 2018, data is utilized. Furthermore, the data has been split into training, testing and validation bit out of which test split is excluded.
References:
1. Ahsan Jalala, Ahmad Salmana, Ajmal Mian, bMark Shortisc, Faisal Shafait., “Fish detection and species classification in underwater environments using deep learning with temporal information”, DOI: 10.1016/j.ecoinf.2020.101088
2. Alexander Gebhard, 2020, “Orca Individual Identification based on Image Classification Using Deep Learning”
3. Kersten D, Mamassian P, Yuille A. Object perception as Bayesian inference. Annu Rev Psychol. 2004; 55:271-304. doi: 10.1146/annurev.psych.55.090902.142005. PMID: 14744217.
4. J. Towers, G. Sutton, T. Shaw, M. Malleson, D. Matkin, B. Gisborne, J. Forde, D. Ellifrit, G. Ellis, J. Ford, et al. Photo-identification catalogue, population status, and distribution of bigg’s killer whales known from coastal waters of british columbia, canada. Fisheries and Oceans Canada, Pacific Biological Station, Nanaimo, BC, 2019.