Interactive Analysis of Multispectral and Hyperspectral Image Data
A multispectral or hyperspectral sensor captures images of high spectral resolution by dividing the light spectrum into many narrow bands. With the advent of affordable and flexible sensors, the modality is constantly widening its range of applications. This necessitates novel tools that allow general and intuitive analysis of the image data. In this work, a software framework is presented that bundles interactive visualization techniques with powerful analysis capabilities and is accessible through efficient computation and an intuitive user interface. Towards this goal, several algorithmic solutions to open problems are presented in the fields of edge detection, clustering, supervised segmentation and visualization of hyperspectral images.
In edge detection, the structure of a scene can be extracted by finding discontinuities between image regions. The high dimensionality of hyperspectral data poses specific challenges for this task. A solution is proposed based on a data-driven pseudometric. The pseudometric is computed through a fast manifold learning technique and outperforms established metrics and similarity measures in several edge detection scenarios.
Another approach to scene understanding in the hyperspectral or a derived feature space is data clustering. Through pixel-cluster assignment, a global segmentation of an image is obtained based on reflectance effects and materials in the scene. An established mode-seeking method provides high-quality clustering results, but is slow to compute in the hyperspectral domain. Two methods of speedup are proposed that allow computations in interactive time. A further method is proposed that finds clusters in a learned topological representation of the data manifold. Experimental results demonstrate a quick and accurate clustering of the image data without any assumptions or prior knowledge, and that the proposed methods are applicable for the extraction of material prototypes and for fuzzy clustering of reflectance effects.
For supervised image analysis, an algorithm for seed-based segmentation is introduced to the hyperspectral domain. Specific segmentations can be quickly obtained by giving cues about regions to be included in or excluded from a segment. The proposed method builds on established similarity measures and the proposed data-driven pseudometric. A new benchmark is introduced to assess its performance.
The aforementioned analysis methods are then combined with capable visualization techniques. A method for non-linear false-color visualization is proposed that distinguishes captured spectra in the spatial layout of the image. This facilitates the finding of relationships between objects and materials in the scene. Additionally, a visualization for the spectral distribution of an image is proposed. Raw data exploration becomes more feasible through manipulation of this plot and its link to traditional displays. The combination of false-color coding, spectral distribution plots, and traditional visualization enables a new workflow in manual hyperspectral image analysis.