Snapshot spectral mosaic imaging sensor architecture (SSI) has been successfully developed, allowing dynamic scenes to be captured using miniaturized platforms. However, SSI systems encounter a core trade-off concerning spatial and spectral resolution due to the assignment of individual spectral bands to each pixel. While the SSI camera manufacturer provides a pipeline to process such data, we propose to process the RAW SSI data directly. In practice, this strategy is much more accurate than post-processing after the pipeline.
In particular, in the first part of our paper (see link below), we propose a low-rank matrix factorization and completion framework which jointly tackles both the demosaicing and the unmixing steps of the SSI data. In particular, we expand the well-known pure pixel assumption to the SSI sensor level and propose two dedicated methods to extract the endmembers. The first one can be seen as a weighted Sparse Component Analysis (SCA) method, while the second one relaxes the abundance sparsity assumption of the former. The abundances are then recovered by applying the naive approach with the fixed extracted endmembers. Finally, we experimentally validate the merits of the proposed methods using synthetically generated data and real images obtained with an SSI camera.
The code is available at: https://github.com/kinan3bb3s/Snapshot_Image_Demosaicing_Unmixing_Completion
You can use it for for research or educational purpose. In that case, please cite:
K. Abbas, M. Puigt, G. Delmaire, G. Roussel, Locally-Rank-One-Based Joint Unmixing and Demosaicing Methods for Snapshot Spectral Images. Part I: a Matrix-Completion Framework, IEEE Transactions on Computational Imaging, vol. 10, pp. 848-862, 2024. https://doi.org/10.1109/TCI.2024.3402322
For any suggestions or questions about this code, please contact: Kinan.3bbas [at] gmail.com and Matthieu.Puigt [at] univ-littoral.fr.