Coded aperture snapshot spectral imager (CASSI), efficiently captures 3D spectral images by sensing 2D projections of the scene. While CASSI offers a substantial reduction in acquisition time, compared to traditional scanning optical systems, it requires a reconstruction post-processing step. Furthermore, to obtain high-quality reconstructions, an accurate propagation model is required. Notably, CASSI exhibits a variant spatio-spectral sensor response, making it difficult to acquire an accurate propagation model. To address these inherent limitations, this work proposes to learn a deep non-linear fully differentiable propagation model that can be used as a regularizer within an optimization-based reconstruction algorithm. The proposed approach trains the non-linear spatially-variant propagation model using paired compressed measurements and spectral images, by employing side information only in the calibration step. From the deep propagation model incorporation into a plug-and-play alternating direction method of multipliers framework, our proposed method outperforms traditional CASSI linear-based models. Extensive simulations and a testbed implementation validate the efficacy of the proposed methodology.
Fig. 3. Simulated measurement $\mathbf{y}_i$ for each degradation level $D_i$. (b) Absolute error of each simulated measurement with respect to the non-degradation scenario $\mathbf{y}_0=\mathbf{Hx}$. (c) Improved measurement obtained with the $\text{DNL}^2$ model. (d) Absolute error of each simulated measurement with respect to the improved measurement. The evident reduction in error across degradation levels underscores its robustness and independence.
Fig. 4. (a) Simulation results and ablation studies of $\lambda_1$ under each degradation level in the ARAD dataset. Note the improvement in the different metrics by increasing the proposed regularizer influence through the coefficient $\lambda_1$, with $\lambda_1=0$ as the baseline PnP with the linear propagation model. (b) Average results of different testing images evaluating the impact of additional noise at the highest degradation level $D_4$ from the ARAD dataset.
.
Hardware experiments
Fig. 5. (a) Average PSNR of the estimated measurements with $\text{DNL}^2$ model through iterations. (b) Comparison between the $\text{DNL}^2$ measurement performance and reconstruction quality using $\lambda=1$. (c) Convergence of the $\text{DNL}^2$ recovery loss. (d-f) Reconstruction quality with real data and ablation studies of the proposed regularizer influence, $\lambda_1$. The quality of the reconstructions shows a significant improvement, with an increase of nearly 3 dB in PSNR compared to the simple propagation model (i.e., $\lambda_1=0$).
Fig. 6. Some recovered spectral bands for an acquired testing spectral image. Note the visual and metric improvement in the false RGB representation and across the spectral bands using the $\text{DNL}^2$ model, i.e., $\lambda_1=1$.
Fig. 7. The input of the $\text{DNL}^2$ model is the spectral image in (a) resulting in a set of spatio-spectrally variant PSFs within the domain of the compressed measurements, which are spectrally plotted in (b) and two of them are zoomed in (d). The linear propagation model PSF is shown in (c).
State-of-the-art comparison
.
Fig. 8. Experimental reconstructions comparison as false RGB and corresponding signatures between Song et al.’s [22] and DNL² propagation model. Note the visual and quantitative improvement for the three testing images and in the corresponding spectral signatures at points A, B, and C.
Supplementary material
.
Fig. 1. Results and ablation studies of $\lambda_1$ using a Fixed~\cite{dabov2007image} and Learned prior~\cite{zhang2021plug} with experimental data.
Fig. 2. Experimental reconstructions comparison as false RGB and corresponding signatures between Song et al.’s~\cite{song2022high} and $\text{DNL}^2$ propagation model with a fixed \cite{dabov2007image} or learned prior \cite{zhang2021plug}.
