On A Class of Greedy Sparse Recovery Algorithms

To accurately solve the sparse recovery problem (2), the very first question to raise is: when does it have a unique solution $\bm{s}^{\star}$? Let $\text{spark}[\bm{Q}]$ denote the smallest number of columns of $\bm{Q}$ that are linearly dependent. It is well known that the following result [30] plays a key role in sparse recovery: Let $\bm{x}$ be any $\kappa$-sparse vector in $\bm{Q}$, that is there exists a $\tilde{\bm{s}}$ with $\|\tilde{\bm{s}}\|_{0}\leq\kappa$ such that $\bm{x}=\bm{Q}\tilde{\bm{s}}$. Then, problem (2) has one and only one solution, $\bm{s}^{\star}=\tilde{\bm{s}}$, if and only if $\bm{Q}$ satisfies

Let $\kappa_{\bm{Q}}$ be the largest integer that is not bigger than $(\text{spark}[\bm{Q}]-1)/2$. Then, any $\kappa$-sparse solution $\tilde{\bm{s}}$ of $\bm{x}=\bm{Q}\bm{s}=\bm{Q}\tilde{\bm{s}}$ is the unique solution of (2) as long as $\kappa\leq\kappa_{\bm{Q}}$, which is assumed in the sequel.

Another very important concept is the restricted isometry property (RIP), introduced in [31], which is defined as follows: a matrix $\bm{Q}$ is said to satisfy the $(\kappa,\delta)$-RIP with $0\leq\delta<1$ if

where $\Sigma_{\kappa}$ denotes the set of all $\kappa$-sparse signals in $\bm{Q}$. It can be seen that if $\bm{Q}$ satisfies the $(2\kappa,\delta)$-RIP, then any $2\kappa$ columns of $\bm{Q}$ are linearly independent, and hence condition (3) holds. Therefore, if $\bm{x}=\bm{Q}\bm{s}^{\star}$ with $\|\bm{s}^{\star}\|_{0}\leq\kappa$, then, the underdetermined system $\bm{x}=\bm{Q}\bm{s}$ has a unique $\kappa$-sparse solution, which is $\bm{s}^{\star}$.

As an NP-hard problem, (2) is not tractable due to the $l_{0}$ norm. To address this, a convex relaxation is commonly employed. In particular, two classes of methods are popularly used:

It should be pointed out that the iteratively re-weighted algorithms provide state-of-the-art sparse recovery performance. However, they are computationally demanding. In particular, IRLS-based algorithms, though much faster than the IRBP-based ones, can encounter numerical issues when the $\bm{Q}$ matrix is ill-conditioned (See (7)). By contrast, OMP is significantly more efficient. The most computationally demanding stage involves solving the least squares problem to update the non-zero entries. A class of iterative methods, such as gradient pursuit (GP) and conjugate gradient pursuit (CGP), can be used to address this issue, as described in [46]. To improve the recovery accuracy and update the index set of the $\kappa$ non-zero entries more efficiently, one approach is to select multiple entries at once, rather than just one as in OMP. This leads to the development of stage-wise OMP (StOMP) [47], compressive sampling match pursuit (CoSaMP) [48], and stagewise weak GP (StWGP) [49]. A comprehensive survey of these issues as well as performance guarantees can be found in [50].

As well-known, the greedy methods can not yield a comparable sparse recovery performance to the iteratively re-weighted ones. Suppose that $\mathcal{I}_{k-1}$ has its $k-1$ indices that all fall within the support $\mathcal{I}_{\bm{s}^{\star}}$. Then, $\bm{r}_{k-1}=\bm{Q}\bm{s}-\bm{Q}(:,\mathcal{I}_{\bm{s}^{\star}})\bm{\beta}_{k-1}\triangleq\bm{Q}\tilde{\bm{s}}$, and this new system has a unique sparsest solution, denoted as $\tilde{\bm{s}}^{\star}$, with $\|\tilde{\bm{s}}^{\star}\|_{0}\leq\kappa$. The $l_{2}$-norm based minimization (8) can not ensure that the obtained index $i_{k}$ falls into $\mathcal{I}_{\bm{s}^{\star}}$ because the cost function is a complicated function of the true indices to be detected and such a function changes with $\bm{\beta}_{k-1}$. This is the main reason why the OMP algorithm cannot generally yield a sparse recovery as accurate as the iteratively re-weighted algorithms, prompting the proposal of its variants such as CoSaMP and StOMP.

The question we ask ourselves: Can we transform the system $\bm{r}_{k-1}=\bm{Q}\tilde{\bm{s}}$ into $\bm{s}$ domain by characterizing its set of solutions and then work directly on the sparse recovery problem? If an estimate of $\tilde{\bm{s}}^{\star}$ is achieved, the next index $i_{k}$ can be selected as the one that corresponds to the largest entry in magnitude of this estimate. Intuitively, by doing so the accuracy of index detection can be improved. This motivates us to deal with the sparse recovery problem on the $\bm{s}$-space, instead of the measurement $\bm{x}$-space.

The main objective of this paper is to derive a class of greedy algorithms that operate directly in the $\bm{s}$-domain and outperform classical greedy algorithms (i.e., OMP, CoSaMP) in terms of sparse recovery accuracy while maintaining similar implementation complexity. Additionally, these algorithms offer greater robustness against the numerical instability caused by $\bm{Q}$ and interferences in the signal $\bm{x}$ compared to the classical algorithms, especially the IRLS-based ones. Our contributions are summarized as follows.

• •

  Based on a characterization of solutions to equation (1), we propose a class of greedy sparse recovery approaches for a given sparsity measure. Such an approach allows us to work directly in the $\bm{s}$-space;

• •

  Two algorithms, denoted as $\text{Alg}_{GL2}$ and $\text{Alg}_{GL1}$, are derived based on $l_{2}$ and $l_{1}$, respectively. Despite having nearly the same computational complexity, $\text{Alg}_{GL2}$ significantly outperforms the classical OMP algorithm and is comparable to the classical BP algorithm in terms of recovery accuracy. Meanwhile, the $\text{Alg}_{GL1}$ algorithm significantly surpasses the classical OMP, BP_C, and CoSaMP algorithms in terms of sparse recovery accuracy;

• •

  An improved version of $\text{Alg}_{GL1}$ has been derived, denoted as the $\text{Alg}_{GLQ}$ algorithm, in which $l_{q}$-minimization is used to enhance the sparsity. Fast variants of the algorithms $\text{Alg}_{GL1}$ and $\text{Alg}_{GLQ}$, denoted as $\text{Alg}^{F}_{GL1}$ and $\text{Alg}^{F}_{GLQ}$, are developed to accelerate $\text{Alg}_{GL1}$ and $\text{Alg}_{GLQ}$ by selecting multiple atoms per iteration;

• •

  It is shown that the proposed algorithms such as $\text{Alg}_{GL1}$, $\text{Alg}_{GLQ}$ and their fast variants can be used to deal with low-rank interferences in the measurements effectively, which owes to the proposed characterization and is a significant advantage over the classical algorithms.

All claims have been validated through extensive numerical simulations on both synthetic and image data.

The paper is organized as follows. In Section 2, we derive a characterization of the solution set for the linear system $\bm{x}=\bm{Q}\bm{s}$. Based on this characterization, we propose a greedy approach to sparse recovery. Two specified algorithms, denoted as $\text{Alg}_{GL2}$ and $\text{Alg}_{GL1}$, are presented in this section. Section 3 analyzes the performance of the proposed algorithms. An improved version of $\text{Alg}_{GL1}$, denoted as $\text{Alg}_{GLQ}$, is derived in Section 4, where the $l_{q}$-minimization is employed to enhance the sparsity. Fast variants of both $\text{Alg}_{GLQ}$ and $\text{Alg}_{GL1}$ are also derived in this section. Numerical examples and simulations are presented in Section 5 to validate the theoretical results, evaluate the performance of the proposed methods, and compare them with some existing approaches. To end this paper, some concluding remarks are given in Section 6.

Suppose that $i_{1}$ is a detected index, expected to be one of the elements in $\mathcal{I}_{\bm{s}^{\star}}$. Let’s consider how to detect the next index, say $i_{2}$, in a greedy manner.

First of all, it follows from $\bm{x}=\bm{Q}\bm{s}$ that the residual is given by $\bm{x}-\beta\bm{Q}(:,i_{1})=\bm{Q}\bm{s}-\beta\bm{Q}(:,i_{1})$ or more simply $\bm{x}-\beta\bm{Q}(:,i_{1})=\bm{Q}\bm{s}_{1}$, where $\bm{s}_{1}$, according to the proposed characterization (9), can be expressed as

where $\bm{s}(\bm{z})=\bm{s}_{0}+\bm{W}\bm{z}\in\mathcal{S}_{0}$, as defined in (9).

Recall that $\bm{Q}=\bm{U}_{1}\bm{\Sigma}\bm{V}_{1}^{\top}$. Together with $\bm{T}_{0}=\bm{\Sigma}^{-1}\bm{U}^{\top}_{1}$ and $\bm{s}_{0}=\bm{V}_{1}\bm{T}_{0}\bm{x}=\bm{V}_{1}\bm{T}_{0}\bm{Q}\bm{s}^{\star}$, we obtain

where $\bar{\bm{Q}}\triangleq\bm{V}_{1}\bm{V}^{\top}_{1}$. Define a set $\mathcal{S}_{1}\triangleq\{\bm{s}_{1}:\bm{s}_{1}=\bm{s}(\bm{z})-\beta\bar{\bm{Q}}(:,i_{1})\}$ that is characterized by $(\bm{z},\beta)$.

Assume that $i_{1}\in\mathcal{I}_{\bm{s}^{\star}}$. We note that problem (12) with $i=i_{1}$ results in $\tilde{\bm{z}}=\bm{z}^{\star}$ and $\tilde{\beta}=\beta^{\star}_{1}$. Then, the corresponding $\bm{s}^{\star}_{1}=\bm{s}^{\star}-\bm{s}^{\star}(i_{1})\bar{\bm{Q}}(:,i_{1})$ is the unique $(\kappa-1)$-sparse and the sparsest element in set $\mathcal{S}_{1}$. This fact implies that to detect the next index in $\mathcal{I}_{\bm{s}^{\star}}$, say $i_{2}$, we can first identify the sparsest element in $\mathcal{S}_{1}$. Then, $i_{2}$ can be determined as the index corresponding to the largest entry in magnitude of $\bm{s}^{\star}_{1}$.

It is based on this observation that we propose the following greedy method, whose $(k+1)$th iteration involves

where $\bar{\bm{Q}}_{k}\triangleq\bar{\bm{Q}}(:,\mathcal{I}_{k})$ with $\mathcal{I}_{k}\triangleq\{i_{1},i_{2},\cdots,i_{k}\}$ being the set of indices detected before, and $\mathcal{I}^{c}_{k}$ its complement.³³3Throughout this paper, $\mathcal{I}_{k}$ denotes a set of indices when $k$ is an integer, while, as defined before, $\mathcal{I}_{\bm{v}}$ denotes the support of a vector $\bm{v}$. Here, $\|.\|_{\chi}$ denotes the $l_{0}$ norm or one of its approximating measures.

Note that the pair $(\tilde{\bm{s}}_{k},\tilde{\bm{\beta}}_{k})$ in (15) corresponds to an $\bm{s}_{k}$ that satisfies $\bm{x}=\bm{Q}\bm{s}_{k}$, where

and $\tilde{\bm{\beta}}^{\ast}_{k}$ is $k$-sparse with $\tilde{\bm{\beta}}^{\ast}_{k}(\mathcal{I}_{k})=\tilde{\bm{\beta}}_{k}$.

Running (15) for $\kappa$ iterations, the $\kappa$-sparse estimate of $\bm{s}^{\star}$, denoted as $\hat{\bm{s}}^{\star}$, is given by $\hat{\bm{s}}^{\star}(\mathcal{I}_{\kappa})=\tilde{\bm{\beta}}_{\kappa}$. It will be shown in the next section that if $\tilde{\bm{s}}_{\kappa}=\bm{0}$, then $\mathcal{I}_{\kappa}=\mathcal{I}_{\bm{s}^{\star}}$, and $\tilde{\bm{\beta}}_{\kappa}=\bm{s}^{\star}(\mathcal{I}_{\kappa}),$ which implies that the unique $\kappa$-sparse solution $\bm{s}^{\star}$ of $\bm{x}=\bm{Q}\bm{s}$ has been obtained.

Although the $\|.\|_{0}$ norm is the best measure to deal with sparsity-related problems, one of its approximating measures has to be used, as the $\|.\|_{0}$ norm is NP-hard and hence not tractable. In the next two subsections, we will further elaborate on the proposed greedy approach (15) with two approximating measures.

Now, consider $\chi=2$, that is, the $l_{2}$-relaxation of $l_{0}$. Denote $\bm{s}_{k}=\bm{s}(\bm{z})-\bar{\bm{Q}}(:,\mathcal{I}_{k})\bm{\beta}$. We have

due to the fact that $\bm{s}_{0}=\bar{\bm{Q}}\bm{s}^{\star}$, where $\bar{\bm{Q}}=\bm{V}_{1}\bm{V}^{\top}_{1}$, as defined in (11), and $\bm{V}_{1}^{\top}\bm{W}={\bf 0}$. Then, the proposed greedy approach (15) turns to

as $\tilde{\bm{z}}_{k}=\bm{0},~{}\forall~{}k$. We name the above approach $\text{Alg}_{GL2}$, where the subscript “_GL2” stands for greedy method with indices selected using $l_{2}$ minimization. We briefly denote the approach as $\hat{\bm{s}}^{\star}=\text{Alg}_{GL2}(\bm{s}_{0},\bm{V}_{1},\kappa)$, where $\bm{s}_{0}=\bm{V}_{1}\bm{T}_{0}\bm{x}$, as mentioned before, is the initial estimate of $\bm{s}^{\star}$.

We realize by noting $\bm{s}_{0}=\bar{\bm{Q}}\bm{s}^{\star}$ that the proposed $\text{Alg}_{GL2}$ algorithm (17) is closely related to the classical OMP algorithm applied to the following higher dimensional system (than $\bm{x}=\bm{Q}\bm{s}$)

It can be seen that the $\text{Alg}_{GL2}$ algorithm differs from the classical OMP applied to (18) in how it detects the next index $i_{k+1}$.

Note that $\bm{s}_{k}=\bm{s}_{0}+\bm{W}\bm{z}-\bar{\bm{Q}}(:,\mathcal{I}_{k})\bm{\beta}$ can be rewritten as $\bm{s}_{k}=\bm{s}_{0}+\bm{\Omega}_{k}\bar{\bm{z}}$, where

With $\chi=1$, the proposed greedy approach (15) becomes

which is a greedy $l_{1}$-based algorithm for sparse recovery.

As seen, such an iterative algorithm involves solving a series of $l_{1}$-minimizations of the form

As shown in [31], the above problem can be converted as a linear programming problem and hence can be solved with one of the standard convex optimization techniques, such as the MATLAB command $linprog.m$. For convenience, we denote the corresponding algorithm as $\text{BP}_{alt}$: $[\tilde{\bm{z}},\tilde{\bm{s}}]=\text{BP}_{alt}(\bm{s}_{0},\bm{\Omega})$ with the residual $\tilde{\bm{s}}=\bm{s}_{0}+\bm{\Omega}\tilde{\bm{z}}$.

The outline of the proposed greedy $l_{1}$-based algorithm (23), denoted as $\text{Alg}_{GL1}$, is then presented in Algorithm 1. For convenience, we denote this algorithm as $\hat{\bm{s}}^{\star}=\text{Alg}_{GL1}(\bm{s}_{0},\bm{W},\bm{V}_{1},\kappa).$

Before proceeding to the next section, we first evaluate the performance of the proposed greedy algorithms through the following numerical experiment. We generate a sequence $\{\bm{x}_{j}\}$ of $J=1,000$ samples with $\bm{x}_{j}=\bm{Q}\bm{s}_{j},~{}\forall~{}j$. Here, $\bm{Q}^{64\times 128}$ - a (column) $l_{2}$-normalized matrix, and $\{\bm{s}_{j}\}$ with $\|\bm{s}_{j}\|_{0}=20$ are generated randomly. We run each of the four algorithms: $\text{OMP}_{C}(\bm{x},\bm{Q},\kappa)$, $\text{BP}_{C}$ - the classical BP method, $\text{Alg}_{GL2}(\bm{s}_{0},\bm{V}_{1},\kappa)$ and $\text{Alg}_{GL1}(\bm{s}_{0},\bm{W},\bm{V}_{1},\kappa)$, where $\bm{s}_{0}=\bm{V}_{1}\bm{T}_{0}\bm{x}$ with $\bm{x}=\bm{x}_{j}$, for all the 1,000 samples $\{\bm{x}_{j}\}$.

Let $\mathcal{I}_{\lambda}[\bm{v}]$ denote the index set of the $\lambda$ largest absolute entries of $\bm{v}$. A recovery of a $\kappa$-sparse $\bm{s}^{\star}$ is considered successful if the solution $\hat{\bm{s}}^{\star}$ obtained by an algorithm satisfies $\mathcal{I}_{\kappa}[\hat{\bm{s}}^{\star}]=\mathcal{I}_{\bm{s}^{\star}}.$ The rate of successful recovery for an algorithm running $J$ samples is defined as $\varrho_{ok}\triangleq J_{ok}/J$, where $J_{ok}$ is the number of successful recoveries out of the $J$ samples.

With the above setting, the successful recovery rates for the four algorithms are given in Table 1.

Let $\hat{\bm{s}}^{\star}$ be an estimate of $\bm{s}^{\star}$, both being $\kappa$-sparse. The signal-to-noise ratio of this estimation is defined as $\varrho_{snr}\triangleq 10\times\text{log}_{10}\frac{\|\bm{Q}\bm{s}^{\star}\|^{2}_{2}}{\|\bm{Q}(\bm{s}^{\star}-\hat{\bm{s}}^{\star})\|^{2}_{2}}~{}~{}~{}(\text{dB})$. When some of the $\kappa$ nonzero entries of $\bm{s}^{\star}$ are very small, these entries have a little contribution to $\bm{x}=\bm{Q}\bm{s}^{\star}$ but it is very difficult to correctly identify the indices of these entries. In this paper, we consider a recovery to be successful if either all the indices of $\bm{s}^{\star}$ are detected or the corresponding $\varrho_{snr}$ exceeds 60 dB.

As understood, greedy methods, including our proposed approach (15), are designed based on minimizing a residual. Consequently, the generated sequence of residuals is non-increasing. Although the residuals converge, most greedy algorithms are expected to yield the unique solution $\bm{s}^{\star}$ within a finite number of iterations.

First of all, we present the following result associated with the proposed greedy approach (15).

We note that $\bm{s}^{\star}-\hat{\bm{s}}_{k}$ is $2\kappa$-sparse when the condition given in the theorem holds and $\text{spark}[\bm{V}^{\top}_{1}]=\text{spark}[\bm{Q}]>2\kappa$ due to the assumption of uniqueness on $\bm{s}^{\star}$. Thus, $\bm{V}^{\top}_{1}(:,\mathcal{I}_{k})$ is of full (column) rank, and hence the above equation leads to $\bm{s}^{\star}-\hat{\bm{s}}_{k}=\bm{0}$. This completes the proof.  

In fact, the claims in Theorem 1 also hold for the classical $\text{OMP}_{C}$ algorithm, in which the upper bound for the number of iterations is set to $\kappa$. This means that $\text{OMP}_{C}$ must select an index that falls within $\mathcal{I}_{\bm{s}^{\star}}$, which precludes the possibility of $\text{OMP}_{C}$ making some mistakes in selecting indices. However, these mistakes can be rectified by taking more than $\kappa$ iterations [50]. In general, for $\mathcal{I}_{k}$ with $\kappa<k\leq 2\kappa$, nothing can guarantee whether $\mathcal{I}_{\bm{s}^{\star}}\subseteq\mathcal{I}_{k}$ holds unless $\|\tilde{\bm{\beta}}_{k}\|_{0}=\kappa$ is verified.

The primary advantage of the proposed greedy approach over classical methods, such as $\text{OMP}_{C}$ and CoSaMP, lies in its index selection strategy. In the proposed greedy approach (15), the next index $i_{k+1}$ is chosen as the one corresponding to the largest absolute entry of the residual $\tilde{\bm{s}}_{k}$. As demonstrated in Remark 2.1 and the surrounding discussion, this selection ensures that $i_{k+1}$ belongs to $\mathcal{I}_{\bm{s}^{\star}}$. In contrast, $\text{OMP}_{C}$ determines $i_{k+1}$ by minimizing $\|\bm{r}_{k}-\beta\bm{Q}(:,i)\|^{2}_{2}$ with respect to $\beta$ for $i\in\mathcal{I}^{c}_{k}$. However, such a selection does not necessarily guarantee that $i_{k+1}$ belongs to $\mathcal{I}_{\bm{s}^{\star}}$, even if $\mathcal{I}_{k}\subseteq\mathcal{I}_{\bm{s}^{\star}}$, as the minimizer of $\|\bm{r}_{k}-\beta\bm{Q}(:,i)\|^{2}_{2}$ may not correspond to an index within $\mathcal{I}_{\bm{s}^{\star}}$.

Furthermore, let $\mathcal{I}_{k}$ be the detected indices and $\mathcal{I}^{\ast}_{k}$ be a subset of $\mathcal{I}_{k}$, which contains $k^{\star}$ elements with $\mathcal{I}^{\ast}_{k}\subseteq\mathcal{I}_{\bm{s}^{\star}}$. Using similar arguments to those in Remark 2.1, we have

for which the residual is

as long as $k^{\star}>k-k^{\star}~{}~{}\Rightarrow~{}~{}k^{\star}>k/2$ holds, where $\bm{s}^{\star}_{k^{\star}}(l)=0,~{}l\in\mathcal{I}^{\ast}_{k}$ and otherwise, $\bm{s}^{\star}_{k^{\star}}(l)=\bm{s}^{\star}(l)$. This implies that with $\chi$ taking $l_{0}$, the index $i_{k+1}$ of the absolutely largest entry of the residual $\bm{s}^{\star}_{k}$ should fall within $\mathcal{I}_{\bm{s}^{\star}}$, provided that more than half of the $k$ detected indices belong to $\mathcal{I}_{\bm{s}^{\star}}$. In other words, when the first few indices are detected correctly, the proposed greed approach tolerates a small number of mistakes in index detection. This is indeed a remarkable property.

As argued above, the key to $\text{Alg}_{GL2}$ outperforming the classical $\text{OMP}_{C}$ algorithm lies in its method of detecting the support $\mathcal{I}_{\bm{s}^{\star}}$. Another contributing factor is related to the numerical properties of the system matrix, including the concept of spark and RIP (see equations (3) $\&$ (4)), and mutual coherence. In fact, let $\bm{x}=\bm{Q}\bm{s}^{\star}$ with $\|\bm{s}^{\star}\|_{0}\leq\kappa$. It was shown in [30] and [45] that both the classical $\text{BP}_{C}$ and $\text{OMP}_{C}$ algorithms can yield the true solution $\bm{s}^{\star}$ from the system $\bm{x}=\bm{Q}\bm{s}$ with $\bm{Q}$ (column) $l_{2}$-normalized, if the following condition holds:

where $\mu[\bm{Q}]$ is the mutual coherence of $\bm{Q}\in\mathbb{R}^{N\times L}$. As shown in [51], for a matrix $\bm{Q}\in\mathbb{R}^{N\times L}$ with $\|\bm{Q}(:,l)\|_{2}=1,~{}\forall~{}l$, $\mu[\bm{Q}]\leq\sqrt{\frac{L-N}{N(L-1)}}\triangleq\underline{\mu}$ and the lower bound is achieved if and only if the Gram matrix $\bm{G}_{\bm{Q}}\triangleq\bm{Q}^{\top}\bm{Q}$ satisfies $\bm{G}_{\bm{Q}}(i,j)=\underline{\mu},\forall~{}i\neq j$. Such a $\bm{Q}$ is called an equiangular tight-frame (ETF).

The condition (25) is just a sufficient condition to ensure a prefect recovery of $\bm{s}^{\star}$. As shown in [52] and [53], the spare recovery accuracy of $\text{OMP}_{C}$ is more closely related to averaged mutual coherence of $\bm{Q}$, denoted as $\mu_{av}[\bm{Q}]$, which is defined as the mean of all the squared off-diagonal entries of $\bm{G}_{\bm{Q}}$ rather than $\mu[\bm{Q}]$. This implies that it is desired for the system matrix $\bm{Q}$ to have its Gram matrix close to the identity matrix $\bm{I}$. Based on this argument, it was shown in [53] and [54] that an optimal $\bm{Q}$ is a tight-frame (TF), whose Gram matrix $\bm{G}_{\bm{Q}}$ is of form $\bm{G}_{\bm{Q}}=\bm{V}\left[\begin{array}[]{cc}\alpha^{2}\bm{I}_{N}&{\bf 0}\\ {\bf 0}&{\bf 0}\end{array}\right]\bm{V}^{\top},$ where $\bm{V}$ is orthonormal. As ETFs exist only for some pairs $(N,L)$, tight-frames provide an alternative for designing sparse recovery systems [55, 56, 57].

Our proposed $\text{Alg}_{GL2}$ algorithm is based on the system $\bm{s}_{0}(\bm{x})=\bar{\bm{Q}}\bm{s}$. As shown in (11), $\bar{\bm{Q}}=\bm{V}_{1}\bm{T}_{0}\bm{Q}$. We note that