5.2 Greedy algorithms  (Page 2/3)

These steps are repeated until convergence. This is known as Orthogonal Matching Pursuit (OMP)  [link] . Tropp and Gilbert  [link] proved that OMP can be used to recover a sparse signal with high probability using compressive measurements. The algorithm converges in at most $K$ iterations, where K is the sparsity, but requires the added computational cost of orthogonalization at each iteration. Indeed, the total complexity of OMP can be shown to be $O\left(MNK\right).$

While OMP is provably fast and can be shown to lead to exact recovery, the guarantees accompanying OMP for sparse recovery are weaker than those associated with optimization techniques . In particular, the reconstruction guarantees are not uniform , i.e., it cannot be shown that a single measurement matrix with $M=CKlogN$ rows can be used to recover every possible $K-$ sparse signal with $M=CKlogN$ measurements. (Although it is possible to obtain such uniform guarantees when it is acceptable to take more measurements. For example, see [link] .) Another issue with OMP is robustness to noise; it is unknown whether the solution obtained by OMP will only be perturbed slightly by the addition of a small amount of noise in the measurements. Nevertheless, OMP is an efficient method for CS recovery, especially when the signal sparsity $K$ is low. A pseudocode representation of OMP is shown below.

Stagewise orthogonal matching pursuit (stomp)

Orthogonal Matching Pursuit is ineffective when the signal is not very sparse as the computational cost increases quadratically with the number of nonzeros $K$ . In this setting, Stagewise Orthogonal Matching Pursuit (StOMP)  [link] is a better choice for approximately sparse signals in a large-scale setting.

StOMP offers considerable computational advantages over ${\ell }_1$ minimization and Orthogonal Matching Pursuit for large scale problems with sparse solutions. The algorithm starts with an initial residual $r_0=y$ and calculates the set of all projections ${\Phi }^T{r}_{k-1}$ at the $k^{th}$ stage (as in OMP). However, instead of picking a single dictionary element, it uses a threshold parameter $\tau$ to determine the next best set of columns of $\Phi$ whose correlations with the current residual exceed $\tau$ . The new residual is calculated using a least squares estimate of the signal using this expanded set of columns, just as before.

Unlike OMP, the number of iterations in StOMP is fixed and chosen before hand; $S=10$ is recommended in [link] . In general, the complexity of StOMP is $O(KNlogN)$ , a significant improvement over OMP. However, StOMP does not bring in its wake any reconstruction guarantees. StOMP also has moderate memory requirements compared to OMP where the orthogonalization requires the maintenance of a Cholesky factorization of the dictionary elements.