# 5.2 Greedy algorithms  (Page 3/3)

 Page 3 / 3

## Compressive sampling matching pursuit (cosamp)

Greedy pursuit algorithms (such as MP and OMP) alleviate the issue of computational complexity encountered in optimization-based sparse recovery, but lose the associated strong guarantees for uniform signal recovery, given a requisite number of measurements of the signal. In addition, it is unknown whether these greedy algorithms are robust to signal and/or measurement noise.

There have been some recent attempts to develop greedy algorithms (Regularized OMP  [link] , [link] , Compressive Sampling Matching Pursuit (CoSaMP)  [link] and Subspace Pursuit  [link] ) that bridge this gap between uniformity and complexity. Intriguingly, the restricted isometry property (RIP), developed in the context of analyzing ${\ell }_{1}$ minimization , plays a central role in such algorithms. Indeed, if the matrix $\Phi$ satisfies the RIP of order $K$ , this implies that every subset of $K$ columns of the matrix is approximately orthonormal. This property is used to prove strong convergence results of these greedy-like methods.

One variant of such an approach is employed by the CoSaMP algorithm. An interesting feature of CoSaMP is that unlike MP, OMP and StOMP, new indices in a signal estimate can be added as well as deleted from the current set of chosen indices. In contrast, greedy pursuit algorithms suffer from the fact that a chosen index (or equivalently, a chosen atom from the dictionary $\Phi$ remains in the signal representation until the end. A brief description of CoSaMP is as follows: at the start of a given iteration $i$ , suppose the signal estimate is ${\stackrel{^}{x}}_{i-1}$ .

• Form signal residual estimate: $e←{\Phi }^{T}r$
• Find the biggest $2K$ coefficients of the signal residual $e$ ; call this set of indices $\Omega$ .
• Merge supports: $T←\Omega \cup \mathrm{supp}\left({\stackrel{^}{x}}_{i-1}\right)$ .
• Form signal estimate $b$ by subspace projection: ${b|}_{T}←{\Phi }_{T}^{†}y$ , ${b|}_{{T}^{C}}←0$ .
• Prune $b$ by retaining its $K$ largest coefficients. Call this new estimate ${\stackrel{^}{x}}_{i}$ .
• Update measurement residual: $r←y-\Phi {\stackrel{^}{x}}_{i}$ .

This procedure is summarized in pseudocode form below.

Inputs: Measurement matrix $\Phi$ , measurements $y$ , signal sparsity $K$ Output: $K$ -sparse approximation $\stackrel{^}{x}$ to true signal representation $x$ Initialize: ${\stackrel{^}{x}}_{0}=0$ , $r=y$ ; $i=0$ while ħalting criterion false do 1. $i←i+1$ 2. $e←{\Phi }^{T}r$ {form signal residual estimate} 3. $\Omega ←\mathrm{supp}\left(\mathbf{T}\left(e,2K\right)\right)$ {prune signal residual estimate} 4. $T←\Omega \cup \mathrm{supp}\left({\stackrel{^}{x}}_{i-1}\right)$ {merge supports} 5. ${b|}_{T}←{\Phi }_{T}^{†}y$ , ${b|}_{{T}^{C}}$ {form signal estimate} 6. ${\stackrel{^}{x}}_{i}←\mathbf{T}\left(b,K\right)$ {prune signal estimate} 7. $r←y-\Phi {\stackrel{^}{x}}_{i}$ {update measurement residual} end while return $\stackrel{^}{x}←{\stackrel{^}{x}}_{i}$ 

As discussed in  [link] , the key computational issues for CoSaMP are the formation of the signal residual, and the method used for subspace projection in the signal estimation step. Under certain general assumptions, the computational cost of CoSaMP can be shown to be $O\left(MN\right)$ , which is independent of the sparsity of the original signal. This represents an improvement over both greedy algorithms as well as convex methods.

While CoSaMP arguably represents the state of the art in sparse recovery algorithm performance, it possesses one drawback: the algorithm requires prior knowledge of the sparsity $K$ of the target signal. An incorrect choice of input sparsity may lead to a worse guarantee than the actual error incurred by a weaker algorithm such as OMP. The stability bounds accompanying CoSaMP ensure that the error due to an incorrect parameter choice is bounded, but it is not yet known how these bounds translate into practice.

## Iterative hard thresholding

Iterative Hard Thresholding (IHT) is a well-known algorithm for solving nonlinear inverse problems. The structure of IHT is simple: starting with an initial estimate ${\stackrel{^}{x}}_{0}$ , iterative hard thresholding (IHT) obtains a sequence of estimates using the iteration:

${\stackrel{^}{x}}_{i+1}=\mathbf{T}\left({\stackrel{^}{x}}_{i}+{\Phi }^{T}\left(y-\Phi {\stackrel{^}{x}}_{i}\right),K\right).$

In  [link] , Blumensath and Davies proved that this sequence of iterations converges to a fixed point $\stackrel{^}{x}$ ; further, if the matrix $\Phi$ possesses the RIP, they showed that the recovered signal $\stackrel{^}{x}$ satisfies an instance-optimality guarantee of the type described earlier . The guarantees (as well as the proof technique) are reminiscent of the ones that are derived in the development of other algorithms such as ROMP and CoSaMP.

## Discussion

While convex optimization techniques are powerful methods for computing sparse representations, there are also a variety of greedy/iterative methods for solving such problems. Greedy algorithms rely on iterative approximation of the signal coefficients and support, either by iteratively identifying the support of the signal until a convergence criterion is met, or alternatively by obtaining an improved estimate of the sparse signal at each iteration by accounting for the mismatch to the measured data. Some greedy methods can actually be shown to have performance guarantees that match those obtained for convex optimization approaches. In fact, some of the more sophisticated greedy algorithms are remarkably similar to those used for ${\ell }_{1}$ minimization described previously . However, the techniques required to prove performance guarantees are substantially different. There also exist iterative techniques for sparse recovery based on message passing schemes for sparse graphical models. In fact, some greedy algorithms (such as those in  [link] , [link] ) can be directly interpreted as message passing methods  [link] .

how environment affect demand and supply of commodity ?
Wht at the criteria for market ?
Amos
what is difference between monitory policy and fiscal policy?
monetary policy is a policy thrust by National Govt(CBN) to influence government spending, purchase &taxes
Frank
necessity of economics
I will say want,choice,opportunity cost,scarcity,scale of preference
Alao
what is monopoly market.How price output are determined under monopoly market
bisham
b) Monopoly market is an impecfect market where s single firm having the innovation to produce a particular commodity.Prices are determined through output since there are no other competitive.
Frank
Monopoly market:firm has market power & does not respond to market price
Frank
Explain the process of price determination under perfect competition market with suitable diagram
Price determination under perfect competition via this process :firms have no market power to influence price rather firms respond to market price.
Frank
price is different from demand- demand is amount of commodity
demand is amount /quantity of commodity a potential buyer is willing to buy at a given price at market
Frank
demand is a desire of customer on commodity with the ability to pay it and willing to buy it at given price of commodity
Harika
demand is price of what
show that shortrun average cost
what is economics
what is money
Mbah
what is money
Mbah
Difine macro economics
agaba
money is a medium of exchange between goods and services,maybe inform of currency.
Wesonga
Economics is study of how human beings strive to satisfy numerous wants using limited available resources.
Wesonga
how do you find the maximum number of workers the firms should employ order to produce where there are increasing returns
Jane
what are implications of computing national income?.
agaba
pl
MUDASIRU
what is the formulae for calculating national income
MUDASIRU
it calculated by value added method
Praveen
classify the production units like agriculture, banking, transport etc
Praveen
money is anything that is generally acceptetable for human
Ogbaji
Estimate the net value added(NVA) at fixed cost by each industrial structure
Praveen
definition of unemployment
what are the causes of unemployment?
The main causes of unemployment are listed below. 1. Frictional unemployment 2. Cyclical unemployment 3. Structural unemployment
assani
We can also categorize the causes on a broader sense as: 1. Political and 2. Social cause As unemployeement root causes are embaded in this two.
Yonathan
would opportunity cost exist if there was no scarcity?
assani
yes just because the opportunity cost arose when there is Alternative to choose among the alternatives.
I am thinking that, if our resources were unlimited, then there wouldn't be any need to forgo some wants. Hence the inexistence if opportunity cost
assani
Politics
Job
politics has done what?
assani
consider time assani
Mary
I'm Emmanuel,...I taught the main cause is the change in gov't.
Emmanuel
...Lack of capital to set up a firm respectively
Emmanuel
🙈
Emmanuel
I would like to bring in Educational levels can also be the cause the cause of the problem respectively
Emmanuel
I think the main causes of unemployment is lack of INFRASTRUCTURAL DEVELOPMENT OVER POPULATION OVER DEPENDENT ON GOVERNMENT LACK OF SELF EMPOWERMENT...
ananti
lack of skills among the new generation is the serious issue.
Vishal
Where I come from , I don't see why education or personal aspects seem to do with unimployment, technically the motivation and eigerness in all works of live is there , dispite the cultural influence and physical bearriors;the thing we lacking is Government Support and open market ethics.
Joe
sorry about that-(repation). We have a over powering ethical political system that's displacing the marketing asspects of economy and causing large scale unemployment right across the board...
Joe
can someone Explain Expansionary Monetary Policy and Contractionary Monetary Policy Using one of the instrument of Monetary Policy? Please am kinda lost here?. ta
using a graph show the case of substitute and compliment goods
can anyone give me a simple explanation to Five Sector Macroeconomics?
Emmanuel
Can someone please define what economics is
economics simply is a social science subject that study human behavior.
dajan
economics is a social science which studies human behaviour as a relationship between ends and scarce means that has alternative uses
Alao
Can someone please tell me how to calculate GDP
Emmanuel
emmanual kapal to calculate GDP (Gross Domestic Product) has three method in calculating it (1)income approach (2) expenditure approach (3) value added method
Alao
thanks Alae
Emmanuel
u are welcome
Alao
in basic terms economics is revered to as battery system, it date back to when Men sees the need to exchange sapless goods and produce to gain , either wealth , basic necessities or to establish trading ties for personal benefit or social asspects in terms of coexistence and continuity, future .
Joe
what is the law of demand
keep other thing constant, when the price increases demand decrease when the price decreases demand increases of the commodity.
sj
all things being equal,quantity demanded decrease as price increase and increase as price decrease
Seth
there's practial joke to it ..." the higher the demand ; scarcity, increase in production and drop in quality"... quite the controversy - for example China vs Europe, United States and we are all boxed up in between somewhere...
Joe
Other thing remain constant the low price of commodity the high quantity of commodity and vice versa is true
Baraka
Explain Effective demand
What is effective demand
Anita
like Modi is in demand...best example of effective demand
Pranav
Don't get you
Anita
Anita you mean you don't get me or who?
Onyeking
level of demand that represents a real intention to purchase by people with the means to pay
Pranav
Difference between extinct and extici spicies
While the American heart association suggests that meditation might be used in conjunction with more traditional treatments as a way to manage hypertension
in a comparison of the stages of meiosis to the stage of mitosis, which stages are unique to meiosis and which stages have the same event in botg meiosis and mitosis
Researchers demonstrated that the hippocampus functions in memory processing by creating lesions in the hippocampi of rats, which resulted in ________.
The formulation of new memories is sometimes called ________, and the process of bringing up old memories is called ________.
Got questions? Join the online conversation and get instant answers!