2.2 Second-order convergence analysis of the lms algorithm and

Adaptive filters Page 1 / 1

Convergence of the mean (first-order analysis) is insufficient to guarantee desirable behavior of the LMS algorithm; the variancecould still be infinite. It is important to show that the variance of the filter coefficients is finite, and to determine how closethe average squared error is to the minimum possible error using an exact Wiener filter.

_{k} 2 d_{k} W^{k} X^{k} 2 d_{k} 2 2 d_{k} X^{k} W^{k} W^{k} X^{k} X^{k} W^{k} r_{dd} 0 2 W^{k} P W^{k} R W^{k}

The minimum error is obtained using the Wiener filter

W_{opt} R P

_{\min} 2 2 r_{dd} 0 2 P R P P R R R P r_{dd} 0 P R P

To analyze the average error in LMS, write in terms of

V^{'} Q W W_{opt}

, where

Q Q R

_{k} 2 r_{dd} 0 2 W^{k} P W^{k} R W^{k} W^{k} R W_{opt} W_{opt} R W^{k} W_{opt} R W_{opt} W^{k} R W_{opt} W_{opt} R W^{k} W_{opt} R W_{opt} r_{dd} 0 V^{k} R V^{k} P R P_{\min} 2 V^{k} R V^{k}_{\min} 2 V^{k} Q Q R Q Q V^{k}_{\min} 2 V^{'k} V^{'k}

_{k} 2_{\min} 2 j 0 N 1_{j} v_{j}^{'k} 2

So we need to know

v_{j}^{'k} 2

, which are the diagonal elements of the covariance matrix of

V^{'k}

, or

V^{'k} V^{'k}

From the LMS update equation $W^{k + 1} W^{k} 2_{k} X^{k}$ we get $V^{'k + 1} W^{'k} 2_{k} Q X^{k}$

^{k + 1} V^{' k + 1} V^{' k + 1} V^{'k} V^{'k} 2_{k} Q X^{k} V^{'k} 2_{k} V^{'k} X^{k} Q 42_{k} 2 Q X^{k} X^{k} Q^{k} 2_{k} Q X^{k} V^{'k} 2_{k} V^{'k} X^{k} Q 42_{k} 2 Q X^{k} X^{k} Q

Note that

_{k} d_{k} W^{k} X^{k} d_{k} W_{opt} V^{'k} Q X^{k}

_{k} Q X^{k} V^{'k} d_{k} Q X^{k} V^{'k} W_{opt} X^{k} Q X^{k} V^{'k} V^{'k} Q X^{k} V^{'k} 0 0 Q X^{k} X^{k} Q V^{'k} V^{'k} Q X^{k} X^{k} Q V^{'k} V^{'k}^{k}

Note that the Patently False independence Assumption was invoked here.

To analyze $_{k} 2 Q X^{k} X^{k} Q$ , we make yet another obviously false assumptioon that $_{k} 2$ and $X^{k}$ are statistically independent. This is obviously false, since $_{k} d_{k} W^{k} X^{k}$ . Otherwise, we get 4th-order terms in $X$ in the product. These can be dealt with, at the expense of a more complicated analysis, if aparticular type of distribution (such as Gaussian) is assumed. See, for example Gardner . A questionable justification for this assumption is that as $W^{k} W_{opt}$ , $W^{k}$ becomes uncorrelated with $X^{k}$ (if we invoke the original independence assumption), which tends to randomize the error signal relative to $X^{k}$ . With this assumption, $_{k} 2 Q X^{k} X^{k} Q_{k} 2 Q X^{k} X^{k} Q_{k} 2$ Now $_{k} 2_{\min} 2 V^{'k} V^{'k}$ so

_{k} 2_{\min} 2 j_{j} V_{j}^{'k} 2_{\min} 2 j_{j}_{jj}^{k}

Thus, becomes

^{k + 1} I 4^{k} 42 j_{j}_{jj}^{k} 42_{\min} 2

Now if this system is stable and converges, it converges to

^{}^{+ 1}

4^{} 42 j_{j}_{jj}_{\min} 2

^{} j_{j}_{jj}_{\min} 2 I

So it is a diagonal matrix with all elements on the diagonal equal:

Then $_{ii}^{}_{ii}^{} j_{j}_{\min} 2$ $_{ii}^{} 1 j_{j}_{\min} 2$ $_{ii}^{}_{\min} 2 1 j_{j}$ Thus the error in the LMS adaptive filter after convergence is

_{} 2_{\min} 2 V^{'} V^{'}_{\min} 2_{\min} 2 j_{j} 1 j_{j}_{\min} 2 1 1 j_{j}_{\min} 2 1 1 tr R_{\min} 2 1 1 r_{xx} 0 N

_{} 2_{\min} 2 1 1 N_{x} 2

1 N_{x} 2

is called the misadjustment factor . Oftern, one chooses

to select a desired misadjustment factor, such as an error 10% higher than theWiener filter error.

2nd-order convergence (stability)

To determine the range for for which converges, we must determine the for which the matrix difference equation converges. $^{k + 1} I 4^{k} 42 j_{j}_{jj}^{k} 42_{\min} 2$ The off-diagonal elements each evolve independently according to $_{ij}^{k + 1} 1 4_{i}_{ij}^{k}$ These terms will decay to zero if $i 4_{i} 2$ , or $1 2_{\max}$

The diagonal terms evolve according to $_{ii}^{k + 1} 1 4_{i}_{ii}^{k} 42_{i} j_{j}_{jj}^{k} 42_{\min} 2_{i}$ For the homoegeneous equation $_{ii}^{k + 1} 1 4_{i}_{ii}^{k} 42_{i} j_{j}_{jj}^{k}$ for $1 4_{i}$ positive,

_{ii}^{k + 1} 1 4_{i}_{iimax}^{k} 42_{i} j_{j}_{jjmax}^{k} 1 4_{i} 42_{i} j_{j}_{jjmax}^{k}

_{ii}^{k + 1}

will be strictly less than

_{jjmax}^{k}

for

1 4_{i} 42_{i} j_{j} 1

42_{i} j_{j} 4_{i}

1 j_{j} 1 tr R 1 N r_{xx} 0 1 N_{x} 2

This is a more rigorous bound than the first-order bounds. Ofter engineers choose

a few times smaller than this, since more rigorous analyses yield a slightly smallerbound.

3 N_{x} 2

is derived in some analyses assuming Gaussian

x_{k}

d_{k}

<< Chapter < Page Page > Chapter >>

Read also:

Get Jobilize Job Search Mobile App in your pocket Now!

100% Free Mobile Applications
Receive real-time job alerts and never miss the right job again

Source: OpenStax, Adaptive filters. OpenStax CNX. May 12, 2005 Download for free at http://cnx.org/content/col10280/1.1

Google Play and the Google Play logo are trademarks of Google Inc.

Notification Switch

Would you like to follow the 'Adaptive filters' conversation and receive update notifications?

Ask

	18 AP Key Terms 18 Cardiovascular System Blood By OpenStax Start Key Terms
	Anthropology Environment, Adaption Subsistence By Richley Crapo Start Assignment
	2 CDL Quiz - Air Brakes Endorsement Part 2 By Jazzycazz Jackson Start Quiz
	6 Arts Society: Theater 6 By Jonathan Long Start Quiz
©flickr: Elliott	Spanish Verbs Subject Pronouns By Mariah Hauptman Start Quiz
	24 PLANT QUIZ 1 By Brooke Delaney Start Exam
	Nutrition Exam By Hannah Sheth Start Quiz
	Computer System Engineering By Robert Morris Start Exam
	Social Media By Brenna Fike Start Quiz
	1 AP 01 Human Body Anatomy Physiology MCQ By OpenStax Start Quiz