4.4 Normalized lms

In "normalized" LMS, the gradient step factor $$ is normalized by the energy of the data vector: $${}_{\mathrm{NLMS}}=\frac{}{X_k^H{X}_k+}$$ where $$ is usually $\frac{1}{2}$ and $$ is a very small number introduced to prevent division by zero if $X_k^H{X}_k$ is very small. $$W_{k+1}=W_k+\frac{1}{X^{H}X}{e}_k{X}_k$$ The normalization has several interpretations

• corresponds to the 2nd-order convergence bound
• makes the algorithm independent of signal scalings
• adjusts $W_{k+1}$ to give zero error with current input: $W_{k+1}{X}_k=d_k$
• minimizes mean effort at time $k+1$