0.6 Nonparametric regression with wavelets  (Page 4/5)

In [link] , Donoho and Johnstone showed that, when using the universal threshold, the following oracle inequality prevails

However, this inequality is not optimal. Donoho and Johnstone looked for the optimal threshold $t^{*}\left(n\right)$ which leads to the smallest possible constant ${\Lambda }_n^{*}$ in place of $2logn+1$ . Such a threshold does not exist in closed form, but can be approximated numerically. For small sample size, it is sensibly smaller than the universal threshold.

Sureshrink procedure

Given the expression [link] for the $l_2$ -risk, it is natural to look for a threshold that minimizes an estimation of this risk.

By minimizing Stein's unbiased estimate of the risk [link] and using a soft thresholding scheme, the resulting estimator, called `SureShrink', is adaptive over a wide range of function spaces including Hölder, Sobolev, and Besov spaces, see "Adaptivity of wavelet estimator" . That is, without any a priori knowledge on the type or amount of regularity of the function, the SURE procedure nevertheless achieves the optimal rate of convergence that one could attain by knowing the regularity of the function.

Other thresholding procedures

We mention some of the other thresholding or shrinkage procedures proposed in the literature.

Instead of considering each coefficient individually, Cai et al. [link] , [link] consider blocks of empirical wavelet coefficients in order to make simultaneous shrinkagedecisions about all coefficients within a block.

Another fruitful idea is to use the Bayesian framework. There a prior distribution is imposed on the wavelet coefficients $d_{jk}$ . This prior model is designed to capture the sparseness of the wavelet expansion. Next, the function is estimated by applying some Bayes rules on the resulting posteriordistribution of the wavelet coefficients, see for example [link] , [link] , [link] , [link] .

Antoniadis and Fan [link] treat the problem of selecting the wavelet coefficients as a penalized least squares issue. Let $W$ be the matrix of an orthogonal wavelet transform and $Y:={\left\{Y_i\right\}}_{i=1}^n$ . The detail coefficients $d:=\left\{d_{jk}\right\}$ which minimize

are used to estimate the true wavelet coefficients. In equation [link] , $p_{\lambda }(·)$ is a penalty function which depends on the regularization parameter $\lambda$ . The authors provide a general framework, wheredifferent penalty functions correspond to different type of thresholding procedures (like, e.g., the soft- and hard- thresholding)and obtain oracle inequalities for a large class of penalty functions.

Other methods include threshold selection by hypothesis testing [link] , cross-validation [link] , or generalizedcross-validation [link] , [link] , which is used to estimated the $l_2$ -risk of the empirical detail coefficients.

Linear versus nonlinear wavelet estimator

In order to differenciate the behaviours of a linear and of a nonlinear wavelet estimator, we consider the Sobolev class $W_q^s\left(C\right)$ defined as

and that we denote $V$ in short. Assume we know that $m$ , the function to be estimated, belongs to $V$ . In the next section, we will release this assumption. The $L_p-$ risk of an arbitrary estimator $T_n$ based on the sample data is defined as $E{∥T_n,-,m∥}_p^p,1\le p<\infty$ , whereas the $L_p-$ minimax risk is given by