0.15 Denoising ii: adapting to unknown smoothness

 Page 1 / 2

Review: denoising in smooth function spaces i - method of sieves

Suppose we make noisy measurements of a smooth function:

${Y}_{i}={f}^{*}\left({x}_{i}\right)+{W}_{i},\phantom{\rule{4pt}{0ex}}i=\left\{1,...,n\right\},$

where

${W}_{i}\stackrel{i.i.d.}{\sim }N\left(0,,,{\sigma }^{2}\right)$

and

${x}_{i}=\left(\frac{i}{n}\right).$

The unknown function ${f}^{*}$ is a map

${f}^{*}:\left[0,1\right]\to \mathbf{R}.$

In Lecture 4 , we consider this problem in the case where ${f}^{*}$ was Lipschitz on $\left[0,1\right].$ That is, ${f}^{*}$ satisfied

$|{f}^{*}\left(t\right)-{f}^{*}\left(s\right)|\le L|t-s|,\phantom{\rule{4pt}{0ex}}\phantom{\rule{5.69054pt}{0ex}}\forall t,s\in \left[0,1\right]$

where $L>0$ is a constant. In that case, we showed that by using a piecewise constant function on a partition of ${n}^{\frac{1}{3}}$ equal-size bins [link] we were able to obtain an estimator ${\stackrel{^}{f}}_{n}$ whose mean square error was

$E\left[\parallel ,{f}^{*},-,{\stackrel{^}{f}}_{n},{\parallel }^{2}\right]=O\left({n}^{-\frac{2}{3}}\right).$

In this lecture we will use the Maximum Complexity-Regularized Likelihood Estimation result we derived in Lecture 14 to extend our denoising scheme in several important ways.

To begin with let's consider a broader class of functions.

Hölder spaces

For $0<\alpha <1,$ define the space of functions

${H}^{\alpha }\left({C}_{\alpha }\right)=\left\{|f|<,{C}_{\alpha },:,\phantom{\rule{2.84526pt}{0ex}},\underset{x,h}{sup},\frac{|f\left(x+h\right)-f\left(x\right)|}{{|h|}^{\alpha }},\le ,{C}_{\alpha }\right\}$

for some constant ${C}_{\alpha }<\infty$ and where $f\in {L}_{\infty }.$ ${H}^{\alpha }$ above contains functions that are bounded, but less smooth than Lipschitz functions. Indeed, the space of Lipschitzfunctions can be defined as ${H}^{1}$ ( $\alpha =1$ )

${H}^{1}\left({C}_{1}\right)=\left\{|f|<,{C}_{1},:,\phantom{\rule{2.84526pt}{0ex}},\underset{x,h}{sup},\frac{|f\left(x+h\right)-f\left(x\right)|}{|h|},\le ,{C}_{1}\right\}$

for ${C}_{1}<\infty .$ Functions in ${H}^{1}$ are continuous, but those in ${H}^{\alpha },\alpha <1,$ are not in general.

Let's also consider functions that are smoother than Lipschitz. If $\alpha =1+\beta ,$ where $0<\beta <1,$ then define

${H}^{\alpha }\left({C}_{\alpha }\right)=\left\{f,\in ,{H}^{1},\left({C}_{\alpha }\right),:,\frac{\partial f}{\partial x},\in ,{H}^{\beta },\left({C}_{\alpha }\right)\right\}.$

In other words, ${H}^{\alpha },1<\alpha <2,$ contains Lipschitz functions that are also differentiable and their derivatives are Hölder smooth with smoothness $\beta =\alpha -1.$

And finally, let

${H}^{2}\left({C}_{2}\right)=\left\{f,:,\frac{\partial f}{\partial x},\in ,{H}^{1},\left({C}_{2}\right)\right\}$

contain functions that have continuous derivatives, but that are notnecessarily twice-differentiable.

If $f\in {H}^{\alpha }\left({C}_{\alpha }\right)$ , $0<\alpha \le 2,$ then we say that $f$ is Hölder $-\alpha$ smooth with Hölder constant ${C}_{\alpha }.$ The notion of Hölder smoothness can also be extended to $\alpha >2$ in a straightforward way.

Note: If ${\alpha }_{1}<{\alpha }_{2}$ then

$f\in {H}^{{\alpha }_{2}}⇒f\in {H}^{{\alpha }_{1}}.$

Summarizing, we can describe Hölder spaces as follows. If ${f}^{*}\in {H}^{\alpha }\left({C}_{\alpha }\right)$ for some $0<\alpha \le 2$ and ${C}_{\alpha }<\infty ,$ then

• $0<\alpha \le 1$                    $|{f}^{*}\left(t\right)-{f}^{*}\left(s\right)|\le {C}_{\alpha }{|t-s|}^{\alpha }$
• $1<\alpha \le 2$                    $\left|\frac{\partial {f}^{*}}{\partial x},\left(t\right),-,\frac{\partial {f}^{*}}{\partial x},\left(s\right)\right|\le {C}_{\alpha }{|t-s|}^{\alpha -1}$

Note that in general there is a natural relationship between the Hölder space containing the function and the approximation classused to estimate the function. Here we will consider functions which are Hölder $-\alpha$ smooth where $0<\alpha \le 2$ and work with piecewise linear approximations. If we were to consider smootherfunctions, $\alpha >2$ we would need consider higher order approximation functions, i.e. quadratic, cubic, etc.

Denoising example for signal-plus-gaussian noise observation model

Now let's assume ${f}^{*}\in {H}^{\alpha }\left({C}_{\alpha }\right)$ for some unknown $\alpha \left(0<\alpha \le 2\right)$ ; i.e. we don't know how smooth ${f}^{*}$ is. We will use our observations

${Y}_{i}={f}^{*}\left({x}_{i}\right)+{W}_{i},\phantom{\rule{4pt}{0ex}}i=\left\{1,...,n\right\},$

to construct an estimator ${\stackrel{^}{f}}_{n}.$ Intuitively, the smoother ${f}^{*}$ is, the better we should be able to estimate it. Can we take advantage ofextra smoothness in ${f}^{*}$ if we don't know how smooth it is? The smoother ${f}^{*}$ is, the more averaging we can perform to reduce noise. In other words for smoother ${f}^{*}$ we should average over larger bins. Also, we will need to exploit the extra smoothnessin our approximation of ${f}^{*}.$ To that end, we will consider candidate functions that are piecewiselinear functions on uniform partitions of $\left[0,1\right].$ Let

is economics important in programming world?
What is oppunity cost
it refers wants that are left unsatisfied in Oder satisfy another more pressing need
Osei
Thank bro
Kudzanayi
is the second altenative to foregone
swedy
How does monopoly and imperfect competion;public goods externalities ass symmetric information, ommon property ressourses; income distribution Merit goods and Macroeconomic growth and stability causes market failure?
Gcina
it is where by a labour moves from one place to another
yh
Osei
another
Amoako
yh
Osei
ok
Amoako
what is mobility of labour
who best defined economics
what are the importance of economics
Asamoah
Aakash
help to know how our government operates in which system
swedy
1. To solve economic problems. 2. To predict economic event. 3. It also offer intellectual training to students.
samuel
How price elasticity can affect the markets of certain goods
Heyy thanks for the teaching
what is labour
Mental and physical ability of human being is considered as labour.
JEYARAM
and usually provided by human
KEMAWOR
is all human effort both physical and mental abilities to work
Asamoah
why is it that most countries in Africa abuse available resources
What is economics
economics is a science which studies human behaviour as a relationship between ends, scarce means which have alternative uses.
prince
Why is scarcity a mind problem in economics
because of the problem of allocation of resources
Osola
unequalled distribution of resources
Agyen
thanks
Alima
scarcity is a mind problem due to circumstances like when a particular product is demand in a high rate at the market.
Saihou
scarcity defines limited in supply relative to the demand them. so scarcity is a mind problem in economics because wants are unlimited while resources are also limited.
prince
it is a mind problem because it's one of the fundamental issue address by economics human wants are unlimited and resources available are limited this makes the study of economics essential
rashid
its a fundamental issue
zahid
it's not a mind problem, I think it's a economic problem, how to allocate scarce resources to satisfy need and want of society
Samiullah
Identify the different sectors in the economy
what is economics
Moses
economic is study of human behaviour according to how they satisfie their numerous want
Osola
Economics is a science which studies human behaviour as a relationship between ends and scares resources.
JEYARAM
economic is study of human behaviour according to how they satisfie their numerous want
Amar
what is the law of demand
the lower the price the higher the quantity demanded vice versa is true
yes
vivek
yes ooh
Asamoah
right
samuel
what are relationship between unemployment and economic growth
no relationship
Awuah
When there is economic growth, there is increased opportunity in employment. When there is no economic growth mean there is recession causing a decline or downsizing in employment opportunities.
Teescou
unemployment or employment determined by the level of economic growth
Osola
Got questions? Join the online conversation and get instant answers!