# 1.11 Least squares estimation in hilbert spaces

 Page 1 / 2
Presents leasts squares estimation in subspaces of Hilbert spaces, with applications.

## Projections with orthonormal bases

Having an orthonormal basis for the subspace of interest significantly simplifies the projection operator.

Lemma 1 Let $x\in X$ , a Hilbert space, and let $S$ be a subspace of $X$ . If $\left\{{b}_{1},{b}_{2},...\right\}$ is an orthonormal basis for $S$ , then the closest point ${s}_{0}\in S$ to $x$ is given by ${s}_{0}={\sum }_{i}⟨x,{b}_{i}⟩{b}_{i}$ .

We begin by noting that

$\begin{array}{cc}\hfill \sum _{i}⟨x,{b}_{i}⟩{b}_{i}& =\sum _{i}⟨x-{s}_{0}+{s}_{0},{b}_{i}⟩{b}_{i}=\sum _{i}⟨x-{s}_{0},{b}_{i}⟩{b}_{i}+\sum _{i}⟨{s}_{0},{b}_{i}⟩{b}_{i}.\hfill \end{array}$

Now, since ${s}_{0}$ is the projection of $x$ onto $S$ , we must have that $x-{s}_{0}\perp S$ , and so for each basis element ${b}_{i}$ we must have $⟨x-{s}_{0},{b}_{i}⟩=0$ . Additionally, since ${s}_{0}\in S$ and $\left\{{b}_{1},{b}_{2},...\right\}$ is an orthonormal basis for $S$ , we must have that ${\sum }_{i}⟨{s}_{0},{b}_{i}⟩{b}_{i}={s}_{0}$ . Thus, we obtain

$\begin{array}{cc}\hfill \sum _{i}⟨x,{b}_{i}⟩{b}_{i}& ={s}_{0},\hfill \end{array}$

proving the lemma.

Consider the case of a communications receiver that records a continuous-time signal $r\left(t\right)=s\left(t\right)+n\left(t\right)$ over $0\le t\le 1$ , where $s\left(t\right)$ is one of $m$ codeword signals $\left\{{s}_{1}\left(t\right),...,{s}_{m}\left(t\right)\right\}$ , and $n\left(t\right)$ is additive white Gaussian noise. The receiver must make the best possible decision on the observed codeword given the reading $r\left(t\right)$ ; this usually involves removing as much of the noise as possible from $r\left(t\right)$ .

We analyze this problem in the context of the Hilbert space ${L}_{2}\left[0,1\right]$ . To remove as much of the noise as possible, we define the subspace $S=\mathrm{span}\left(\left\{{s}_{1}\left(t\right),...,{s}_{m}\left(t\right)\right\}\right)$ . Anything that is not contained in this subspace is guaranteed to be part of the noise $n\left(t\right)$ . Now, to obtain the projection into $S$ , we need to find an orthonormal basis $\left\{{e}_{1}\left(t\right),...,{e}_{n}\left(t\right)\right\}$ for $S$ , which can be done for example by applying the Gram-Schmidt procedure on the vectors $\left\{{s}_{1}\left(t\right),...,{s}_{m}\left(t\right)\right\}$ . The projection is then obtained according to the lemma as

$\begin{array}{cc}\hfill {r}_{S}\left(t\right)& =\sum _{i=1}^{n}⟨r\left(t\right),{e}_{i}\left(t\right)⟩{e}_{i}\left(t\right),\hfill \end{array}$

where $⟨r\left(t\right),{e}_{i}\left(t\right)⟩={\int }_{0}^{1}r\left(t\right){e}_{i}\left(t\right)dt$ .

After the projection is obtained, an optimal receiver proceeds by finding the value of $k$ that minimizes the distance

$\begin{array}{cc}\hfill {d}_{2}\left({r}_{S}\left(t\right),{s}_{k}\left(t\right)\right)& ={\int }_{0}^{1}{|{r}_{S}\left(t\right)-{s}_{k}\left(t\right)|}^{2}dt={\int }_{0}^{1}{r}_{S}{\left(t\right)}^{2}dt+{\int }_{0}^{1}{s}_{k}{\left(t\right)}^{2}dt-2{\int }_{0}^{1}{r}_{S}\left(t\right){s}_{k}\left(t\right)2dt;\hfill \end{array}$

note here that the first term does not depend on $k$ , so it suffices to find the value of $k$ that minimizes the “cost”

$\begin{array}{cc}\hfill {c}_{k}& :={\int }_{0}^{1}{s}_{k}{\left(t\right)}^{2}dt-2{\int }_{0}^{1}{r}_{S}\left(t\right){s}_{k}\left(t\right)2dt=⟨{s}_{k}\left(t\right),{s}_{k}\left(t\right)⟩-2⟨{r}_{S}\left(t\right),{s}_{k}\left(t\right)⟩\hfill \\ & =⟨{s}_{k}\left(t\right),{s}_{k}\left(t\right)⟩-2〈\sum _{i=1}^{n},⟨r\left(t\right),{e}_{i}\left(t\right)⟩,{e}_{i},\left(t\right),,,{s}_{k},\left(t\right)〉\hfill \\ & =⟨{s}_{k}\left(t\right),{s}_{k}\left(t\right)⟩-2\sum _{i=1}^{n}⟨r\left(t\right),{e}_{i}\left(t\right)⟩⟨{e}_{i}\left(t\right),{s}_{k}\left(t\right)⟩.\hfill \end{array}$

In practice, the codeword signals are designed so that their norms $\parallel {s}_{k}{\left(t\right)\parallel }_{2}=\sqrt{⟨{s}_{k}\left(t\right),{s}_{k}\left(t\right)⟩}$ are all equal. This design choice reduces the problem above to finding the value of $k$ that maximizes the score

$\begin{array}{cc}\hfill {c}_{k}^{\text{'}}& :=\sum _{i=1}^{n}⟨r\left(t\right),{e}_{i}\left(t\right)⟩⟨{e}_{i}\left(t\right),{s}_{k}\left(t\right)⟩.\hfill \end{array}$

Thus, the receiver can be designed according to the diagram in [link] .

## Least squares approximation in hilbert spaces

Let ${y}_{1},...,{u}_{n}$ be elements of a Hilbert space $X$ and define the closed, finite-dimensional subspace of $X$ given by $S=\mathrm{span}\left({y}_{1},...,{y}_{n}\right)$ . We wish to find the best approximation of $x$ in terms of the vectors ${y}_{i}$ , that is, the linear combination ${\sum }_{i=1}^{n}{a}_{i}{y}_{i}$ with the smallest error $e=x-{\sum }_{i=1}^{n}{a}_{i}{y}_{i}$ . To measure the size of the error, we use the induced norm $\parallel e\parallel =∥x,-,{\sum }_{i=1}^{n},{a}_{i},{y}_{i}∥$ .

To solve this problem, we rely on the projection theorem: we are indeed looking for the closest point to $x$ in $S=\mathrm{span}\left({y}_{1},...,{y}_{n}\right)$ . The projection theorem tells us that the closest point ${s}_{0}={\sum }_{i=1}^{n}{a}_{i}{y}_{i}$ must give $x-{s}_{0}\perp S$ , i.e., $e\perp S$ , which implies in turn that $〈x,-,{\sum }_{i=1}^{n},{a}_{i},{y}_{i},,,{y}_{j}〉=0$ for all $j=1,...,n$ . The requirement can be rewritten as $⟨x,{y}_{j}⟩=〈{\sum }_{i=1}^{n},{a}_{i},{y}_{i},,,{y}_{j}〉={\sum }_{i=1}^{n}{a}_{i}⟨{y}_{i},{y}_{j}⟩$ for each $j=1,...,n$ . These requirements can be collected and written in matrix form as

where we get a research paper on Nano chemistry....?
what are the products of Nano chemistry?
There are lots of products of nano chemistry... Like nano coatings.....carbon fiber.. And lots of others..
learn
Even nanotechnology is pretty much all about chemistry... Its the chemistry on quantum or atomic level
learn
da
no nanotechnology is also a part of physics and maths it requires angle formulas and some pressure regarding concepts
Bhagvanji
Preparation and Applications of Nanomaterial for Drug Delivery
revolt
da
Application of nanotechnology in medicine
what is variations in raman spectra for nanomaterials
I only see partial conversation and what's the question here!
what about nanotechnology for water purification
please someone correct me if I'm wrong but I think one can use nanoparticles, specially silver nanoparticles for water treatment.
Damian
yes that's correct
Professor
I think
Professor
Nasa has use it in the 60's, copper as water purification in the moon travel.
Alexandre
nanocopper obvius
Alexandre
what is the stm
is there industrial application of fullrenes. What is the method to prepare fullrene on large scale.?
Rafiq
industrial application...? mmm I think on the medical side as drug carrier, but you should go deeper on your research, I may be wrong
Damian
How we are making nano material?
what is a peer
What is meant by 'nano scale'?
What is STMs full form?
LITNING
scanning tunneling microscope
Sahil
how nano science is used for hydrophobicity
Santosh
Do u think that Graphene and Fullrene fiber can be used to make Air Plane body structure the lightest and strongest. Rafiq
Rafiq
what is differents between GO and RGO?
Mahi
what is simplest way to understand the applications of nano robots used to detect the cancer affected cell of human body.? How this robot is carried to required site of body cell.? what will be the carrier material and how can be detected that correct delivery of drug is done Rafiq
Rafiq
if virus is killing to make ARTIFICIAL DNA OF GRAPHENE FOR KILLED THE VIRUS .THIS IS OUR ASSUMPTION
Anam
analytical skills graphene is prepared to kill any type viruses .
Anam
Any one who tell me about Preparation and application of Nanomaterial for drug Delivery
Hafiz
what is Nano technology ?
write examples of Nano molecule?
Bob
The nanotechnology is as new science, to scale nanometric
brayan
nanotechnology is the study, desing, synthesis, manipulation and application of materials and functional systems through control of matter at nanoscale
Damian
Is there any normative that regulates the use of silver nanoparticles?
what king of growth are you checking .?
Renato
What fields keep nano created devices from performing or assimulating ? Magnetic fields ? Are do they assimilate ?
why we need to study biomolecules, molecular biology in nanotechnology?
?
Kyle
yes I'm doing my masters in nanotechnology, we are being studying all these domains as well..
why?
what school?
Kyle
biomolecules are e building blocks of every organics and inorganic materials.
Joe
how did you get the value of 2000N.What calculations are needed to arrive at it
Privacy Information Security Software Version 1.1a
Good
Got questions? Join the online conversation and get instant answers! By OpenStax By Michael Sag By OpenStax By OpenStax By OpenStax By Brooke Delaney By Vanessa Soledad By Mahee Boo By Stephen Voron By Bonnie Hurst