1.11 Hilbert spaces and separable vector spaces

Hilbert spaces

Hilbert Spaces

A Hilbert space $$ is a closed, normed linear vector space which contains all of its limitpoints: if $\{x_n\}$ is any sequence of elements in $$ that converges to $x$ , then $x$ is also contained in $$ . $x$ is termed the limit point of the sequence.

orthogonal

If $$ is a subspace of $$ , the vector $x$ is orthogonal to the subspace $$ for every $y\in$ , $x\cdot y=0$ .

We now arrive at a fundamental theorem.

Let $$ be a Hilbert space and $$ a subspace of it. Any element $x\in$ has the unique decomposition $x=y+z$ , where $y\in$ and $z$ is orthogonal to $$ . Furthermore, $(x-y)=\min\{(x-)\mid \in \}$ : the distance between $x$ and all elements of $$ is minimized by the vector $y$ . This element $y$ is termed the projection of $x$ onto $$ .

Geometrically, $$ is a line or a plane passing through the origin. Any vector $x$ can be expressed as the linear combination of a vector lying in $$ and a vector orthogonal to $y$ . This theorem is of extreme importance in linear estimation theoryand plays a fundamental role in detection theory.

Separable vector spaces

separable

A Hilbert space $$ is said to be separable if there exists a set of vectors $\{{}_i\}$ , $i=\{1, , \mathrm{elementsof}\}$ , that express every element $x\in$ as

$x=\sum_{i=1}$∞ x i i

where $x_i$ are scalar constants associated with ${}_i$ and $x$ and where "equality" is taken to mean that the distancebetween each side becomes zero as more terms are taken in the right. $$\lim_{m\to }m\to$$∞ x i 1 m x i i 0

The set of vectors $\{{}_i\}$ are said to form a complete set if the above relationship is valid. A complete set is said to form a basis for the space $$ . Usually the elements of the basis for a space are taken to be linearlyindependent. Linear independence implies that the expression fo the zero vector by a basis can only be madeby zero coefficients.

In general, the upper limit on the sum in is infinite. For the previous example , the upper limit is finite. The number of basis vectors that is required to express every element of a separable space in terms of is said to be the dimension of the space. In this example , the dimension of the space is $N$ . There exist separable vector spaces for which the dimension isinfinite.

orthonormal

The basis for a separable vector space is said to be an orthonormal basis if the elements of the basis satisfy thefollowing two properties:

• The inner product between distinct elements of the basis is zero (i.e., the elements of the basis are mutuallyorthogonal).
  $$\forall i, j, i\neq j\colon {}_i\cdot {}_j=0$$
• The norm of each element of a basis is one (normality).
  $$\forall i, i=\{1, \}\colon ({}_i)=1$$

For example, the basis given above for the space of $N$ -dimensional column matrices is orthonormal. For clarity, two facts must be explicitlystated. First, not every basis is orthonormal. If the vector space is separable, a complete set of vectors can be found;however, this set does not have to be orthonormal to be a basis. Secondly, not every set of orthonormal vectors canconstitute a basis. When the vector space $L^2$ is discussed in detail, this point will be illustrated.

Despite these qualifications, an orthonormal basis exists for every separable vector space. There is an explicitalgorithm - the Gram-Schmidt procedure - for deriving an orthonormal set of functions from a complete set.Let $\{{}_i\}$ denote a basis; the orthonormal basis $\{{}_i\}$ is sought. The Gram-Schmidt procedure is:

• 1.

  ${}_1=\frac{{}_1}{({}_1)}$ . This step makes ${}_1$ have unit length.

• 2.

  ${}_2^{}={}_2-({}_1\cdot {}_2){}_1$ . Consequently, the inner product between ${}_2^{}$ and ${}_1$ is zero. We obtain ${}_2$ from ${}_2^{}$ forcing the vector to have unit length.

• 2'.

  ${}_2=\frac{{}_2^{}}{({}_2^{})}$ .

The algorithm now generalizes.

• K.

  ${}_k^{}={}_k-\sum_{i=1}^{k-1} ({}_i\cdot {}_k){}_i$

• K'.

  ${}_k=\frac{{}_k^{}}{({}_k^{})}$

By construction, this new set of vectors is an orthonormal set. As the original set of vectors $\{{}_i\}$ is a complete set, and, as each ${}_k$ is just a linear combination of ${}_i$ , $i=\{1, , k\}$ , the derived set $\{{}_i\}$ is also complete. Because of the existence of this algorithm, a basis for a vector space is usually assumed to beorthonormal.

A vector's representation with respect to an orthonormal basis $\{{}_i\}$ is easily computed. The vector $x$ may be expressed by:

The mathematical representation of a vector (expressed by equations and can be expressed geometrically. This expression is ageneralization of the Cartesian representation of numbers. Perpendicular axes are drawn; these axes correspond to theorthonormal basis vector used in the representation. A given vector is representation as a point in the "plane" with thevalue of the component along the ${}_i$ axis being $x_i$ .

An important relationship follows from this mathematical representation of vectors. Let $x$ and $y$ by any two vectors in a separable space. These vectors are represented with respectto an orthonormal basis by $\{x_i\}$ and $\{y_i\}$ , respectively. The inner product $x\cdot y$ is related to these representations by: $$x\cdot y=\sum_{i=1}$$∞ x i y i This result is termed Parseval's Theorem . Consequently, the inner product between any two vectors can becomputed from their representations. A special case of this result corresponds to the Cartesian notion of the length of avector; when $x=y$ , Parseval's relationship becomes: $$(x)=\sqrt{\sum_{i=1} }$$∞ x i 2 These two relationships are key results of the representation theorem. The implication is that any inner product computedfrom vectors can also be computed from their representations. There are circumstances in which the latter computation ismore manageable than the former and, furthermore, of greater theoretical significance.