2.1 Dual spaces

Definition

The set of linear functionals $f$ on $X$ is itself a vector space:

• $(f+g)\left(x\right)=f\left(x\right)+g\left(x\right)$ (closed under addition),
• $\left(af\right)\left(x\right)=af\left(x\right)$ (closed under scalar multiplication),
• $z\left(x\right)=0$ is linear and $f+z=f$ (features an addition zero).

Definition 1 The dual space (or algebraic dual) $X^{*}$ of $X$ is the vector space of all linear function on $X$ .

Example 1 If $X={\mathbb{R}}^N$ then $f\left(x\right)={\sum }_{i=1}^N{a}_i{x}_i\in X^{*}$ and any linear functional on $X$ can be written in this form.

Definition 2 Let $X$ be a normed space. The normed dual $X^{*}$ of $X$ is the normed space of all bounded linear functionals with norm $\parallel f\parallel ={sup}_{\parallel x\parallel \le 1}\left|f\left(x\right)\right|$ .

Since in the sequel we refer almost exclusively to the normed dual, we will abbreviate to “dual” (and ignore the algebraic dual in the process).

Example 2 Let us find the dual of $X={\mathbb{R}}^N$ . In this space,

From Example [link] , a linear functional $f$ can be written as:

There is a one-to-one mapping between the space $X$ and the dual of $X$ :

For this reason, ${\mathbb{R}}^n$ is called self-dual (as there is a one-to-one mapping ${\left({\mathbb{R}}^n\right)}^{*}⟷{\mathbb{R}}^n$ ). Using the Cauchy-Schwartz Inequality, we can show that

with equality if $a$ is a scalar multiple of $x$ . Therefore, we have that $\parallel f\parallel =\parallel a\parallel$ , i.e., the norms of $X$ and $X^{*}$ match through the mapping.

Theorem 1 All normed dual spaces are Banach.

Since we have shown that all dual spaces $X^{*}$ have a valid norm, it remains to be shown that all Cauchy sequences in in $X^{*}$ are convergent.

Let ${\left\{f_n\right\}}_{n=1}^{\infty }\subseteq X^{*}$ be a Cauchy sequence in $X^{*}$ . Thus, we have that for each $ϵ>0$ there exists $n_0\in {\mathbb{Z}}^{+}$ such that if $n,m>n_0$ then $\parallel f_n\left(x\right)-f_m\left(x\right)\parallel \to 0$ as $n,m\to \infty$ . We will first show that for an arbitrary input $x\in X$ , the sequence ${\left\{f_n\left(x\right)\right\}}_{n=1}^{\infty }\subseteq R$ is Cauchy. Pick an arbitrary $ϵ>0$ , and obtain the $n_0\in {\mathbb{Z}}^{+}$ from the definition of Cauchy sequence for ${\left\{f_n^{*}\right\}}_{n=1}^{\infty }$ . Then for all $n,m>n_0$ we have that $|f_n\left(x\right)-f_m\left(x\right)|=|(f_n-f_m)\left(x\right)|\le \parallel f_n\left(x\right)-f_m\left(x\right)\parallel ·\parallel x\parallel \le ϵ\parallel x\parallel$ .

Now, since $R=\mathbb{R}$ and $R=\mathbb{C}$ are complete, we have that the sequence ${\left\{f_n\left(x\right)\right\}}_{n=1}^{\infty }$ converges to some scalar $f^{*}\left(x\right)$ . In this way, we can define a new function $f^{*}$ that collects the limits of all such sequences over all inputs $x\in X$ . We conjecture that the sequence $f_n\to f^{*}$ : we begin by picking some $ϵ>0$ . Since ${\left\{f_n\left(x\right)\right\}}_{n=1}^{\infty }$ is convergent to $f^{*}\left(x\right)$ for each $x\in X$ , we have that for $n>n_{0,x}$ , $|f_n\left(x\right)-f_n^{*}\left(x\right)|\le ϵ$ . Now, let $n_0^{*}={sup}_{x\in X,\parallel x\parallel =1}{n}_{0,x}$ . Then, for $n>n_0^{*}$ ,

Therefore, we have found an $n_0^{*}$ for each $ϵ$ that fits the definition of convergence, and so $f_n\to f^{*}$ .

Next, we will show that $f^{*}$ is linear and bounded. To show that $f^{*}$ is linear, we check that

To show that $f^{*}$ is bounded, we have that for each $ϵ>0$ there exists $n_0^{*}\in {\mathbb{Z}}^{+}$ (shown above) such that if $n>n_0^{*}$ then

This shows that $f^{*}$ is linear and bounded and so $f^{*}\in X^{*}$ . Therefore, ${\left\{f_n^{*}\right\}}_{n=1}^{\infty }$ converges in $X^{*}$ and, since the Cauchy sequence was arbitrary, we have that $X^{*}$ is complete and therefore Banach.

The dual of ${\ell }_p$

Recall the space ${\ell }_p=\{(x_1,x_2,...):{\sum }_{i=1}^{\infty }|x_i{|}^p<\infty \}$ , with the ${\ell }_p$ -norm ${\left|\right|x\left|\right|}_p=({\sum }_{i=1}^{\infty }|x_i{{|}^p)}^{1/p}$ . The dual of ${\ell }_p$ is ${\left({\ell }_p\right)}^{*}={\ell }_q$ , with $q=\frac{p}{p-1}$ . That is, every linear bounded functional $f\in {\left({\ell }_p\right)}^{*}$ can be represented in terms of $f\left(x\right)={\sum }_{n=1}^{\infty }{a}_n{x}_n$ , where $a=(a_1,a_2,...)\in {\ell }_q$ . Note that if $p=2$ then $q=2$ , and so ${\ell }_2$ is self-dual.

Bases for self-dual spaces

Consider the example self-dual space ${\mathbb{R}}^n$ and pick a basis $\{e_1,e_2,...,e_n\}$ for it. Recall that for each $a\in {\mathbb{R}}^n$ there exists a bounded linear functional $f_a\in {\left({\mathbb{R}}^n\right)}^{*}$ given by $f_a\left(x\right)=⟨x,a⟩.$ Thus, one can build a basis for the dual space ${\left({\mathbb{R}}^n\right)}^{*}$ as $\{f_{e1},f_{e2},...,f_{en}\}$ .

Riesz representation theorem

Since it is easier to conceive a dual space by linking it to elements of a known space (as seen above for self duals), we may ask if there are large classes of spaces who are self-dual.

Theorem 2 (Riesz Representation Theorem) If $f\in H^{*}$ is a bounded linear functional on a Hilbert space $H$ , there exists a unique vector $y\in H$ such that for all $x\in H$ we have $f\left(x\right)=⟨x,y⟩$ . Furthermore, we have $\left|\right|f\left|\right|=\left|\right|y\left|\right|$ , and every $y\in H$ defines a unique bounded linear functional $f_y\left(x\right)=⟨x,y⟩$ .

Thus, since there is a one-to-one mapping between the dual of the Hilbert space and the Hilbert space ( $H^{*}\approx H$ ), we say that all Hilbert spaces are self-dual. Pick a linear bounded functional $f\in H^{*}$ and let $\mathcal{N}\subseteq H$ be the set of all vectors $x\in H$ for which $f\left(x\right)=0$ . Note that $\mathcal{N}$ is a closed subspace of $H$ , for if $\left\{x_n\right\}\subseteq \mathcal{N}$ is sequence that converges to $x\in H$ then, due to the continuity of $f$ , $f\left(x_n\right)\to f\left(x\right)=0$ and so $x\in \mathcal{N}$ . We consider two possibilities for the subspace $\mathcal{N}$ :

• If $N=H$ then $y=0$ and $f\left(x\right)=⟨x,0⟩=0$ .
• If $N\ne H$ then we can partition $H=\mathcal{N}+{\mathcal{N}}^{\perp }$ and so there must exist some $z_0\in {\mathcal{N}}^{\perp }$ , $z_0\ne 0$ . Then, define ${\displaystyle z=\frac{z_o}{\left|f\right(z_o\left)\right|}}$ to get $z\in {\mathcal{N}}^{\perp }$ such that $f\left(z\right)=1$ . Now, pick an arbitrary $x\in H$ and see that
  $$f(x-f(x\left)z\right)=f\left(x\right)-f\left(x\right)f\left(z\right)=f\left(x\right)-f\left(x\right)=0.$$
  Therefore, we have that $x-f\left(x\right)z\in \mathcal{N}$ . Since $z\perp \mathcal{N}$ , we have that
  $$\begin{array}{cc}\hfill ⟨x-f\left(x\right)z,z⟩& =0,\hfill \\ \hfill ⟨x,z⟩-f\left(x\right)⟨z,z⟩& =0,\hfill \end{array}$$
  and so
  $${\displaystyle f\left(x\right)=\frac{⟨x,z⟩}{⟨z,z⟩}=\frac{⟨x,z⟩}{{\left|\right|z\left|\right|}^2}=〈x,,,\frac{z}{{\left|\right|z\left|\right|}^2}〉.}$$
  Thus, we have found a vector $y=\frac{z}{{\parallel z\parallel }^2}$ such that $f\left(x\right)=⟨x,y⟩$ for all $x\in H$ . Now, to compute the norm of the functional, consider
  $$\left|f\right(x\left)\right|=|⟨x,y⟩|\le \left|\right|x\left|\right|·\left|\right|y\left|\right|,$$
  with equality if $x=\alpha y$ for $\alpha \in R$ . Thus, we have that $\parallel f\parallel =\parallel y\parallel$ .

We now show uniqueness: let us assume that there exists a second $\widehat{y}\ne y$ , $\widehat{y}\in H$ , for which $f\left(x\right)=⟨x,\widehat{y}⟩$ for all $x\in H$ . Then we have that for all $x\in H$ ,

This contradicts the original assumption that $y\ne \widehat{y}$ . Therefore we have that $y$ is unique, completing the proof.

Linear functional extensions

When we consider spaces nested inside one another, we can define pairs of functions that match each other at the overlap.

Definition 3 Let $f$ be a linear functional on a subspace $M\subseteq X$ of a vector space $X$ . A linear functional $F$ is said to be an extension of $f$ to $N$ (where $N$ is another subspace of $X$ that satisfies $M\subseteq N\subseteq X$ ) if $F$ is defined on $N$ and $F$ is identical to $f$ on $M$ .

Here is another reason why we are so interested in bounded linear functionals.

Theorem 3 (Hahn-Banach Theorem) A bounded linear functional $f$ on a subspace $M\subseteq X$ can be extended to a bounded linear functional $F$ on the entire space $X$ with

The proof is in page 111 of Luenberger.