1.2 Normed spaces

Distances and metrics allow us to evaluate how different two signals are from each other. Norms allow us to evaluate how “big”, “important”, or “interesting” a given signal is.

Definition 1 Assume $X$ is a linear vector space. A norm on $X$ is a function $\parallel ·\parallel :X\to \mathbb{R}$ with the following properties for all $x,y\in X$ and $a\in R$ :

1. $\parallel x\parallel \ge 0$ , (non-negativity)
2. $\parallel x\parallel =0$ if and only if $x=0$ (zero norm for zero vector)
3. $\parallel a·x\parallel =\left|a\right|·\parallel x\parallel$ , (scaling)
4. $\parallel x+y\parallel \le \parallel x\parallel +\parallel y\parallel$ . (triangle inequality)

$\parallel x\parallel$ is read as the norm of $x$ or length of $x$ .

Intuitively, one can say that $\parallel x\parallel$ is the distance between $x$ and the zero vector (more on this soon).

Definition 2 A vector space $X$ with a norm $\parallel ·\parallel$ is called a normed linear vector space $(X,\parallel ·\parallel )$ (or a normed space for brevity).

Definition 3 Let $(X,\parallel ·\parallel )$ be a normed space. The induced metric or induced distance is given by $d_I(x,y)=\parallel x-y\parallel$ .

Definition 4 If a normed space is complete under the induced metric, then it is called a Banach space .

All norms induce distances, but not all distances are induced by norms.

Example 1 Consider the distance

Let us assume that there exists a norm ${\parallel x\parallel }_i=d^{\text{'}}(x,0)$ that would induce this distance. We would then have for $x\ne 0$ and $\alpha \notin \{-1,0,1\}$ that ${\parallel \alpha x\parallel }_i=d^{\text{'}}(\alpha x,0)=1$ and ${\parallel x\parallel }_i=d^{\text{'}}(x,0)=1$ , which contradicts ${\parallel \alpha x\parallel }_i={\left|\alpha \right|\parallel x\parallel }_i$ . Thus ${\parallel ·\parallel }_i$ is not a valid norm.

In contrast, here are some examples of valid norms.

Example 2 The vector space $X=C\left[T\right]$ accepts the norm ${\parallel x\parallel }_{\infty }={sup}_{t\in T}\left|x\left(t\right)\right|$ . The induced distance is $d_i(x,y)={sup}_{t\in T}|x\left(t\right)-y\left(t\right)|=d_{\infty }(x,y)$ ; it is straightforward to prove properties (1–4). We previously showed that the metric space $(C\left[T\right],d_{\infty })$ is complete, and so $(C\left[T\right],\parallel ·{\parallel }_{\infty })$ is a Banach space.

Example 3 The vector space $X={\mathbb{R}}^n$ accepts the norm ${\parallel x\parallel }_2={\left({\sum }_{i=1}^n,{\left|x_i\right|}^2\right)}^{1/2}$ . The induced metric is $d_i(x,y)={\parallel x-y\parallel }_2={\left({\sum }_{i=1}^n,{|x_i-y_i|}^2\right)}^{1/2}=d_2(x,y)$ , the Euclidean distance. Thus ${\parallel ·\parallel }_2$ is known as the Euclidean norm. The spaces $({\mathbb{R}}^n,d_2)$ are Banach spaces for all values of $n$ .

Example 4 The vector space $X=[0,1)$ accepts the norm $\parallel x\parallel =\left|x\right|$ . The induced metric $d_i(x,y)=|x-y|=d_0(x,y)$ is the standard metric for the reals. Since we have previously shown that $(X,d_0)$ is not a complete vector space, then the space $\left(\right(0,1],\parallel ·\parallel )$ is not a Banach space.