<< Chapter < Page | Chapter >> Page > |
We also must define the notion of an extremum in an arbitrary normed space.
Definition 1 Let $f$ be a real-valued functional defined on $\Omega \subseteq X$ where $X$ is a normed space. A point ${x}_{0}\in \Omega $ is a local/relative minimum of $f$ on $\Omega $ if $f\left({x}_{0}\right)\le f\left(x\right)$ for all $x\in \Omega $ such that $\parallel x-{x}_{0}\parallel <\u03f5$ for some $\u03f5>0$ .
Definition 2 Let $f$ be a real-valued functional defined on $\Omega \subseteq X$ where $X$ is a normed space. A point ${x}_{0}\in \Omega $ is a local maximum of $f$ on $\Omega $ if $f\left({x}_{0}\right)\ge f\left(x\right)$ for all $x\in \Omega $ such that $\parallel x-{x}_{0}\parallel <\u03f5$ for some $\u03f5>0$ .
Definition 3 Let $f$ be a real-valued functional defined on $\Omega \subseteq X$ where $X$ is a normed space. A point ${x}_{0}\in \Omega $ is a local strict minimum of $f$ on $\Omega $ if $f\left({x}_{0}\right)<f\left(x\right)$ for all $x\in \Omega $ such that $\parallel x-{x}_{0}\parallel <\u03f5$ for some $\u03f5>0$ .
Definition 4 Let $f$ be a real-valued functional defined on $\Omega \subseteq X$ where $X$ is a normed space. A point ${x}_{0}\in \Omega $ is a local strict maximum of $f$ on $\Omega $ if $f\left({x}_{0}\right)>f\left(x\right)$ for all $x\in \Omega $ such that $\parallel x-{x}_{0}\parallel <\u03f5$ for some $\u03f5>0$ .
It turns out the notion of a gradient is intrinsically linked to the directional derivatives we have introduced.
Definition 5 Let $X$ be a Hilbert space and $f:X\to R$ . If $f$ is a Fréchet differentiable functional, then for each $x\in X$ there exists a vector in $X$ such that $\delta f(x;h)=\u27e8h,\nabla f\left(x\right)\u27e9$ for all $h\in X$ ; the vector $\nabla f\left(x\right)$ is called the gradient of $f$ at $x$ , and can be written as a functional $\nabla f:X\to X$ .
This definition can be seen to correspond to an application of the Riesz representation theorem to the Fréchet derivative $\delta f(x;h)$ , which is a linear bounded functional on $h$ .
Example 1 We know now that:
By the Cauchy-Schwarz Inequality, we have:
If $h=\nabla f\left(x\right)$ then $\delta f(x;h)$ is maximized.
Example 2 Recall that if $f:{\mathbb{R}}^{n}\to \mathbb{R}$ , then
Theorem 1 Let $f:X\to \mathbb{R}$ have a Gâteaux differential on $X$ . A necessary condition for $f$ to have an extremum at ${x}_{0}\in X$ is that $\delta f({x}_{0};h)=0$ for all $h\in X$ . Alternatively, if $X$ is a Hilbert space, we can write $\u27e8h,\nabla f\left({x}_{0}\right)\u27e9=0$ for all $h\in X$ , which implies $\nabla f\left({x}_{0}\right)=0$ .
Suppose ${x}_{0}$ is a local minimum. Then there exists $\u03f5>0$ such that if $\parallel x-{x}_{0}\parallel <\u03f5$ then $f\left({x}_{0}\right)\le f\left(x\right)$ . Fix $h\ne 0$ and let $\theta =\frac{\u03f5}{\parallel h\parallel}$ . Next, consider $x={x}_{0}+\alpha h$ . For $\alpha \in (-\theta ,\theta )$ :
Therefore, $\delta f({x}_{0};h)=0$ for arbitrary nonzero $h$ . Now since $\delta f(x;h)$ is linear on $h$ we must have $\delta f(x;h)=0$ for $h=0$ . Therefore, the equality is true for all $h\in X$ .
Definition 6 A point at which $\delta f(x;h)=0$ for all $h\in X$ is called a stationary point of $f$ .
Notification Switch
Would you like to follow the 'Signal theory' conversation and receive update notifications?