3.4 Second order conditions

When the objective function and constraints $f:{\mathbb{R}}^n\to \mathbb{R}$ , $g_i:{\mathbb{R}}^n\to \mathbb{R}$ , it is easy to check whether an extremum $x_0$ is a maximum or a minimum of the functional. We appeal to second-order differentials, known as Hessians.

Definition 1 The $n\times n$ Hessian matrix $F\left(x\right)$ for the functional $f\left(x\right):{\mathbb{R}}^n\to \mathbb{R}$ has entries

Lemma 1 Let $\mathcal{L}\left(x\right)$ be the Hessian of the Lagrangian $L(x,\lambda )$ and let $x_0$ be an extremum. If $d^T\mathcal{L}\left(x_0\right)d\ge 0$ for all $d\in {\tilde{T}}_{\Omega }\left(x_0\right)$ , then $x_0$ is a minimizer. If $d^T\mathcal{L}\left(x_0\right)d\le 0$ for all $d\in {\tilde{T}}_{\Omega }\left(x_0\right)$ , then $x_0$ is a maximizer.

Example 1 Find the extremum of $f\left(x\right)={x_2}^2+x_2{x}_3+x_1{x}_3$ subject to $x_1+x_2+x_3=3$ and determine whether it is a maximum or a minimum.

To begin, we write the optimization's equality constraint:

The objective function can be written in the form

where the matrix $A$ has entries given by $A_{ij}=a_{ij}$ . Thus, for our example the resulting matrix is

The gradient for this function is given by

where $B=A+A^{*}$ . We can also rewrite the inequality constraint as $g\left(x\right)={\mathbf{1}}^{T}x-3$ , where $\mathbf{1}$ denotes a vector with entries equal to one of appropriate size. Therefore, its gradient is equal to $\nabla g\left(x\right)=\mathbf{1}.$ The resulting gradient of the Lagrangian is set to zero to obtain the solution:

Solve for $\lambda$ from ${\underline{1}}^{T}x=3$ to obtain $\lambda =-3/2$ . Therefore, the optimization's solution is

We can solve for the Hessians of $F\left(x\right)$ and $G\left(x\right)$ :

We therefore obtain that the Hessian of the Lagrangian is equal to

At this point we need to check if the product $h^T\mathcal{L}\left(x_0\right)h$ is positive or negative for all $h\in {\tilde{T}}_{\Omega }\left(x_0\right)$ , the tangent space defined as

It is easy to see that $h\in {\tilde{T}}_{\Omega }\left(x_0\right)$ if and only if $h_1+h_2+h_3=0$ . To begin, we check whether the eigenvalues of $\mathcal{L}\left(x_0\right)$ are all positive or negative: a calculation returns $\{-1.1701,0.6889,2.4812\}$ . Since neither case occurred, we have to specifically consider the case in which $h_1+h_2+h_3=0$ :

It turns out that we can find $h\in {\tilde{T}}_{\Omega }\left(x_0\right)$ for which the value on the left hand side may be positive or negative. Therefore, this is neither a maximum or a minimum, and we have found an inflection point.