# 1.4 Orthogonality

 Page 1 / 1
Definition of orthogonal vectors, sets, and subspaces; properties and benefits.

Recall that a set $S$ is a basis for a subspace $X$ if $\left[S\right]=X$ and $S$ has linearly independent elements. If $S$ is a basis for $X$ then each $x\in X$ is uniquely determined by $\left({a}_{1},{a}_{2},...,{a}_{n}\right)$ such that ${\sum }_{i=1}^{n}{a}_{i}{s}_{i}=x$ . In this sense, we could operate either with $x$ itself or with the vector $a=\left[{a}_{1},...{a}_{n}\right]={R}^{n}$ . One would wonder then whether particular operations can be performed with a representation $a$ instead of the original vector $x$ .

Example 1 Assume $x,y\in X$ have representations $a,b\in {R}^{n}$ in a basis for $X$ . Can we say that $〈x,,,y〉=〈a,,,b〉$ ?

For the particular example of $X={L}_{2}\left[0,1\right]$ , $S=\left\{1,t,{t}^{2}\right\}$ so that $\left[S\right]=Q$ , the set of all quadratic functions supported on $\left[0,1\right]$ . Pick $x=2+t+{t}^{2}$ and $y=1+2t+3{t}^{2}$ . One can see then that if we label ${s}_{1}=1$ , ${s}_{2}=t$ , ${s}_{3}={t}^{2}$ , then the coefficient vectors for $x$ and $y$ are $a=\left[2\phantom{\rule{3.33333pt}{0ex}}1\phantom{\rule{3.33333pt}{0ex}}1\right]$ and $b=\left[1\phantom{\rule{3.33333pt}{0ex}}2\phantom{\rule{3.33333pt}{0ex}}3\right]$ , respectively. Let us compute both inner products:

$\begin{array}{cc}\hfill 〈x,,,y〉& ={\int }_{0}^{1}x\left(t\right)y\left(t\right)dt={\int }_{0}^{1}\left(2+t+{t}^{2}\right)\left(1+2t+3{t}^{2}\right)dt=\frac{187}{20}\approx 9.35,\hfill \\ \hfill 〈a,,,b〉& =2+2+3=7.\hfill \end{array}$

Since $7\ne 9.35$ , we find that we fail to obtain the desired equivalence between vectors and their representations.

While this example was unsuccessful, simple conditions on the basis $S$ will yield this desired equivalence, plus many more useful properties.

Several definitions of orthogonality will be useful to us during the course.

Definition 1 A pair of vectors $x$ and $y$ in an inner product space are orthogonal (denoted $x\perp y$ ) if the inner product $〈x,,,y〉=0$ .

Note that 0 is immediately orthogonal to all vectors.

Definition 2 Let $X$ be an inner product space. A set of vectors $S\subseteq X$ is orthogonal if $〈x,,,y〉=0$ for all $x,y\in S,x\ne y$ .

Definition 3 Let $X$ be an inner product space. A set of vectors $S\subseteq X$ is orthonormal if $S$ is an orthogonal set and $\parallel s\parallel =\sqrt{⟨s,s⟩}=1$ for all $s\in S$ .

Definition 4 A vector $x$ in an inner product space $X$ is orthogonal to a set $S\subseteq X$ (denoted $x\perp S$ ) if $x\perp y$ for all $y\in S$ .

Definition 5 Let $X$ be an inner product space. Two sets ${S}_{1}\subseteq X$ and ${S}_{2}\subseteq X$ are orthogonal (denoted ${S}_{1}\perp {S}_{2}$ ) if $x\perp y$ for all $x\in {S}_{1}$ and $y\in {s}_{2}$ .

Definition 6 The orthogonal complement ${S}^{\perp }$ of a set $S$ is the set of all vectors that are orthogonal to $S$ .

## Benefits of orthogonality

Why is orthonormality good? For many reasons. One of them is the equivalence of inner products that we desired in a previous example. Another is that having an orthonormal basis allows us to easily find the coefficients ${a}_{1},...{a}_{n}$ of $x$ in a basis $S$ .

Example 2 Let $x\in X$ and $S$ be a basis for $X$ (i.e., $\left[S\right]=X$ ). We wish to find ${a}_{1},...{a}_{n}$ such that $x={\sum }_{i=1}^{n}{a}_{i}{s}_{i}$ . Consider the inner products

$\begin{array}{cc}\hfill 〈x,,,{s}_{i}〉& =〈\sum _{j=1}^{n},{a}_{j},{s}_{j},,,{s}_{i}〉=\sum _{j=1}^{n}{a}_{j}〈{s}_{j},,,{s}_{i}〉,\hfill \end{array}$

due to the linearity of the inner product in the first term. If $S$ is orthonormal, then we have that for $i\ne j$ $〈{s}_{j},,,{s}_{i}〉=0$ . In that case the sum above becomes

$\begin{array}{cc}\hfill 〈x,,,{s}_{i}〉& ={a}_{i}〈{s}_{i},,,{s}_{i}〉={a}_{i}{\parallel {s}_{i}\parallel }^{2}={a}_{i},\hfill \end{array}$

due to the orthonormality of $S$ . In other words, for an orthonormal basis $S$ one can find the basis coefficients as ${a}_{i}=⟨x,{s}_{i}⟩$ .

If $S$ is not orthonormal, then we can rewrite the sum above as the product of a row vector and a column vector as follows:

$〈x,,,{s}_{i}〉=\left[\begin{array}{cccc}〈{s}_{1},,,{s}_{i}〉& 〈{s}_{2},,,{s}_{i}〉& \cdots & 〈{s}_{n},,,{s}_{i}〉\end{array}\right]\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ ⋮\\ {a}_{n}\end{array}\right].$

We can then stack these equations for $i=1,...,n$ to obtain the following matrix-vector multiplication:

$\underset{\beta }{\underbrace{\left[\begin{array}{c}〈x,,,{s}_{1}〉\\ 〈x,,,{s}_{2}〉\\ ⋮\\ 〈x,,,{s}_{n}〉\end{array}\right]}}=\underset{G}{\underbrace{\left[\begin{array}{cccc}〈{s}_{1},,,{s}_{1}〉& 〈{s}_{2},,,{s}_{1}〉& \cdots & 〈{s}_{n},,,{s}_{1}〉\\ 〈{s}_{1},,,{s}_{2}〉& 〈{s}_{2},,,{s}_{3}〉& \cdots & 〈{s}_{n},,,{s}_{2}〉\\ ⋮& ⋮& \ddots & ⋮\\ 〈{s}_{1},,,{s}_{n}〉& 〈{s}_{2},,,{s}_{n}〉& \cdots & 〈{s}_{n},,,{s}_{n}〉\end{array}\right]}}\underset{a}{\underbrace{\left[\begin{array}{c}{a}_{1}\\ {a}_{2}\\ ⋮\\ {a}_{n}\end{array}\right]}}.$

The nomenclature given above provides us with the matrix equation $\beta =G·a$ , where $\beta$ and $G$ have entries ${\beta }_{i}=〈x,,,{s}_{i}〉$ and ${G}_{i,j}=〈{s}_{j},,,{s}_{i}〉$ , respectively.

Definition 7 The matrix $G$ above is called the Gram matrix (or Gramian) of the set $S$ .

In the particular case of orthonormal $S$ , it is easy to see that $G=I$ , the identity matrix, and so $a=\beta$ as given earlier. For invertible Gramians $G$ , one could compute the coefficients in vector form as $a={G}^{-1}\beta$ . For square matrices (like $G$ ), invertibility is linked to singularity.

Definition 8 A singular matrix is a non-invertible square matrix. A non-singular matrix is an invertible square matrix.

Theorem 1 A matrix is singular if $G·x=0$ for some $x\ne 0$ . A matrix is non-singular if $G·x=0$ only for $x=0$ .

The link between this notion of singularity and invertibility is straightforward: if $G$ is singular, then there is some $a\ne 0$ for which $G·a=0$ . Consider the mapping $y=G·x$ ; we would also have $y=G\left(x+a\right)$ . Since $x\ne x+a$ , one cannot “invert” the mapping provided by $G$ into $y$ .

Theorem 2 $S$ is linearly independent if and only if $G$ is non-singular (i.e. $Gx=0$ if and only if $x=0$ ).

Proof: We will prove an equivalent statement: $S$ is linearly dependent if and only if $G$ is singular, i.e., if and only if there exists a vector $x\ne 0$ such that $Gx=0$ .

$\left(⇒\right)$ We first prove that if $S$ is linearly dependent then $G$ is singular. In this case there exist a set $\left\{{a}_{i}\right\}\subseteq R$ , with at least one nonzero, such that ${\sum }_{i=1}^{n}{a}_{i}{s}_{i}=0$ . We then can write $〈{\sum }_{i=1}^{n},{a}_{i},{s}_{i},,,{s}_{j}〉=⟨0,{s}_{j}⟩=0$ for each ${s}_{j}$ . Linearity allows us to take the sum and the scalar outside the inner product:

$\sum _{i=1}^{n}{a}_{i}〈{s}_{i},,,{s}_{j}〉=0.$

We can rewrite this equation in terms of the entries of the Gram matrix as ${\sum }_{i=1}^{n}{a}_{i}{G}_{ji}=0$ . This sum, in turn, can be written as the vector inner product

$\left[\begin{array}{ccc}{G}_{j1}& \cdots & {G}_{1n}\end{array}\right]\left[\begin{array}{c}{a}_{1}\\ ⋮\\ {a}_{n}\end{array}\right]=0,$

which is true for every value of $j$ . We can therefore collect these equations into a matrix-vector product:

$\begin{array}{c}\hfill \left[\begin{array}{ccc}{G}_{11}& \cdots & {G}_{1n}\\ ⋮& \ddots & ⋮\\ {G}_{n1}& \cdots & {G}_{nn}\end{array}\right]\left[\begin{array}{c}{a}_{1}\\ ⋮\\ {a}_{n}\end{array}\right]=\left[\begin{array}{c}0\\ ⋮\\ 0\end{array}\right].\end{array}$

Therefore we have found a nonzero vector $a$ for which $Ga=0$ , and therefore $G$ is singular. Since all statements here are equalities, we can backtrack to prove the opposite direction of the theorem $\left(⇐\right)$ .

## Pythagorean theorem

There are still more nice proper ties for orthogonal sets of vectors. The next one has well-known geometric applications.

Theorem 3 (Pythagorean theorem) If $x$ and $y$ are orthogonal ( $x\perp y$ ), then ${\parallel x\parallel }^{2}+{\parallel y\parallel }^{2}={\parallel x+y\parallel }^{2}$ .

Proof:

${\parallel x+y\parallel }^{2}=⟨x+y,x+y⟩=⟨x,x⟩+⟨x,y⟩+⟨y,x⟩+⟨y,y⟩$

Because $x$ and $y$ are orthogonal, $⟨x,y⟩=⟨y,x⟩=0$ and we are left with $⟨x,x⟩={\parallel x\parallel }^{2}$ and $⟨y,y⟩={\parallel y\parallel }^{2}$ . Thus: ${\parallel x+y\parallel }^{2}={\parallel x\parallel }^{2}+{\parallel y\parallel }^{2}.$

explain and give four Example hyperbolic function
The denominator of a certain fraction is 9 more than the numerator. If 6 is added to both terms of the fraction, the value of the fraction becomes 2/3. Find the original fraction. 2. The sum of the least and greatest of 3 consecutive integers is 60. What are the valu
1. x + 6 2 -------------- = _ x + 9 + 6 3 x + 6 3 ----------- x -- (cross multiply) x + 15 2 3(x + 6) = 2(x + 15) 3x + 18 = 2x + 30 (-2x from both) x + 18 = 30 (-18 from both) x = 12 Test: 12 + 6 18 2 -------------- = --- = --- 12 + 9 + 6 27 3
Pawel
2. (x) + (x + 2) = 60 2x + 2 = 60 2x = 58 x = 29 29, 30, & 31
Pawel
ok
Ifeanyi
on number 2 question How did you got 2x +2
Ifeanyi
combine like terms. x + x + 2 is same as 2x + 2
Pawel
Mark and Don are planning to sell each of their marble collections at a garage sale. If Don has 1 more than 3 times the number of marbles Mark has, how many does each boy have to sell if the total number of marbles is 113?
Mark = x,. Don = 3x + 1 x + 3x + 1 = 113 4x = 112, x = 28 Mark = 28, Don = 85, 28 + 85 = 113
Pawel
how do I set up the problem?
what is a solution set?
Harshika
find the subring of gaussian integers?
Rofiqul
hello, I am happy to help!
Abdullahi
hi mam
Mark
find the value of 2x=32
divide by 2 on each side of the equal sign to solve for x
corri
X=16
Michael
Want to review on complex number 1.What are complex number 2.How to solve complex number problems.
Beyan
yes i wantt to review
Mark
use the y -intercept and slope to sketch the graph of the equation y=6x
how do we prove the quadratic formular
Darius
hello, if you have a question about Algebra 2. I may be able to help. I am an Algebra 2 Teacher
thank you help me with how to prove the quadratic equation
Seidu
may God blessed u for that. Please I want u to help me in sets.
Opoku
what is math number
4
Trista
x-2y+3z=-3 2x-y+z=7 -x+3y-z=6
can you teacch how to solve that🙏
Mark
Solve for the first variable in one of the equations, then substitute the result into the other equation. Point For: (6111,4111,−411)(6111,4111,-411) Equation Form: x=6111,y=4111,z=−411x=6111,y=4111,z=-411
Brenna
(61/11,41/11,−4/11)
Brenna
x=61/11 y=41/11 z=−4/11 x=61/11 y=41/11 z=-4/11
Brenna
Need help solving this problem (2/7)^-2
x+2y-z=7
Sidiki
what is the coefficient of -4×
-1
Shedrak
the operation * is x * y =x + y/ 1+(x × y) show if the operation is commutative if x × y is not equal to -1
A soccer field is a rectangle 130 meters wide and 110 meters long. The coach asks players to run from one corner to the other corner diagonally across. What is that distance, to the nearest tenths place.
Jeannette has $5 and$10 bills in her wallet. The number of fives is three more than six times the number of tens. Let t represent the number of tens. Write an expression for the number of fives.
What is the expressiin for seven less than four times the number of nickels
How do i figure this problem out.
how do you translate this in Algebraic Expressions
why surface tension is zero at critical temperature
Shanjida
I think if critical temperature denote high temperature then a liquid stats boils that time the water stats to evaporate so some moles of h2o to up and due to high temp the bonding break they have low density so it can be a reason
s.
Need to simplify the expresin. 3/7 (x+y)-1/7 (x-1)=
. After 3 months on a diet, Lisa had lost 12% of her original weight. She lost 21 pounds. What was Lisa's original weight?
why its coecients must have a power-law rate of decay with q > 1/p. ?