Section 6.

2: Orthogonal Sets
De…nition. A set of vectors f~u1 ; ~u2 ; :::; ~up g is said to be an orthogonal set if each vector
is orthogonal to others, i.e., ~ui ? ~uj for any i 6= j:
Example 6.2.1. Show that (a) in Rn ; the standard basis f~e1 ; ~e2 ; :::; ~en g is an orthogonal
set, and (b) the following set is an orthogonal set:
2 3 2 3 2 3
3 1 1
~u1 = 415 ; ~u2 = 4 2 5 ; ~u3 = 4 45 :
1 1 7

Solution. (a) ~ei ~ej = 0 if i 6= j: (b) we check by direct calculations:

2 3 2 3
3 1
~u1 ~u2 = 14 5 4 2 5 = 0;
1 1
2 3 2 3
3 1
~u1 ~u3 = 14 5 4 45 = 0
1 7
2 3 2 3
1 1
~u2 ~u3 = 4 2 5 4 45 = 0:
1 7
Theorem. Any orthogonal set is linearly independent.
Proof. Suppose f~u1 ; ~u2 ; :::; ~up g is an orthogonal set, and suppose

c1~u1 + c2~u2 + ::: + cp~up = ~0:

We dot-multiplying ~ui on both sides of the equation and obtain

ci~ui ~ui = 0 =) ci = 0:

De…nition. A basis of a subspace is said to be an orthogonal basis if it is an orthogonal

set.
Theorem. Let B= f~u1 ; ~u2 ; :::; ~up g be an orthogonal basis for a subspace W: Then, for
each w
~ 2 W; its coordinate [w]
~ B relative to this orthogonal basis can be expressed as
2 3
c1
6 c2 7 w~ ~ui
6 7
~ B = 6 .. 7 ; ci =
[w] ; i = 1; 2; :::; p: (1)
4.5 k~ui k2
cp

In other words,
w
~ = c1~u1 + c2~u2 + ::: + cp~up ; (2)

1
Proof. Consider expression (2). All we need to do is to derive formula for ci in (1). To this
end, we dot-multiply (2) by ~ui :

w
~ ~ui = (c1~u1 + c2~u2 + ::: + cp~up ) ~ui = ci~ui ~ui

since ~uj ~ui = 0 unless j = i: It follows that

w
~ ~ui
w
~ ~ui = ci~ui ~ui =) ci = :
k~ui k2

Example 6.2.2. We know from Example 1 that

2 3 2 3 2 3
3 1 1
4 5 4 5
~u1 = 1 ; ~u2 = 2 ; ~u3 = 4 45
1 1 7

form an orthogonal basis for R3 : Find the coordinate of

2 3
6
~ = 1 5 relative to this basis.
w 4
8

Solution. Note that if the basis were not orthogonal, then we have to proceed as follows:
solving linear system:
c1~u1 + c2 ~u2 + c3~u3 = w
~
or 2 32 3 2 3
3 1 1 c1 6
41 2 4 c2 = 1 5 :
5 4 5 4
1 1 7 c3 8
Since the basis is indeed orthogonal, we use formula (1):
2 3 2 3 2 3 2
3 6 3
w
~ ~u1 4 5 4 5 415
c1 = = 1 1 =1
k~u1 k2 1 8 1
2 3 2 3 2 3 2
1 6 1
w
~ ~u2 4 5 415 425
c2 = = 2 = 2
k~u2 k2 1 8 1
2 3 2 3 2 3 2
1 6 1
w
~ ~u3 4 5 415 4 45
c3 = = 4 = 1
k~u2 k3 7 8 7

2
 2 3
1
[w]
~ f~u1 ;~u2 ;~u3 g = 4 25 :
1
Orthogonal Projections.
Given a vector ~u; the orthogonal projection of ~y onto ~u; denoted by y^ =Proj~u (~y ) ; is
de…ned as the vector parallel to ~u such that

~y = y^ + ~z; ~z ? ~u; y^==~u:

Since y^ is parallel to ~u; we have y^ = ~u: Hence

~y = ~u + ~z; ~z ? ~u:

Dot-multiplying by ~u; we …nd

~y ~u
~y ~u = ( ~u + ~z) ~u = ~u ~u =) = :
~u ~u
~y ~u
Proj~u (~y ) = ~u = ~u (3)
~u ~u
In general, for any given subspace W; y^ =ProjW (~y ) is de…ned as the vector in W such
that
(~y y^) ? W:
In other words, any vector ~y can be decomposed into two components: one is the projection
y^ on W (which is in W ) and another component perpendicular to W: Suppose that W has
an orthogonal basis B= f~u1 ; ~u2 ; :::; ~up g : Then we may write, since y^ 2 W;

y^ = P rojW (~y ) = c1~u1 + c2~u2 + ::: + cp~up

~y = y^ + ~z = c1~u1 + c2~u2 + ::: + cp~up + ~z; ~z ? W:

By dot-multiplying by ~ui ; we …nd

~y ~ui
~y ~ui = (c1~u1 + c2~u2 + ::: + cp~up + ~z) ~ui = ci~ui ~ui =) ci = :
~ui ~ui
Therefore,
~y ~ui
y^ = P rojW (~y ) = c1~u1 + c2~u2 + ::: + cp~up ; ci = : (4)
~ui ~ui
Example 6.2.3. Let
2 3 2 3 2 3
3 1 2
~u1 = 1 ; ~u2 = 2 ; ~y = 3 5
4 5 4 5 4
1 1 1

3
From Example 1 above, we know that ~u1 and ~u2 form an orthogonal basis for W = Span f~u1 ; ~u2 g :
Find (a) Proj~u1 (~y ) ; (b) Proj~u2 (~y ) , (c) ProjW (~y ) :
Solution. (1) By (3),
2 3 2 3
3 3
~y ~u1 6+3 14 5 8 4 5
P roj~u1 (~y ) = ~u1 = 1 = 1
~u1 ~u1 11 11
1 1

(2) Analogously,
2 3 2 3
1 1
~y ~u2 2+6 14 5 3
P roj~u2 (~y ) = ~u2 = p 2 =p 425
~u2 ~u2 6 6 1
1

(3) Using (4) and answer from part (1) & (2)
2 3 2 3
3 1
~y ~u1 ~y ~u2 8 4 5 3
P rojW (~y ) = ~u1 + ~u2 = P roj~u1 (~y ) + P roj~u2 (~y ) = 1 +p 425
~u1 ~u1 ~u2 ~u2 11 6 1
1

We conclude from this example that, in general, suppose that W has an orthogonal
basis B= f~u1 ; ~u2 ; :::; ~up g : Then

P rojW (~y ) = P roj~u1 (~y ) + P roj~u2 (~y ) + ::: + P roj~up (~y ) :

De…nition. A set B= f~u1 ; ~u2 ; :::; ~up g is said to be an orthonormal set if it is an orthogonal
set and if each vector is a unit vector, i.e.,
1 if i = j
~ui ~uj = ij =
0 if i =
6 j:
Example 6.2.4. Show that
2 3 2 3 2
3
3 1 1
1 1 1
~u1 = p 415 ; ~u2 = p 4 2 5 ; ~u3 = p 4 45
11 1 6 1 66 7

form an orthonormal set.

Solution. Direct computation show

~u1 ~u1 = ~u2 ~u2 = ~u3 ~u3 = 1

~u1 ~u2 = ~u2 ~u3 = ~u1 ~u3 = 0:

Theorem. Let U = [~u1 ; ~u2 ; :::; ~un ] be a n n matrix with columns ~u1 ; ~u2 ; :::; ~un . Suppose
that the columns of U form an orthonormal set. Then
1
U = U T ; i:e:; U U T = U T U = I:

4
We call it orthonormal matrix.
Proof. We observe that U may be written as
2 3 2 3
u11 u12 ::: u1n u1j
6 u21 u22 ::: u2n 7 6 u2j 7
U =6 7 6 7
4 ::: ::: ::: ::: 5 = [~u1 ; ~u2 ; :::; ~un ] ; ~uj = 4 ::: 5
un1 un2 ::: unn unj

and since (~u1 )T = [u11 ; u21 ; :::; un1 ] is a row-vector,

2 3 2 3
u11 u21 ::: un1 (~u1 )T
6 u12 u22 ::: un2 7 6 T7
UT = 6 7 = 6 (~u2 ) 7 :
4 ::: ::: ::: ::: 5 4 ::: 5
u1n u2n ::: unn (~un )T

Since, by orthonormality, (~u1 )T ~u1 = ~u1 ~u1 = 1; (~u2 )T ~u1 = ~u2 ~u1 = 0; ... we have
2 3 2 3 2 3
(~u1 )T (~u1 )T ~u1 (~u1 )T ~u2 ::: (~u1 )T ~un 1 0 ::: 0
6 (~u2 )T 7 6 T T T 7
::: (~u2 ) ~un 7 6 06 1 ::: 07
UT U = 6 7 [~u1 ; ~u2 ; :::; ~un ] = 6 (~u2 ) ~u1 (~u2 ) ~u2 = 7
4 ::: 5 4 ::: ::: ::: ::: 5 4::: ::: ::: :::5
T
(~un ) (~un ) ~u1 (~un )T ~u2
T
::: (~un )T ~un 0 0 ::: 1

Note that the same technique may be used to calculate the inverse of a matrix A =
[u1 ; u2 ; :::; un ]; where the column vectors ~u1 ; ~u2 ; :::; ~un form an orthogonal set, but not ortho-
normal set. In this case,
2 3 2 3
(~u1 )T ~u1 (~u1 )T ~u2 ::: (~u1 )T ~un ~u1 ~u1 0 ::: 0
6 (~u2 )T ~u1 (~u2 )T ~u2 ::: (~u2 )T ~un 7 6 0 ~u2 ~u2 ::: 0 7
A A=6
T
4 :::
7=6 7:
::: ::: ::: 5 4 ::: ::: ::: ::: 5
(~un )T ~u1 (~un )T ~u2 ::: (~un )T ~un 0 0 ::: ~un ~un

So 2 3 1
~u1 ~u1 0 ::: 0
6 0 ~u2 ~u2 ::: 0 7
6 7 AT A = I;
4 ::: ::: ::: ::: 5
0 0 ::: ~un ~un

5
i.e.,
2 3 1
~u1 ~u1 0 ::: 0
6 0 ~
u 2 ~ u2 ::: 0 7
A 1
=6
4 :::
7 AT
::: ::: ::: 5
0 0 ::: ~un ~un
2 3 2 3
1 (~u1 )T
6 ~u1 ~u1 0 ::: 0 72 T3 6 ~u ~u 7
6 7 (~u1 ) 6 1 T1 7
6 1 76 6 (~u2 ) 7
6 0 ::: 0 7 6 (~u2 ) 7 6
T7
6 ~u ~u 7 :
7
=6 ~u2 ~u2 74 =
6 ::: ::: ::: ::: 7 ::: 5 6 2
6 ::: 7
27
6 7 T 6 7
4 1 5 (~un ) 4 (~un )T 5
0 0 :::
~un ~un ~u ~u
n n

Example 6.2.5. We know from previous examples that

2 3 2 3 2 3
3 1 1
1 1 1
~u1 = p 415 ; ~u2 = p 4 2 5 ; ~u3 = p 4 45
11 1 6 1 66 7

form an orthonormal basis, but

2 3 2 3 2 3
3 1 1
~v1 = 1 ; ~v2 = 2 5 ;
4 5 4 ~v3 = 4 45
1 1 7
form only an orthogonal basis. Set
2 3
3 1 1
p p p
6 11 6 66 7
6 1 2 47
6p p p 7
U = [~u1 ; ~u2 ; ~u3 ] = 6 7
6 11 6 66 7
4 1 1 7 5
p p p
11 6 66
2 3
3 1 1
4
V = [~v1 ; ~v2 ; ~v3 ] = 1 2 45 :
1 1 7
The …rst matrix U is an orthogonal matrix, and
2 3
3 1 1
p p p
6 11 11 11 7
6 1 2 1 7
6 p 7
U 1 = UT = 6 p p 7:
6 6 6 67
4 1 4 7 5
p p p
66 66 66

6
The matrix is not an orthonormal matrix. However,
2 32 3 2 3
3 1 1 3 1 1 11 0 0
V T V = 4 1 2 1 5 41 2 45 = 4 0 6 0 5 :
1 4 7 1 1 7 0 0 66

Therefore, 2 3 1
11 0 0
40 6 05 V T V = I;
0 0 66
or
2 3 1
11 0 0
V 1
=40 6 05 VT
0 0 66
2 32 3
11 1 0 0 3 1 1
=4 0 6 1 0 54 1 2 15
0 0 66 1 1 4 7
2 3
3 1 1
6 11 11 11 7
6 1 1 17
=6
6 6
7:
4 1 3 67
2 75
66 33 66