Gockenbach M.S. Partial Differential Equations. Analytical and Numerical Methods
Подождите немного. Документ загружается.
48
Chapter
3.
Essential
linear
algebra
4.
For
each
of the
following
matrices
A,
determine
if Ax = b has a
unique
solution
for
each
b,
that
is,
determine
if A is
nonsingular.
For
each matrix
A
which
is
singular,
find a
vector
b
such
that
Ax = b has a
solution
and a
vector
c
such
that
Ax
—
c
does
not
have
a
solution.
5.
Suppose
A
€
R
nxn
and b €
R
n
,
b
^
0. Is the
solution
set of the
equation
Ax
=
b,
The
reader should note
that
C^[a,
b]
is a
subspace
of
C
2
[a,
b].
(a)
Determine
the
null space
of
L
m
.
a
subspace
of
R
n
?
Why or why
not?
6. Let D
:
C
l
[a,b]
—»•
C[a,
b]
be the
derivative operator. What
is the
null space
of
Dl
7. Let
L
:
C
2
[a,
b]
—»
C[a,
b]
be the
second derivative operator. What
is the
null
space
of L?
8. As
shown
in
Chapter
2,
differential equations involving
the
second derivative
operator
are
sometimes paired with mixed boundary conditions.
Define
the
set
48
[
1
-1]
(a)
A =
-2
4
(b)
A = [!
~]
(c)
A =
[~
~]
Chapter
3.
Essential linear algebra
4.
For each of
the
following matrices
A,
determine if
Ax
= b has a unique
solution for each b,
that
is, determine
if
A
is
nonsingular. For each matrix
A which
is
singular, find a vector b such
that
Ax
= b has a solution and a
vector
c such
that
Ax
= c does not have a solution.
(e)
A
~
[
1
2
-11
2
-1
1
4
-2
-2
1
5.
Suppose A E Rnxn and
bERn,
b
'"
O.
Is the solution set of
the
equation
Ax=b,
{x
ERn
:
Ax
=
b}
,
a subspace of R
n
? Why or why not?
6.
Let D : CI[a,
b]
-+
C[a,
b]
be the derivative operator.
What
is
the
null space
of
D?
7.
Let L : C
2
[a,
b]
-+
C[a,
b]
be
the
second derivative operator.
What
is
the null
space of
L?
8.
As
shown in Chapter
2,
differential equations involving the second derivative
operator are sometimes paired with mixed boundary conditions. Define the
set
C![a,b] = {u E C
2
[a,b]
:
u(a)
=
~~(b)
=
O}
and define
Lm
: C![a, b]-+
C[a,
b]
by
The
reader should note
that
C![a,
b]
is a subspace of C
2
[a,
b].
(a) Determine
the
null space of
Lm.
3.2.
Existence
and
uniqueness
of
solutions
to Ax = b 49
(b)
Explain
how to
solve
L
m
u
= f for
w,
when
/ is a
given
function
in
C[a,b].
(Hint: Integrate twice.)
Is
this
differential
equation solvable
for
every
/ £
C[a,
&]?
9.
Repeat Exercise
8, but
with
10.
Consider
the
following
system
of
nonlinear equations:
This system
can be
written
as f (x) = b,
where
f : R
2
—>
R
2
is
denned
by
and
(a)
Show
that
f
(x)
= b has
exactly
two
solutions. (Hint:
A
graph
is
useful.)
(b)
Show
that
the
only solution
to
f(x)
= 0 is x = 0.
(Yet,
as
Part
lOa
shows,
the
solution
to f
(x)
= b is not
unique.)
This
example illustrates
that
the
properties
of
linear systems
do not
necessar-
ily
carry over
to
nonlinear systems.
11. Let D
:
C
l
[a,
b]
->
C[a,b]
be the
differentiation
operator:
(a)
Show
that
the
range
of D is all of
C[a,
b].
(b)
Part
lla
is
equivalent
to
saying
that,
for
every
/
e
C[a,
b],
the
(differen-
tial) equation
Du = f has a
solution.
Is
this solution unique?
Why or
why
not?
12.
Let LN be
defined
as in
Example 3.14,
and
suppose
/
e
C[Q,l]
satisfies
Show
that
/
e
n(L
n
}.
3.2. Existence
and
uniqueness
of
solutions
to
Ax
= b
49
(b) Explain how to solve
Lmu
= f for u, when f
is
a given function in
C[a,
b].
(Hint: Integrate twice.) Is this differential equation solvable for
every
f E
C[a,
b]?
9.
Repeat Exercise
8,
but
with
C~[a,b]
= {u E C
2
[a,b]
~~(a)
=
u(b)
=
o}
and
Lm
:
C~.[a,
b]
-+
C[a,
b]
defined by
cPu
Lm
u
= -
dX2·
10.
Consider the following system of nonlinear equations:
xi
+
x~
= 1,
X2
-
xi
=
o.
This system can be written as f(x) =
b,
where
f:
R2
-+
R2
is
defined by
and
[
2 2
f(x) =
Xl
+
X~
X2
-
Xl
b=[~].
(a) Show
that
f(x) = b has exactly two solutions. (Hint: A graph
is
useful.)
(b) Show
that
the only solution
to
f(x) = 0
is
x =
o.
(Yet, as
Part
lOa
shows,
the
solution to f(x) = b
is
not unique.)
This example illustrates
that
the properties of linear systems do not necessar-
ily
carryover
to nonlinear systems.
11.
Let D : C
I
[a,
b]
-+
C[a,
b]
be the differentiation operator:
Df
=
df.
dx
(a) Show
that
the
range of D
is
all of
C[a,
b].
(b)
Part
11a
is
equivalent
to
saying
that,
for every f E
C[a,
b],
the (differen-
tial) equation
Du
= f has a solution. Is this solution unique? Why or
why not?
12.
Let
LN
be defined as in Example 3.14, and suppose f E
C[O,
l] satisfies
1£
f(x)
dx
=
o.
Show
that
f E
R(Ln).
50
Chapter
3.
Essential linear algebra
The
reader should write
out a
specific
example
if
this
is not
clear (see Exercise
1). The
quantities
£i,#2,
• • •
i
x
n
are
scalars
and
YI,
V2,....,
v
n
are
vectors;
an ex-
pression such
as
XiVi
+
#2V2
+
cldots
+
x
n
v
n
is
called
a
linear combination
of the
vectors
YI
,
v
2
,...,
v
n
because
the
vectors
are
combined using
the
linear operations
of
addition
and
scalar multiplication.
When
A £
R
nxn
is
nonsingular, each
b £
R
n
can be
written
in a
unique
way
as
a
linear combination
of the
columns
of A
(that
is, the
equation
Ax = b has a
unique
solution).
The
following
definition
is
related.
Definition
3.23.
Let V be a
vector
space,
and
suppose
vi,V2,..
.,v
n
are
vectors
in V
with
the
property
that
each
v 6 V can be
written
in a
unique
way as a
lin-
ear
combination
of
(YI,
v
2
,...,
v
n
}.
Then
{vi,
V2,...,
v
n
)
is
called
a
basis
of
V.
Moreover,
we say
that
n is the
dimension
of
V.
A
vector space
can
have many
different
bases,
but it can be
shown
that
each
contains
the
same number
of
vectors,
so the
concept
of
dimension
is
well-defined.
We
now
present several examples
of
bases.
Example
3.24.
The
standard
basis
for
R
n
is
(ei,e2,...
,e
n
}
;
where
every
entry
of
GJ
is
zero
except
the
jth, which
is
one. Then
we
obviously
have,
for any x 6
R
n
,
Example
3.25.
An
alternate
basis
for R
3
is
{vi,V2,V3J
;
where
It may not be
obvious
to the
reader
why one
would
want
to use
this
basis
instead
of
the
much simpler
basis
{61,62,63}.
However,
it is
easy
to
check
that
3.3
Basis
and
dimension
The
matrix-vector
product
Ax is
equivalent
to a
linear combination
of the
columns
of
the
matrix
A. If A has
columns
YI,
V2,...,
v
n
,
then
and
it is not
hard
to see
that this representation
is
unique.
For
example,
for x £
R
3
;
50
Chapter
3.
Essential
linear
algebra
3.3
Basis and dimension
The
matrix-vector product
Ax
is
equivalent
to
a linear combination of
the
columns
of
the
matrix
A.
If
A has columns VI,
V2,
...
, V
n
,
then
The
reader should write
out
a specific example if this
is
not clear (see Exercise
1).
The
quantities
XI,X2,
•••
,Xn
are scalars
and
VI,V2,
...
,Vn are vectors;
an
ex-
pression such as
Xl
Vl
+ X2
V2
+ cldots +
Xn
V n is called a linear combination
of
the
vectors Vl,
V2,
...
,
Vn
because
the
vectors are combined using
the
linear operations
of
addition
and
scalar multiplication.
When A E
RDXD
is nonsingular, each b E
RD
can be
written
in a unique way
as a linear combination
of
the
columns of A
(that
is,
the
equation
Ax
= b has a
unique solution).
The
following definition
is
related.
Definition
3.23.
Let V
be
a vector space, and suppose Vl,
V2,
...
,
Vn
are
vectors
in
V with the property that each V E V can
be
written
in
a unique way
as
a lin-
ear combination
of
{Vl,
V2,···,
v
n
}.
Then {Vb
V2,
...
, v
n
}
is called a basis
of
V.
Moreover,
we
say that n is the dimension
of
V.
A vector space can have many different bases,
but
it
can be shown
that
each
contains
the
same number
of
vectors, so
the
concept of dimension is well-defined.
We
now present several examples of bases.
Example
3.24.
The standard basis for
RD
is
{el,e2,
...
,e
n
}, where every entry
of
ej
is zero except the
jth,
which is one. Then
we
obviously have, for any x E
RD,
and
it
is
not
hard to see that this representation is unique. For example, for x E R
3
,
Example
3.25.
An
alternate basis for R3 is {Vb
V2,
V3},
where
It
may
not
be
obvious to the reader why one would want to use this basis instead
of
the much simpler basis
{el,
e2, e3} . However,
it
is easy to check that
Vl
.
V2
= 0,
Vl
.
V3
= 0, V2·
V3
= 0,
3.3. Basis
and
dimension
51
and
this property makes
the
basis
{vi,V2,vs}
almost
as
easy
to use as
{61,62,63}.
We
explore
this topic
in the
next section.
Example
3.26.
The set
P
n
is the
vector
space
of all
polynomials
of
degree
n or
less
(see Exercise 3.1.3).
The
standard
basis
is
{l,x,x
2
,...
,x
n
}.
To see
that this
is
indeed
a
basis,
we first
note that
every
polynomial
p €
P
n
can be
written
as a
linear
combination
of
1,
x,
x
2
,...,
x
n
:
(this
is
just
the
definition
of
polynomial
of
degree
n).
Showing that this representa-
tion
is
unique
is a
little subtle.
If we
also
had
then,
subtraction
would
yield
for
every
x.
However,
a
nonzero polynomial
of
degree
n can
have
at
most
n
roots,
so it
must
be the
case
that
(CQ
— do)
4-
(GI
—
di)x
+ • • • +
(c
n
—
d
n
)x
n
is the
zero
polynomial.
That
is,
CQ
—
do,
ci
=
di,...,
c
n
=
d
n
must hold.
Example
3.27.
An
alternate
basis
for
P^
is
(the
advantage
of
this
basis
will
be
discussed
in
Example
3.39
in the
next section).
To
show
that this
is
indeed
a
basis,
we
must
show that, given
any
p(x]
—
CQ+cix+C2X
2
,
there
is a
unique
choice
of the
scalars
ao,ai,fl2
such that
This
equation
is
equivalent
to the
three linear equations
The
reader
can
easily
verify
that this system
has a
unique solution,
regardless
of
the
values
0/co,ci,C2.
Example
3.28.
Yet
another
basis
forP?
is
{1/1,1/2,1^3},
where
3.3.
Basis
and
dimension
51
and
this property makes the basis {VI,
V2,
V3}
almost
as
easy to use
as
{el,
e2, e3}.
We explore this topic
in
the
next
section.
Example
3.26.
The set P
n
is the vector space
of
all polynomials
of
degree n or
less (see Exercise
3.1.3). The standard basis is
{1,x,x
2
,
...
,x
n
}.
To
see that this
is indeed a basis, we first note that every polynomial
pEP
n can
be
written as a
linear combination
of
1,
X,
x
2
,
...
,
xn:
p(x)
=
co'
1 +
CIX
+
C2X
2
+ ... +
cnx
n
(this is
just
the definition
of
polynomial
of
degree
n).
Showing that this representa-
tion is unique is a little subtle.
If we also had
p(x)
=
do'
1 +
dlx
+ d
2
x2 + ... +
dnxn,
then, subtraction would yield
(eo
-
do)
+
(Cl
-
ddx
+ ... +
(c
n
-
dn)x
n
= 0
for every
x.
However, a nonzero polynomial
of
degree n can have
at
most
n roots,
so
it
must
be
the case that
(eo
-
do)
+
(CI
-
ddx
+ ... +
(c
n
-
dn)x
n
is the zero
polynomial. That is,
Co
=
do,
Cl
= d
1
,
...
, C
n
= d
n
must
hold.
Example
3.27.
An
alternate basis for P
2
is
{
I
x-~
X2_X+~}
,
2'
6
(the advantage
of
this basis will
be
discussed in Example 3.39
in
the
next
section).
To
show that this is indeed a basis, we
must
show that, given any
p(x)
=
Co
+CIX+C2X2,
there is a unique choice
of
the scalars ao,
aI,
a2
such that
ao
. 1 +
al
(x
-
~)
+
a2
(X2 - X +
~)
=
Co
+
CIX
+
C2X2.
This equation is equivalent to the three linear equations
1 1
ao
-
'2a1
+
6'a2
=
Co,
al
-
a2
=
CI,
a2
=
C2·
The reader can easily verify that this
system
has a unique solution, regardless
of
the
values
of
eo,Cl,C2·
Example
3.28.
Yet another basis for P
2
is
{L
1
,
L
2
,
L
3
},
where
LI(X)
= 2
(x
-~)
(x
-1),
L2(X) =
-4x(x
- 1),
L3
(x) =
2x
(x
-
~)
.
52
Chapter
3.
Essential
linear
algebra
// we
write
x\
=
Q,
#
2
=
1/2,
and
x%
=
I,
then
the
property
holds.
From this
property,
the
properties
of a
basis
can be
verified
(see Exercise
5).
There
are two
essential properties
of a
basis
{vi,
v
2
,...,v
n
}
of a
vector space
V.
First,
every vector
in V can be
represented
as a
linear combination
of the
basis
vectors. Second, this representation
is
unique.
The
following
two
definitions provide
concise
ways
to
express these
two
properties.
Definition
3.29.
Let V be a
vector
space,
and
suppose
{YI,
v
2
,...,
v
n
}
is a
col-
lection
of
vectors
in
V.
The
span
of
{YI,
v
2
,...,
v
n
}
is the set of all
linear combi-
nations
of
these vectors:
Thus,
one of the
properties
of a
basis
{YI,
v
2
,...,
v
n
}
of a
vector space
V is
that
V
=
span{vi,v
2
,...,v
n
}.
The
reader should also
be
aware
that,
for any
vectors
vi,V2,...,v
n
in a
vector
space
V,
span{vi,
V2,...,
v
n
}
is a
subspace
of V
(possibly
the
entire space
V,
as in
the
case
of a
basis).
Definition
3.30.
A set
of
vectors
{YI,
v
2
,...,
v
n
}
is
called
linearly independent
if
the
only
scalars
c\,
c
2
,...,
c
n
satisfying
It can be
shown
that
the
uniqueness
part
of the
definition
of a
basis
is
equiva-
lent
to the
linear independence
of the
basis vectors. Therefore,
a
basis
for a
vector
space
is a
linearly independent spanning set.
A
third quality
of a
basis
is the
number
of
vectors
in
it—the
dimension
of the
vector space.
It can be
shown
that
any two of
these properties imply
the
third.
That
is, if V has
dimension
n,
then
any two of the
following
statements about
{vi,
v
2
,...,
v
fc
}
imply
the
third:
spon{vi,v
2
,...,v
n
}
=
{o!iVi+Q!2V2
+
h
a
n
v
n
:
c*i,
a
2
,
• •
•,
a
n
e
R}
52
Chapter
3. Essential linear algebra
If
we write
Xl
=
0,
X2 =
1/2,
and
X3
=
1,
then
the property
L (
)
{
1, i =
j,
i
Xj
= 0, i
¥-
j
(3.9)
holds. From this property, the properties
of
a basis can
be
verified (see Exercise 5).
There are two essential properties of a basis
{v
1 , V
2,
...
, V n} of a vector space
V. First, every vector in V can
be
represented as a linear combination of the basis
vectors. Second, this representation
is
unique.
The
following two definitions provide
concise ways
to
express these two properties.
Definition
3.29.
Let
V
be
a vector space,
and
suppose
{Vl,
V2,
...
, V n}
is
a col-
lection
of
vectors
in
V.
The span
of
{Vl,
V2,
...
, v
n
} is the
set
of
all linear combi-
nations
of
these vectors:
Thus, one of the properties of a basis
{v!,
V2,
...
, V n} of a vector space V
is
that
The
reader should also be aware
that,
for any vectors
Vl,
V2,
...
,
Vn
in a vector
space
V, span
{Vl'
V2,
...
, V n}
is
a subspace of V (possibly the entire space V, as in
the
case of a basis).
Definition
3.30.
A
set
of
vectors
{Vl'
V2,
...
, V n} is called linearly independent
if
the
only scalars
Cl,
C2,
•••
,C
n
satisfying
are
Cl
= C2 = ... = C
n
=
o.
It
can be shown
that
the
uniqueness
part
of
the
definition of a basis is equiva-
lent
to
the
linear independence of
the
basis vectors. Therefore, a basis for a vector
space is a linearly independent spanning set.
A
third
quality of a basis
is
the number of vectors in
it-the
dimension
of
the
vector space.
It
can
be
shown
that
any two of these properties imply the third.
That
is, if V has dimension
n,
then
any two of
the
following statements
about
{Vl,
V2,
...
,
vd
imply the third:
• k
=n;
•
{Vl,
V2,
...
,
vd
is
linearly independent;
9
If
the
importance
of
this question
is not
apparent
to the
reader
at
this point,
it
will
be
after
he
or she
reads
the
next
two
sections.
10
This
notation means that
A is the
matrix whose columns
are the
vectors
vi,
V2,...,
v
n
.
3.3. Basis
and
dimension
53
Thus
if
{vi,
v
2
,...,
Vfc}
is
known
to
satisfy
two of the
above properties, then
it is a
basis
for
V.
Before
leaving
the
topic
of
basis,
we
wish
to
remind
the
reader
of the
fact
indicated
in the
opening paragraphs
of
this section, which
is so
fundamental
that
we
express
it
formally
as a
theorem.
Theorem
3.31.
Let A be an
n
x
n
matrix. Then
A is
nonsingular
if and
only
if
the
columns
of
A
form
a
basis
for
R
n
.
Thus, when
A is
nonsingular,
its
columns
form
a
basis
for
R
n
,
and
solving
Ax
=
b is
equivalent
to finding the
weights
that
express
b as a
linear combination
of
this
basis.
This fact answers
the following
important
question.
9
Suppose
we
have
a
basis
YI,
v
2
,...,
v
n
for
R
n
and a
vector
b G
R
n
.
Then,
of
course,
b is a
linear combination
of the
basis vectors.
How do we find the
weights
in
this linear
combination?
How
expensive
is it to do so
(that
is, how
much work
is
required)?
To find the
scalars
#1,
#2,
• •
•,
x
n
in the
equation
we
define
10
and
solve
via
Gaussian elimination.
The
expense
of
computing
x can be
measured
by
count-
ing
the
number
of
arithmetic
operations—the
number
of
additions, subtractions,
multiplications,
and
divisions—required.
The
total
number
of
operations required
to
solve
Ax = b is a
polynomial
in n, and it is
convenient
to
report just
the
leading
term
in the
polynomial, which
can be
shown
to be
(the lower-order terms
are not
very significant when
n is
large).
We
usually express
this
saying
that
the
operation count
is
("on
the
order
of
(2/3)n
3
").
In the
next section,
we
discuss
a
certain special type
of
basis
for
which
it is
much
easier
to
express
a
vector
in
terms
of the
basis.
3.3.
Basis
and
dimension
53
Thus if
{Vl'
V2,
.•.
,
vd
is
known to satisfy two of the above properties, then it
is
a
basis for
V.
Before leaving
the
topic of basis,
we
wish to remind
the
reader of the fact
indicated in the opening paragraphs of this section, which is so fundamental
that
we
express it formally as a theorem.
Theorem
3.31.
Let
A
be
an n x n matrix. Then A is nonsingular
if
and only
if
the columns
of
A form a basis for R n .
Thus, when A
is
nonsingular, its columns form a basis for Rn, and solving
Ax
= b is equivalent to finding the weights that express b
as
a linear combination
of
this basis. This fact answers the following important question.
9
Suppose
we
have a basis
Vl,
V2,
...
,
Vn
for Rn and a vector
bERn.
Then, of course, b
is
a
linear combination of the basis vectors.
How
do
we
find the weights in this linear
combination?
How
expensive
is
it
to
do so
(that
is, how much work
is
required)?
To
find the scalars
Xl,
X2,
..•
,X
n
in the equation
we
define
lO
and
solve
Ax=b
via Gaussian elimination. The expense of computing x can be measured by count-
ing the number of arithmetic
operations-the
number of additions, subtractions,
multiplications,
and
divisions-required. The total number of operations required
to
solve
Ax
= b
is
a polynomial in n,
and
it
is
convenient to report just
the
leading
term in
the
polynomial, which can be shown to be
2 3
-n
3
(the lower-order terms are not very significant when n
is
large).
We
usually express
this saying
that
the operation count
is
("on the order of
(2/3)n
3
").
In the next section,
we
discuss a certain special type of basis for which it
is
much easier to express a vector in terms of the basis.
91f
the
importance of this question is not apparent
to
the
reader
at
this
point,
it
will be after
he
or
she reads
the
next
two sections.
lOThis
notation
means
that
A is
the
matrix
whose columns are
the
vectors
VI,
V2,
..•
,
Vn.
and
verify
that
they
are
equal,
(b)
Let A 6
R
nxn
and x
e
R
n
,
and
suppose
the
columns
of A are
2.
Is
a
basis
for R
3
?
(Hint:
As
explained
in the
last
paragraphs
of
this section,
the
three given vectors
form
a
basis
for
R
n
if and
only
if Ax = b has a
unique
solution
for
every
b 6
R
n
,
where
A is the
3x3
matrix whose columns
are the
three
given vectors.)
3. Is
is
a
basis
for
P%,
the
space
of
polynomials
of
degree
2 or
less. (Hint:
Verify
directly
that
the
definition
holds.)
5.
Show
that
{L
1?
L
2
,
L
3
},
defined
in
Example 3.28,
is a
basis
for
P
2
-
(Hint:
Use
(3.9)
to
show
that
54
Chapter
3.
Essential linear
algebra
Exercises
so
that
the
(i,
j)-entry
of A is
(vj)j.
Compute both
(Ax)j
and
(#iVi
+
x
2
V2
+
1-
x
n
v
n
)i,
and
verify
that
they
are
equal.
4.
Show
that
a
basis
for
R
3
?
(See
the
hint
for the
previous exercise.)
Compute bote ax and
54
Chapter
3.
Essential
linear
algebra
Exercises
1.
(a) Let
A
~
[
-~ -~
-n,
x
~
[
-n
Compute
both
Ax
and
and verify
that
they are equal.
(b) Let
A
ERn
x n and x
ERn,
and
suppose
the
columns of A are
2.
Is
so
that
the
(i,j)-entry
of A
is
(vjk
Compute both (AX)i and
(XlVI
+
X2V2
+ ... +
XnVn)i,
and verify
that
they are equal.
a basis for R3? (Hint:
As
explained in
the
last paragraphs of this section,
the
three given vectors form a basis for Rn if and only if
Ax
= b has a unique
solution for every b
ERn,
where A
is
the 3 x 3 matrix whose columns are the
three given vectors.)
3. Is
a basis for R3? (See
the
hint for
the
previous exercise.)
4.
Show
that
{X2
+
1,
X +
1,
X2
- X +
I}
is
a basis for
P2,
the space of polynomials of degree 2 or less. (Hint: Verify
directly
that
the definition holds.)
5.
Show
that
{L
I
,L
2
,L
3
},
defined in Example 3.28, is a basis for P
2
.
(Hint: Use
(3.9)
to
show
that
holds for every p E P
2
.)
3.4
Orthogonal
bases
and
projections
At
the end of the
last section,
we
discussed
the
question
of
expressing
a
vector
in
terms
of a
given basis. This question
is
important
for the
following
reason,
which
we
can
only describe
in
general terms
at the
moment: Many problems
that
are
posed
in
vector spaces admit
a
special basis,
in
terms
of
which
the
problem
is
easy
to
solve.
That
is, for
many problems, there exists
a
special basis with
the
property
that
if all
vectors
are
expressed
in
terms
of
that
basis, then
a
very simple calculation
will
produce
the final
solution.
For
this reason,
it is
important
to be
able
to
take
a
vector (perhaps expressed
in
terms
of a
standard basis)
and
express
it in
terms
of a
different
basis.
In the
latter
part
of
this
section,
we
will study
one
type
of
problem
for
which
it is
advantageous
to use a
special basis,
and we
will
discuss another such
problem
in the
next section.
It is
quite easy
to
express
a
vector
in
terms
of a
basis
if
that
basis
is
orthogonal
We
wish
to
describe
the
concept
of an
orthogonal basis
and
show some important
examples.
Before
we can do so, we
must introduce
the
idea
of an
inner product,
which
is a
generalization
of the
Euclidean
dot
product.
The dot
product plays
a
special role
in the
geometry
of R
2
and R
3
. The
reason
for
this
is the
fact
that
two
vectors
x,y
in
R
2
or
R
3
are
perpendicular
if
and
only
if
Indeed,
one can
show
that
where
9 is the
angle between
the two
vectors (see Figure 3.2).
From elementary Euclidean geometry,
we
know
that,
if x and y are
perpen-
dicular,
then
(the Pythagorean theorem). Using
the dot
product,
we can
give
a
purely algebraic
proof
of the
Pythagorean theorem.
By
definition,
3.4. Orthogonal
bases
and
projections
55
6.
Let V be the
space
of all
continuous,
complex-valued
functions
defined
on the
real line:
Define
W to be the
subspace
of V
spanned
by
e
lx
and e
lx
,
where
i =
^/^l.
Show
that
{cos
(x),
sin
(x}}
is
another basis
for
W.
(Hint:
Use
Euler's
formula:
3.4. Orthogonal bases and projections 55
6. Let V
be
the
space of all continuous, complex-valued functions defined on
the
real line:
V =
{f
: R
-+
C : f
is
continuous}.
Define W
to
be
the
subspace of V spanned by ei:v
and
e-i:v,
where i =
A.
Show
that
{cos (x), sin (x) } is
another
basis for W. (Hint: Use Euler's formula:
e
iIJ
= cos
(0)
+ i sin
(0).)
3.4 Orthogonal
bases
and projections
At
the
end of
the
last section,
we
discussed
the
question
of
expressing a vector in
terms
of a given basis. This question is
important
for
the
following reason, which
we
can only describe in general terms
at
the
moment: Many problems
that
are
posed in vector spaces
admit
a special basis, in terms of which
the
problem is easy
to
solve.
That
is, for many problems, there exists a special basis with
the
property
that
if all vectors are expressed in
terms
of
that
basis,
then
a very simple calculation
will produce
the
final solution. For this reason,
it
is
important
to
be
able
to
take a
vector (perhaps expressed in terms of a
standard
basis)
and
express
it
in terms of a
different basis. In
the
latter
part
of
this section,
we
will
study
one type
of
problem
for which
it
is advantageous
to
use a special basis,
and
we
will discuss another such
problem in
the
next section.
It
is quite easy
to
express a vector
in
terms of a basis if
that
basis
is
orthogonal.
We
wish
to
describe
the
concept of
an
orthogonal basis
and
show some
important
examples. Before
we
can do so,
we
must introduce
the
idea of
an
inner product,
which is a generalization
of
the
Euclidean dot product.
The
dot
product
plays a special role in
the
geometry of R2
and
R3.
The
reason for this is
the
fact
that
two vectors
x,
y
in
R2
or
R3
are perpendicular if
and
only if
x·y
=0.
Indeed, one can show
that
x . y =
Ilxllllyll
cos (0),
where 0 is
the
angle between
the
two vectors (see Figure 3.2).
From elementary Euclidean geometry,
we
know
that,
if x
and
y are perpen-
dicular,
then
Ilx
+
yW
=
IIxl1
2
+
IIyl12
(the
Pythagorean
theorem). Using
the
dot product,
we
can give a purely algebraic
proof
of
the
Pythagorean
theorem.
By
definition,
so
Ilull=~,
Ilx
+
Yl12
=
(x
+
y)
.
(x
+
y)
=x,x+x·y+y·x+y·y
=x·x+2x·y+y·y
=
IIxl12
+
2x
. y +
IIYI12.
56
Chapter
3.
Essential linear
algebra
Figure
3.2.
The
angle
between
two
vectors.
This calculation shows
that
holds
if and
only
if x • y = 0.
Seen
this way,
the
Pythagorean theorem
is an
algebraic property
that
can be
deduced
in
R
n
,
n > 3,
even though
in
those spaces
we
cannot visualize vectors
or
what
it
means
for
vectors
to be
perpendicular.
We
prefer
to use the
word
orthogonal
instead
of
perpendicular: Vectors
x and y in
R
n
are
orthogonal
if x • y
=
0.
In the
course
of
solving
differential
equations,
we
deal with
function
spaces
in
addition
to
Euclidean spaces,
and our
methods
are
heavily dependent
on the
existence
of an
inner
product—the
analogue
of the dot
product
in
more general
vector spaces. Here
is the
definition:
Definition
3.32.
Let V
be
a
real
vector
space.
A
(real)
inner product
on V is a
function,
usually
denoted
(•,•)
or
(-,-)v>
taking
two
vectors
from
V and
producing
a
real
number.
This
function must
satisfy
the
following
three
properties:
1.
(u,
v) = (v, u) for all
vectors
u and v;
2.
(cm +
/?v,
w) =
a(u,
w) +
/?(v,
w) and (w, cm +
/?v)
=
a(w,
u) +
/?(w,
v) for
all
vectors
u,
v, and
w,
and all
real
numbers
a and
/3;
3.
(u, u)
>
0 for all
vectors
u, and (u, u) = 0
if
and
only
if
u is the
zero vector.
It
should
be
easy
to
check
that
these properties hold
for the
ordinary
dot
product
on
Euclidean
n-space.
Given
an
inner product
space
(a
vector space with
an
inner product),
we
define
orthogonality just
as in
Euclidean space:
two
vectors
are
orthogonal
if and
only
if
their
dot
product
is
zero.
It can
then
be
shown
that
the
Pythagorean theorem holds
(see
Exercise
3).
56 Chapter
3.
Essential linear algebra
y
e
x
Figure
3.2.
The angle between two vectors.
This calculation shows
that
holds if and only if x . y =
O.
Seen this
way,
the Pythagorean theorem
is
an
algebraic property
that
can be
deduced in RD, n > 3, even though in those spaces
we
cannot visualize vectors or
what
it
means for vectors to be perpendicular.
We
prefer
to
use
the
word orthogonal
instead of perpendicular: Vectors x and y in
RD
are orthogonal if
x·
y =
O.
In the course of solving differential equations,
we
deal with function spaces
in addition to Euclidean spaces, and our methods are heavily dependent on the
existence of
an
inner
product-the
analogue of the dot product in more general
vector spaces. Here
is
the definition:
Definition
3.32.
Let V
be
a
real
vector space. A (real) inner product on V is a
function, usually denoted (',
.)
or (.,
.)
v,
taking two vectors from V and producing
a
real
number. This function
must
satisfy the following three properties:
1.
(u, v) = (v, u) for all vectors u and
Vi
2.
(au
+ fiv,
w)
=
a(u,
w)
+ fi(v,
w)
and (w,
au
+ fiv) =
a(w,
u)
+ fi(w, v) for
all vectors
u,
v,
and
w,
and all real numbers a and fii
3.
(u,
u)
2':
0 for all vectors
u,
and
(u,
u)
= 0
if
and only if u is the zero vector.
It
should be easy to check
that
these properties hold for the ordinary dot
product on Euclidean n-space.
Given
an
inner
product space
(a
vector space with
an
inner product),
we
define
orthogonality
just
as in Euclidean space: two vectors are orthogonal if and only if
their dot product
is
zero.
It
can then be shown
that
the
Pythagorean theorem holds
(see Exercise 3).
3.4. Orthogonal
bases
and
projections
57
and so
This
formula
shows
that
it is
easy
to
express
a
vector
in
terms
of an
orthogonal
basis. Assuming
that
we
compute
(vi,
vi),
(v2,
V2),...,
(v
n
,
v
n
)
once
and for
all,
it
requires
just
n
inner products
to find the
weights
in the
linear combination.
In the
case
of
Euclidean
n-vectors,
a dot
product requires
2n
—
1
arithmetic
operations
(n
multiplications
and n
—
I
additions),
so the
total
cost
is
just
An
orthogonal
basis
for an
inner product space
V is a
basis
{vi,
v
2
,...,
v
n
}
with
the
property
that
(that
is,
every vector
in the
basis
is
orthogonal
to
every other vector
in the
basis).
We
now
demonstrate
the first
special property
of an
orthogonal basis. Suppose
{vi,
v
2
,...,
v
n
}
is an
orthogonal basis
for an
inner product space
V and x is any
vector
in
V.
Then there exist scalars
ai,
0:2,
• •
•,
a
n
such
that
To
deduce
the
value
of
Q:J,
we
take
the
inner product
of
both sides
of
(3.10) with
Vi:
The
last
step
follows
from
the
fact
that
every inner product
(v^,
YJ)
vanishes except
(vj,Vj).
We
then obtain
If
n is
large, this
is
much less costly
than
the
O(2n
3
/3)
operations required
for a
nonorthogonal basis.
We
also remark
that
if the
basis
is
orthonormal—each
basis
vector
is
normalized
to
have length
one—then
(3.11)
simplifies
to
Example
3.33.
The
basis
{vi,v
2
,v
3
}
forH
3
,
where
3.4. Orthogonal bases and projections 57
An orthogonal basis for
an
inner product space V is a basis
{VI,
V2,
...
, v
n
}
with
the
property
that
i
=I-
j '*
(Vi,Vj)
= 0
(that
is, every vector in
the
basis is orthogonal
to
every other vector in
the
basis).
We
now demonstrate
the
first special property of
an
orthogonal basis. Suppose
{VI,
V2,
...
, v
n
}
is
an
orthogonal basis for
an
inner product space V
and
x
is
any
vector in
V.
Then
there exist scalars
aI,
a2,
...
,
an
such
that
(3.10)
To deduce
the
value of
ai,
we
take
the
inner product of
both
sides of (3.10) with
Vi:
(Vi,X) = (Vi,aIVI + a2v2 + ... +
anv
n
)
=
al
(Vi,
vd
+
a2
(Vi,
V2)
+ ... + an(Vi, V
n
)
= ai(Vi,
Vi).
The
last step follows from
the
fact
that
every inner product
(Vi,
V
j)
vanishes except
(Vi,
Vi).
We
then
obtain
(Vi,
x)
ai
= (
)'
i = 1,2,
...
, n,
Vi,
Vi
and
so
(VI,X)
(V2,X)
(vn,x)
X =
VI
+
V2
+ ... + V
n
.
(VI,
VI)
(V2,
V2)
(V
n
,
v
n
)
(3.11)
This formula shows
that
it
is easy
to
express a vector in terms of
an
orthogonal
basis. Assuming
that
we
compute (VI,
vt),
(V2,
V2),
...
,
(V
n
, V
n
) once
and
for all,
it
requires
just
n inner products
to
find
the
weights in
the
linear combination.
In
the
case of Euclidean n-vectors, a dot product requires
2n - 1 arithmetic operations (n
multiplications
and
n - 1 additions), so
the
total
cost is
just
If
n is large, this
is
much less costly
than
the
O(2n
3
/3) operations required for a
nonorthogonal basis.
We
also remark
that
if
the
basis
is
orthonormal-each basis
vector is normalized
to
have length
one-then
(3.11) simplifies
to
(3.12)