Numerical Analysis : [ MA214 ]

Lecture 2

Instructor: Prof. Tony J. Puthenpurakal

Department of Mathematics
Indian Institute of Technology Bombay

[email protected]

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Recall I

1 Suppose x ∗ is an approximation to exact value x. Then

|x − x ∗ | = Absolute error

|x − x ∗ |
= relative error, ( provided x 6= 0)
|x|
2 x ∗ is said to be approximate x to t significant digits if
x − x∗ 
 
−t
 x  ≤ 5 × 10
 

3 Things which create loss of significant digits

(a) Subtraction of nearly equal quantities
(b) division by number which is close to zero
4 Once an error has been committed it contaminates subsequent results.
Error propagation is studied in terms of two related concepts:
1 condition
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Recall II
2 instability
5

Condition ↔ sensitivity of f (x) to changes in x

∗
 
 |f (x)−f (x )| 
|f (x)| ∗
= max : |x − x | small
 |x−x ∗ | 
|x|
 0 
 f (x)x 
≈ 
 f (x) 


Example
√
(a) f (x) = x is well conditioned.
10
(b) f (x) = 1−x 2 is ill conditioned near 1.

(c) I gave an example where f = f1 (f2 (f3 (f4 ))) with f well conditioned but
f3 ill-conditioned. This also creates lot of error.
6 Last time I gave a spectacular example of instability.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

 Today we first discuss Polynomials

p(x) = a0 + a1 x + a2 x 2 + · · · + an x n
This is called power form. This may lead to loss of significant digits.

Example:- ( We work with 4 sig digits )

−2
Suppose p is a straight line such that p(6000) = 13 , p(6001) = 3 .

p(x) = 6000 − x , in 4 sig digits

This gives p(6000) = 0 and p(6001) = −1

Remedy:- ( Shift of center )

p(x) = 3.333E −1 − (x − 6000)

p(6000) = 3.333E −1 and p(6001) = −6.667E −1

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Nested form of polynomial

p(x) = a0 + a1 (x − c) + a2 (x − c)2 + · · · + an (x − c)n

Computing directly requires more multiplications and additions. Better to
compute it as

p(x) = a0 + (x − c){a1 + a2 (x − c) + · · · + an (x − c)n−1 }

= a0 + (x − c){a1 + (x − c){a2 + a3 (x − c) + · · · + an (x − c)n−2 }}
= a0 + (x − c){a1 + (x − c){a2 + (x − c){a3 + · · · + an (x − c)n−3 }}

Bonus of nested form is preservation of significant digits

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Example ( 4 Significant digits )

p(x) = x 3 − 6.1x 2 + 3.2x + 1.5

p(4.71) = −14.26 , ( correct upto 4 sig digits )
However if you directly compute

p(4.71) = (4.71)3 − 6.1(4.71)2 + 3.2(4.71) + 1.5

= −14.23 , ( correct upto 3 sig digits )

In nested form

p(x) = x(x 2 − 6.1x + 3.2) + 1.5

= x(x(x − 6.1) + 3.2) + 1.5
p(4.71) ≈ −14.26

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

 We now discuss ”Interpolation”

We have f (x1 ), f (x2 ), · · · , f (x6 ). We need to approximate f (a).

Idea:- Fit a curve ”passing through”

(x1 , f (x1 )), (x2 , f (x2 )), · · · , (x6 , f (x6 ))
and then approximate f (a).

Question:- Which curve to fit ?

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Weirstass Approximation Theorem

One classical and non-trivial result is the following.

Theorem ( Weirstass Approximation Theorem )

Suppose f : [a, b] −→ R is a continuous function. For each  > 0, there
exists a polynomial p(x) such that

|f (t) − p(t)| <  , for all t ∈ [a, b]

Remark:- The polynomial constructed for proving this theorem has slow
convergence. So it is ineffective in practice.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

We might be tempted to use Taylor polynomial

Example:- f (x) = x1 , x ∈ [1, 4]

1
f 0 (x) = −
x2
2
f 00 (x) = (−1)2
x3
..
.
k!
f (k) (x) = (−1)k
x k+1
So nth Taylor polynomial
n n
X f (k) (1) X
Tn (x) = (x − 1)k = (−1)k (x − 1)k
k!
k=0 k=0

n 2 3 4 5 6 7
Tn (3) 3 -5 11 -21 43 -85

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

 Problem

Given n + 1 distinct points x0 , x1 , · · · , xn in [a, b] and a function

f : [a, b] −→ R, does there exists a polynomial p(x) of degree ≤ n which
interpolates f (x) at the points x0 , x1 , · · · , xn , i.e. p(x) satisfies

p(xi ) = f (xi ) , for i = 0, 1, 2, · · · , n

We prove that there exists a unique polynomial which does the job.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Lagrange Polynomials

Given x0 , x1 , · · · , xn distinct points

n
Y x − xi
lk (x) = , k = 0, 1, · · · , n
xk − xi
i=0,i6=k

Example:-
n = 2 and x0 , x1 , x2 are distinct points

(x − x1 )(x − x2 )
l0 (x) =
(x0 − x1 )(x0 − x2 )
(x − x0 )(x − x2 )
l1 (x) =
(x1 − x0 )(x1 − x2 )
(x − x0 )(x − x1 )
l2 (x) =
(x2 − x0 )(x2 − x1 )

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

x0 , x1 , · · · , xn are (n + 1) distinct points
n
Y x − xi
lk (x) = , k = 0, 1, · · · , n
xk − xi
i=0,i6=k

Properties:-
1 l (x) is a degree n-polynomial, for all k = 0, 1, · · · , n
k
2 l (x ) = 1
k k
3 l (x ) = 0 for i 6= k
k i
X n
Pn (x) = f (xi )li (x)
i=0

Then Pn (x) is a polynomial of degree ≤ n and

Pn (xi ) = f (xi ), for all i = 0, 1, · · · , n

Thus Pn (x) is an interpolating polynomial.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Proposition
Let p(x), q(x) interpolate f (x) in x0 , · · · , xn . Then p(x) = q(x)

Proof.
degree p(x) ≤ n
degree q(x) ≤ n
So h(x) = p(x) − q(x) is a polynomial of degree ≤ n.

h(xi ) = p(xi ) − q(xi ) , for i = 0, 1, · · · , n

= f (xi ) − f (xi )
= 0

Thus h(x) has n + 1 zeros x0 , · · · , xn .

Therefore h(x) = 0. Thus p(x) = q(x)

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Example
x f (x)
2 6.931E −1
3 1.099
4 1.386
Approximate f (3.2).
Ans.:- P2 (x) = f (2)l0 (x) + f (3)l1 (x) + f (4)l2 (x) interpolates f (x). Recall
(x − x1 )(x − x2 )
l0 (x) =
(x0 − x1 )(x0 − x2 )
(3.2 − 3)(3.2 − 4)
Thus l0 (3.2) = = −8E −2
(2 − 3)(2 − 4)
Similarly
(3.2 − 2)(3.2 − 4)
l1 (3.2) = = 9.6E −1
(3 − 2)(3 − 4)
(3.2 − 2)(3.2 − 3)
l2 (3.2) = = 1.2E −1
(4 − 2)(4 − 3)
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
f (3.2) ≈ (6.931E −1)(−8E −2) + (9.6E −1)(1.099) + (1.2E −1)(1.386)
= 1.166

The function

f (x) = log(x)
f (3.2) = log(3.2)
= 1.163

Thus our approximation is correct upto 3 significant digits.

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Definition
n
X
Pn (x) = f (xk )lk (x)
k=0

is called ”Lagrange” form of interpolating polynomial.

Problem with Lagrange form of interpolating polynomial

Suppose we have found Pn (x) by interpolating f (x) at points

x0 , x1 , · · · , xn .

Suppose we also know f (xn+1 ). Then we can form Pn+1 (x) by

interpolating f (x) at x0 , x1 , · · · , xn , xn+1 .

There is no obvious relation between the Lagrange form of Pn (x) and

Pn+1 (x).

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Newton form of interpolating polynomial
We write
Pn (x) = a0 + a1 (x − x0 ) + a2 (x − x0 )(x − x1 ) + · · · + an (x − x0 ) · · · (x − xn−1 )

Important remark:- x − xn does not appear in the last term above.

Set
q(x) = a0 + a1 (x − x0 ) + a2 (x − x0 )(x − x1 ) + · · ·
+ an−1 (x − x0 ) · · · (x − xn−2 )
Pn (x) = q(x) + an (x − x0 ) · · · (x − xn−1 )
Note q(xi ) = Pn (xi ) = f (xi ), i = 0, 1, · · · , n − 1
Also degree q(x) ≤ n − 1. By uniqueness of interpolating polynomial
q(x) = Pn−1 (x)
So
Pn (x) = Pn−1 (x) + an (x − x0 ) · · · (x − xn−1 )

Note :- aInstructor:
n =coefficient
Prof. Tony J.of x n in Pn (x).
Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Definition
f [x0 , x1 , · · · , xn ] =: an
f [x0 , x1 , · · · , xn ] is called the nth divided difference of f (x) at the points
x0 , x1 , · · · , xn .

we write
Pn (x) = f (x0 ) + f [x0 , x1 ](x − x0 )
+ f [x0 , x1 , x2 ](x − x0 )(x − x1 )
+ ···
..
.
+ f [x0 , x1 , x2 , · · · , xn ](x − x0 )(x − x1 ) · · · (x − xn−1 )

Thus to determine Pn (x) we only have to find

f [x0 , x1 ], f [x0 , x1 , x2 ], · · · , f [x0 , x1 , x2 , · · · , xn ]
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Determining f [x0 , x1 , x2 , · · · , xn ]

f [x0 ] = f (x0 )
P1 (x1 ) = f (x1 )
f (x1 ) = f (x0 ) + f [x0 , x1 ](x1 − x0 )
f (x1 ) − f (x0 )
So f [x0 , x1 ] =
x1 − x0

Proposition
f [x1 , x2 , · · · , xk ] − f [x0 , x1 , · · · , xk−1 ]
f [x0 , x1 , x2 , · · · , xk ] =
xk − x0

Proof.
Let Pi (x) = polynomial of degree ≤ i which agrees with f (x) at the points
x0 , x1 , x2 , · · · , xi .
Let qk−1 (x) = polynomial of degree ≤ k − 1 which agrees with f (x) at the
points x1 , x2 , · · · , xk .
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Proof Continue
Set
x − x0 xk − x
p(x) = qk−1 (x) + Pk−1 (x)
xk − x0 xk − x0
Note: degree p(x) ≤ k
p(x0 ) = f (x0 ) and p(xk ) = f (xk ) and for 1 ≤ i ≤ k − 1
xi − x0 xk − xi
p(xi ) = f (xi ) + f (xi ) = f (xi )
xk − x0 xk − x0
By uniqueness of interpolating polynomial
p(x) = Pk (x)

f [x0 , x1 , · · · , xk−1 ] = Coeff of x k in Pk (x)

Coeff of x k−1 in qk−1 (x) Coeff of x k−1 in Pk−1
= −
xk − x0 xk − x0
f [x1 , x2 , · · · , xk ] − f [x0 , x1 , · · · , xk−1 ]
=
xk − x0
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Table of divided differences
x f (x) f [, ] f [, , ] f [, , , ]
x0 f (x0 ) f [x0 , x1 ] f [x0 , x1 , x2 ] f [x0 , x1 , x2 , x3 ]
x1 f (x1 ) f [x1 , x2 ] f [x1 , x2 , x3 ]
x2 f (x2 ) f [x2 , x3 ]
x3 f (x3 )
Example:- Fill the table and approximate f (3.2)
x f (x) f [, ] f [, , ] f [, , , ]
2 6.931E −1 4.059E −1 −5.945E −2 9.15E −3
3 1.099 2.87E −1 −3.2E −2
4 1.386 2.23E −1
5 1.609

P2 (x) = f (2) + f [2, 3](x − 2) + f [2, 3, 4](x − 2)(x − 3)

= 6.931E −1 + 4.059E −1(x − 2) − 5.945E −2(x − 2)(x − 3)
P2 (3.2) = 1.166
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Exact value f (3.2) = log(3.2) = 1.163

P3 (x) = P2 (x) + 9.15E −3(x − 2)(x − 3)(x − 4)

P3 (3.2) = 1.166 + 9.15E −3(1.2)(0.2)(−0.8) = 1.164

Rx
Example 2:- f (x) = 0 sin(t 2 )dt

x f (x) f [, ] f [, , ] f [, , , ]
0.8 1.657E −1 6.62E −1 6.1E −1 −2.5E −1
0.9 2.319E −1 7.84E −1 5.35E −1
1.0 3.103E −1 8.91E −1
1.1 3.994E −1

P2 (x) = f (0.8) + f [0.8, 0.9](x − 0.8) + f [0.8, 0.9, 1.0](x − 0.8)(x − 0.9)
= 1.657E −1 + 6.62E −1(x − 0.8) + 6.1E −1(x − 0.8)(x − 0.9)
P2 (0.85) = 1.973E −1
f (0.85) = 1.974E −1

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

 P3 (x) = P2 (x) + f [0.8, 0.9, 1.0, 1.1](x − 0.8)(x − 0.9)(x − 1.0)
P3 (0.85) = P2 (0.85) − 2.5E −1(0.05)(−0.05)(−0.15)
P3 (0.85) = 1.973E −1

Remarks:-
1 In the above example error of P3 (x) was same as that P2 (x).
2 It is possible that interpolating error to increase if we increase number
of points.
error en (x) = f (x) − Pn (x)
i.e. it is possible that

maxx∈[a,b] |en+1 (x)| > maxx∈[a,b] |en (x)|

See example 2.4 page 44 of your textbook Conte & de Boor

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
The error of the interpolating polynomial

Let x0 , x1 , x2 , · · · , xn be n + 1 points and f (x0 ), f (x1 ), f (x2 ), · · · , f (xn ) be

function values
f : [a, b] −→ R.

Pn (x) = polynomial which interpolates f (x) at x0 , x1 , x2 , · · · , xn

error en (x) = f (x) − Pn (x)

Let x ∈ [a, b] be distinct from x0 , x1 , x2 , · · · , xn .

We need to estimate en (x).

Let Pn+1 (x) = polynomial which interpolates f (x) at the points

x0 , x1 , x2 , · · · , xn , x ( n + 2 points )

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

 n
Y
Pn+1 (x) = Pn (x) + f [x0 , x1 , · · · , xn , x] (x − xj )
j=0
f (x) = Pn+1 (x) by definition
en (x) = f (x) − Pn (x)
= Pn+1 (x) − Pn (x)
n
Y
= f [x0 , x1 , · · · , xn , x] (x − xj )
j=0

Thus to estimate error we need to approximate

(a) f [x0 , x1 , · · · , xn , x]
(b) nj=0 (x − xj )
Q

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2

Theorem
Let f : [a, b] −→ R be a continuous and k times differentiable in (a, b). If
x0 , x1 , · · · , xk are k + 1 distinct points in [a, b], then there exists ξ ∈ (a, b)
such that
f (k)(ξ)
f [x0 , x1 , · · · , xk ] =
k!

Proof.
k=1
f (x1 ) − f (x0 )
f [x0 , x1 ] = = f 0 (ξ) by M.V.T.
x1 − x0
ek (x) = f (x) − Pk (x)
(k)
has k + 1 zeros x0 , x1 , · · · , xk . So ek (x) will have a zero say ξ.
(k) (k)
0 = ek (ξ) = f (k) (ξ) − Pk (ξ)
0 = f (k) (ξ) − k!f [x0 , x1 , · · · , xk ]
f (k)(ξ)
So f [x0 , x1 , · · · , xk ] =
Instructor: Prof. Tony J. Puthenpurakal
k!
Numerical Analysis : [ MA214 ] Lecture 2
Corollary

en (x) = f (x) − Pn (x)

n
f (n+1)(ξ) Y
= (x − xj )
(n + 1)!
j=0

Estimating:-
n
Y
Ψn+1 (x) = (x − xj )
j=0

It is possible to choose x0 , x1 , · · · , xn in [a, b] such that |Ψn+1 (x)| is as

small as possible.

This choice of points are called Chebyshev points of [a, b].

( Unfortunately it is not in syllabus )

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
Osculatory interpolation
Sometimes we have the following situation
We have x0 , x1 , · · · , xk and
f (x0 ), f (x1 ), f (x2 ), · · · , f (xk )
f 0 (x0 ), f 0 (x1 ), f 0 (x2 ), · · · , f 0 (xk )
We need a polynomial p(x) such that
p(xi ) = f (xi ) and p 0 (xi ) = f 0 (xi ) , for i = 0, 1, · · · , n
Note:- deg p(x) ≤ 2n + 1
Example where this situation arise
dy
= g (x, y ) , y (x0 ) = y0
dx
x0 y (x0 ) y 0 (x0 ) = g (x0 , y0 ) Remark
x1 y (x1 ) y 0 (x1 ) = g (x1 , y1 ) y (xi ) is
.. .. ..
. . . calculated by some
xn y (xn ) y 0 (xn ) = g (xn , yn ) Numerical Method
Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2
 f (x1 )−f (x0 )
Remark:- f [x0 , x1 ] = x1 −x0 . So lim f [x0 , x1 ] = f 0 (x0 )
x1 →x0

Definition
f [x0 , x0 ] = f 0 (x0 )

Example:- f (1) = 0, f 0 (1) = 1, f (2) = 6.931E −1, f 0 (2) = 0.5.

We need a cubic polynomial P3 (x) such that P3 (1) = f (1), P30 (1) = f 0 (1),
P3 (2) = f (2), P30 (2) = f 0 (2).

n f (x) f [, ] f [, , ] f [, , , ]
y0 = 1 0 1 -0.3069 0.1137
y1 = 1 0 6.931E −1 -0.1931
y2 = 2 6.931E −1 0.5
y3 = 2 6.931E −1

P3 (x) = 0 + 1(x − 1) − 0.3069(x − 1)2 + 0.1137(x − 1)2 (x − 2)

Instructor: Prof. Tony J. Puthenpurakal Numerical Analysis : [ MA214 ] Lecture 2