Finite Differences for Differential Equations 28 Finite Differences for Differential Equations 29

B OUNDARY VALUE P ROBLEMS (I)

Solving a TWO POINT BOUNDARY VALUE PROBLEM with D IRICHLET
B OUNDARY C ONDITIONS :
d2y
=g for x (0, 2)
dx2
y(0) = y(2) = 0
PART II
Finitedifference approximation:
Secondorder center difference formula for the interior nodes:
Finite Difference Methods for Differential y j+1 2y j + y j1
= g j for j = 1, . . . , N
h2
Equations where h = 2
N+1 and x j = jh
Endpoint nodes:
y0 = 0 = y2 2y1 = h2 g1
yN+1 = 0 = 2yN + yN1 = h2 gN

Tridiagonal algebraic system solved very efficiently with the

T HOMAS ALGORITHM (a version of the Gaussian elimination)

Finite Differences for Differential Equations 30 Finite Differences for Differential Equations 31

B OUNDARY VALUE P ROBLEMS (II)

Solving a TWO POINT BOUNDARY VALUE PROBLEM with N EUMANN
B OUNDARY C ONDITIONS :
d2y
dx2
=g for x (0, 2) B OUNDARY VALUE P ROBLEMS (III)
dy dy
(0) = (2) = 0
dx dx
In order to retain the SECOND ORDER ACCURACY in the approximation of
Finitedifference approximation: the Neumann problem need to use higher-order formulae at endpoints, e.g.
Secondorder center difference formula for the interior nodes: y2 + 4y1 3y0 1
y0 = = 0 = y0 = (y2 + 4y1 )
y j+1 2y j + y j1 2h 3
= g j for j = 1, . . . , N
h2
The first row thus becomes
Firstorder Forward/Backward Difference formulae to reexpress
2 2
endpoint values: y1 y0 y2 y1 = h2 g1
= 0 = y0 = y1 3 3
h
yN+1 yN S ECOND ORDER ACCURACY RECOVERED!
= 0 = yN+1 = yN
h
Firstorder only DEGRADED ACCURACY !

Tridiagonal algebraic system Is there any problem? Where?

Finite Differences for Differential Equations 32 Finite Differences for Differential Equations 33

B OUNDARY VALUE P ROBLEMS (IV)

C OMPACT S TENCILS stencils based on three grid points (in every
direction) only: {x j+1 , x j , x j1 } at the j th node
Is is possible to obtain higher (then second) order of accuracy on compact
stencils? YES !
B OUNDARY VALUE P ROBLEMS (V)
d2 y
Consider the central difference approximation to the equation dy2
=g
C OMPACT F INITE D IFFERENCE S CHEMES
y j+1 2y j + y j1 h2 (iv)
y j + O (h4 ) = g j A DVANTAGES :
h2 12
2 (iv)
Increased accuracy on compact grids
h
Re-express the error term 12 y j using the equation in question:
DRAWBACKS:
h2 (iv) h2 h2 g j+1 2g j + g j1 h2 (iv)
 
yj = gj = g + O (h 4
) need to be tailored to the specific equation solved
12 12 12 h2 12 j
can get fairly complicated for more complex equations
Inserting into the original finitedifference equation:
y j+1 2y j + y j1 g j+1 2g j + g j1
= gj + + O (h4 )
h2 12
Slight modification of the RHS = FOURTH ORDER ACCURACY !!!

Finite Differences for Differential Equations 34 Finite Differences for Differential Equations 35

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

G ENERAL R EMARKS C HARACTERIZATION OF I NTEGRATION
M ETHODS
Consider the following C AUCHY PROBLEM :
dy
= f (y,t) with y(t0 ) = y0 ACCURACY unlike in the Boundary Value Problems, there is no terminal
dt
condition and approximation errors may accumulate in time; consequently, a
The independent variable t is usually referred to as TIME . relevant characterization of accuracy is provided by the GLOBAL ERROR
Equations with higherorder derivatives can be reduced to systems of (global error) = (local error) (# of time steps),
firstorder equations
rather than the LOCAL ERROR .
Generalizations to systems of ODEs straightforward
STABILITY unlike in the Boundary Value Problems, where boundedness
When the RHS function does not depend on y, i.e., f (y,t) = f (t), of the solution at final time is enforced via a suitable terminal condition , in
solution obtained via a QUADRATURE Initial Value Problems there is a priori no guarantee that the solution will
Assume uniform timesteps ( h is constant ) remain bounded.
Finite Differences for Differential Equations 36 Finite Differences for Differential Equations 37

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

M ODEL P ROBLEM E XPLICIT E ULER S CHEME (I)
S TABILITY of various numerical schemes is usually analyzed by applying
these schemes to the following LINEAR MODEL :
Consider a Taylor series expansion
dy
= y = (r + ii )y with y(t0 ) = y0 ,
dt h2
y(tn+1 ) = y(tn ) + hy (tn ) + y (tn ) + . . .
which is stable when r <= 0 . 2
2 h2 3 h3 Using the ODE we obtain
 
y(t): = y0 et = 1 + h +
E XACT SOLUTION + + . . . y0
2 6 dy
y = =f
M OTIVATION consider the following ADVECTION DIFFUSION PDE : dt
dy df
u u 2 u y = = = ft + f fy
+c a 2 = 0 dt dt
t x x
Neglecting terms proportional to second and higher powers of h yields the
Taking Fourier transform yields (k is the wavenumber):
E XPLICIT E ULER M ETHOD
d uk
+ c i k uk + a k2 uk = 0
dt yn+1 = yn + h f (yn ,tn )
where
the real term a k2 uk represents DIFFUSION Retaining higherorder terms is inconvenient, as it requires differentiation of
the imaginary term c i k uk represents ADVECTION f and does not lead to schemes with desirable stability properties.

Finite Differences for Differential Equations 38 Finite Differences for Differential Equations 39

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

E XPLICIT E ULER S CHEME (II) E XPLICIT E ULER S CHEME (III)

Stability (for the model problem)

L OCAL ERROR analysis:
yn+1 = yn + hyn = (1 + h)yn
yn+1 = (1 + h) yn + [O (h2 )]
Thus, the solution after n time steps
G LOBAL ERROR analysis:
yn = (1 + h)n y0 , n y0 = = 1 + h
T
(global error) = Ch N = Ch = C h
2 2
h For large n, the numerical solution remains stable iff
Thus, the scheme is || 1 = (1 + r h)2 + (i h)2 1
locally secondorder accurate
CONDITIONALLY STABLE for real
globally (over the interval [t0 ,t0 + Nh]) firstorder accurate
UNSTABLE for imaginary
Finite Differences for Differential Equations 40 Finite Differences for Differential Equations 41

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

I MPLICIT E ULER S CHEME (I) I MPLICIT E ULER S CHEME (II)

I MPLICIT S CHEMES based on approximation of the RHS that involve Stability (for the model problem):
f (yn+1 ,t), where yn+1 is the unknown to be determined yn+1 = yn + hyn+1 = yn+1 = (1 h)1 yn
I MPLICIT E ULER S CHEME obtained by neglecting second and 
1
n
1
yn+1 = y0 , n y0 = =
higherorder terms in the expansion: 1 h 1 h
h2 || 1 = (1 r h)2 + (i h)2 1
y(tn ) = y(tn+1 ) hy (tn+1 ) + y (tn+1 ) . . .
2
 Implicit Euler scheme is thus stable for
dy 
Upon substitution dt t = f (yn+1 ,tn+1 ) we obtain
n+1 all stable model problems
yn+1 = yn + h f (yn+1 ,tn+1 ) most unstable model problems

The scheme is R EMARK : When solving systems of ODEs of the form y = A (t)y, each
locally SECOND ORDER accurate implicit step requires solution of an algebraic system: yn+1 = (I hA )1 yn

globally (over the interval [t0 ,t0 + Nh]) FIRST ORDER accurate Implicit schemes are generally hard to implement for nonlinear problems

Finite Differences for Differential Equations 42 Finite Differences for Differential Equations 43

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

C RANK N ICHOLSON S CHEME (I) L EAPFROG S CHEME (I)
Obtained by approximating the formal solution of the ODE
Rt
yn+1 = yn + tnn+1 f (y,t) dt using the TRAPEZOIDAL QUADRATURE :
L EAPFROG as an example of a TWO STEP METHOD :
h
yn+1 = yn + [ f (yn ,tn ) + f (yn+1 ,tn+1 )] yn+1 = yn1 + 2 h yn
2
The scheme is C HARACTERISTIC EQUATION for the AMPLIFICATION FACTOR (yn = n y0 )
locally THIRD ORDER accurate
2 2 h 1 = 0
globally (over the interval [t0 ,t0 + Nh]) SECOND ORDER accurate
where roots give the amplification factors:
Stability (for the model problem):
2 h2
+ . . . = eh + O (h3 )
p
1 + h 1 = h + 1 + 2 h2 1 + h +
!
h 2 2
yn+1 = yn + (yn+1 + yn ) = yn+1 = yn
2 1 h
2
p 2 h2
!n 2 = h 1 + 2 h2 (1 h + . . . ) = eh + O (h3 )
1 + h 1 + h 2
yn+1 = 2
y0 , n y0 = = 2
1 h
2 1 h
2
Thus, the scheme is
|| 1 = (h) 0 locally THIRD ORDER accurate
S TABLE for all model ODEs with stable solutions globally (over the interval [t0 ,t0 + Nh]) SECOND ORDER accurate
Finite Differences for Differential Equations 44 Finite Differences for Differential Equations 45

I NITIAL VALUE P ROBLEMS

I NITIAL VALUE P ROBLEMS M ULTISTEP P ROCEDURES
L EAPFROG S CHEME (II) General form of a MULTISTEP PROCEDURE :
p q
j yn+ j = h j f (yn+ j ,tn+ j )
j=1 j=1
Stability for diffusion problems ( = r ):
q with characteristic polynomials
1 = h + 1 + 2r h2 > 1 for all h > 0 p (z) = p z p + p1 z p1 + + 0
q (z) = q zq + q1 zq1 + + 0
Thus the scheme is UNCONDITIONALLY UNSTABLE for diffusion problems!
Stability for advection problems ( = ii ): if p > q EXPLICIT SCHEME

1
if p q IMPLICIT SCHEME
21/2 = 1 (!!!) for h <
|i | A (, ) procedure converges uniformly in [a, b], i.e.,
Thus, the scheme is CONDITIONALLY UNSTABLE and NON DIFFUSIVE for limh0 maxtn [a,b] |yn y(tn )| = 0 if:
advection problems! the following consistency conditions are verified: (1) = 0 and
(1) = (1) ( CONSISTENCY CONDITION )
Q UESTION analyze dispersive (i.e., related to arg()) errors of the
leapfrog scheme. all roots of the polynomial (z) are such that |zi | 1 and the roots with
|zk | = 1 are simple ( STABILITY CONDITION )

Finite Differences for Differential Equations 46 Finite Differences for Differential Equations 47

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

RUNGE K UTTA M ETHODS (I) RUNGE K UTTA M ETHODS (II)
General form of a FRACTIONAL STEP METHOD :
RK4 an ODE workhorse:
yn+1 = yn + 1 h k1 + 2 h k2 + 3 h k3 + . . .
h h h
where yn+1 = yn + k1 + (k2 + k3 ) + k4
6 3 6
k1 = f (yn ,tn ) h
k1 = f (yn ,tn ) k2 = f (yn + k1 ,tn+1/2 )
2
k2 = f (yn + 1 hk1 ,tn + 1 h)
h
k3 = f (yn + 2 hk1 + 3 hk2 ,tn + 2 h) k3 = f (yn + k2 ,tn+1/2 ) k4 = f (yn + hk3 ,tn+1 )
2
. The amplification factor:
.
.
Choose i , i and i to match as many expansion coefficients as possible in 2 h2 3 h3 4 h4
= 1 + h + + +
h2 h3 2 6 24
y(tn+1 ) = y(tn ) + hy (tn ) + y (tn ) + y (tn ) . . .
2 6 Thus, stability iff || 1
y = f
ACCURACY:
y = ft + f fy
eh = + O (h5 )
y = ftt + ft fy 2 f fyt + f 2 fyt + f 2 fyy
Thus, the scheme is
RungeKutta methods are SELF STARTING with fairly good stability and locally FIFTH ORDER accurate
accuracy properties. globally (over the interval [t0 ,t0 + Nh]) FOURTH ORDER accurate
Finite Differences for Differential Equations 48 Finite Differences for Differential Equations 49

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

RUNGE S P RINCIPLE L AX E QUIVALENCE T HEOREMa
Consider an I NITIAL VALUE P ROBLEM
Let (k + 1) be the order of the local truncation error; denote Y (t, h) an du
= L u with u(t0 ) = u0
approximation of the exact solution y(t) computed with the step size h; then dt
at t = t0 + 2nh: and assume that it is wellposed, i.e., it admits solutions which are unique
y(t) Y (t, h) C 2 n hk+1 = C(t t0 )hk and stable

y(t) Y (t, 2h) C n (2h)k+1 = C(t t0 )2k hk Consider a numerical method defined by a finitedifference operator C(h)
such that the approximate solution is given by
Subtracting:
Y (t, 2h) Y (t, h) C(t t0 )(1 2k )hk uh (nh) = C (h)n u0 , n = 1, 2, . . .
Thus, we can obtain an estimate of the ABSOLUTE ERROR based on solution The above method is CONSISTENT iff
C (h)I
is a convergent approximation
h
with two stepsizes only: of the operator L
Y (t, h) Y (t, 2h) L AX T HEOREM For a CONSISTENT difference method STABILITY is
y(t) Y (t, h)
2k 1 equivalent to CONVERGENCE
a For
Runges principle is very useful for ADAPTIVE STEP SIZE REFINEMENT a more technical discussion, see $ 5.2 in Atkinson & Han (2001)

Finite Differences for Differential Equations 50 Finite Differences for Differential Equations 51

I NITIAL VALUE P ROBLEMS I NITIAL VALUE P ROBLEMS

C ONSERVATION P ROPERTIES (I) C ONSERVATION P ROPERTIES (II)
Is ACCURACY and S TABILITY all that matters? Implicit Euler
 1  1 1
C ONSERVATION P ROPERTIES conservation by the numerical method || =  = q = 1 2i h2 + < 1 for all h

1 ii h 1 + i h
2 2 2
(i.e., in the discrete sense) of various invariants the original equation may
possess The scheme is thus DISSIPATIVE (i.e., not conservative)
R EMARK conservation properties are particularly relevant for solution FourthOrder RungeKutta
of Hamiltonian / hyperbolic systems
 
2i h2 3i h3 4i h4  1
 q
|| = 1 + ii h i + = 576 86i h6 + 8i h8

Example conservation of the solution norm:  2 6 24  24

In the continuous setting (assume u = |u|ei ) = 1

1 6 6
h + < 1 for small h
144 i
d|u|
= 0 = |u(t)| = |u0 |, The scheme is thus DISSIPATIVE (i.e., not conservative)


du
dt
= ii u
dt
d
= i , Leapfrog |1/2 | 1 for all h < |1 |
i
dt The scheme is thus CONSERVATIVE for all timesteps for which it is
In the discrete setting: |uh (nh)| = |uh ((n 1)h)| = = |uh (0)| stable!!! Leapfrog is an example of a SYMPLECTIC INTEGRATOR which are
Necessary and sufficient condition for discrete conservation: h, |(h)| = 1 designed to have good conservation properties.
Finite Differences for Differential Equations 52 Finite Differences for Differential Equations 53

F INITE D IFFERENCES FOR PDE S

R EVIEW
Classification of linear PDEs in 2D: consider u : 2 R and A, B,C R
such that
2 u 2 u 2 u
A +B +C 2 + f (x, y, u) = 0
x2 x y y
E LLIPTIC P ROBLEMS : B2 4AC < 0
Poisson equation: F INITE D IFFERENCES FOR PDE S
2 u 2 u E LLIPTIC P ROBLEMS
+ = g(x, y)
x2 y2
PARABOLIC P ROBLEMS : B2 4AC = 0 See Homework Assignment # 1 ...
Heat equation:
u 2 u 2 u
 
=a + + g(x, y)
t x2 y2
H YPERBOLIC P ROBLEMS : B2 4AC > 0
Wave equation:
2 u 2 u 2 u
 
=a + + g(x, y)
t 2 x2 y2

Finite Differences for Differential Equations 54 Finite Differences for Differential Equations 55

F INITE D IFFERENCES FOR PDE S

u 2 u F INITE D IFFERENCES FOR PDE S
PARABOLIC P ROBLEMS t = x2 (I) u 2 u
PARABOLIC P ROBLEMS t = x2 (II)
C RANK N ICHOLSON M ETHOD (x j = jx, j = 1, . . . , M, t = nt,
n = 1, . . . , N): M ETHOD
 2 n un 2un +un
allow for a more general approximation in time of the RHS ( [0, 1])
spatial derivative: xu2 = j+1 (x)j 2 j1 + O ((x)2 )
j
un+1
"  n+1  2 n #
u unj 2 u u
 n+1
j 1
time derivative: = + O (t) = + (1 ) + O (t)
t j t 2 x2 j x2 j
un+1
" n+1 n #
u unj 2 u 2 u
 n+1 
j 1
= + O (t) = + + O ((t)2 )
t j t 2 x2 j x2 j
special cases
t  n+1  = 0 = E XPLICIT METHOD : U n+1 = A0U n
un+1 unj = u j+1 2un+1 + un+1 n n n 2 2


j1 + u j+1 2u j + u j1 + O (x) + (t)
j
2(x)2 j
= 21 = C RANK N ICHOLSON METHOD (see previous slide)
t = 1 = I MPLICIT METHOD: A1U n+1 = U n
thus, defining r = (x)2
,we have at every time step n
Stability:
run+1 n+1
j+1 + 2(1 + r)u j run+1 n n n
j1 = ru j+1 + 2(1 r)u j + ru j1 t 1
The E XPLICIT S CHEME is STABLE for r = 2(x)2
< 2
which for U n = [un1 , . . . , unM ]T can be written as an algebraic system
The C RANK N ICHOLSON and I MPLICIT S CHEME are STABLE for all r
(2I A)U n+1 = (2I + A)U n , where A is a tridiagonal matrix
Finite Differences for Differential Equations 56

F INITE D IFFERENCES FOR PDE S

2 u 2 u
H YPERBOLIC P ROBLEMS t 2 = x2 (I)
n unj+1 2unj +unj1
2 u

Spatial derivative: x2
= (x)2
+ O ((x)2 )
j

Time derivative:
un+1 2unj + un1
n n
2 u 2 u
 
j j
= + O ((t)2 ) = + O ((t)3 )
t 2 j (t)2 x2 j

(t)2 n (t)2
 
n1
un+1 n
unj + O (x)2 + (t)4



j = u j+1 + u j1 u j + 2 1
(x)2 (x)2

(t)2
Stability for (x)2
1

R EMARK : need two initial conditions!