The Discrete Fourier Transform

Abhiram Ranade

June 21, 2021

Convolution using Divide and Conquer
Convolution:
Input: Numbers a0 , . . . , an−1 , b0 , . . . P
, bm−1
Output: c0 , . . . , cm+n−2 where ck = i,j|i+j=k ai bj

Alternative definition:
Pn−1 “Polynomial
Pm−1 multiplication”
i
Let A(x) = i=0 ai x , B(x) = j=0 bj x j
Then ck are coefficients of C (x) = A(x)B(x)

Time to convolve using ”direct” method: O(mn)

Today: Time O(n log n) Assume n ≥ m

Polynomial multiplication is related to integer multiplication. Fastest integer

multiplication algorithm is based on polynomial multiplication algorithms.

Convolution is very important in Signal Processing.

The basic signal processing problem: Input Signal → ”Filter” → Output signal

Action of a ”linear time invariant” filter:

Input: Unit height pulse at time 0
Output: Pulses of heights bj at time j, for j = 0, . . . , m − 1.
b0 , . . . , bm−1 = ”Impulse response”
Input: Height a pulse at time t.
Output: (Scaling + shifting) Pulses of height abj at time t + j, for j = 0, . . . , m − 1.

Input: Pulses of height a0 , a1 , a2 , . . . , an−1 at times 0, . . . , n − 1

Output: Superposition/sum of effects of each input pulse
a0 at t = 0 → a0 b0 a0 b1 a0 b2 a0 b3 . . . a0 bt ...
a1 at t = 1 → a1 b0 a1 b1 a1 b2 . . . a1 bt−1 . . .
a2 at t = 2 → a2 b0 a2 b1 . . . a2 bt−2 . . .
...
Output at time t: pulse of height a0 bt + a1 bt−1 + a2 bt−2 + . . .
For t = 0, . . . , m + n − 2
Output is a convolution of input and impulse response
Alternate representation of polynomials
Theorem: There is a unique degree d polynomial through (x0 , y0 ), (x1 , y1 ), . . . , (xd , yd )
if xi are distinct.

(x0 , A(x0 )), (x1 , A(x1 )), . . . , (xn−1 , A(xn−1 )) is ”point-value representation”.
a0 , . . . , an−1 is ”coefficient representation”

Coefficient → point-value : Evaluate!

point-value → Coefficient : Lagrange Interpolation

Alternate algorithm for multiplying degree n − 1 polynomials A, B:

1. Evaluate A, B at 2n − 1 distinct points : point-value representation (redundant)
2. Multiply the values of A, B at each point to get corresponding values of C = AB.
3. Interpolate the values of C at 2n − 1 points to get coefficient representation.
Because C has degree 2n − 2, 2n − 1 point value pairs suffice.
Need fast evaluation/interpolation.
Pause point:
1. Suppose you are given the price of a commodity for n successive days,
p0 , . . . , pn−1 . The m day simple moving average is just the average of the last m
days including the current day, for days 0 through m + n − 1. Express this as a
convolution. For defining moving averages: assume that the price for days -1, -2,
... the price is 0.
2. Conside A(x) = x + 3, B(x) = 2x + 1. (a) Is ((0,3),(1,4)) a point value
representation for A? (b) For what y is ((0,1),(1,y)) a point value representation
for B? (c) Is ((0,3),(1,4y)) a point value representation for C?

Solution: Proceed only after trying.

by a polynomial i pi x i . The moving average will
P
1. The prices will be represented
Pm−1
be represented by a filter j=0 x i /m. Note that because of the very simple form
of the coefficients the convolution can be easily performed in O(m + n) time. So
this formulation is not particularly useful.
2. (a) Yes. (b) y=2*1+1=3. (c) No. Since C has degree 2 we need 3 point value
pairs to represent it.
Can we evaluate/interpolate quickly?
Naive evaluation method: Evaluating degree n − 1 polynomial at 1 point takes O(n)
time. At 2n − 1 points: O(n2 ) time.

Main Result 1: It is possible to evaluate a degree n − 1 polynomial at n carefully

chosen points in time O(n log n).

Main Result 2: It is possible to construct the coefficients of a degree n − 1 polynomial

given the values at n carefully chosen points in time O(n log n).
Proofs soon.

So evaluating at 2n − 1 points and interpolating from 2n − 1 points can be done in

time O(n log n).

Convolution time: Evaluate + multiply + Interpolate :

O(n log n) + O(n) + O(n log n) = O(n log n)
Polynomial evaluation algorithm: Overview
A simple trick for efficiently evaluating a polynomial at x, −x
I Will take less work that separate evaluations at x and at −x

The trick can be extended recursively!

I Which points to evaluate the polynomial at will become clear as we work this out.

Complex roots of unity will play an important role.

√ √
I ωn = exp 2π −1/n = cos 2π/n + −1 sin 2π/n is a principal nth root of 1.

I ωn0 , ωn1 , ωn2 , . . . , ωnn−1 are all nth roots of 1.

n/2 √ √
I ωn = cos(2π/n)(n/2) + −1 sin(2π/n)(n/2) = cos π + −1 sin π = −1
I nth roots of 1 are square roots of n/2th roots
q of 1.
i 1/2
ωn = exp(2πi/n) = exp(2πi/(n/2)) = ωn/2 i
q
i+n/2
ωn = −ωni = − ωn/2 i
Choosing P
the points at which to evaluate A
ai x i i ai x i
P P
A(x) = i A(−x) = i even ai x − i odd

a2i y i a2i+1 y i
P P
”even(A)”: EA (y ) = i ”odd(A)”: OA (y ) = i

A(x) = EA (x 2 ) + xOA (x 2 ) A(−x) = EA (x 2 ) − xOA (x 2 )

Value of EA , OA at x 2 → value of A at ±x Time O(1)

√
Values of EA , OA at v → value of A at ± v Time O(1)

Values of n/2 coefficient polynomials EA , OA at v0 , v1 , . . . , vn/2−1

→ Values of n coeff poly A at 2 square roots each of v0 , v1 , vn/2−1 Time O(n)

Values of n/4 coeff poly EEA , OEA , EOA , OOA at w0 , w1 , . . . , wn/4−1

→ Values of E , O at 2 square roots of w0 , w1 , . . . , wn/4−1 Time O(n)
→ Value of A at 4 4th roots of each of w0 , w1 , . . . , wn/4−1 Time O(n)

Value of n 1-coeff polynomials {E , O}log

A
n
at u0 → . . . → Val of A at n nth roots of u0 .
log n → steps. Each O(n) time. Total O(n log n) Choose u0 = 1, n = power of 2.
Recap

Evaluate n 1-coefficient polynomials {E , O}log

A
n
at 1. constant polynomials
Evaluate n/2 2-coefficient polynomials {E , O}log n−1 at ±1. degree 1 poly
Evaluate n/4 4-coefficient polynomials {E , O} log n−2 at 4th roots of 1. degree 3 poly
...
Evaluate n/2 coefficient polynomials EA , OA at n/2th roots of 1. degree n/2 − 1 poly
Evaluate n coefficient polynomial A at at nth roots of 1. degree n − 1 poly

Computation in last step: Given values of EA , OA at x 2 = ωn/2

i

2 2 2 2
q = EA (x ) + xOA (x ), A(−x) = EA (x ) − xO(x )
Calculate A(x)
i+n/2
where x = ωn/2 i = ωni , and −x = ωn
i+n/2
Thus we have A(ωni ) = EA (ωn/2
i ) + ω i O(ω i ),
n n/2 A(ωn i ) − ω i O(ω i )
) = EA (ωn/2 n n/2

Now we can write a recursive program.

Evaluate(a0 , a1 , . . . , an−1 , ωn ){ // Returns A[0..n − 1] with A[i] = A(ωni )
1. Base case: if n = 1 return a0
2. E = Evaluate(a0 , a2 , . . . , an−2 , ωn2 ) E [i] = E (ωn2i ) = E (ωn/2
i )

3. O = Evaluate(a1 , a3 , . . . , an−1 , ωn2 ) O[i] = O(ωn2i ) = O(ωn/2

i )

4. for i = 0 to n/2 − 1
5. A[i] = E [i] + ωni O[i]
//Need A[i] = A(ωni ) = E (ωn/2
i ) + ω i O(ω i ) = E [i] + ω i O[i]
n n/2 n

6. A[n/2 + i] = E [i] − ω i O[i]

i+n/2 i ) − ω i O(ω i ) = E [i] − ω i O[i]
//Need A[i + n/2] = A(ωn ) = E (ωn/2 n n/2 n
7. end for
8. Return A // array

T (n) = 2T (n/2) + O(n)

So T (n) = O(n log n)

PolynomialPEvaluation as matrix vector multiplication
n−1 n−1
A(ω i ) = i j ij
P
j=0 aj (ω ) = j=0 aj ω ω = ωn
 
a0
  a1 
.. ..
  
 
.  . 
 .i     .. 
A(ω ) = ω 0 ω i . . . ω ij . . . ω i(n−1)   
  aj 
.. ..
  
 
. .  .. 
 . 
an−1
Interpolation = multiplying lhs by inverse of above matrix, M(ω).
Matrix M(ω) has orthogonal columns! Next
√
Each column has norm n Next

Matrix inverse = n1 M(ω)∗ = n1 M(ω −1 ) Next

Multiplication by inverse is similar to multiplication by original.

Interpolation time is also O(n log n).
Conjugation, orthogonality and norm
ith row of M(ω) is: Ri = [. . . ω ik . . .]

To find its norm, we have to multiply by conjugate transpose

Conjugate of ω is ω −1 Argand diagram
Conjugate of ω ik is ω −ik .

Ri Ri∗ = ω ik (ω −ik ) = n
P
k
√
Hence norm = n.
Pn−1
For i 6= j : Ri Rj∗ = ω ik (ω −jk ) = (i−j)k
P
k k=0 ω
i−j n−1
= (ωωi−j)−1 Sum of geometric series
=0 ω in = ω jn = 1

Hence orthogonal.
Remarks

I Values at roots of unity = Discrete Fourier Transform of coefficients.

I Algorithm is called FFT : Fast Fourier Transform. Cooley-Tukey 1965.
Like many great ideas, it was apparently known to Gauss!
I Many ideas. Multiple representations. Exercising your freedom carefully (where to
evaluate). Divide and conquer.
I Version exists for ring of integers modulo a number. Useful for doing arithmetic.
I Reading: [Dasgupta, Papadimitriou, Vazirani].
Exercises

1. How many recursive calls are executed by the procedure Evaluate for n = 4?
2. Draw the recursion tree showing the arguments to the recursive calls. Assume
a0 , a1 , a2 , a3 = 0, 1, 2, 3.
Please read further only after trying hard.
1. 7
2. Arguments given in parenthesis. i = square root of -1.
(0,1,2,3,i)
(0,2, -1) (1,3,-1)
(0,1) (2,1) (1,1) (3,1)