Scribd
Upload
44 views2 pages

Adaptive Systems - Problem Class 4: The LMS Algorithm

This document contains 3 problems related to adaptive filtering and the LMS algorithm. Problem 1 involves deriving the gradient descent algorithm, deriving the LMS algorithm from gradient descent using instantaneous estimates, and showing that LMS behaves like gradient descent on average. Problem 2 looks at properties of gradient descent for a white noise input, introducing misalignment vector, deriving a stability bound on step size, and convergence time constants. Problem 3 generalizes to a non-white noise input, using eigenvalue decomposition to represent the autocorrelation matrix and decoupling modes of gradient search, finding new stability bounds and time constants.

Uploaded by

Khanh Nam
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
44 views2 pages

Adaptive Systems - Problem Class 4: The LMS Algorithm

This document contains 3 problems related to adaptive filtering and the LMS algorithm. Problem 1 involves deriving the gradient descent algorithm, deriving the LMS algorithm from gradient descent using instantaneous estimates, and showing that LMS behaves like gradient descent on average. Problem 2 looks at properties of gradient descent for a white noise input, introducing misalignment vector, deriving a stability bound on step size, and convergence time constants. Problem 3 generalizes to a non-white noise input, using eigenvalue decomposition to represent the autocorrelation matrix and decoupling modes of gradient search, finding new stability bounds and time constants.

Uploaded by

Khanh Nam
Copyright
© Attribution Non-Commercial (BY-NC)
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 2

http://www.spsc.tugraz.

at/courses/adaptive

Adaptive Systems - Problem Class 4 The LMS Algorithm


Signal Processing and Speech Communication Laboratory Graz University of Technology Inffeldgasse 16c/II http://www.spsc.tugraz.at May 9, 2007

Problem 4.1 - Relation of the gradient descent and the LMS algorithm
w[n] x[n] H (z ) [n] d

+
d[n]

c[n]

y [n]

e[n]

Figure 1: System identication. The shown system identication scenario will be used to derive the relations between the gradient descent and the LMS algorithm. (a) Derive the gradient descent algorithm. (b) Starting from the gradient descent algorithm, derive the LMS algorithm by using instantaneous estimates for the autocorrelation matrix Rxx and the cross-correlation vector p. (c) Start from the LMS algorithm and show that the LMS algorithm behaves on average as the gradient descent algorithm, i.e., E {c[n]} of the LMS = c[k ] of the gradient descent algorithm.

http://www.spsc.tugraz.at/courses/adaptive

Problem 4.2 - Gradient descent algorithm for a white-noise input


We will investigate properties of the gradient descent algorithm, since the LMS algorithm behaves on average as the gradient descent algorithm. (a) Introduce the misalignment vector v[n] in the gradient descent algorithm. (b) Derive a stability bound on the step size paramter . (c) Derive the convergence time contants k .

Problem 4.3 - Gradient descent algorithm for a non white-noise input


(a) Use the eigenvalue decomposition to represent the autocorrelation matrix Rxx . (b) Decouple the modes of the gradient serach algorithm. What is the stability bound now? What are the time-constants k ?

You might also like