Audio and Speech Processing Topic4 Acoustic Echo Cancellation

1
Audio and Speech ProcessingTopic-4Acoustic
Echo Cancellation

• Marc Moonen/Koen Eneman/Geert Rombouts
• Dept. E.E./ESAT, K.U.Leuven
• marc.moonen_at_esat.kuleuven.be
• homes.esat.kuleuven.be/moonen/

2
Outline

• Introduction
• Acoustic Echo Cancellation (AEC) problem
  applications
• Speech signals and acoustic channels
• Adaptive filtering algorithms for AEC
• NLMS
• Frequency domain adaptive filters
• Affine Projection Algorithm (APA)
• Control algorithm
• Post-processing
• Loudspeaker non-linearity

3
Introduction

• AEC problem/applications
• Suppress echo
• to guarantee normal conversation conditions
• to prevent the closed-loop system from becoming
  unstable
• Applications
• teleconferencing
• hands-free telephony
• handsets

4
Introduction

• AEC standardization
• ITU-T recommendations (G.167) on acoustic echo
  controllers state that
• the input/output delay of the AEC should be
  smaller than 16 ms
• far-end signal suppression should reach 40 dB for
  teleconferencing systems and 45 dB for hands-free
  telephony, if no near-end signal is present
• in presence of near-end signals the suppression
  should be at least 25 dB
• many other requirements

5
Introduction

• Speech Signals
• wideband (several octaves)
• highly non-stationary
• large dynamic range
• sometimes noise-like, sometimes quasi periodic,
  sometimes silence
• mean frequency envelope of about 6dB/octave

6
Introduction

• Room Acoustics (I)
• Propagation of sound waves in an acoustic
  environment results in
• signal attenuation
• spectral distortion
• Propagation can be modeled
• quite well as a linear filtering
• operation
• Nonlinear distortion mainly stems from the
  loudspeakers. Its effect is
• typically of second order, therefore (often)
  not taken into account.

7
Introduction
Room Acoustics (II) The linear filter modeling
the acoustic path H(z) between loudspeaker and
microphone is represented by the acoustic impulse
response

• Observe that
• first there is a dead time
• then come the direct path impulse
• and some early reflections, which
• depend on the geometry of the room
• finally there is an exponentially decaying tail
  called reverberation, coming from multiple
  reflections on walls, objects,... The amount of
  reverberation mainly depends on the reflectivity
  of the room and not on the geometry.

8
Introduction

• Room Acoustics (III)
• To characterize the reflectivity of a recording
  room the reverberation
• time RT60 is defined
• as the time which the sound pressure level or
  intensity needs
• to decay to - 60dB of its original value.
• for a typical office room RT60 is between 100 and
  400 ms. for a church RT60 can be several
  seconds
• Acoustic room impulse responses are highly
  time-varying !!!!

ESAT speech laboratory RT60 ? 120 ms
Begijnhofkerk Leuven RT60 ? 3730 ms
Original speech signal
9
Introduction

• Acoustic Impulse Response FIR or IIR ?
• If the acoustic impulse response is modeled as
• an FIR filter ? many hundreds to several
  thousands of filter taps are needed
• an IIR filter ? filter order can be reduced, but
  still hundreds of filter coeffs (num. denom.)
  may be needed (sigh!)
• Hence FIR models are typically used in practice
  because...
• they are guaranteed to be stable.
• in a speech comms set-up the acoustics are highly
  time-varying, hence adaptive filtering techniques
  are called for (see DSP-2)
• FIR adaptive filters simple adaptation rules,
  no local minima,..
• IIR adaptive filters more complex adaptation,
  local minima

10
Introduction
Conventional AEC Techniques

• directional loudspeakers and microphones
• voice controlled switching, loss control
• howling control stability margin improvement of
  the closed loop by
• frequency shifting
• using comb filters
• removing resonant peaks
• nonlinear post-processing, e.g. center clipping

11
Outline

• Introduction
• Acoustic Echo Cancellation (AEC) problem
  applications
• Speech signals and acoustic channels
• Adaptive filtering algorithms for AEC
• NLMS
• Frequency domain adaptive filters
• Affine Projection Algorithm (APA)
• Control algorithm
• Post-processing
• Loudspeaker non-linearity

12
Adaptive filtering algorithms for AEC

• Basic set-up
• Adaptive filter produces a model for acoustic
  room impulse response
• an estimate of the echo contribution in
  microphone signal, which is
• then subtracted from the microphone signal
• Thanks to adaptivity
• time-varying acoustics can be tracked
• performance superior to performance of
  conventional techniques

13
Adaptive filtering algorithms for AEC

• Algorithms to be discussed
• (Fast) Recursive Least Squares
• Normalized LMS
• Subband adaptive filtering
• Frequency-domain adaptive filter (FDAF)
• Partitioned block frequency-domain adaptive
  filter (PB-FDAF)
• Affine projection algorithm (APA)
• Fast affine projection algorithms

14
Adaptive Filtering Algorithms RLS

• Standard RLS (see DSP-II)
• Complexity depends quadratically on the
  filter length N ? RLS currently not taken into
  consideration for AEC
• Fast-RLS (see DSP-II)
• Complexity depends linearly on N. Cheapest
  Fast-RLS variants are still roughly 4 times more
  expensive than LMS ? may gain importance in the
  field of AEC in coming years

15
Adaptive Filtering Algorithms NLMS

• NLMS update equations
• in which
• N is the adaptive filter length, ? is the
  adaptation stepsize,
• ? is a regularization parameter and k is the
  discrete-time index

16
Adaptive Filtering Algorithms NLMS

• Pros and cons of NLMS
• cheap algorithm O(N)
• small input/output delay ( 1 sample)
• for colored far-end signals (such as speech)
  convergence of the NLMS algorithm is slow
  
• (cfr lambda_max versus lambda_min, etc., see
  DSP-II)
• large N then means even slower convergence
• NLMS is thus often used for the cancellation of
  short echo paths

17
Adaptive Filtering Algorithms

• As some input/output delay is acceptable in AEC
  (cfr ITU..), algorithms can be derived that are
  even cheaper than NLMS, by exchanging
  implementation cost for extra processing delay,
  sometimes even with improved performance
• Subband adaptive filtering (SBAF)
• Frequency-domain adaptive filtering (FDAF)
• Partitioned Block FDAF (PB-FDAF)

cost reduction optimal (stepsize) tuning for
each subband/frequency bin separately results
in improved performance
18
Adaptive Filtering Algorithms SBAF

• M (simpler (?)) subband modeling problems instead
  of one (more complicated) fullband modeling
  problem
• N-fold downsampling results in complexity
  reduction
• (ideally N-fold shorter subband filters
  (?))
• Perfect reconstruction analysis-synthesis filter
  bank guarantees distortion-free desired near-end
  speech signal (see DSP-II)
• 
  (M4, N3)

19
Adaptive Filtering Algorithms Block-LMS

• To derive the frequency-domain adaptive filter
  the BLMS algorithm is considered first

in which
N is filter taps, L is block length, n is block
time index
20
Adaptive Filtering Algorithms Block-LMS

• Both the BLMS convolution and correlation
  operation are computationally demanding. They can
  be implemented more efficiently in the frequency
  domain using fast convolution techniques, i.e.
  overlap-save/overlap-add

convolution
overlap-save
correlation
with
DFT matrix
21
Adaptive Filtering Algorithms FDAF

• Overlap-save FDAF

Will only work if
(M is FFT-size)
22
Adaptive Filtering Algorithms FDAF

• Typical parameter setting for the FDAF
• FDAF is functionally equivalent to BLMS
• FDAF is significantly cheaper than (B)LMS
• for a typical parameter settingIf N1024
  
• - Input/output delay is equal to 2L-12N-1,
  which may be unacceptably large for realistic
  parameter settings e.g. if N1024 and
  fs8000Hz ? delay is 256 ms !

(estimate only, in practice lt19)
23
Adaptive Filtering Algorithms PB-FDAF

• Overlap-save PB-FDAF N-tap full-band filter
  split into (N/P) filter sections, P-taps each,
  then apply overlap-save to each section, etc. (P
  takes the place of N).

24
Adaptive Filtering Algorithms PB-FDAF

• Typical parameter setting
• PB-FDAF is intermediate between LMS and FDAF
  (P/N1)
• PB-FDAF is functionally equivalent to BLMS
• PB-FDAF is cheaper than LMS If N1024,
  PL128, M256
• Input/output delay is 2L-1 which can be chosen
  small, in the example above the delay is 32 ms,
  if fs8000Hz
• used in commercial AECs

(estimate)
25
Adaptive Filtering Algorithms PB-FDAF

• PS Instead of a simple stepsize ?, subband
  dependent stepsizes can be applied
• stepsizes dependent on the subband energy
  (subband normalization)
• convergence speed increased at only a small extra
  cost
• PS PB-FDAF algorithm can be simplified by
  leaving
• out of the weight updating equation
  (unconstrained updating)

26
Adaptive Filtering Algorithms APA
Affine Projection Algorithm intermediate
between RLS and NLMS, complexity- as well as
performance-wise
NLMS (delta0)
APA
if ?1
P last a-posteriori errors are 0
a-posteriori error is 0
27
Adaptive Filtering Algorithms APA
NLMS (P1) Graphical Representation
optimal filter
?1
initial filter coefficients
28
Adaptive Filtering Algorithms APA
APA (P2) Graphical Representation
?1
29
Adaptive Filtering Algorithms APA
Problem with APA near-end noise amplification
is echo-signal
is near-end noise
orthogonal
contains sorted singular values on diagonal
, multiplied by , appears as noise in the
filter weights 
Solution replace by
in update formula
(regularization, similar to delta in
NLMS-formula)
30
Adaptive Filtering Algorithms APA
Effect on near-end noise amplification
Smaller if more regularization
Effect on adaptation speed
Slower if more regularization
31
Adaptive Filtering Algorithms Fast-APA
APA complexity, i.e.O(P.N), may be reduced to
(roughly) LMS complexity, i.e. O(N)
1. Recursive  error vector calculation
(delta0)
Ex mu1, then lower components were already
nulled _at_ time k-1
2. Delayed filter vector update accumulate
filter adaptations based on vector x_k, apply
only when x_k leaves  the X_k matrix (at
time kP-1)
Ignore steps 2 3
3. Recursive updating scheme for inverse in
32
Outline

• Introduction
• Acoustic Echo Cancellation (AEC) problem
  applications
• Speech signals and acoustic channels
• Adaptive filtering algorithms for AEC
• NLMS
• Frequency domain adaptive filters
• Affine Projection Algorithm (APA)
• Control algorithm
• Post-processing
• Loudspeaker non-linearity

33
Control Algorithm

• Adaptation speed (? ) should be adjusted
• to the far-end signal power, in order to avoid
  instability of the adaptive filter? stepsize
  normalization (e.g. NLMS)
• to the amount of near-end activity, in order to
  prevent the filter to move away from the optimal
  solution (see DSP-II)
• ? double-talk detection

Double talk refers to the situation where both
the far-end and the near-end speaker are active.
34
Control Algorithm

• 3 modes of operation
• near-end activity (single or double talk) (Ed
  large) ?
• no near-end activity, only far-end activity (Ex
  large, Ed small) ?
• no near-end activity, no far-end activity (Ex
  small, Ed small) ?

? FILT
? FILTADAPT
? NOP

• Ex is short-time energy of
• the far-end signal (p.52)
• Ed is short-time energy of
• the desired signal

35
Control Algorithm

• Double-talk Detection (DTD)
• Difficult problem detection of speech during
  speech
• Desired properties
• limited number of false alarms
• small delay
• low complexity
• Different approaches exist in the literature
  which are based on
• energy
• correlation
• spectral contents

36
Control Algorithm

• Energy-based DTD
• Compare short-time energy of far-end and
  near-end channel Ex and Ed
• method 1 If Ed gt ? Ex ? double talk ? is a
  well-chosen threshold
• method 2

If ? gt 1 ? double talk
37
Post-processing

• Error suppression obtained in practice will
• be limited to /- 30 dB, due to
• non-linearities in the signal path (loudspeakers)
• time-variations of the acoustic impulse responses
• finite length of the adaptive filter
• local background noise
• failing double-talk detection
• A post-processing unit is added to further reduce
  the residual signal, e.g. center clipping

38
Loudspeaker Non-linearity

• If loudspeaker non-linearity is significant
  (e.g. consumer applications), then this should be
  compensated for
• Solution-1 Non-linear model (fixed) in
  cancellation path

x
Non-linear model
Adaptive filter
y
d
e
39
Loudspeaker Non-linearity

• Solution-2 Inverse non-linear model in forward
  path
• advantageif successful, also improves
  loudspeaker
• characteristic/sound
  quality..

x
Inverse non-linear model
Adaptive filter
y
d
e
40
PS AEC Application

• Selective volume control

Control PA volume independently of
environmental noise.
This is AEC-type problem, but now filter-output
is of interest. 44.1kHz instead of 8kHz (times
5), hence complexity times 25 ! (explain)