FYP0202 Advanced Audio Information Retrieval System - PowerPoint PPT Presentation
1 / 13
Title: FYP0202 Advanced Audio Information Retrieval System
1
FYP0202Advanced Audio InformationRetrieval
System
- By Alex Fok, Shirley Ng
2
Outline
- Overview
- Read in the raw speech
- MFCC processing
- Detect the audio scene change
- Audio Clustering
- Interleave Audio Clustering
- Conclusion
3
Overview
- Automatic segmentation of an audio stream and
automatic clustering of audio segments have quite
a bit of attention nowadays. - Example, in the task of automatic transcription
of broadcast news, the data contains clean
speech, telephone speech, music segments, speech
corrupted by music or noise.
4
Overview (cont)
- We would like to SEGMENT the audio stream into
homogenous regions according to speaker identity. - We would like to cluster speech segments into
homogeneous clusters according to speaker
identity.
5
Step1Read in the raw speech
- Read in a mpeg file as input
- Convert the file from .mpeg format to .wav format
- Because the MFCC library only process on .wav
file
6
Step2MFCC processing
- A wav is viewed as frames, each contains
different features - We make use of the MFCC library to convert the
wav to MFCC features for processing - We extract 24 features for each frames
- The result are stored in feature vectors
Frame1
Frame 2
Frame 3
7
Step3 Detect the audio scene change
- Make use of the feature vector to detect the
audio scene change - The input audio stream will be modeled as
Gaussian process - Model selection criterion called BIC (Bayesian
Information Criterion) is used to detect the
change point
8
Step3 Detect the audio scene change
- Denote Xi (i 1,,N) as the feature vector of
frame i - N is the total number of frame
- mi mean of mean vector of frame i
- ?i full covariance matrix of frame i
- R(i) N log ? - N1 log ?1 - N2 log ?2
- ?, ?1, ?2 are the sample covariance matrices from
all the data, from x1,,xi, from xi1,,Xn
respectively
9
Step3 Detect the audio scene change
- BIC(i) R(i) constant
- If there is only one change point, then the frame
with highest BIC score is the change point - If there are more than one change point, just
simple extend the algorithm
10
Step 4Audio Clustering
- As we want to speed up the audio detecting, so we
just roughly find the change point. - As a result, there maybe some wrongly calculated
change point. - In this part, we try to combine the wrongly
segmented neighbor segments - Compare with neighbor segments, if they are
speech of the same person, then combine it.
11
Step5Interleave Audio Clustering
- Group all the segments of the same speaker into
one node. - Before
- After
Speaker 1
Speaker 1
Speaker 2
Combined Speaker1
Speaker 1
Speaker 1
Speaker 2
12
Conclusion
- We would like to make a precise and speedy engine
that recognize the identity of speaker in a wave
file. - We would like to group the same speaker in the
wave.
13
Conclusion (cont)
- Instead of making local decision based on
distance between fixed size sample, we expand the
decision as wide as possible - Avoid the respectively calculation by using
dynamic programming. - Detection algorithm can detects acoustic changing
points with reasonable detestability.
Recommended
CrystalGraphics Presentations
