ESTIMATION OF CONVERSATIONAL QUALITY OF IP TELEPHONY SERVICES

1
ESTIMATION OF CONVERSATIONAL QUALITY OF IP
TELEPHONY SERVICES

• Akira TAKAHASHI
• NTT Service Integration Laboratories
• Service assessment group

2
What are the Quality Factors in VoIP?
conversational MOS
listening MOS
delay
echo
distortion
Network packet loss
Network delay
Coding distortion
Electrical/ acoustic echo
Jitter-buffer overflow
Jitter-buffer delay
IP networks
Conversational factors must be taken into
account.
3
Needs in quality assessment

• Quality of VoIP services should be evaluated in
  terms of subjective quality.
• Subjective quality assessment is timeconsuming
  and expensive.
• Subjective quality assessment is not applicable
  to in-service quality management, which requires
  real-time assessment.

Objective means for estimating subjective quality
4
What do we have right now?
Need to develop an opinion model that is accurate
enough to be applied to benchmarking and/or
management.
5
What is the E-model?

• computational model for estimating conversational
  quality defined in ITU-T Recommendation G.107.
• based on NTTs OPINE technology developed in
  the 1980s.
• 21 parameters represent terminal and network
  characteristics.

R Ro Is Id Ie,eff A
Advantage factor
distortion
Equipment impairment factor
echo, delay
loudness
Delay impairment factor
noisiness
Simultaneous impairment factor
Basic signal-to-noise ratio
6
What is the goal of our study?
Delay, echo, and noise measurement
Opinion model
P.862
Objective quality models
P.563
P.VTQ
G.113
Listening quality index
Develop a method that integrates various quality
measurement technologies and estimates
conversational quality.
7
Proposed model
Modeling effects of delay and echo
Echo loudness
Echo impairment
Idte
Conversational MOS
One-way delay

Delay impairment
Idd
Codec
Combined impairment
LQd
Speech distortion
Ie,eff
Packet loss rate

Modeling interaction between effects of delay and
distortion
Noise impairment
Noise level
Modeling effects of noise
8
Which input did we use?
9
How well does the proposed method
work? -Experiment 1-

• Subjective assessment method conversational ACR
• Conversational task free conversation
• Duration of conversations 1 min./condition
• Number of subjects 40 Japanese
• Terminal characteristics normal handsets
• Variables
• codec G.711 PLC, GSM-EFR, G.723.1
• one-way delay 100, 500 ms
• talker echo (TELR) 30, 65 dB
• noise floor -62.2, -55.5, -51.8 dBm0p

10
How well does the proposed method
work? -Experiment 2-

• Subjective assessment method conversational ACR
• Conversational task free conversation
• Duration of conversations 1 min./condition
• Number of subjects 40 Japanese
• Terminal characteristics normal handsets
• Variables
• codec GSM-EFR, G.723.1
• one-way delay 150/160, 250, 350, 500 ms
• talker echo (TELR) 30, 45, 65 dB
• packet loss rate 0, 3, 5, and 10 (random loss)

11
How well does the proposed method
work? -Experiment 3-

• Subjective assessment method conversational ACR
• Conversational task free conversation
• Duration of conversations 1 min./condition
• Number of subjects 40 Japanese
• Terminal characteristics normal handsets
• Variables
• codec G.711, G.711 PLC, G.729
• one-way delay 100 - 400 ms
• talker echo (TELR) 35 to 55 dB
• packet loss rate 0, 1, 3, and 5 (random loss)

12
How well does the proposed method work? -
Scatter plot -
Exp. 1 Exp. 2 Exp. 3
R 0.75
R 0.83
R 0.86
Subjective MOS
Subjective MOS
Subjective MOS
Estimated MOS
Estimated MOS
Estimated MOS
(b) Proposed (original)
(a) E-model
(c) Proposed (enhanced)
The proposed model outperforms the E-model, so
enhancement works well.
13
How well does the proposed method work? -
Cross-correlation -
14
Which input did we use?
15
Proposed model
Modeling effects of delay and echo
Echo loudness
Echo impairment
Idte
Conversational MOS
One-way delay

Delay impairment
Idd
Codec
Combined impairment
LQd
Use of P.862 PESQ and P.834
Speech distortion
Ie,eff
Packet loss rate

Modeling interaction between effects of delay and
distortion
Noise impairment
Noise level
Modeling effects of noise
16
Does the proposed method work with objective
quality measures like P.862?
R 0.85
Subjective MOS
Estimated MOS
Yes! The proposed method in conjunction with
P.862 maintains the same accuracy as the method
with G.113 App. I.
17
In conclusion, ..
To develop an opinion model that can be applied
to quality benchmarking and management, we.

• Enhanced the originally proposed model
• Modeled the effect of noise floor.
• Modified the function that takes into account the
  interaction between delay and speech distortion.
• Verified that the proposed enhancement is
  effective
• We conducted extensive subjective quality
  assessment in a conversational way.
• The enhanced model outperformed the original
  model, as well as the E-model
• The enhanced model also works well with the
  objective quality model (PESQ).

18
Thank you for your attention.

• Akira TAKAHASHI
• NTT, Japan

19
ESTIMATION OF CONVERSATIONAL QUALITY OF IP
TELEPHONY SERVICES

• Akira TAKAHASHI
• NTT, Japan