Design Process for Auditory Interfaces

1
Design Processfor Auditory Interfaces

• Sylvain Daudé and Laurence Nigay
• HCI team of the CLIPS-IMAG Lab.(Grenoble,
  France)

2
Purpose

• Auditory display interlinking location
• Unified design framework for auditory interfaces

Information visualization
Computer music
Sonification
Psycho-acoustics
HCI
DSP
3
Motivations

• Design framework
• existing for data visualization...
• ... but not for data sonification
• Double interest of a design framework
• classification of existing results and systems
• help to the auditory interfaces designer

4
Visualization processChi 1998
Data transformation
Documentslist
Analytical abstraction Data view
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Visualization transformation
Visualization abstraction Abstract visual space
Visual realization transformation
View Space view
5
Visualization process Chi 1998
Data
Data transformation
Documentslist
Analytical abstraction Data view
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Visualization transformation
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Visualization abstraction Abstract visual space
2D space
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Visual realization transformation
View Space view
6
Visualization process Chi 1998
Data
Data transformation
Analytical abstraction Data view
Visualization transformation
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Visualization abstraction Abstract visual space
2D space
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Bienvenue àIHM2002 Bienvenue àIHM2002
Visual realization transformation
View Space view
Deformed view
7
Sonification process
Data transformation
Analytical abstraction Data view
Pixels columns
Sonification transformation
Sonic abstraction Abstract sound space
Aud. display transformation
Sonic rendering
8
Sonification process
Data
Data transformation
Analytical abstraction Data view
Gray level volumeOrdinate pitch
Sonification transformation
. . .
Sonic abstraction Abstract sound space
Sounds sequence
Aud. display transformation
Sonic rendering
9
Sonification process
Data
Data transformation
Analytical abstraction Data view
Sonification transformation
. . .
Sonic abstraction Abstract sound space
Sounds sequence
Aud. display transformation
Sonic rendering
10
Data transformation
Data

• Common with visualization
• Dependent on
• the data
• the data semantics
• the user task
• ex AROMA

Data transformation
Analytical abstraction Data view
Sonification transformation
Sonic abstraction Abstract sound space
Aud. display transformation
Sonic rendering
11
Sonification transformation abstract sound space
Data

• Abstract sound space
• euclidean space
• 1 point if no spatialization
• abstract sound
• duration
• during which acoustical cues vary
• Two sonification levels
• 1 data element -gt 1 abstract sound
• Coordination of the abstract sounds

Data transformation
Analytical abstraction Data view
Sonification transformation
Sonic abstraction Abstract sound space
Aud. display transformation
Sonic rendering
12
Sonification transformation abstract sound space
Data

• Abstract sound space
• euclidean space centered on user
• 1 point if no spatialization
• abstract sound
• duration
• acoustical cues varying in time
• Two sonification levels
• 1 data element -gt 1 abstract sound
• Coordination of the abstract sounds

Data transformation
Analytical abstraction Data view
Sonification transformation
Sonic abstraction Abstract sound space
Aud. display transformation
Sonic rendering
13
Sonification transformation representing a
single element

• 3 independent mappings

Data element
Abstract sound
- semantics - structure - value
- semantics - structure - value
1
2
3
14
Sonification transformation representing a
single element

• 3 independent mappings

• Special cue spatial position
• Semantics independent from semantics of sound
• Analogical spatialization ?
• Analogy with a phenomenon
• Sonic cathedral
• Non-sonic Diary in the Sky

Data element
Abstract sound
- semantics - structure - value
- semantics - structure - value
1
2
3
15
Sonification transformation representing a set
of data

• Two sonification levels
• 1 single data element ? 1 sound
• 1 set of data ? 1 set of sounds
• Coordination of the sounds of the sound set
• in time
• in space

Space
Stereosound
Diary in theSky
Different places
Pixelscolumn
Speechsynthesis
Same place
Time
Simultaneous
Alternate
16
Auditory display transformationphysical device

• 4 steps transformation
• (i) chosing a view point
• (ii) sound space -gt device parameters
• (iii) device parameters -gt sound signal
• (iv) sound signal -gt physical sound

Data
Data transformation
Analytical abstraction Data view
Sonification transformation
Sonic abstraction Abstract sound space
Aud. display transformation
Sonic rendering
17
Auditory display transformationphysical device

• (iv) Physical device
• sound precision
• degree of spatiality
• geometrical distribution of the loudspeakers
• degree of sharing
• liberty of movement
• degree of immersion
• Ex SoundbeamNeckset

Data
Data transformation
Analytical abstraction Data view
Sonification transformation
Sonic abstraction Abstract sound space
Aud. display transformation
Sonic rendering
18
Contributions a. classification

• Example the  earcons category 

19
Contributions b. help for the designer

• Identification of design steps
• Identification of design parameters for each step
  
• guide for the design
• foundation for design heuristics
• context-aware sonifications

20
Perspectives

• Design heuristics
• Links with a software architecture
• Platform for sonification design based on this
  process

21
Thank you for your attention

• Any question ?

22
(No Transcript)