Interactively Evolving User Interfaces

1
Interactively Evolving User Interfaces

• Juan Quiroz
• Committee
• Dr. Sushil Louis
• Dr. Sergiu Dascalu
• Dr. Swatee Naik

2
Outline

• Motivation
• Related work GAs, IGAs, UI Design
• User fatigue in IGAs
• UI evolution
• Experiments
• Results
• Future Work

3
Motivation

• User interface design is a complex, expensive,
  time consuming process
• Iterative process
• Users and contexts of use are numerous
• Streamline and improve UI design
• End-user customization

4
IGA for UI Evolution

• IGA to explore the space of UIs
• Creativity and insight
• Evolution is guided by both the user preferences
  and coded guideline metrics
• Pick best and worst

5
Genetic Algorithms

• Population based search technique
• Natural selection
• Survival of the fittest

6
Interactive Genetic Algorithms (IGAs)

• Fashion design (Kim 2000)
• Micromachine design (Kamalian 2005)
• Music, editorial design (Takagi 2001)
• Traveling salesman problem (Louis 1999)

7
User Evaluation Subjective Fitness
100
75
15
80
20
8
38
53
82
8
User Evaluation Ranking
9
User Evaluation Tournaments
A
B
Fitness A gt Fitness B
10
User Fatigue in IGAs

• GAs tend to rely on
• Large populations
• Many generations
• Suboptimal solutions
• Noisy fitness landscapes

11
Alleviating User Fatigue

• Use small population sizes
• Display a subset of population
• Accelerate convergence through prediction (Llora
  2005)

12
UI Design Support

• GUI toolkits and libraries

13
Guidelines of Style

• Microsoft, Apple, Java, KDE, Gnome
• Define a common look and feel for applications
• Discuss the use of color, layout of widgets, the
  use of fonts
• Interpreting the guidelines is itself a challenge
• Too vague or too specific

14
Ambiguity in Guidelines

• Use color to enhance the visual impact of your
  widgets Apples Human Interface Guidelines

15
XUL User Interfaces

• XML User Interface Language
• Mark-up language for UIs
• Buttons, textboxes, sliders
• menubars, toolbars

16
Related Work UI Evolution

• Evolution of style sheets (Monmarché et al)
• Font and links color, paragraph spacing, font
  family, font decoration
• We allow both the user and the coded guidelines
  to guide the evolution

17
Related Work User Fatigue

• SVMs to combat user fatigue (Llora 2005)
• Kamalian et al. (2005)
• User evaluation every tth generation
• Demote or promote reaction to individuals
• Validity constraint is used to determine viable
  and meaningful designs

18

• Interactive Genetic Algorithms in UI Design

19
Lagoon MoveTo Panel
20
UI Representation

• Two chromosomes
• Widget chromosome
• Layout chromosome

21
Genetic Operators for UI IGA

• Single point crossover
• Bit flip mutation
• PMX partial mapped crossover
• Swap mutation

22
Widget Color

• RGB color model
• Red (255, 0, 0), Green (0, 255, 0), Blue
  (0, 0, 255)
• 224 color space for each widget
• HSV
• Same gamut as RGB
• No significant efficiency difference in RGB and
  HSV

23
Fitness Evaluation

• Ask the user to select the best and worst UIs
  from the subset displayed
• Interpolate the subjective fitness of individuals
  in population
• Compute the objective metrics taken from
  guidelines of style
• Fitness w1 subjective
• w2 objective

24
1 Fitness Evaluation
Best
Worst
25
2 Subjective Fitness Interpolation

• Compare to best
• Compare to worst

26
3. Objective Fitness Computation

• High contrast between widget colors and
  background color
• Low contrast between widget colors

27

• Fitness w1 subjective fitness
• w2 objective fitness

28
Research Questions

• Which selection type is the most effective for
  this problem?
• Who from the population do we display for user
  evaluation?
• How often should we ask for user input?

29
Experimental Setup

• Greedy simulated user
• 30 independent runs, 200 generations
• Population size of 100
• Roulette wheel vs. tournament
• Display method comparison best 10, random 10,
  best 5 and worst 5
• User input every 1, 5, 10, 20, 40, 80 generations

30
Which selection type is the most effective for
this problem?
31
Who from the population do we display for user
evaluation?
Fitness Convergence
Convergence to Blue UIs
32
How often do we ask for user input?
User input every tth generation Fitness
convergence
High values of t
Low values of t
33
How often do we ask for user input?
User input every tth generation Convergence to
blue UIs
High values of t
Low values of t
34
Experimental Setup Actual Users

• Three users
• 30 generations
• Pick the one they like the best and the one they
  like the least
• 5 sessions
• User input every 1, 3, 5, 10, 15 generations
• 30, 10, 6, 3, and 2 user evaluations respectively

35
Results
36
What leads to the drop in average performance?

• Two sessions with a user
• Comparison to user selected worst turned on
• Comparison to user selected worst turned off
• Always pick the same UI as the best
• Ask for user input every 3 generations

37
What leads to the drop in average performance?
38
Generated UIs Simulated User
39
Generated UIs Simulated User
40
Generated UIs User3
41
Generated UIs User3
42
Future Work

• Ask user to select the best/worst UI only
• Varying the frequency of user input during a
  session
• Convergence acceleration with neuroevolution or
  SVMs
• Integration with a GUI toolkit or library
• User studies!
• Task completion
• Explore color representations
• Specify the type of data that needs to be
  represented

43
Contributions

• We can use IGAs to evolve UIs
• Our simulated user and actual users are able to
  effectively bias the evolution of UIs
• UIs reflect coded guidelines of style
• Reduce user fatigue
• Interpolation technique
• Asking for less user input

44
Demo
45
Questions?

• www.cse.unr.edu/quiroz
• quiroz_at_cse.unr.edu