Understanding Visualization through Spatial Ability Differences

1
Understanding Visualization through Spatial
Ability Differences

• Maria C. Velez, Deborah Silver and Marilyn
  Tremaine
• Rutgers University
• 2005

2
What this Talk is About

• Different people have been shown to have a lot of
  trouble with 3D visualizations
• To investigate this issue further, we ran an
  experiment comparing measured spatial skills to
  abilities to understand visualizations
• The results suggested key problems individuals
  were having and ways in which we can make the
  visualizations understandable by a larger audience

3
Motivation

• Issues in Visualization Understanding

Classic 3D visualizations (2D projection and
slices) have been found to be suboptimal for
tasks like understanding shape and 3D space
layout.
Examples of conventional visualization displays
used in medical and weather imaging
4
Previous Solutions to Visualization Difficulties
5
Focus What Makes a Visualization Difficult?

• We use differences in human spatial abilities to
  understand the problems that affect peoples
  understanding of a visualization
• Controlling for human variability makes effects
  detectable
• Looking at extremes helps us understand normal
  behavior
• Our questions
• Does everyone solve problems similarly?
• Do they make the same error?
• How does diversity in the population affect
  performance in the visualization?
• Do the solutions proposed help everyone equally?

6
Research Approach

• Select a set of cognitive skills that are likely
  to play a role in visualization understanding
• Measure these cognitive skills with standardized
  tests using a group of subjects selected for
  their variability
• Measure the level of visualization understanding
  of the subjects (via one type of prototypical
  visualization test)
• Match the visualization performance results to
  the standardized test results
• Examine the properties of the visualization for
  both successful and unsuccessful comprehensions
  for each spatial ability subgroup
• Examine the error distributions of the wrong
  answers for each spatial ability subgroup

7
Human Spatial Abilities

• What are spatial abilities?
• Skills involving the retrieval, retention and
  transformation of visual information in a spatial
  context.

Spatial Orientation Spatial Location Memory
Targeting Spatial Visualization Disembedding
Spatial Perception

• Are there other relevant cognitive factors ?

Visual Memory Perceptual Speed
8
Standardized Tests

• Measuring spatial abilities
• We measured spatial abilities through the Kit of
• Factor-Referenced Cognitive Tests available at
  ETS.

• Spatial Orientation Cube Comparison Test

• Spatial Visualization Paper Folding Test

• Disembedding Hidden Patterns Test

• Visual Memory Shape Memory Test

• Perceptual Speed Identical Figures Test

9
The Visualization Test

• Goal Examine the comprehension of a
  prototypical visualization orthogonal
  projection
• Basic visualization without bells and whistles
• Easy to learn by untrained experiment
  participants
• Use geometrical (geon-like and compounded) and
  common realistic objects

10
The Visualization Test Screen 1
Mentally form an image of the object and its
alignment
11
The Visualization Test Screen 2
Select the object that represents the object
creating the projections
This should be your answer
12
Experiment Design

• Measures of performance
• Accuracy percentage of correct answers
• Analysis Time Time spent analyzing the objects
  projections (seconds)
• Selection Time Time spent selecting the answer
  (seconds)
• Analysis Time and Selection Time are measured
  independently
• Experiment Participants (selected for
  variability)
• 56 paid participants, 50 percent female
• Average age 21 years (range 18 to 31 years)
• 84 undergraduate, 16 graduate students

13
Experimental Method

• Experiment Procedure
• Five paper-based cognitive factor tests
• General instructions and five practice questions
• Computer-based visualization test 60 minutes to
  complete 38 questions
• Debriefing explaining the purpose of the
  experiment

14
Road Map to Analysis
Correlation
Correlation
Visualization properties results
Differences
Spatial ability groups results
Analysis of subjects errors (Case by case
analysis)
15
Expected Relation Between Spatial Ability and
Visualization Performance
Positive correlation - Negative correlation
16
Spatial Test Results

• Analysis
• Pearson correlation analysis between performance
  and scores in standardized tests

• Results

Not Expected
Expected

• Implications
• Visualization comprehension on diverse
  populations affected by spatial ability diversity
• Paper tests were time constrained which may have
  affected the time correlations

17
Visualization Properties Results

• For the objects properties (i.e. surfaces, edges
  and vertices) we calculated
• Total count in the original 3D object
• Distinct properties that would be visible in a
  wireframe rendering of the projection.
• Visible properties in a uniformly shaded object

Visible edges 4
Distinct edges 7
Total count of edges 12
18
Visualization Properties Results

• Analysis
• Pearson correlation analysis between performance
  and property counts and ratios

• Results

• Implications
• The hidden geometric properties make
  visualization understanding cognitively harder
  and thus, more time consuming
• Rotation of objects and animation will help
  users comprehension
• Complex objects require slower animations to give
  viewer time to extract information

19
Other Visualization Properties Results

• A learning curve was not detected (see figure?)

• Accuracy was affected by choices that differed
  from the correct answer by small differences in
  orientation

• No significant performance differences were found
  between geometric and realistic objects

20
Spatial Ability Groups Results

• No significant property differences between
• All questions with high percentage of correct
  answers
• All questions with high percentage of incorrect
  answers

BUMMER!
Our Next Step is to Look at the Data in More
Detail

• Divide-up participants 3 groups and selected the
  High Spatial (HS) and Low Spatial (LS) ability
  participants (based on Paper Folding Test).
• Knowing a source of variability and looking at
  the extremes helps to make the effect visible

21
Spatial Ability Groups Results

• Analysis
• Are there properties that only high spatial
  people use?
• Compare Properties of Questions Answered
  Correctly by high spatial participants to
  Properties of all Questions
• Results
• Total Number of Edges and Total Number of
  Vertices were found significantly higher in
  questions which the high spatial participants
  answered correctly
• The Ratio of Distinct to Visualized Surfaces was
  found significantly higher in questions answered
  correctly by high spatial participants
• Implications
• High spatial participants understand more complex
  objects and can process a higher number of hidden
  properties

22
Analysis of Subjects Errors

• Analysis
• Create a bar graph showing distribution of
  answers for each question
• Analyze the questions where distributions clearly
  not evenly distributed
• Interesting results for further analysis
  (possible strategies)

Frequency of answers High spatial ability 6 Low
spatial ability 8
23
Issues

• Experiment trials organized according to what was
  believed to be trial difficulty. This
  organization was wrong.
• There was ambiguity in the answers that
  participants had to choose between, in particular
  because participants were allowed to rotate the
  answers, they were not able to see the
  differences in orientation between two possible
  answers
• Only projection visualization was studied and
  thus, the results cannot be readily extrapolated
  to many other 3D visualizations
• The object properties manipulated in the
  questions were horizontal and vertical alignment.
  Future studies will include properties such as
  size, shape (sides), aspect ratio.

24
Summary of Results

• Spatial abilities are related to 3D visualization
  comprehension
• Problem solution time was not found to be related
  to visualization accuracy
• Counts of geometric properties affected
  visualization accuracy for low spatial subjects,
  and time of analysis for everyone
• The hidden geometric properties in the
  visualization affect visualization accuracy for
  low spatial subjects
• Small rotation differences are difficult to
  detect in a visualization
• A case by case analysis suggests that high
  spatial and low spatial ability participants use
  different strategies

25
Larger Implications of Research

• Visualization designers can use measures of
  cognitive ability to help understand what makes
  visualizations hard/easy to comprehend
• Using interactive rotation and animations is
  likely to help users better understand
  visualizations
• Visualization difficulty may be highly variable
  for a diverse population
• There exist educated people who cannot understand
  simple 3D visualizations

26
Acknowledgments

• Thanks to our reviewers for their comments
• This research is supported by the National
  Science Foundation through the SGER grant
  0503680

27
(No Transcript)