Methodology and Explanation XX50125

1
Methodology and ExplanationXX50125

• Lecture 2 Experiments
• Dr. Danaë Stanton Fraser

2
Lectures

• Introduction to Methods
• Experiments
• Interviews/questionnaires/surveys
• Usability testing
• Ethnography

3
Aim

• To answer a question or test an hypothesis that
  predicts a relationship between two or more
  events, known as variables.
• E.g.
• Will people be quicker and more accurate at
  completing a task collaborating at a distance
  using just verbal instructions or with verbal
  instructions with gesture input?
• Will spatial knowledge be superior following
  exploration of a VR simulation of a building or
  from exploring a model of a building?

4
Variables

• Such hypotheses are tested by manipulating one or
  more of the variables.
• The variable that is manipulated is called the
  independent variable (the conditions to test this
  variable are setup independently before the
  experiment starts). In the study mentioned media
  type (VR vs model) is the independent measure
• The dependent variable would be accuracy of
  spatial information (ie time to reach point A
  from point B) because the time to carry out the
  task depends on the media explored.
• What are the IV and DV for the gesture study?

5
Variables and Conditions

• In order to test an hypothesis the experimenter
  sets up the experimental conditions e.g
• Condition 1 Pretest Explore model Post test
• Condition 2 Pretest Explore VR simulation Post
  test
• Control condition against which to compare the
  results
• What are the conditions in the gesture study?

6
Participants

• Within participants design
• Counterbalancing required
• Between participants design
• Two drawbacks no. of participants needed,
  individual differences
• Advantage no order effects
• Matched pairs design (on characteristics such as
  gender and expertise)
• Difficult to match across all variables

7
Practicalities

• Where will the experiment be carried out?
• How will the equipment be set-up?
• How does one introduce participants to study?
• What scripts are need to standardise the
  procedure?
• Always include pilot study

8
Data Collection and Analysis

• Data performance measures are taken e.g
  response times, no. of errors, angle estimation
• Use graphs
• The data should be averaged across conditions to
  examine any differences
• Statistical tests such as t-tests and ANOVAs can
  reveal if the differences are significant
• Software packages such as SPSS are often used
• If there is no significant difference then the
  hypothesis is refuted.

9
Data captured

• Record quantative results
• However often also able to gain qualitative data
• e.g dialogue turns, gesture and non verbal
  behaviour
• Good practice to collect both qualitative and
  quantative data

10
Summary of Experimental Design

• State a causal hypothesis
• Manipulate independent variable
• Assign participants randomly to groups
• Use systematic procedures to test hypothesised
  causal relationships
• Use specific controls to ensure validity

11
Quasi-Experimental Design

• State a causal hypothesis
• Separate participants based on some
  characteristic e.g level of mobility as child,
  gender, occupation
• Cannot assign participants randomly to groups

12
A Variety of Experimental Designs

• (Intervention) - Post test
• Pre test - (Intervention) - Post test
• Pretest - (Intervention) - Post test - Delayed
  Post Test
• Interrupted time-series
• Cross-over

13
Preparation for Practical 2/3

• Topic Examining the use of gestures in
  co-located collaboration.
• We will design and run a small experimental study
  to examine gesture in co-located collaboration
• In preparation for the practical please
• Read up on experimental design
• e.g Robson, C. (1994). Experimental Design and
  Statistics in Psychology. Aylesbury, UK. Penguin
  Psychology.
• Coursework 1
• Write up the experiment carried out in Practicals
  2 and 3.
• Deadline for report Friday 18th November.