A1: Surveys for Behavioural Experiments

1
A1 Surveys for Behavioural Experiments

• Peter Jones, CTS, UCL
• Regine Gerike, TUD
• Giorgia Servente, Polito, TO

2
Workshop Context

• Many countries are considering using VBC
  programmes to influence (travel) behaviour
• How should such programmes be monitored and
  evaluated?
• Are there methodological problems unique to VBC?
• Are there best practice recommendations?
• What are the methodological research challenges?

3
SURVEY DESIGN PROCESS
Existing Travel and Consumption Patterns
Need for Change Policy OBJECTIVES?
Voluntary Behaviour Change Intervention
POST Travel and Consumption Patterns
4
SURVEY DESIGN PROCESS
Existing Travel and Consumption Patterns
Need for Change Policy OBJECTIVES?
CONTROL Group
Voluntary Behaviour Change Intervention

• What were the IMPACTS of the intervention?
• On Subjects
• On Others

POST Travel and Consumption Patterns
5
SURVEY DESIGN PROCESS
Existing Travel and Consumption Patterns
Need for Change Policy OBJECTIVES?
CONTROL Group
Voluntary Behaviour Change Intervention

• What were the IMPACTS of the intervention?
• On Subjects
• On Others

WHAT Led to the Observed Changes? PROCESS
POST Travel and Consumption Patterns
6
Key Issues - 1

• What are we trying to measure?
• What are our objectives? (reduce car use, save
  energy, increase physical activity)
• What about secondary impacts? (local economy,
  traffic accidents)
• Are we seeking just to measure change or
  understand reasons for people to/not to change?

7
Key Issues - 2

• Ways of measuring behaviour
• Take up of marketing information
• Reported behaviour (trips, durations, etc)
• Direct individual consumption measures
  (odometers, GPS, etc) reported or relayed
• Aggregate area measures (bus passengers, traffic,
  etc)

8
Key Issues - 3

• How to identify real change
• Need for control groups difficult!
• Potential biases in reporting, especially with
  VBC programmes
• Retrospective vs. pre-determined
• Key importance of question framing
• Greater risks where respondents associate survey
  with VBC programme
• Dangers of using panel surveys

9
Key Issues - 4

• How to assess statistical significance
• Systematic vs. random errors
• What level of significance do we really need?
• Large samples are expensive to collect
• Possible scope for meta analysis?
• Importance of response rate
• Higher gt lower mean trip rate

10
Key Issues - 5

• When should impacts be measured?
• Allow time for effects to settle down (e.g. after
  expiry of free bus tickets)
• Match before and after seasonally
• How long do impacts last?
• May increase over time (snowball effect)..?

11
Key Issues - 6

• How can we understand causes of changes and
  willingness to change?
• Trips as multi-dimensional packages
• Need to ground in psychological theory (e.g.
  Theory of Planned Behaviour)
• Important role for qualitative research
• ..All of which can help inform best type of VBC
  interventions

12
Some Key Research Areas

• Accuracy/role of meter readings
• How best to integrate data sources
• Defining effective control groups
• Cultural effects (survey and behaviour)
• Deeper understanding of change processes
• Importance of interviewer effects?
• Greater use of experimentation (reduced need for
  precision in evaluating practical applications?)
• Tool kit for VBS evaluation low cost options
  for developing countries