Evaluation - Did it really work?

1
Evaluation - Did it really work?

• Effective public relations measurement and
  evaluation
• Presented by Dr Tom Watson, Associate Professor
  in Communication, Charles Sturt University

2
Flying on one instrument?
3
Agenda

• The communications process
• What is evaluation?
• Models of evaluation
• Brief case studies

4
How PR is practiced

• Grunigs four models
• One-way
• Press agentry
• Informational
• Two-way
• Asymmetrical
• Symmetrical

5
Communications process

• Presentation
• ?
• Attention
• ?
• Comprehension
• ?
• Acceptance
• ?
• Retention
• ?
• Action - Outcome
• McGuire (1984)

6
Communication and evaluation

• Presentation
• Attention
• Comprehension
• Acceptance
• Retention
• Action

• Output (L1)
• Out-growth (L2)
• Outcome (L3)
• Lindenmann 1993

7
Types of evaluation

• Summative final impact
• Formative in progress assessment
• Should be integrated into all stages of programs

8
What can be measured?

• Success or failure of strategy
• Message distribution and reception
• Understanding, favourability and acceptance
• Trends

9
What else?

• Internal
• Staff attitudes value of newsletters, video,
  team briefings
• SWOT
• External
• Sponsorship impact
• Issues management and prediction
• Stakeholder attitudes

10
Case studies

• PricewaterhouseCoopers (PwC) accountants and biz
  advisers
• DIMIA Harmony Day
• Polestar major printing group
• Volvo - XC90 launch

11
PwC

• Why arent we getting coverage?
• asked the bosses
• Merger of PW with CL not seen as equal a
  takeover
• Change in approach

12
PwC lessons

• Charts and graphics improve story acceptance and
  favourable write-up
• CEO in interviews increases favourability
  voice of the organisation
• Benchmark coverage to find whos writing about
  what

13
DIMIA

• Monitored media coverage of events
• Moved from celebrity to community basis
• More outside metros
• Increased participation

14
Polestar

• Europes biggest print group
• Going through major changes
• Monitored media coverage
• Able to see strategy and tactics working

15
Polestar mid-2001
16
Polestar early 2003
17
Volvo new car launch
18
UK Launch program

• Competing against BMW and Mercedes
• Strategy was to have continuing campaign instead
  of launch blip
• Closely targeted 25 to 44, mid to high income,
  degree educated, adventurous
• Total of 609,000 people (1 of pop.)

19
PR-led campaign

• Media relations drove enquiries to internet and
  dealers
• More interest than cars to sell All ordered
  before sale date announced
• Advertising ran for one month and cancelled
  saving of c.6m
• PR helped sell the required number of cars

20
PR works!
21
Evaluation

• Three methods of evaluation used
• Responses to internet and then dealers
• Reach of coverage to target audience 96 of
  target group saw it once 90 twice
• Sales

22
Evaluation your future

• Forget the Holy Grail, its about relationships
  (Hon Grunig 1999)
• Most PR campaigns dont influence behaviour.
  (McCoy Hargie 2003)
• Avoid dead-ends like management language for
  evaluation, e.g. ROI (Watson 2005)

23
References and Reading

• Hon LC Grunig JE (1999). Guidelines for
  measuring relationships in public relations.
  www.instituteforpr.com
• Lindenmann, W.K. (1993) An effectiveness
  yardstick to measure public relations success.
  Public Relations Quarterly, 38 (1)
• McCoy M Hargie (2003) Implications of mass
  communication theory for asymmetric public
  relations evaluation. Journal of Communication
  Management 7 (4)
• McGuire, WJ 1984 Attitudes and Attitude Change in
  Lindzey, G. Aronson, E. Ed. The Handbook of
  Social Psychology Vol.2 3rd Edn, Random House
• Walker, G (1997) Public relations practitioners
  use of research, measurement and evaluation,
  Australian Journal of Communication 24 (2)
• Watson T (2005) ROI or evidence-based PR the
  language of public relations evaluation. PRism 3,
  http//praxis.massey.ac.nz/issue_3.html
  (forthcoming)
• Watson T Noble P (2005) Evaluating PR, Kogan
  Page, London
• Watson T Simmons P (2004) Public Relations
  Evaluation Survey of Australian Practitioners,
  ANZCA04, July 2004

24
Web sources

• www.instituteforpr.com
• www.carma.com.au

25
Up and away!
26
Key thinking

• PR is concerned with ill-defined problems. It
  cannot be evaluated by a single method or metric
  because it does not deal with simple issues
• Likewise, business in general recognises that a
  simple, single financial measure does not give an
  accurate reflection of a companys true worth.
• Gregory Watson 2006 1

27
Policy and resources

• Research in the UK sponsored by CIPR
• Produced by Henley Management College
• Looked at practitioner and academic literature.

28
CIPR policy 1

• PR is part of the management task and is subject
  to the same disciplines, such as the need to set
  direction, allocate and manage resources, and
  monitor progress.

29
CIPR policy 2

• Measurement and evaluation are problematic in all
  areas of management. Complexity is a key factor
  and in business, it is difficult to separate
  out the effect of one area of management such as
  public relations.

30
CIPR policy 3

• However, the situation can be eased by better
  planning and objective setting where precise,
  measurable objectives allow for enlightened
  judgements of progress in campaigns and their
  management.

31
CIPR policy 4

• PR can be measured and evaluated in terms of
• Contribution to social and economic development
• Contribution to management, leadership and
  organisational performance by aiding better
  decision-making and avoiding mistakes
• Being a process and part of programme development
  and implementation
• The contribution of individual practitioners

32
CIPR policy 5

• Available methods, research based, provide
  information that is good enough for decision
  making for PR programmes
• Existing methods are adequate for measuring the
  contribution of PR.
• Each method has limitations, so a raft of
  measures appropriate to particular situations
  needs to be employed.

33
The CIPR resources

• Go to www.cipr.co.uk/research

34
Most Admired Companies

• A range of evaluation metrics were used between
  four and eight.
• Most frequent were informal and/or qualitative
  such as journalist feedback and discussions with
  stakeholders.
• Just checking everything is on track.
• Gregory, Morgan Kelly 2005