Consultation

1
Consultation

• principles and methods

2
Why consult?

• There are several reasons for consulting
• because we have to
• because we look good if we do
• because we look bad if we dont
• it gives us a warm, caring feeling
• it gives them a warm, cared-for feeling
• to give staff something to do
• to help us make better decisions.

These may all be reasons in practice they may or
may not be legitimate.
This should be our main reason for consulting in
most cases.
3
Approaching consultation

• Six basic principles

4
1. Research methods
Consultation is an attempt to gather and
interpret information on what people think. This
makes it a form of research.

• Consultation should be based on sound research
  methodology, informing the
• problems to be addressed,
• data collection methods used,
• actual collection of the data,
• analysis of the data, and
• the interpretation of the findings.
• The biggest danger is that the results will
  reflect our preconceptions.

5
2. Clarity of purpose

• Consultation should begin with a clear
  understanding of
• what we wish to find out, and
• what we are going to do with it.
• On one level, we consult because we have to.
• But consultation is both costly and risky, and
  should be conducted
• to bring together those who deliver services and
  those who receive them,
• to increase everyones understanding of the
  issues, and
• ultimately to allow more effective use of
  resources, through more informed decisions.

6
3. Supporting action

• The ultimate test of consultation is whether the
  findings make a difference to what we DO.
• If we engage people in consultation, and then do
  not act on the findings, then we have failed.
• At the outset of a consultation exercise we
  should ask ourselves
• who are the decision makers who we want to be
  informed by the consultation? and
• what are the sort of outcomes that could be
  affected by the results.
• Pre-consultation - finding out whats on the
  table.

7
4. Triangulation

• Triangulation is the use of two or more methods
  or sources of data to illuminate a problem.
• If two methods point to the same conclusion, the
  conclusion may be strengthened.
• (The more diverse the methods, the better.)
• If two methods point to different conclusions,
  this will need to be explained in terms of
• possible biases in methods
• inherent conflicts or contradictions in views.

8
5. Fulfilling expectations

• Community consultation may raise expectations of
  action.
• If these expectations are not fulfilled, this may
• lead to negative attitudes towards the agencies
  conducting the consultation, and
• reduce willingness to engage in future
  consultation.
• To minimise the risk of this happening, we should
• negotiate what can change as a result of
  consultation, and
• be realistic - dont promise what cant or wont
  be delivered.

9
6. Genuine partnership
Community consultation is the essence of
partnership - not just between agencies, but
engaging the community on an equal basis.
Contrasting models of consultation

• MODEL ONE
• we define the issues,
• we ask you what you think, (possibly through a
  questionnaire survey), then
• we decide what you need.

• MODEL TWO
• we will not set the agenda, nor assume that we
  know what the issues are,
• we will find out what you think the issues are,
  then
• work with you to generate and enable solutions.

PARENT-CHILD PATRON-CLIENT CONSULTANT-PATIENT TOP-
DOWN
COMMUNITY-CENTRED EMPOWERMENT BOTTOM-UP
10
Consultation methods

• quantitative and qualitative

11
A restricted and selective view

• There is a wide range of consultation methods.
• I shall talk about two broad areas
• quantitative - questionnaire-based surveys
• qualitative - focus groups.

• Why?
• time constraints,
• my own direct experience,
• both are important for fear of crime,
  reassurance
• general lessons for how they work together.

12
Sample surveys

• Medium to large samples (200 to 10,000), usually
  drawn by some standard of representativeness.
• Uses standardised questionnaires, either
• self-completion (usually postal), or
• interview (face-to-face, or by telephone)
• Allows us some degree of confidence in the
  reliability of
• general conclusions drawn,
• comparisons between data sets, and
• comparisons within the data set.

13
Sample surveys

• Medium to large samples (200 to 10,000), usually
  drawn by some standard of representativeness.
• Uses standardised questionnaires, either
• self-completion (usually postal), or
• interview (face-to-face, or by telephone)
• Allows us some degree of confidence in the
  reliability of
• general conclusions drawn,
• comparisons between data sets, and
• comparisons within the data set.

14
General conclusions drawn
Top level findings on key indicators should be
qualified by a confidence interval.
Example Out of a sample of 200 people, 35 said
they feel unsafe This is the result for the
sample we are interested in the
population. The sample result (35) gives us a
best estimate for the population. But this is
an approximation, and has a margin of error. In
this case, the margin of error is 6.6 this is
the confidence interval. This means that
somewhere between 28.4 and 41.6 of the
population would say they feel safe if we had
asked them all. We cant be absolutely sure even
of that - we are 95 confident.
15
Sample surveys

• Medium to large samples (200 to 10,000), usually
  drawn by some standard of representativeness.
• Uses standardised questionnaires, either
• self-completion (usually postal), or
• interview (face-to-face, or by telephone)
• Allows us some degree of confidence in the
  reliability of
• general conclusions drawn,
• comparisons between data sets, and
• comparisons within the data set.

16
Comparisons between data sets

• Changes in key indicators may reflect
• underlying changes in the population, or
• random fluctuations due to sampling error.

• They should be tested for statistical
  significance, which
• tells us the likelihood that a finding could have
  occurred randomly,
• so that we can say with some confidence (95
  again) whether there appears to have been a
  genuine, underlying change.

17
Comparisons between data sets
Example 2002 35 of a sample of 200 said they
feel unsafe. 2003 32 of a sample of 240 said
they feel unsafe. Conclusion People now feel
safer. But is this an underlying change, or a
random fluctuation? The confidence interval gives
us a clue. In 2002, the true (population)
figure lay between 28.4 and 41.6. The 2003
figure (32) lies within this range, and so we
should not conclude that there has been an
underlying change. Statistical significance
testing allows us to say with specified
confidence (again, 95) whether there has been an
underlying change. (The calculation is more
involved than this, but the principle is the
same.)
18
Sample surveys

• Medium to large samples (200 to 10,000), usually
  drawn by some standard of representativeness.
• Uses standardised questionnaires, either
• self-completion (usually postal), or
• interview (face-to-face, or by telephone)
• Allows us some degree of confidence in the
  reliability of
• general conclusions drawn,
• comparisons between data sets, and
• comparisons within the data set.

19
Comparisons within the data set

• The statistical analysis of data should involve
  breaking down the sample into comparison groups,
  and comparing them on key indicators.
• These breakdowns can be by
• demographic factors (sex, age, marital status,
  etc), or
• key questions about behaviour or experiences.

20
Comparisons within the data set
Example 2002 35 (70) of a sample of 200 said
they feel unsafe. 25 (50) had seen a police
officer on patrol in the past week. QUESTION is
seeing a police officer linked with safety? seen
patrol 20 feel unsafe (10 out of 50) not seen
patrol 40 feel unsafe (60 out of 150)
CONCLUSION people feel safer if they have seen
patrol. But what if seen patrol 38 feel
unsafe (19 out of 50) not seen patrol 34 feel
unsafe (51 out of 150). Again, statistical
significance will be needed to sort this out.
21
Sample surveys

• Medium to large samples, usually drawn by some
  standard of representativeness.
• Allows us some degree of confidence in the
  reliability of
• general conclusions drawn,
• comparisons between data sets, and
• comparisons within the data set.

22
Sample surveys

• Our ability to draw confident conclusions will
  depend very heavily on
• samples that are representative, and large enough
  to be subdivided for analysis,
• sound questionnaire design, asking relevant
  questions, while minimising bias, and
• appropriate statistical analysis, bringing
  different questions together to reach an informed
  interpretation of what the findings mean.

23
Sample surveys

• Sample surveys can be
• general population surveys,
• surveys of customer groups,
• panel surveys,
• omnibus surveys,
• before-and-after evaluations, and
• surveys of hard-to-reach groups.

We shall now look at each in turn.
24
General population surveys

• Surveys representing all residents of a specified
  area.
• Usually addressing specified issues, such as
• perceptions of levels of crime and disorder,
• feelings of safety,
• satisfaction of strategies to deal with crime and
  disorder.

• strengths
• potentially representative of an area
• can identify demographic variations.

• weaknesses
• broad-brush approach, often with no clear
  purpose
• can lack practical focus, and be merely
  interesting
• respondents may not have considered the issues,
  and may not have opinions
• random sampling can be expensive.

25
Surveys of customer groups

• Population/sample defined by a shared experience,
  such as
• victims of burglary, or
• tenants who have reported damage to social
  landlords.
• Usually addressing service issues and customer
  satisfaction.

• strengths
• focus on individuals experiences - people will
  usually hold an opinion
• easy to generate random samples
• can give actionable results (if appropriately
  analysed).

• weaknesses
• only captures those who have reported - must
  beware of over-generalising
• short time-span for memory.

26
Panel surveys

• Sample of people recruited as
• demographically representative of the population,
  and
• questioned regularly on an issue, or range of
  issues.
• Panel will usually be refreshed over the course
  of two years.

• strengths
• people questioned regularly, so get more
  considered responses
• people commit themselves to the survey, so can be
  asked more detailed or open-ended questions
• sensitive measure of change over time.

• weaknesses
• possible bias arising from repeated questioning
• needs careful management and administration.

27
Omnibus surveys
Form of general population survey, where
different organisations buy space to ask
questions of a large, representative
sample. Often run by large market research
agencies.

• strengths
• pooled resources giving economies of scale
• uses sampling frames and expertise of large,
  established agencies
• surveys that cover a wide range of topics may be
  more interesting to complete, and give better
  response rates.

• weaknesses
• limited numbers of questions, which limits depth
  of analysis
• surveys that cover a wide range of topics may not
  be conducive to careful reflection on your
  questions.

28
Before-and-after evaluations

• Surveys repeated to measure change, either
• using the same people, or
• drawing separate samples from the same
  population,
• in order to assess the effects of an
  intervention, or new method of working.

• strengths
• ideal for measuring change attributable to
  changes in practice (panel surveys are the only
  other way of doing this)
• can give actionable findings.

• weaknesses
• needs careful interpretation - guard against
  temptation to draw simplistic causal
  interpretations.

29
Surveys of hard-to-reach groups

• Surveys of groups of people, who
• have some shared characteristic, and
• who tend to be missed by other sample survey
  methods.
• For example, young men, homeless people, victims
  of homophobic abuse.

• strengths
• gives another side to the picture, that is
  easily overlooked
• sends message of inclusive approach to
  consultation.

• weaknesses
• populations are, by definition, hard to reach -
  creative approach to sampling needed
• because of this, need to guard against
  over-generalisation.
• difficult to decide on or select which groups to
  survey (and which groups not to survey) - danger
  of arbitrariness.

30
Focus Groups

• Focus groups
• are a method of consultation
• involve a group of participants - typically 8-12
  people
• discussing a designated topic - the focus of
  the group
• facilitated by a moderator and co-moderator.

31
Focus Groups

• Advantages
• very rich source of data
• allow the exploration of deeper issues
• that are difficult to address by more
  quantitative methods
• give access to the meaning and motivation behind
  what people say and do.

32
Focus Groups

• Disadvantages
• do not give statistically reliable or
  representative findings (they sacrifice breadth
  for depth)
• require skilled moderators if they are to be run
  successfully
• yield data that are very difficult and
  time-consuming to analyse.

33
Focus Groups

• Use focus groups
• as a source of data in their own right, for
  issues that cannot be addressed by quantitative
  consultation methods
• in conjunction with quantitative consultation
  methods, to inform questionnaire design
• in conjunction with quantitative consultation
  methods, to flesh out or add depth to statistical
  findings.