Measurement, Data and Information for Residential Aged Care

1
Measurement, Data and InformationforResidential
Aged Care
2
Overview

• Section One
• An introduction to measurement, data and
  information
• Measurement, data and information defined
• Why measure?
• Some background to measurement practices

3
Measurement, Data and Information Defined

• Measurement
• to ascertain the extent, dimensions, quality of
  something
• Data
• known or available facts or figures
• Information
• knowledge communicated or received concerning
  some fact or circumstance

4
Turning Data Into Useful Information and Knowledge
5
Why Measure?

• Benefits derived from collecting, analysing and
  reporting data and information are
• Provides the home with the ability to demonstrate
  its current level of performance to all
  stakeholders and against stakeholder needs
• Enables the home to gather information about the
  needs of stakeholders in an ongoing way
• Provides information for strategic and
  improvement planning
• Enables the home to monitor how well its plans
  are being implemented

6
The Link between Data and Information,
Self-Assessment and Continuous Improvement

• As a regular and cyclical process, rigorous
  measurement and analysis of results
• Enables the staff and management of the home to
  understand their current performance for example
  through results of resident satisfaction,
  resident outcomes, clinical indicators, audit
  results, other survey results
• Ensures staff and management of the home have up
  to date knowledge of their current level of
  achievement against the Accreditation Standards
• As part of self-assessment or routine performance
  review assists the home to identify areas where
  improvements could be made

7
The Link between Data and Information,
Self-Assessment and Continuous Improvement

• Is an input into planning processes enabling
  decisions abut improvement, priorities, goals and
  resourcing
• Also informs the homes management of the results
  achieved through improvement projects previously
  implemented
• Ensures that staff and management of the home
  have up to date evidence available to support
  their accreditation application and evidence
  provided during a support contact

8
Data and Information As Part of Self Assessment
9
Background to Measurement Practices

• The CSIRO found the following problems with
  collection and use of information. This package
  aims to touch on these problems.
• Little relationship existed between the strategic
  intents of the organisation and what was actually
  measured and reported
• Information usually focused only on financial
  performance
• Information did not support assessment and
  management of performance
• Information was largely operational
• Ambiguity about what was being measured
• Lack of capability to provide the appropriate
  analysis
• Lack of understanding of variation and statistics

10
Overview

• Section Two
• Deciding what to measure
• Levels of measurement
• Start the thinking about measurement with what
  the home strives to achieve
• Process drive achievement understanding
  processes leads to other appropriate measures
• Measurement and the expected outcomes of the
  Accreditation Standards
• Where measurement associated with improvements
  fits in

11
Levels of Measurement

• It is important to be clear about what measures
  are important and for what reason. For instance
• How the the home performs overall in its major
  areas of care and service
• Enable staff to manage a particular job or
  process such as care planning or the catering
  services
• To track the success of a quality improvement
  project

12
Levels of Measurement

• The following steps simplify the approach to
  measurement
• Define what is important to the residents, other
  stakeholders and the home
• Identify what the residential aged care home
  wants to achieve
• Look at the major processes of the home and how
  they are undertaken
• Decide what information will assist staff and
  managers to assess performance at the various
  levels

13
Measurement What the Home Strives to Achieve

• It is important to begin with a review of what
  overall achievements the home aims for.
  Information about critical areas for achievement
  could be found through
• Review of the homes strategic plans and major
  objectives
• Review of the mission and vision statements
• Workshops and discussion between the managers and
  the board

14
Resident Focus

• Resident satisfaction and the satisfaction of
  other stakeholders are generally important high
  level measures for a home. Good management
  practice also focuses on the needs, expectations
  and satisfaction of other key stakeholders such
  as
• Family members, carers and/or resident
  representatives
• Staff
• Visiting professionals
• The owners of the home
• Local and professional communities
• Other agencies

15
Understanding Processes, Outputs and Outcomes

• Results are what we aim for
• Systems and processes help us achieve them!

16
When a Problem Arises
17
Measurement from a Process Perspective

• There are three sorts of measures that may be
  relevant to the residential aged care homes
  major processes
• Measurement of outcome
• Measurement of output
• Measurement within the process itself

18
Measurement of Outcome

• Outcome measures provide answers to the
  following probing questions
• You did thisand so?
• And the result of this effort was?
• Did this benefit the residents?the staff?
• They provide information about the ultimate
  results achieved.

19
Measurement Around Output

• Output measures are a step removed from the
  measures of outcome and can sometimes be seen as
  predictors of outcome.
• That is if results of outputs measures are
  positive the results of the measurement of
  outcome are also likely to be positive.

20
Measurement Inside the Processes

• Examples of commonly used in-process measures
  include
• Staff compliance rates (to procedure and/or
  protocols) for example
• - compliance to residents needs assessment
  procedures
• - compliance to medication procedures
• - compliance to cooking procedures and times
• - compliance to occupational health and safety
  procedures such as hand washing and universal
  precautions

21
Measurement and the Expected Outcomes of the
Accreditation Standards

• The issue of effectiveness is the link between
  processes and outcome measures. Where the home
  has robust information around outcomes,
  effectiveness may be relatively easy to assess.
  Without this data, the home will need other ways
  of demonstrating that care and services are
  effective.

22
Where Measurement Associated With Improvement
Activities Fits In

• Measurement can be used at various levels and to
  suit various purposes
• Routine high level measures of performance for
  management and staff
• Routine measurement to enable the management of
  processes
• Targeted measurement for quality improvement
  projects

23
Where Measurement Associated With Improvement
Activities Fits In

• Each set of measures suits a particular purpose
  and as such
• The various sets of measures should be considered
  differently and appropriately
• A comprehensive measurement system would most
  likely include measures to enable review of each
  of the three aspects of management described
  above
• It is important to distinguish between the
  measures being used

24
Measurement and Quality Improvement

• The development of appropriate measures and
  target for each quality improvement project
  enables the home to readily assess whether its
  improvement efforts are successful and, if
  measured over a defined period, will also show
  whether improvements have been sustained.
• Without such measurement and analysis it is
  difficult for a home to demonstrate the
  effectiveness of its improvements and the
  benefits achieved for residents and others.

25
Overview

• Section Three
• Considerations in data collection
• Aspects of effective data management
• Aspects of effective data collection
• Sampling
• Types of data and data collection tools
• Planning for routine data collection

26
Aspects of Effective Data Management

• The appropriate collection of data, covered in
  this section, is one of the important aspects of
  a data management system that ensures that the
  data and information does in fact provide value
  to the users.
• Effective data management includes the
  presentation, analysis and interpretation of the
  data and information and these are covered in the
  next section

27
Aspects of Effective Data Collection

• Major aspects of data collection include
• Clarity about the relevance and importance of the
  measure
• The definition of how the measure will be made or
  taken
• Clarity about who will collect the measure, where
  and when
• Definition of how the results will be reported
  in what format and to whom
• Clarity about who has responsibility to respond
  to the results of the measurement

28
The Relevance and Importance of the Measure

• If the purpose of any measure is not clear, or
  it is obvious that the results are difficult to
  understand or act onwhy measure it? Try asking
• What major area of care and/or service delivery
  does this measure relate to?
• Is it clear how it relates?
• Will it effectively tell us about the area of
  care and/or service delivery we are interested
  in?
• Are there other measures that would provide more
  accurate information about the area under study?
• Are there additional measures that would provide
  more complete information about the area under
  study?

29
Definition About How the Measure Will be Made or
Taken

• The definition could include details of
• Any tools required for the measurement
• The appropriate level of detail required in the
  measurement
• Instructions about timing
• Any associated information that should be
  collected at the same time
• How to record the measurement
• How to store the measurement once recorded

30
Who Will Perform the Measurement?

• Some measurements could be taken easily by any
  staff member and others may require specialised
  skills and knowledge. The detail can include
• The designated staff responsible for collecting
  the measurement
• Where they will source the measurement
• Detail around the choice of subject
• Detail around timing and sampling

31
Reporting of the Measurement

• Clarity about reporting could include details of
• How often the results are to be reported
• Who they are reported to such as the department
  manager, the quality committee, the director of
  nursing, the management committee, or other
  forums
• In what format they will be reported for instance
  as
• - raw data
• - averages, ranges
• - in tables
• - in run charts
• - other formats as required

32
Identifying who has Responsibility to Respond to
the Results

• The person responsible will vary according to
  the particular measures being evaluated. For
  instance
• The Chief Executive Officer, Director of Nursing,
  Care Manager or other senior member of staff are
  likely to be responsible for review of, and
  response to, the residential aged care homes
  outcome measures and other high level data
• Service managers such as unit managers, catering
  managers and a support services manager may be
  responsible for the review of, and response to,
  process measures used to monitor their individual
  activities

33
Sampling

• There are several methods of sampling, four of
  which are described
• Incidental Sampling
• The person conducting the measurement chooses
  whoever they wish. It may result in bias.
  Though easy, it is not recommended.
• Stratified Sampling
• Involves identifying sub-groups within the
  population and choosing an appropriate number
  from each category.

34
Sampling

• Random Sampling
• Random sampling minimises the introduction of
  bias and provides a sound basis for generalising
  the results found to the whole population. It is
  considered one of the best types of sampling.
• Systematic Sampling
• Systematic sampling involves choosing a subject
  at regular intervals from the whole population of
  subjects for instance every fifth or tenth
  subject from a list. It is not truly random but
  approaches this.

35
What is a Good Approach to Sampling?

• To decide the best overall approach for any
  particular study, audit or other assessment
  consider
• Are there particular sub-groups that are
  important to this measurement? If so, note this.
• Attempt to choose the subjects (residents, staff,
  files) as randomly as possible, choosing numbers
  from any relevant sub-groups proportionally.
• If sub-groups are not important in a particular
  study, attempt to choose files, residents or
  other subjects at random.

36
Sample Size

• Attempt to review approximately 10-20 of cases
  for any audit, review or assessment.
• Be flexible in the approach. For instance if the
  audit or review shows unusual findings or mixed
  results, you may want to sample more subjects to
  develop a clearer view and understanding of the
  findings.
• When sampling from a sub-group attempt wherever
  possible to have a minimum number (say five) in
  each individual sub-group.
• An inclusive approach where all subjects are in
  the study, is entirely appropriate where the home
  has considered the importance of the assessment
  and plans the appropriate resources for the data
  collection.

37
Types of Data and Data Collection Tools

• Quantitative Data
• Can be counted. The results can be described
  and analysed numerically.
• Examples of data collection tools to gather
  quantitative data include audits, check sheets,
  tick sheets, count sheets.
• Qualitative Data
• Is generally concerned with individuals and
  their opinions, thoughts, feelings, experiences
  and other feedback.
• Examples of qualitative data collection tools
  are surveys, questionnaires, one to one
  interviews and focus groups.

38
Audits

• An audit can be defined as
• a planned, independent, and documented
  assessment to determine whether agreed-upon
  requirements are being met. Arter, D.R, 94
• Some considerations before developing any audit
• Be clear about the objectives of the audit
• Ensure that this element of care and/or service
  is important and warrants its own auditing
  approach never audit for the sake of auditing
• Be clear about exactly what you will do with the
  results

39
Collecting Data on Opinions

• Steps in developing a questionnaire
• Define what it is you want to ask about
• Draft the tool (such as the survey or
  questionnaire, interview process)
• Test the tool
• Refine the tool based on the findings from the
  test
• Conduct the process and review
• It is clear that this process follows the simple
  Plan-Do-Check-Act cycle common in any quality
  improvement exercise.

40
Planning for Routine Data Collection

• Most residential aged care homes collect a range
  of data using a range of tools. It is a useful
  exercise to list the major data collection
  activities and schedule them. Preparing such a
  schedule provides a number of advantages such as
• Communication to staff, managers and others about
  the range of measurement activities
• Providing a timetable for staff and managers for
  their particular audits and activities
• Providing a platform for review of the
  measurement of activities

41
Reviewing the Data that is Collected and
Reported

• Some useful questions to review measurement
  activities.
• For each audit, survey or or other data
  collection exercise ask
• How does this measurement activity fit with the
  information needs of the residential aged care
  home? How important is it?
• How well does the data and information meet the
  needs of the user?
• What do the results show?

42
Overview

• Section Four
• Presentation and analysis of data
• Aspects of effective data management
• Effective presentation and analysis of data
• Presentation and analysis of quantitative data
• Case Study and exercise

43
Aspects of Effective Data Management

• This section addresses the final cornerstone of
  an effective data management system the way
  that data are presented and analysed.
• Turning data into useful information and
  knowledge.

44
Effective Presentation and Analysis of Data
The choice of the format for presentation of data
relies on two important considerations the type
of data and the audience.

• Quantitative
• Tables
• Bar charts histograms
• Run charts
• Control charts
• Pie charts
• Pareto charts
• Scatter diagrams

• Qualitative
• Flowcharts
• Cause and effect diagrams

45
Effective Presentation and Analysis of Data

• Whichever format is chosen it is important that
  data and information are presented as clearly and
  accurately as possible with attention to standard
  considerations for data presentation such as
• Clear title for the data and information set
• Clear dates and other identifying information
• The origin of the data clearly identified on the
  data and information set (for instance this data
  was collected from the north wing of XYZ Nursing
  Home)
• Details of who collected the data
• Consideration to what features are required to
  enable the user to appropriately analyse the
  information.

46
Presentation and Analysis of Quantitative Data

• Single data points for results (such as a
  solitary figure in a table in a monthly report)
  rarely provide meaningful information to the
  user.
• The ability to analyse results is strengthened
  when comparative information is also provided
  along with the current results.
• Whatever the presentation format chosen, the
  chart or table should be clean and uncluttered.
• The baseline against which to compare current
  results (say the median and range) can be
  calculated from the previous 12-18 months data if
  these results appear to have been relatively
  stable.

47
Presentation and Analysis of Quantitative Data

• If significant improvements are achieved over
  time, and sustained, then the baseline will need
  to be recalculated to accurately represent the
  current system and its results. A new standard
  has been set.
• If comparing your residential aged care homes
  results with those of others ensure that the
  details of data collection and presentation are
  the same. This simple point lies at the root of
  much dissatisfaction with many data comparison
  programs.
• It may be more valuable to focus on refining and
  developing the homes own internal measurement
  system to best meet its own needs before
  embarking on external comparative programs.

48
Analysis of Qualitative Data

• Important Note!
• Where results for a particular month are in line
  with what has been previously achieved, this
  indicates the homes performance is stable. That
  is, with all else being equal and nothing
  changing, the results can be expected to
  continue.
• It is quite another question whether the results
  are acceptable. The use of an acceptable range
  or a target strengthens the ability to determine
  if results achieved are acceptable.

49
Presentation and Analysis of Qualitative Data

• Qualitative data includes such things as staff
  ideas, resident ideas and suggestions, and the
  process knowledge and understanding of staff and
  managers.
• Presentation formats that are useful for
  qualitative data include
• Flow charts
• Cause and effect diagrams
• Note whilst surveys and questionnaires collect
  qualitative data (such as opinions and
  perception) the analysis of survey and
  questionnaire results commonly involves giving
  some numeric number to the responses. As such,
  numerical results for survey responses are
  presented and analysed as for quantitative data.