Performance measures for integrated child health information systems

1
Performance measures for integrated child health
information systems

• Alan R. Hinman, MD, MPH
• February 20, 2005

2
Outline of presentation

• Review development of principles and core
  functions
• Review development of performance measures
• Review performance measures
• Are they useful?
• Could you measure them?
• Review system functionality checklist (if time
  permits)

3
Principles Core Functions Workgroup Meeting,
May 2003

• Goal Develop a draft Model of Practice
  (Framework) for integrating newborn screening
  systems with other related early child health
  information systems that includes a comprehensive
  set of core functions, activities and services
• Objective To gain agreement on the format of
  the Model of Practice and draft core functions

4
Core Workgroup MeetingParticipants - 1

• Delton Atkinson, NCHS
• Tonya Diehn, IA
• John Eichwald, UT
• Jennifer Heberer, ME
• Therese Hoyle, MI
• Pam King, OK
• Robert Cossack, MA
• Donna Williams, NNSGRC
• Amy Zimmerman, RI

5
Core Workgroup MeetingParticipants - 2

• HRSA/MCHB
• Deborah Linzer
• Michele Lloyd-Puryear
• Marie Mann

• AKC/PHII
• Sherry Bolden
• Nicole Fehrenbach
• Alan Hinman
• Janet Kelly
• David Ross
• Kristin Saarlas

6
Principles underlying integrated child health
information systems

• Purpose 1
• Security confidentiality 5
• Technology serving stakeholder needs 8
• Quality assurance evaluation 3
• Financing 2
• Total 19

7
Core Functions of Integrated Child Health
Information Systems

• Confidentiality security 5
• Establishing maintaining client records 4
• Service functionality 6
• Technical functionality 4
• Reports 3
• Total 22

8
Desirable functions of integrated child health
information systems

• Establishing maintaining client records 3
• Service functionality 2
• Technical functionality 1
• Reports 2
• Total 8

9
Recommendations for Change

• Surveyed SPRANS grantees in Feb. 2004
• Reviewed results of survey in last workgroup
  meeting
• Visited six SPRANS grantees in Aug/Sept. (UT, OR,
  WA, IN, MO, RI)
• JPHMP supplement published Nov 2004

10
Results

• Site visit participants thought the core
  functions list contained functions, requirements,
  and performance measures
• Functions that contained references to time were
  changed or eliminated
• Functions that resembled requirements were moved
  to a draft requirements list
• Revisions discussed by Jan 2005 workgroup
  (handout)

11
Performance Measures Workgroup Meeting, March 2004

• Objective - Develop a draft set of performance
  measures to measure an integrated child health
  information systems ability to perform core
  functions
• Process Review principles and core functions of
  integrated CHIS and discuss possible indicators
  of performance
• Outcome System functionality checklist and
  Performance measures

12
Performance Measures Workgroup Meeting March 2004

• Pam King, OK
• Robert Kossack, MA
• Garland Land, MO
• Charles Rothwell, NCHS
• Sherry Spence, OR
• Donna Williams, NNSGRC

• Ellen Amore, RI
• Paul Biondich, IN
• Richard Harward, UT
• Jennifer Heberer, ME
• Gary Hoffman, WI
• Therese Hoyle, MI

13
Performance Measures Workgroup Meeting March 2004

• HRSA/MCHB
• Michael Kogan
• Jeffrey Koshel
• Deborah Linzer
• Michele Lloyd-Puryear
• Marie Mann
• Jacob Tenenbaum
• Karen Thiel

• AKC/PHII
• Nicole Fehrenbach
• Alan Hinman
• John Kiely
• Patricia Richmond
• Kristin Saarlas
• Ellen Wild

14
Subsequent steps

• Same as with Principles and Core Functions
• Pilot Site Selection and Performance Measure
  Modification
• Ranked sites by core functions
• Invited during March 2004 Work Group meeting
• Confirmed participation
• Survey 5 sites and March 2004 workgroup
• Site visits

15
January 2005 Workgroup Mtg

• Purpose to review and assess recommendations to
  refine principles, core functions, and
  performance measures and to review and assess
  proposed study design for a pilot study of
  performance measures for integrated child health
  information systems

16
Participants, Jan 2005 Wkgp mtg

• Ellen Amore RI
• Pat deHart WA
• Richard Harward UT
• Gary Hoffman WI
• Therese Hoyle MI
• Pam King OK

• Bob Kossack MA
• Garland Land MO
• Maureen Mitchell VA
• Lori Sanchez CO
• Sherry Spence OR
• Donna Williams TX

17
Participants, Jan 2005 Wkgp mtg

• HRSA
• Mary Kay Kenney
• Debbie Linzer
• Michelle Lloyd- Puryear
• Marie Mann
• Bonnie Strickland
• Jack Tenenbaum
• Elizabeth Walkup

• PHII
• Nicole Fehrenbach
• Alan Hinman
• John Kiely
• Kim Koporc
• Ellen Wild

18
Revised performance measures
19
PM 1A - Percent of newborns with a record in the
integrated child health information system (ICHIS)
20
PM 1A

• N Number of live births occurring in the
  jurisdiction that have a record entered into the
  system
• D Total number of live births occurring in
  jurisdiction during a specified time interval
  (MM/DD/YYYY MM/DD/YYYY)

21
PM 1B - Percent of records of live births
occurring in the jurisdiction that were
established within 0-2, 3-7, 8-14, 15-30, and gt30
days of birth

22
PM 1B

• N Number of live births that have a record in
  ICHIS entered into the system by 2 days of
  birth (3-7 days, 8-14 days, 15-30 days, gt30)
• D Total number of live births occurring in the
  jurisdiction that have a record established in
  ICHIS during a specified time interval
  (MM/DD/YYYY MM/DD/YYYY) (Same as PM 1A num)

23
PM 2A - Percent of records that include data on
dried blood spot screening, hearing screening,
immunization, and vital registration
24
PM 2A

• N Number of records for resident children
  included in the denominator that include data on
  dried blood spot screening, hearing screening,
  vital registration, and one or more immunization
  events
• D Total number of live births occurring in the
  jurisdiction that have a record established in
  ICHIS during a specified time interval
  (MM/DD/YYYY MM/DD/YYYY) (Same as PM 1A num)

25
PM 2B - Percent of records that include data on
the four program elements (dried blood spot
screening, hearing screening, immunization, and
vital registration) within 90 days of birth
26
PM 2B

• N Number of records from the denominator that
  include these data within 90 days of birth
• D Number of records for children that include
  data on dried blood spot screening, hearing
  screening, vital registration, and one or more
  immunization events (Same as PM 2A num)

27
PM 3A - Percent of records with immunization
information available
28
PM 3A

• N Number of children in the denominator with
  records with an immunization event, other than
  first Hep B administered in the hospital,
  recorded in ICHIS
• D Total number of live births occurring in the
  jurisdiction that have a record established in
  ICHIS during a specified time interval
  (MM/DD/YYYY MM/DD/YYYY) (Same as PM 2A denom)

29
PM 3B - Percent of records with immunization
information available within 30 days of
administration

30
PM 3B

• N Number of immunization events recorded in
  ICHIS within 30 days of the date of
  administration
• D Total number of immunization events recorded
  in the integrated system in a specified time
  interval (MM/DD/YYYY MM/DD/YYYY)

31
PM 4A - Percent of records with newborn dried
blood spot screening information available
32
PM 4A

• N Number of records for children with newborn
  dried blood spot screening information recorded
  in ICHIS
• D Total number of live births occurring in the
  jurisdiction that have a record established in
  ICHIS during a specified time interval
  (MM/DD/YYYY MM/DD/YYYY) (Same as PM 2A denom)

33
PM 4B - Percent of records with newborn dried
blood spot screening information available within
two days of report from laboratory

• (Note Once initial newborn dried blood spot
  screening results are complete)

34
PM 4B

• N Number of records for children with newborn
  dried blood spot screening information recorded
  in ICHIS within two days of report from
  laboratory
• D Total established records in ICHIS for live
  births occurring in the jurisdiction with newborn
  dried blood spot screening results in a specified
  time interval (MM/DD/YYYY MM/DD/YYYY) (Same as
  PM 4A num)

35
PM 5A - Percent of records with newborn hearing
screening information available
36
PM 5A

• N Number of records for children with hearing
  screening information recorded in ICHIS
• D Total number of live births occurring in the
  jurisdiction that have a record established in
  ICHIS during a specified time interval
  (MM/DD/YYYY MM/DD/YYYY) (Same as PM 2A denom)

37
PM 5B - Percent of records with newborn hearing
screening information available within specified
time intervals for screening
38
PM 5B

• N Number of records for children with newborn
  hearing screening information recorded in ICHIS
  within 0-2, 3-7, 8-14, 15-30, and gt30 days of
  birth
• D Total number of established records in ICHIS
  with a hearing screen for children born in the
  jurisdiction in a specified time interval
  (MM/DD/YYYY MM/DD/YYYY) (Same as PM 6A num)

39
PM 6 - Percent of children with hearing results
of refer who have been evaluated by 6 months of
age and are enrolled in an early intervention or
other appropriate program OR found not to have
hearing loss
40
PM 6

• N From the denominator, number of records for
  children who were referred for confirmatory
  hearing evaluation and were found not to have
  hearing loss OR found to have hearing loss and
  were enrolled in an early intervention or other
  appropriate program within 6 months of age.
• D All resident live births occurring in the
  jurisdiction who have been referred for hearing
  screening in a specified time interval
  (MM/DD/YYYY MM/DD/YYYY).

41
PM 7A - Percent of children with a non-normal
congenital hypothyroidism screening results who
have been evaluated and are under appropriate
management by 30 days of birth
42
PM 7A

• N From the denominator, number of records of
  children who have been evaluated and are under
  appropriate management by 30 days of birth
• D All resident live births occurring in the
  jurisdiction with non-normal congenital
  hypothyroidism screening results within specified
  time interval (MM/DD/YYYY - MM/DD/YYYY)

43
PM 7B - Percent of children with non-normal
hemoglobin screening results who have been
evaluated and are under appropriate management
by 6 months of age

•  

44
PM 7B

• N From the denominator, the number of records of
  children who have been evaluated and are under
  appropriate management by 6 months of age
• D All resident live births occurring in the
  jurisdiction with non-normal hemoglobin screening
  results in a specified time interval (MM/DD/YYYY
  - MM/DD/YYYY)

45
Questions to the group

• Are these performance measures reasonable/useful?
• How would you modify them?
• Could you measure them?
• What other performance measures should be
  considered?

46
(No Transcript)
47
System Functionality ChecklistConfidentiality
and privacy

• Written policies and procedures on
• Parental notification and access
• Opt-out of sharing information
• Training of users
• Written user agreements
• Penalties for violation
• Functional audit trails
• Statement of compliance with states
  interpretation of HIPAA requirements

48
System Functionality ChecklistSecurity

• Minimum system requirements as defined by state
  or national standards (e.g., firewalls,
  encryption, time-out features, changing password
  protection)

49
System Functionality ChecklistService
functionality

• Whenever a record is viewed, needed services from
  all program components are identified to
  authorized users
• The system is capable of generating
  reminders/recalls for all program components
• The system is generating reminders/recalls for
  all program components

50
System Functionality ChecklistTechnical
functionality - 1

• Written procedures for disaster recovery
• Real-time access to enter and retrieve all
  information to which user has authorized access
• If yes, access for different categories of users
• Record includes both negative and positive
  results
• Record includes recommendations for action as
  appropriate

51
System Functionality ChecklistTechnical
functionality - 2

• System has a mechanism for identifying an
  individual child across programs
• System has a mechanism for consolidating separate
  sets of information on an individual child
• There is systematic review of the completeness of
  information in the integrated system

52
System Functionality ChecklistReports - 1

• System can generate official records and reports
  for participating programs
• System can generate an individual child health
  profile with respect to immunization, NDBS, EHDI,
  and vital registration
• System can generate aggregate reports (or data
  for reports) combining information for all
  program elements and for different population
  groups for program planning and monitoring

53
System Functionality ChecklistReports - 2

• System can import data using national standards
  (e.g., HL7 transport, LOINC)
• System can export data using national standards
  (e.g., HL7 transport, LOINC)