Quality Assurance and Head Start Craig T. Ramey Distinguished Professor and Director Georgetown Univ

1
(No Transcript)
2
QA as a concept maintaining acceptable
standards by systematic examination and
judgement. A means to promote excellence of
goods or services.
3
Quality can be measured and quantified.
4
Head Start is Americas commitment to Equality
for young children and their families. They
deserve our best efforts.
5
The National Reporting System is a key process
in establishing a practical quality assurance
system for Head Start.
6
The advisory group to the National Reporting
System is supportive of a commitment to a fair
and practical system that will help teachers,
directors, and senior officials continue to
improve their programs.
7
Key consensus points

• recognize and plan for language diversity
• clearly specify the data analysis and reporting
  format
• keep child assessment battery to a useful minimum

8
Key consensus points (continued)

• provide adequate training and technical
  assistance for quality assurance implementation.
• develop a plain English guide to the National
  Reporting System
• improve measurement instruments as needed

9
NATIONAL REPORTING SYSTEM TIMELINE

• April 2002
• May 2002
• June 2002

• President Bush announces Good Start, Grow Smart
• Focus Group of Program Managers Experts
• ACF/NICHD Workshop on School Readiness Measures

10
NATIONAL REPORTING SYSTEM TIMELINE

• July 2002
• Sept. 2002
• Dec. 2002

• Discussion Session on National Reporting System
  (NRS)
• Award Contract for NRS Development and Field Test
• NRS Technical Work Group Meeting I

11
NATIONAL REPORTING SYSTEM TIMELINE

• Jan. 2003
• Feb. 2003

• Associate Commissioners National Meeting of
  Directors
• Focus Groups of Head Start Directors
• Technical Work Group Meeting II

12
NATIONAL REPORTING SYSTEM TIMELINE

• Language Diversity Group Consultation
• Field Test in 36 Programs
• Public Comment on Data Collection

• March 2003
• April 2003

13
NATIONAL REPORTING SYSTEM TIMELINE

• May 2003
• June 2003

• Technical Work Group Meeting III
• Review Field Test Outcomes
• National Satellite Broadcast

14
Head StartNational Reporting System

• Overview of Child Assessment

15
Purpose

• To provide local programs and the national
  program office with indicators of the progress
  that children are making

16
Purpose

• in acquiring pre-reading and early numeracy
  skills that are important for success in
  elementary school.

17
Overview of Assessment

• All 4- and 5-year-old children will be given the
  same one-on-one assessment.

18
Overview of Assessment

• Fall and spring assessments will be conducted to
  report gains over course of Head Start year.

19
Overview of Assessment

• Childrens individual scores will be aggregated
  and reported at program level.

20
Contextual Information WillAlso Be Collected

• For example
• Program type
• Student demographics

21
Contextual Information WillAlso Be Collected

• For example
• Proportions of children who are English-language
  learners
• Proportions of children who have diagnosed
  disabilities

22
Criteria for Assessment Tasks

• Critical stepping-stones on the path to
  achievement in elementary school
• Can be enhanced by Head Start services

23
Criteria for Assessment Tasks

• Skills that Congress and the President expect
  children to learn in Head Start

24
Technical Adequacy of Measures

• Internal consistency and predictive validity of
  each instrument have been examined for technical
  adequacy

25
Criteria for Assessment Tasks

• Can be reliably measured in a relatively brief
  child assessment conducted by a Head Start
  teacher or staff member

26
Five Assessment Components

• Comprehension of Spoken English
• Vocabulary
• Letter Naming
• Phoneme Deletion
• Early Math Skills

27
Comprehension of Spoken English

• Addresses the following Congressional goals
• Develops ability to understand and use language

28
Comprehension of Spoken English

• Addresses the following Congressional goals
• Uses increasingly complex and varied vocabulary

29
Comprehension of Spoken English

• Addresses the following Congressional goals
• Progresses in understanding and speaking English
  (for non-English speakers)

30
Comprehension of Spoken English

• Enjoyable tasks that help to establish rapport
  with child

31
Comprehension of Spoken English

• Tasks used as an English-proficiency screener for
  determining language of assessment

32
Vocabulary

• Addresses Congressional goal Understands
  increasingly complex and varied vocabulary

33
Vocabulary

• Children receive picture-vocabulary items
  selected for content and appropriate range of
  difficulty

34
Vocabulary Content Areas

• Body parts
• Emotions and feelings
• Activities of daily living

35
Vocabulary Content Areas

• Work activities and objects
• Plants and animals

36
Letter Naming

• Addresses Congressional goal Identifies at
  least 10 letters of the alphabet, especially
  those in own name

37
Letter Naming

• Provides estimate of the number of letters Head
  Start children know at the end of Head Start

38
Phoneme Deletion

• Addresses Congressional goal Phonological
  awareness

39
Early Math

• Addresses Congressional goal Numbers and
  operations

40
Early Math

• Items selected for varied content and difficulty
  level appropriate for preschoolers

41
Early Math Content Areas

• Numbers
• Shapes
• Relative size judgments

42
Early Math Content Areas

• Simple measurement
• Ability to solve simple word problems involving
  counting or basic addition or subtraction

43
Spanish Version of NRS Assessment Battery

• Used to assess Spanish-speaking children

44
Spanish Version of NRS Assessment Battery

• Efforts continuing to expand battery to assess
  children in other languages

45
Assessment Components
46
Assessment Components Vocabulary

• 24 picture-vocabulary items
• Human body parts
• Emotions and feelings
• Activities of daily living
• Work-related activities
• Plants, animals, habitats

47
Assessment Components Letter Naming

• Children are shown all 26 letters (30 in Spanish
  version)
• Provides estimate of the number of letters Head
  Start children know at the end of Head Start

48
Assessment Components Early Math

• Number identification
• Shape recognition
• Relative size judgments
• Simple measurement
• Simple word problems

49
Assessment Components Spanish Version

• Spanish versions of all subtests are administered
  to track progress of Spanish-speaking children,
  whether or not they are assessed in English.

50
Assessment Components Spanish Version

• Spanish-speaking children who score at or above
  the cutoff on Simon Says and Art Show also
  receive English assessment in a separate session.

51
Assessment Components Spanish Version

• Spanish-speaking children who score at or above
  cutoff on Tio Simon and Exposicion de Arte
  receive Spanish assessment

52
Development Field Test

• 36 Head Start Programs (including 2 Migrant
  Seasonal and 2 American Indian programs)
• 101 centers
• 156 classrooms
• 1,430 children

53
Development Field Test

• Field test found measures were appropriate for 4-
  and 5- year old Head Start children and provided
  reliable data
• Average levels of performance were in the middle
  to upper range of scores on all subtests

54
Development Field Test

• One task (La Casita) was improved based on field
  test feedback TWG input
• One task (Elision) was removed for further
  development

55
Development Field Test Future Priorities

• Site visits to 40 programs to observe Year I
  implementation.
• Create clear, useful forms for reporting and
  using NRS information.
• Improve and expand child assessment, including
  versions in other languages.

56

• Good Start, Grow Smart
• an opportunity to strengthen Head Start

57

• Head Start National Reporting System Technical
  Work Group Directory
• Jason L. Anthony, Ph.D., Ed.S. University of
  Houston
• Clancy Blair, Ph.D. Pennsylvania State
  University
• Margaret (Peg) Burchinal, Ph.D. The University
  of North Carolina at Chapel Hill
• Richard Clifford, Ph.D. The University of North
  Carolina at Chapel Hill
• Linda Espinosa, Ph.D. National Institute for
  Early Education Research
• Nicholas Ialongo, Ph.D. Johns Hopkins
  University
• Graciela Italiano-Thomas, Ed.D. Centro de la
  Familia de Utah
• Jacqueline Jones, Ph.D. Educational Testing
  Service
• Ann P. Kaiser, Ph.D. Vanderbilt University
• Samuel J. Meisels, Ed.D. Erikson Institute
• Fred Morrison, Ph.D. University of Michigan
• Robert C. Pianta, Ph.D. University of Virginia
• Craig Ramey, Ph.D. Georgetown University
• W. Douglas Tynan, Ph.D. ABPP Wilmington,
  Delaware
• Jane Wiechel Ohio Department of Education

58

• Head Start National Reporting System Technical
  Work Group Directory
• FEDERAL STAFF
• Wade Horn, Ph.D. Assistant Secretary, US
  Department of Health and Human Services
• Windy Hill Head Start Bureau, US Department of
  Health and Human Services
• Doug Klafehn Head Start Bureau, US Department
  of Health and Human Services
• Tom Schultz Head Start Bureau, US Department of
  Health and Human Services
• Michelle Plutro Head Start Bureau, US
  Department of Health and Human Services
• Howard Rolston, Ph.D. ACF, US Department of
  Health and Human Services
• Kyle Snow, Ph.D. NIH, US Department of Health
  and Human Services
• Head Start National Reporting System Language
  Diversity Group
• Linda Espinosa, Ph.D. National Institute for
  Early Education Research
• Victoria R. Fu, Ph.D. Virginia Polytechnic
  Institute State University
• Vera F. Gutierrez-Clellen, Ph.D. San Diego
  State University
• Graciela Italiano-Thomas, Ed.D. Centro de la
  Familia de Utah
• Mary Lou de Leon Siantz, Ph.D., RN, FAAN
  Georgetown University

59

• Nicholas Zill, Project Director
• Kwang Kim, Peggy Hunker, John Brown,
• Philip Fletcher, Norma Ruffat, Alberto Sorongon,
• Westat
• Ruth Hubbell McKey, Project Director
• Helena Wallin-Miller, Peter Liu, Joanna De Wolfe
• Xtria
• Tom Schultz, Project Officer
• Administration for Children and Families,
  
• U.S. DHHS