Evidence Based Academic Enrichment: WHAT WORKS

1
Evidence Based Academic Enrichment WHAT WORKS???

• PRESENTED BY-
• CYNTHIA BELLIVEAU, Ph.D.
• Director, Pennsylvania Service Learning Alliance
• University of Pennsylvania
• P 117 Franklin Building Annex
• Philadelphia, Pa 19104
• 215 573-6535
• bellivea_at_pobox.upenn.edu
• www.paservicelearning.org

2
Monitoring Question?????

• CONTRACT MANAGEMENT, GOAL
• SETTING, SUSTAINABILITY
• The program has implemented academic enrichment
  activities that are evidence based.
• - Program materials
• - Rationale for choosing specific activities
• - Research Documentation

3
Objectives of Workshop

• Understand the NCLB Legislations focus on
  rigorous scientific research
• Understand the basics of scientific research
  (evidence based)
• Be able to assess educational programs for
  scientific rigor
• Thoughtfully plan for educational programming
  within your 21 CCLC

4
No Child Left Behind

• NCLB requires educational practitioners
  (including after-school programs) to use
  scientifically-based research to guide their
  decisions about which educational interventions
  to implement.

5
No Child Left Behind

• Under No Child Left Behind, federal support is
  targeted to those educational programs that have
  been demonstrated to be effective through
  rigorous scientific research.

6
What does scientifically based research really
mean?

• To say that an instructional program or practice
  is grounded in scientifically based research
  means there is reliable evidence that the program
  or practice works. For example, to obtain
  reliable evidence about a reading strategy or
  instructional practice, an experimental study may
  be done that involves using an experimental/contro
  l group design to see if the method is effective
  in teaching children to read.

7
Why this focus on Scientifically Based Research?

• Not enough time, and not enough money to waste on
  programs that DONT WORK!!!!!!!!!!!!!!
• Only way to know something works (really) is to
  research the impact in rigorous ways.

8
Step-by-Step Process

• How to evaluate whether an educational
  intervention is supported by rigorous evidence!

9
First Is the intervention backed by strong
evidence of effectiveness?

• Randomized control Trials showing effectiveness
• trials that are in two or more school
  settings
• well designed and including an after-school
  similar to
• implemented yours.
• STRONG EVIDENCE

10
Second If the evidence is not backed by strong"
evidence, is it backed by possible evidence of
effectiveness?

• Types of Studies for Possible Evidence
• Randomized controlled studies which are good (but
  not strong).
• Comparison-group studies in which the
  intervention and comparison groups are very
  closely matched in academic achievement,
  demographics and other characteristics.

11
Types of Studies Which are NOT Considered Strong
Evidence

• Pre-post studies.
• Comparison-group studies in which the
  intervention and comparison groups are not
  closely matched.
• Meta-analyses that include the results of
  lower-quality studies.

12
Key Characteristics Of Reliable Research

• Scientific MethodA hypothesis about what works,
  or how it works, is formulated a treatment group
  and control group are used in a study to try to
  disprove the hypothesis.
• ReplicatedSeveral studies find the same result.
• GeneralizedStudy findings can be applied
  broadly, to students other than the ones studied.
• Meets Rigorous StandardsThe study's design,
  measures, and interpretation of results meet
  rigorous standards of peer review.
• Convergent findingsResults found using different
  approaches all point to the same conclusion.

13
Randomized Control Trials the GOLD STANDARD.

• Randomized controlled trials are
• studies that randomly assign
• individuals to an intervention group
• or to a control group, in order to
• measure the effects of the
• intervention. When you see this type
• of design you can be reasonably
• certain the study is rigorous.

14
What is a randomized controlled trial?

• Randomized controlled trials are studies that
  randomly assign individuals to an intervention
  group or to a control group, in order to measure
  the effects of the intervention.

15
Where can you find these well researched programs?
16
What Works Website!!

• On an ongoing basis, the What Works Clearinghouse
  (WWC) collects, screens, and identifies studies
  of the effectiveness of educational interventions
  (programs, products, practices, and policies).
  They review the studies that have the strongest
  design, and report on the strengths and
  weaknesses of those studies against the WWC
  Evidence Standards so that you know what the best
  scientific evidence has to say.

17
What Works Clearinghouse

• The WWC does not endorse any interventions nor
  does it conduct field studies. The WWC releases
  study, intervention, and topic reports. A study
  report rates individual studies and designs to
  give you a sense of how much you can rely on
  research findings for that individual study. An
  intervention report provides all findings that
  meet WWC Evidence Standards for a particular
  intervention. Each topic report briefly describes
  the topic and each intervention that the WWC
  reviewed.

18
What Works Website

• http//www.whatworks.ed.gov/

19
Promising Practices Website

• http//www.promisingpractices.net
• Highlights programs and practices
• that credible research indicates are
• effective in improving outcomes for
• children, youth, and families.

20
International Campbell Collaboration Website

• http//www.campbellcollaboration.org
• Offers a registry of systematic
• reviews of evidence on the effects of
• interventions in the social, behavioral
• and educational arenas.

21
Social Programs That Work Website

• www.evidencebasedprograms.org
• Offers a series of papers developed
• by the Coalition for Evidence-Based
• Policy on social programs that are
• backed by rigorous evidence of
• effectiveness.

22
After-School Exchange

• http//www.thirteen.org/edonline/afterschool/index
  .html
• Not specifically about research
• based curriculums, but great ideas
• for moving from the school day
• lesson to an after school lesson.

23
The Goal Moving to PA Standards and PSSA Success

• AdvancedThe Advanced Level reflects superior
  academic performance. Advanced work indicates an
  in-depth understanding and exemplary display of
  the skills included in the Pennsylvania Academic
  Content Standards.
• ProficientThe Proficient Level reflects
  satisfactory academic performance. Proficient
  work indicates a solid understanding and adequate
  display of the skills included in the
  Pennsylvania Academic Content Standards.
• BasicThe Basic Level reflects marginal academic
  performance. Basic work indicates a partial
  understanding and limited display of the skills
  included in the Pennsylvania Academic Content
  Standards. This work is approaching satisfactory
  performance, but has not been reached.  There is
  a need for additional instructional opportunities
  and/or increased student academic commitment to
  achieve the Proficient Level.
• Below BasicThe Below Basic Level reflects
  inadequate academic performance.  Below Basic
  work indicates little understanding and minimal
  display of the skills included in the
  Pennsylvania Academic Content Standards. There is
  a major need for additional instructional
  opportunities and/or increased student academic
  commitment to achieve the Proficient Level.

24
Pa. Department Education ACADEMIC STANDARDS

• Language Arts
• Math
• Arts and Humanities
• Environment and Ecology
• Science and Technology
• Family Consumer Science
• Geography
• Economics
• History
• Civics and Government

25
For Example Language Arts Standards

• 1.1 Learning to Read Independently
• 1.2 Reading Critically in All Content Areas
• 1.3 Reading, Analyzing and Interpreting
  Literature
• 1.4 Types of Writing
• 1.5 Speaking and Listening
• 1.6 Characteristics and Function of the English
  Language
• 1.7 Research

26
State Standards and Assessment Anchors

The Assessment Anchors are one of the many tools
the Pennsylvania Department of Education has
developed to better align curriculum, instruction
and assessment practices throughout the state. 
PDE first released the Assessment Anchors for
Mathematics and Reading in the spring of
2004.  The Assessment Anchors, like the
Standards, are dynamic documents and will be
adjusted periodically.
27
Assessment Anchors are tools to

• Design the PSSA
• Connect PSSA to classroom practice
• Clarify the PA Standards in Reading and
  Mathematics

28
Why do we need Assessment Anchors?

• Clarity
• Focus Alignment
• Grade appropriate skills
• Curricular flow
• Rigor
• Manageability

29
How to Read Assessment Anchor Coding
M8.E.1.1
Math
Assessment Anchor
Grade Level
Reporting Category
30
What is Standards in Practice?

• SIP is a six-step, team-based professional
  development strategy that can be used to align
  any assignment or program with state content
  standards.

31
Six Step Model

• The teacher describes the assignment.
• The team analyzes the demands of the task.
• The team identifies the standards that apply to
  the assignment.
• The team generates a rough scoring guide.
• The team scores the student work, using the guide
• The team discusses a) revisions to the assignment
  and/or re-teaching without repeating.

32
Website for Additional Information about the PA
State Standards

• http//www.growparents.com/paelementary/

33
After School activities can and SHOULD be as
educational as classroom lessons, but they also
need to

• Be informal
• Be active
• Be FUN!
• Be delivered in small "chunks" -- brief sessions
  that include warm-up activities, follow up, and
  transitions for younger children
• Provide opportunities for social growth and
  development
• Teach kids to work in groups

34
Key principles of developing an after-school
curriculum

• Structure
• Variety
• Choice
• Student Input
• Student Needs particularly - ACADEMIC!
• Logistics (time, staff, environment)
• Budget

35
1) Provide Structure with Flexibility

• Provide Structure with Flexibility One of the
• most important things to remember in designing a
• curriculum is that children need a balance
  between
• structured activities (such as building a model
  of a
• bridge with a group of classmates, or working
  with a
• partner to write a scene for a play) and
• unstructured time, where they are free to choose
• their own activities and partners. Most school
  days
• are highly structured, and kids need some time,
• particularly in the after school hours, to unwind
  and
• follow their own inclinations. Time spent in
• unstructured activities will help them to focus
  on the
• structured activities you offer.

36
2) Plan a Variety of Activities

• When it comes time to plan your activities, try
  to provide a wide range of opportunities for
  learning. Include a mix that develops academic,
  social, physical, and emotional skills, while
  giving your students the chance to develop
  hobbies, skills, and interests they might not
  otherwise be able to explore. Community service
  projects and other hands-on activities can help
  your students cultivate positive character
  traits, and enhance their sense of community.

37
2) Plan a Variety of Activities

• The after school hours provide an ideal time and
  place to get your students excited about learning
  to pursue their own interests, and it helps them
  develop self-confidence as they explore new
  talents in areas that may not be addressed by the
  regular school curriculum. Make sure the
  activities you offer are fun and engaging, no
  matter what they are designed to teach. Most kids
  are tired after a long day at school, and they
  will be best able to absorb the content of a
  lesson if it looks more like play and less like a
  traditional classroom lesson

38
Step 3 Give Students Choices

• You should structure your program so
• students have daily choices about
• how they will spend their time. This
• encourages students to take
• responsibility for their own learning,
• and allows the program staff to
• better meet the needs of all their
• students.

39
Step 4- Provide Opportunities for Student Input

• By working with students to develop
• curriculum activities, you will allow
• them to develop a sense of ownership
• of the program, develop
• responsibility, and select activities
• that reflect their interests as they
• help plan and lead activities.

40
Step 5 - Pay Attention to the Particular Needs of
Your Program

• A thorough knowledge of the needs
• and desires of your students, staff,
• families, and community will allow
• you to fine-tune activities for the best
• possible fit with your unique
• population.

41
Step 6 - Pay Attention to Time, Environmental,
and Staffing Constraints

• Make sure activities work within the
• time allotted for the program, and for
• the environment in which it will be
• performed. There should be a
• sufficient number of qualified staff to
• meet the requirements of the activities.

42
Step 7 - Pay Attention to the Budget

• Make sure activities work within the
• budget allotted for the program.