No Child Left Behind Subgrant Application Project Planning

1
No Child Left BehindSub-grant Application
Project Planning Evaluation

• John Cradler
• Ruthmary Cradler
• Educational Support Systems
• cradler_at_earthlink.net

2
Evaluation and NCLB Grants

• Central to the design of the project
• Document project implementation
• Document student outcomes
• Document change in teaching
• Inform future project improvements
• Increase future funding possibilities

3
3
Typical evaluation issues

• Lack of evidence about outcomes and impact of
  projects
• Lack of evidence relating project outcomes to
  project activities
• Over emphasis on subjective and anecdotal results

4
4
Project Evaluation Process
5
5
Project Planning and Evaluation
I. System
III. Instructional
II. Instructional
IV. Learning
Conditions
Outcomes
Consequences
Interventions
NCLB Intervention
Instructional
District
Student
Conditions
Change
Change

Academic

Develop

Behavioral

Implement
Technology
School

Tech-use

Integration
Conditions
Evaluate
6
I. Evaluation ReadinessA. Evaluation
Pre-planning

• 1. Project faculty administrators understand
  the necessity to evaluate
• 2. Sufficient resources have been made available
  for evaluation
• 3. Evaluation planning is part of project
  planning
• 4. NCLB Evaluation Guidelines inform the
  development of the evaluation plan

6
7
I. Evaluation Readiness B. Project Readiness

• 1. Context for the project is described
• 2. Teacher technology standards are considered
• 3. Faculty needs resources are identified
• 4. Population(s) served are defined
• 5. Objectives are clear related to needs

7
8
I. Evaluation Readiness B. Project Readiness
(cont.)

• 6. Activities are linked to objectives
• 7. Activities are scheduled
• 8. Project products, services, etc. are defined
• 9. Project personnel are identified
• 10. Training resources are described

8
9
I. Evaluation Readiness C. Develop Evaluation
Questions

• 1. Questions address each project objective
• 2. Questions can be answered by the evaluation
  strategies and assessments
• 3. Questions address extent of project
  implementation (inputs)
• 4. Questions address outcomes (outputs)
  resulting from project interventions (inputs

9
10
II. Input Evaluation Questions A.
Implementation Conditions
The extent to which. . .

• 1. Faculty stakeholders support project
• 2. Project is coordinated with existing programs
• 3. Resources to implement are available
• 4. Adequate time for project activities is
  provided
• 5. Partner support resources are evidenced

11
11
II. Input Evaluation Questions B.
Implementation Progress
The extent to which. . .

• 1. Activities were implemented as planned
• 2. Professional development was implemented as
  planned
• 3. State curriculum standards were applied
• 4. Faculty implemented project specific
  activities

12
12
II. Input Evaluation Questions B.
Implementation Progress (cont.)
The extent to which. . .

• 5. Mid-course adjustments were made as needed
• 6. Faculty observed engaging in project
  activities
• 7. Resources practices with greatest influence
  were identified
• 8. Activities unanticipated or unique to
  particular projects were implemented

13
13
III. Output Evaluation Questions A.
Instructional Change
The extent to which. . .

• 1. Faculty integrate technology into curriculum
• 2. Technology resources changed instruction
• 3. Technology resources supported the use of
  state curriculum standards
• 3. Unanticipated faculty changes occurred

14
14
III. Output Evaluation Questions B. Student
Change
The extent to which. . .

• 1. Increases in student performance were
  documented
• 2. Student use of technology increased learning
  opportunities
• 3. Student motivation was increased as a result
  of the project
• 4. Unanticipated student change was observed

15
15
IV. Evaluation Analysis Strategies B. Data
Analysis Reporting

• 1. Implementation conditions are related to level
  of implementation
• 2. Interventions are related to outcomes
• 3. Barriers and facilitating factors are
  discussed
• 4. Ongoing end-project recommendations are made
  
• 5. Reports are prepared for stakeholders
• 6. Analysis determines promising practices for
  future implementation

18
16
Causal Map for Overall Project Evaluation
GEPRA Indicators Technology Standards (four
pillars/ISTE) Federal Evaluation Guidelines Goal
s Objectives
21
17
Technology Integration Analysis

Analysis of technology in teaching and learning
in relation to instructional approach
Technology
Technology
Authentic
Authentic
Driven
Driven
Performance
Performance
Advanced
Advanced
project-based
project-based
Low Education
Low Education
Advanced
Advanced
research
research
High technology
Integration
Integration
technology
technology
evaluation
evaluation
technology
technology
Research
Research
Of highly infused
Of highly infused
integral to
integral to
technology
technology
delivering
delivering
systemic
systemic
traditional
traditional
education reform
education reform
instruction
instruction
initiatives
initiatives
Computer
Computer
Technology
Technology
assisted
assisted
as a supplement
as a supplement
instruction
instruction
Authentic
Authentic
to systemic
to systemic
Weak
Weak
Performance
Performance
supplementing
supplementing
Limited research
Limited research
project-based
project-based
education reform
education reform
traditional
traditional
Low technology
With low
research
With low
research
initiatives
initiatives


evaluation with
evaluation with
technology
technology
instruction
instruction
very limited
very limited
skill-based
skill-based
technology
technology
learning
learning
infusion
infusion
Passive learning
Engaged learning
Educational Support Systems, 1998
22