NCATE Professional Development WebConference Series

1
NCATE Professional Development Web-Conference
Series

• Reviewers Workshop Fall 2006
• Deidre Alves M.Ed.
• November 3, 2006

2
Welcome Reviewers!

• Please remember to set-up your audio
• 145pm- First audio sound check
• 155pm- Final audio sound check
• 200pm- Session will begin

3
Session Agenda

• 1. Tech Check
• 2. Eluminate Product Features
• 3. Conference Goals
• 4. Key Resources
• 5. Electronic Retrieval and Posting Process
• 6. The Program Recognition Report Template

4
Session Agenda (cont.)

• 7. The Decision
• 8. Program and Unit Relationship
• 9. Communication with Your Team
• 10. Timelines and the rolling review
• 11. Q A
• 12. Upcoming Web-Events
• 13. Conclusion

5
Introduction to Features on Elluminate

• Audio check
• Polling feature
• Synchronous text message feature
• On-air feature

6
Conference Goals

• to provide information on how to evaluate and
  submit final recognition reports
• to provide professional development training to
  serve the needs of program reviewers
• to provide a forum to enhance collaboration among
  NCATE/SPA reviewers

7
Key Resources

• www.ncate.org (Program Reviewers)
• NCATE Guidelines for Reviewers
• SPA Template
• SPA Worksheets

8
Electronic Retrieval and Posting Process

• I have been assigned a review
• now what?

9

• You will first need to access the password
  protected site
• URL
• ID
• PASSWORD

10

• URL http//notes.ncate.org/progrev/progreportF06.
  nsf
• ID and PASSWORD
• Sent via email

11

• Once in the site
• You will see your name and program assignments

12

• Program Report
• vs.
• Program Recognition Report

13

• Click on the Institution to view Program Report
  on the Password Protected site
• Access SPA Program Recognition Report template on
  the NCATE site (save to desktop)
• Attach the completed SPA Program Recognition
  Report with Decision to the Password Protected
  site

14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17

• Some items to note

18

• All reviewers on the team should complete
  individual reports and contribute to the final
  report

19

• If the report requires revision by the
  institution in the next semester, the same team
  is assigned

20

• If assigned a revised report
• The Program Report and SPA Recognition Report
  have been made available at

21

• URL (old archived reports)
• http//notes.ncate.org/progrev/progreportf05.nsf
• -or-
• http//notes.ncate.org/progrev/progreports05.nsf
• -or-
• http//notes.ncate.org/progrev/progreports06.nsf

22

• We have talked about the PROCESS of accessing the
  Program Report and Program Recognition Report---
• Lets talk about the CONTENT of the Program
  Recognition Report

23
The SPA Program Recognition Report Template

• Templates available at www.ncate.org
• Click program reviewers then resources for
  reviewing
• Template format is consistent across SPAs

24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27

• Examples of well-written Summary of Strength
  statements
• emphasis on reflective practitioner throughout
  education coursework
• use of an action research project that focuses
  candidates on their effect on student
  learning
• beginnings of a comprehensive program
  assessment system that when refined should
  provide useful, current information on candidate
  success for improving the program and tracking
  candidate progress

28

• Characteristics of a well written Summary of
  Strength statement
• Evidence based
• Clear
• Objective
• Finds a strength even in a weak program

29
(No Transcript)
30

• Met- comment not required
• Not Met- comment is required

31

• Examples of well-written comments for an unmet
  standard
• Standard 2.4. Candidate use practices designed
  to assist students in developing habits of
  critical thinking and judgment. NM
• The assessment cited for this standard has
  insufficient specific reference to critical
  thinking and judgment rather it is focused on
  planning lessons that improve student
  understanding through the use of various reading
  processes.

32

• There are no references to media or technologies
  in cited assessments. This standard does not
  focus on texts.
• The program cites assessment 5, the rubric and
  data table based on the state assessment, as
  providing evidence that this standard has been
  met. However, reviewers found no mention of
  building family and community relationships in
  this assessment tool. Also, reviewers found no
  scoring guides or rubrics in the information
  provided to help understand how a candidate is
  rated as distinguished, proficient or
  developing. Again, there was no information
  specific to the relationship of the state
  assessment and the NAEYC standards, including
  standard 2

33

• Why was this comment well written?
• CLEAR
• OBJECTIVE STATEMENTS
• RELATES DIRECTLY TO THE STANDARD
• DIRECT

34
(No Transcript)
35

• An example of a well-written Part C.1-
  Candidates knowledge of content
• The institution indicated assessments to
  address areas of content knowledge essential for
  elementary teachers. However, evidence from
  Assessment 2 was not available other
  assessments provided only limited data for some
  areas of content knowledge (i.e., science and
  social studies) and no evidence was available to
  indicate knowledge of the arts, health education,
  and physical education.

36

• An example of a well-written Part C.2-
  Candidates ability to understand and apply
  pedagogical and professional content knowledge,
  skills, and dispositions
• The institution indicated Assessments 3, 4,
  and 5 address ACEIs standards applicable to
  pedagogical knowledge and skills. However, the
  elements of the scoring rubrics and the candidate
  data presented only indirectly or narrowly
  addressed most of these standards. It was
  difficult for reviewers to see relationships from
  the information and candidate data presented that
  was clearly directed toward and aligned with the
  applicable standards.

37

• An example of a well written Part C.3- Candidate
  effects on P-12 student learning

There was no evidence to indicate candidates
collect data on students for the purpose of
subsequent specific planning and implementation
of instruction.
38

• More Examples
• C.1Candidates knowledge of content
• The Evidence of Meeting Standards chart is
  particularly helpful in making connections
  between the assessments and the standards.
• Assessments show multiple ways in which content
  is covered in the program.
• C.2Candidates ability to understand and apply
  pedagogical and professional content knowledge,
  skills, and dispositions
• All cited assessments give an opportunity for
  pre-service teachers to begin to learn the
  language of teaching.
• Effectively developed assessments in this area
  make the program strong.
• C.3Candidate effects on P-12 student learning
• Analysis of student learning is clearly covered
  in the assessments. Many of the projects are
  cross-referenced and cover content, pedagogy as
  well as student learning impact standards.

39

• Well-written statements for part C-
• CLEAR/ DIRECT
• INFORMATIVE
• OBJECTIVE
• STATEMENTS not PRESCRIPTIONS

40
(No Transcript)
41

• Example of a well-written Part D
• Evidence that assessment results are evaluated
  and applied to the improvement of candidate
  performance and strengthening of the program (as
  discussed in Section V of the program report)
• Supportive evidence is clear for all
  assessments and there is a procedure in place for
  the evaluation and application of that evidence
  for the improvement of candidate performance and
  strengthening of the program.

42

• The program appears to understand that it has
  considerable work to do some significant
  weaknesses have been identified and the program
  indicates it knows it must work on these. From
  this realization can come significant change and
  strengthening of the program. This review should
  assist in the process as well.
• The university has provided evidence that they
  have made adjustments to their program based on
  data collected from the first cohort of
  candidates. In one example from the Contextual
  Statement, an analysis was made of the match
  between content expectations of the program and
  of candidates performance on the Praxis II exam
  as a way to determine monitor trends that may
  indicate the need for adjustment to the program.

43

• A well-written Part D comment
• Evidence based
• Active on-going evaluation of data
• Improvement driven proof

44
(No Transcript)
45
(No Transcript)
46

• C O N D I T I O N S
• (only for Rec with Conditions rating)

47

• This report functions as a contract between
  NCATE-SPA and the Institution.
• In order for an Institution to retain Recognition
  after a Recognized with Conditions decision has
  been granted

48

• THE CONDITIONS BOX
• MUST BE FILLED OUT

49

• Write in the SPECIFIC conditions that need to
  be met by the institution on the next report
  submission.

50

• The revised team that looks at this when it is
  re-submitted will zero in on this conditions
  section to determine whether they were met or not
• High Stakes!

51

• An example of a well-written CONDITION
• Assessments 1 and 2 The program needs to
  address candidate knowledge of content by
  standard (academic discipline) and by category
  (sub scores) on the Praxis II exam. They must
  also present aggregated candidate data (grades)
  in a table or chart of candidate scores from high
  to low for each standard.
• Assessment 5 Candidate impact on student
  learning needs to be addressed by the degree to
  which secondary students learned from candidate
  instruction. The unit plan assignment could well
  yield such data if the candidate taught it in a
  secondary classroom and conducted something as
  simple as a pre and post test on the unit.
• Standard 3.2 The program must identify the
  instructor(s) of the Social Studies Methods
  course(s) and indicate their backgrounds in
  social studies education or in one of the
  disciplines.
• Data from all assessments for the next year must
  be collected and analyzed.
• Additional procedures for evaluating post
  baccalaureate candidates need to be implemented
  that ensure standards and indicators are met.
• Concerns cited under Standard 1 must be
  addressed.

52

• Characteristics of a well-written Conditions
  statement
• CLEAR
• PRECISE
• OBJECTIVE
• STANDARDS BASED
• EVIDENCE BASED

53
The Decision

• Program Report Decisions
• Nationally Recognized
• Nationally Recognized w/Conditions
• Not Nationally Recognized

54

• Data
• Standards
• Assessments (narrative, scoring guide (rubric)
• Alignment (standards, assessments, scoring guide,
  data)
• Tone
• Sections A-G

55

• Assessments should bring to life the
  standards---standards in action
• Standards should not decorate the assessment

56

• SPA reviewers will make one of the following
  decisions based on your program report
• The program is nationally recognized.
• The program is nationally recognized with
  conditions.
• Insufficient data
• Insufficient alignment
• Poor assessment, scoring guides, etc
• 80 rule
• The program is not nationally recognized a new
  program report may be submitted.

57

• Data Rule
• Until Spring 07, the following rule will apply
  If the program has at least one semesters data
  on at least five assessments and everything else
  is appropriate then a reviewer can recognize the
  program.
• If less than five then the reviewer should
  recognize with conditions.

58
A note about tone

• Reviewer attitude in report decisions is
  critical---
• Remember that you want to be as helpful as
  possible to the institutionstating the problem
  in clear objective languageavoid a God-like tone

59

• Reviewers are asked to accept the integrity of
  the program and review it for the purpose of
  determining if the information submitted meets
  the SPA standards
• Do not try to redesign the programs
• Objectively evaluate the QUALITY of the
  performance assessments and rubrics with respect
  to the alignment w/ SPA standards
• POSITIVE PERSPECTIVE
• Should include strengths and weaknesses
• Adapted from IRA- Guidelines for IRA/NCATE
  Program Reviewers October 4, 2006

60

• The BIG question!

61

• DOES THIS PROGRAM MEET THE STANDARDS
• ?

62
Program and Unit Relationship

• How does my Program Recognition Decision fit
  into the Accreditation Process?

63

• The BOE (Board of Examiners)-
• will use these final report findings (across
  programs) at an institution to evaluate whether
  NCATE Standard 1 and 2 have been met

64
Communication With Your Team

• Lead makes contact
• Team sets work schedule
• Conference call capability

65
TIMELINESUMMARY
66
NCATE Deadline

• Final Posting to Institution
• By 2/15 Initial Submissions
• By 3/15 Revised Submissions
• By 4/15 Latest Revised
  Submissions

67
Q A

• To pose a question
• Click hand icon
• Push talk icon to begin and end audio
• Please state name and SPA affiliation
• Questions will be taken in the order they
  are received

68
Upcoming Web Events

• Series Foci Target Audience Date Time
• Program (Pilot) Institutions 8/10
  1000am
• Semester Launch SPA Coordinators 10/10
  200pm
• Newbie Workshop New Reviewers
  10/24 200pm
• Leads Procedures Lead Reviewers
  10/27 200pm
• Reviewer Guidelines Reviewers
  11/3 200pm
• Final Audit Procedures SPA Coordinators
  12/1 200pm

69
Conclusion

• Web-Conference Archive
• Next Professional Development Web Event
• December 1st at 200pm-
• SPA Coordinators Forum
• Update on New Program Report Database
• Brief Survey

70
BRIEF SURVEY

• A- Extremely
• B- Somewhat
• C- Not at all
• 1. Overall, how helpful was the Elluminate Web
  Conference Presentation?
• 2. How useful was the Eluminate Web conference
  product?
• 3. How clear was the information presented at the
  Web conference?
• 4. How comfortable do you feel with the NCATE
  Reviewer process after attending the
  conference?
• 5. How organized was the correspondence in
  getting you ready to attend the conference
  (emails, e-flyers, confirmations, configuration
  information)?

71
Program Review Staff

• Margie Crutchfield Ph.D.
• Associate Vice President, Program Review
• Deidre Alves M.Ed.
• Accreditation Associate, Program Review
• Jaye Bishop
• Accreditation Associate, Program Review
• Monique Thomason, Accreditation Assistant,
  Program Review
• James Convery, Director Information Systems
• Julien Goichot, Webmaster