The Value of NCATE

1
The Value of NCATE

• A Study of Research Institutions and NCATE
  Accreditation
• AACTE, Friday, February 8, 2008
• Emerson J. Elliott
• NCATE, February 2008

2
Initial report from a study

• Research institutions that are accredited by
  NCATE
• Interviews of deans, NCATE coordinators and
  faculty
• Perceptions of NCATE accreditation

3
Topics in this session

• Setting for the study
• Who participated
• Interview questions
• Response themes
• Summing up

4
A. Setting for the study

• AACTE Fall 2004 study of deans
• CADREI fall 2006, and AACTE committee
• Streamline
• Reduce burden and cost
• Address time commitment required of BOE and UAB
  members
• Dovetail program review and unit review
• Make more collaborative, less punitivemore like
  some other specialized accreditors
• Brought to the NCATE Executive Board

5
A. Continued, Setting

• Executive Board also set goals for NCATEs
  management plan (Fall 2006-Spring 2007)
• Reduce the burden of accreditation
• Improve service to institutions
• Increase the value and perceived value of the
  accreditation process to institutions
• And NCATE was concerned that
• Research universities are the source of new
  knowledge in educator preparation. and for NCATE
  unit and specialized professional standards. . .

6
A. Continued, Setting

• But faculty from research universities are
  infrequently involved with NCATE
• Research university faculty are often not among
  the participants in standards writing
• Andprior to a change in 2007research university
  faculty had participated rarely as examiners or
  on NCATEs policy boards
• 315 session at AACTE today
• Arranged meeting with research university deans,
  January 2007

7
A. Continued, Setting NCATE meeting with
research universities

• Outcomes
• Symbiotic relationship between research on
  teaching and learning produced by research
  universities and use of that research to improve
  educator preparation
• Continue discussions with CADREI and AACTE
• Urge more BOE and board candidates from research
  institutions
• Streamlining to ensure an efficient and effective
  system
• As one way to support these outcomes, and to
  ground the value goal in the management plan,
  NCATE decided to ask the research institutions
  themselves what they think about NCATE
  accreditation.

8
B. Who participated?

• Five research institutions accredited by NCATE
• Five interviews in each institution
• Dean
• Coordinator
• Three faculty designated by the dean
• 4 associate deans
• 4 department chairs
• 6 program directors and/or SPA coordinators
• 1 assessment coordinator and doctoral student

9
B. continued, Who participated Interviews

• All conducted by telephone
• All interviewees gave explicit permission to
  record the interview
• All interviews followed the same interview
  questions
• All interviewees were promised anonymity
• All transcribed

10
B. Continued, Who participated Selected
institutions

• All in the East of the Mississippi
• Three are private not-for-profit, two are public
• Three participated in the NCATE program review
  process, two did not
• All five are doctorate granting, four are
  Carnegie high R D (10 to 150 million in
  2005), one is Carnegie very high (over 250
  million in 2005)
• Two have around 20 million in social sciences R
  D in 2005
• Enrollments for 2004 range from around 12,000 to
  more than 42,000

11
B. Continued, Who participated What is not
reflected in the participating institutions?

• Institutions that NCATE does not accredit
• Institutions in the West
• All NCATE accredited research
  institutionsselected from two recent cohorts of
  UAB action
• Sample is small
• But
• Still found a range of differences across
  institutions and individuals

12
B. Continued,Who participated Accreditation
experience

• All had accreditation visits in 2004-2006
• One was having an initial visit
• Four were continuing visits
• Two had all standards met
• Two required focused visits related to standards
  1 and 2 and are now fully accredited
• One has a coming focused visit related to
  standards 1 and 2

13
B. Continued, Who participated Time warp
14
C. Interview Questions

• Overall topics were
• Value that respondents associate with NCATE
  accreditation
• How NCATE does its job
• Advice about how accreditation could be improved
• Findings from 3 questions and then 6
  themestopics that recurred in the interview
  responses

15
C. Continued, questions (1) Overall reactions

• 72 (18 respondents) positive
• Positive, very positive, useful learning
  experience, important to do
• Includes 3 deans
• 28 (7 respondents) negative
• Technical and prescriptive requirements
• Time and resource intensive
• Too much assessment, compliance, SPA instructions
  changed
• Includes 2 deansdifferent experiences

16
C. Continued, questions (1) Overall reactions

• Additional observations
• Many at our institution think we dont need
  national accreditationwe have really high
  standards
• The process allowed for considerable flexibility
  in applying standards
• The team were true professionals
• The self-study aspect is useful

17
C. Continued, questions (2) Most helpful

• MOST helpful
• Deansreflection, self study, faculty
  collaboration
• Coordinatorsassessment
• Facultyfitting assessments to standards,
  collegial activities among the faculty

18
C. Continued, questions (3) Least helpful

• LEAST helpful
• Deanstime, cost, burden, team visit
• Coordinatorsmixed Praxis, changing rules,
  duplicate NCATE and state requirements, lack of
  research institution team members
• Faculty
• Assessmentsconfusion, rubrics for standards,
  changing requirements
• BOE teamcomposition, logistics, use of web
  information

19
D. Response themes

• Frequently recurring topics that were not
  explicit interview questions
• Research institution self descriptions
• BOE teams
• Program review and SPAs
• Defining evidence for NCATE
• TEAC references
• Statescontrasting perspectives

20
D. Continued, themes (1) Research institutions

• 19 comments, 12 respondents, all 5 institutions
  24 points
• Character of the institutions
• Accreditation is not the way they think
• Candidates arrive at the graduate level and have
  completed subject content courses
• Specialization
• Emphasis on scholarship and research
• Always under study
• Professional life of the faculty
• Own projects and funding
• NCATE takes time from grant writing
• Faculty change courses all the time, and their
  assessments
• Little motivation to volunteer for NCATE work
• BOE teams often dont understand their qualities

21
D. Continued, themes (2) BOE teams

• 48 references (largest number), 18 respondents in
  5 institutions.
• Distinction between one institution that
  characterized its visit as disastrous and all
  other

22
D. Continued, themes (2) BOE teams

• Positive statements about BOE teams--12
• The teamtrue professionals, collegial,
  business-like, outstanding team, chair from a
  large research university, respected what we do,
  team understood who we are, all in this together,
  good team, worked hard

23
D. Continued, themes (2) BOE teams

• Negative statements about BOE teams
• The team in all other institutions6
• lack of preparation, did not use web-based
  exhibits, lack of research university peers,
  overemphasis on logistics
• The team in the difficult visit institution16
• failed to ask for information, did not listen,
  did not use provided documentation, tense exit
  interview, members not prepared, did not make
  good use of the Sunday evening poster session

24
D. Continued, themes (2) BOE teams

• Recommendations11 of the 48 comments
• Chair/members from research institutions (8)
• Train members about research institutions
• Assure that teams arrive better prepared
• To use electronic data
• For more collegial interaction
• To use time not just to find data, but to
  clarify and elaborate

25
D. Continued, themes (3) Program Review and SPAs

• 22 references from 10 respondents in all 5
  institutions total of 51 points made
• Inconsistencies and changes20
• Across SPAs and between SPAs and NCATEsome would
  accept GPA or Praxis data, others would not
• Changes during the accreditation process
• Certain SPAs are difficult to work with
• Lack of fit between the standards and graduate
  level initial preparation

26
D. Continued, themes (3) Program review and SPAs

• Concurrences with the program review process14
• More consistent use of assessments
• Agreement on just 6 to 8 assessments
• Writing for national recognition helped to
  redirect programs
• Faculty worked hard because theres this element
  of pride
• Our programs gotten better
• Comments by reviewers were fair, right on the
  money

27
D. Continued, themes (3) Program review and SPAs

• Complaints9
• Praxis data not aligned cant get sub scores
• Limited feedback
• Took too long
• Just wanted numbers
• Others2
• Respondents own institution made changes
• Faculty would rather fight with their
  professional organization than adopt standards

28
D. Continued, themes (3) Program review and
SPAs Recommendations6

• More consistency
• Equivalent demands for all SPAs
• More coordination across SPAs and between SPAs
  and NCATE
• Limit the number of assessments
• Better fit with graduate level preparation
• Write standards for initial preparation that
  recognize prior candidate preparation in subject
  content
• Provide more examples of evidence for other
  professional preparation
• Reconsider way decisions are made about SPA
  standards
• Need more were in this together, less
  territoriality, more at-large partners

29
D. Continued, themes (4) Evidence

• 24 references by 10 respondents from 4
  institutions
• 18 comments on evidence generally
• NCATE is too prescriptive
• too much reliance on numbers and tables
• Not a research base to back up some data
  requirements
• Institutions use grades as evidence of
  accomplishment
• Assessment data cannot be aggregated meaningfully
  across different levels of programs
• 6 comments on evidence for diversity
• Numbers fail to inform the goal of cultural
  competence

30
D. Continued, themes (4) Evidence
Recommendations

• Use a broader definition, not just numbers on
  tables
• Let institutions decide
• More anthropologicallook at what were doing and
  how faculty and our publics interpret that
• Let institutions self-define their mission, and
  ask for reporting of evidence within broad
  parameterslimiting the number of assessments
• Address particular issues
• Take on measuring of student learning based on
  what good teaching and learning are
• Focus on ethnic diversity and cultural
  competence

31
D. Continued, themes (5) TEAC

• 10 references made by 6 respondents from 4
  institutions total of 13 points
• 7 points were set in a context of perceived
  problems with NCATE
• NCATE is overly prescriptive
• Some required pieces of data are not supported by
  research
• Differences between SPAs and NCATE frustrate
  faculty
• NCATE is bean counting
• NCATE is burdensome, tedious, time intensive,
  costly
• Important qualities of a program cannot be
  captured in numbers
• Standards dont align with the way the state is
  going

32
D. Continued, themes (5) TEAC

• Three respondents ventured impressions of TEAC or
  recognized it as an alternative
• There were 3 recommendations
• NCATE should look at what it does from a
  business perspective, but leave room for
  compromise
• TEAC and NCATE representatives should work out a
  whole new accreditation system. . . Good aspects
  of both, but neither is a perfect system
• BUT dont go so far in another direction that we
  dont have a set of standards that people have to
  step up to meet

33
D. Continued, themes (6) States

• 13 references by 6 respondents from just 2
  institutions
• Starkly different portraits of states, so state
  influence is a strong factor in accreditation, at
  least for some institutions
• NCATE parallels the state, should keep
  coordinated, make state exam pass rate count for
  NCATE, can defer to our state on state
  assessments, would not have completed NCATE
  accreditation if state had not insisted
• Negativity about NCATE is really directed at the
  state state report is different only in minor
  waysa duplicate, unnecessary, requirement
  disparity in the standards that regular and
  alternative programs are held to but state
  refuses to acknowledge

34
E. Summing up

• Respondents found much that was good, e.g.
• I like what Im hearing about web-based
  submissions
• Electronic submission is very helpful to organize
  data, other information,
• Positive that NCATE is interested in what
  approved programs are saying and thinking
• Preparing for NCATE made the regional
  accreditation easy

35
E. Summing up

• They made recommendations that NCATE needs to
  sift and consider. Particularly,
• Evidence
• Teams and recruitment
• Program standards and standards decisions
• Regional accreditor experiences
• Continuing accreditation, candidate data only