KSDENCATE Unit Assessment System 102104

1
KSDE/NCATE Unit Assessment System10/21/04

• Paul Adams
• Anschutz Professor of Education
• Professor of Physics

2
A few thoughts

• Education is what survives when what has been
  learned has been forgotten B.F. Skinner, New
  Scientitst, 21 May 1964, Education in 1984
• Any attempt to reform the university without
  attending to the system of which it is an
  integral part is like trying to do urban renewal
  in New York City from the twelfth story up. Ivan
  Illich, Austrian scologist, Deschooling Society,
  Ch 3
• Education made us what we are Claude-Adrien
  Helvetius (1715-71) French philosopher. Discours
  XXX, Ch. 30

3
What is an Assessment System?

• According to Webster (10th Ed.)
• Assess to determine the importance, size, or
  value of
• System an organized or established procedure
• So an organized established procedure of
  determining the importance and value of OUR
  preparation of initial and advanced candidates

4
The System

• Instruments what we are using to analyze the
  unit
• Alignment the system supports assessment and
  evaluation of the NCATE standards, KSDE standards
  (program), and the Conceptual Framework
• Procedures when, where, and who reports data
• Sampling appropriate (fair, consistent,
  non-bias) measures are made on candidates,
  courses, programs, and unit operations
• Technology data storage and acquisition into
  coherent reports
• Improvement regular and systematic analysis of
  the data to improve courses, programs,
  performance, operations, and assessment system

5
A Big Picture View of KSDE/NCATE Accreditation

• KSDE is primarily focused on how our PROGRAMS
  prepare candidates, how the program assesses
  candidate attainment of standards, and plans for
  improvement of PROGRAMS based on collected data
• NCATE subsumes KSDE information and primarily is
  focused on how our UNIT (read UNIVERSITY AS A
  WHOLE) assesses and improves faculty, funding,
  school partnerships, clinical experiences, etc.

6
NCATE Parts

• THE STANDARDS
• Standard 1 Candidate Knowledge, Skills,
  Dispositions (KSDE Data Here)
• Standard 2 Assessment System Unit Evaluation
• Standard 3 Field Experience Clinical Practice
• Standard 4 Diversity
• Standard 5 Faculty Qualifications, Performance,
  Development
• Standard 6 Unit Governance Resources

7
A Simple Picture for the System
8
The Real Deal

• Warning The following diagram may scare you,
  confuse you, and be downright nasty

9
(No Transcript)
10
Action by responsible parties
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Database Elements Initial Transition Point 1
15
Database Elements Initial Transition Point 2
16
Database Elements Initial Transition Point 3
17
Database Elements Initial Transition Point 4
18
So What is Next?

• Learning ..
• the system and how it substantiates attainment of
  the stated goals of the Conceptual Framework
• WHEN, WHERE, and HOW to REPORT data
• how to validate and develop reliability for all
  measures used refer to the one page cover sheet
  that was asked for last year
• how to use the instruments (e.g. KPA) (TRAINING!)
• how to distill the data from the database (Help
  us Lance!)
• how to interpret the information and ACT ON IT to
  improve PROGRAMS and UNIT
• how to write it all up from the program level to
  the unit

19
Program Data Fall 2003 Fall 2004
Unit Data AY 2003-2004
Revisit in March 05
Data Retreat
Meaning ACTION
20
Data Retreat

• Review the data for each element of the
  conceptual framework at each transition point for
  advanced and initial
• Using the available evidence report
• Are we being successful or not?
• What do we need to watch and monitor?
• What are specific changes to programs, unit,
  operations, and the assessment system to improve
  during the next year?

21
The TASK

• The task is to address for both initial and
  advanced programs
• A. Is the UNIT preparing candidates and
  professionals in the field (examined at each
  transition point) who are meeting performance
  standards at each element of the conceptual
  framework?
• YES or NO or MOSTLY with the answer supported by
  triangulation of the evidence. Disaggregate
  Secondary from Elementary if there is a
  difference in programs
• B. What areas of concern or monitoring are
  indicated by the data?
• Support with evidence.
• C. What are the recommended actions to improve
  the unit programs, courses, procedures, and
  assessment?
• -- Specific actions coupled with measures to
  determine the impact with a rationale for the
  action.
• The REPORT 5 min - A written report (1 to 2
  pages) with TWO elements
• a) A summary of how we are doing on the
  conceptual framework element assigned to your
  group supported with a tabular or graphical
  representation of the data you used to form your
  position for each transition point, advanced and
  initial.
• b) Recommend actions 1 to 3 in priority order.

22
Deeper System Details

• The following tables show specific
  responsibilities
• NOTE These are UNIT DATA COLLECTION ELEMENTS,
  Programs are collecting other elements in
  response to the KSDE Standards
• EACH PROGRAM needs a similar chart or reflect the
  same information in the KSDE Program Chart
• These tables are under revision!

23
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL

24
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)

25
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
26
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
27
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
28
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
29
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
30
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
31
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
32
Table 1 KSDE/NCATE UNIT ASSESSMENT
RESPONSIBILITIES INITIAL LEVEL (Continued)
33
Data Retreat

• WHEN Friday, October 8, 8 to Noon (cancel
  class!)
• WHERE Sternberg Museum Classroom
• WHAT How to
• Training on instruments you are using
• How to request the data you need in a useable
  format
• How to write a program report and represent the
  data
• Resources and helps