Goalaware tools: Integrating course management

1
Goal-aware toolsIntegrating course management
assessment systems

• Syracuse University rSmart
• Joe Shedd Janice Smith
• Sean Keesler Hannah Reeves
• Chris Coppola
• December 8, 2006

2
SUs teacher candidate assessment data are
derived from various sources

• Class assignments submitted in hard copy or via
  various course-management systems (Dialogue,
  Blackboard, WebCT)
• Field assessments completed in hard copy or
  on-line by university supervisors host teachers
• Feedback and ratings on candidate portfolios
  submitted on-line or in hard copy
• Independent ratings of candidates by faculty
  (e.g., disposition ratings, overall summative
  ratings, etc.) completed in hard copy
• State certification exams reported in hard copy
  but available digitally
• and re-entered in a common database

3
SUs solution to integrate data the systems
that generate them by

• Developing a single Sakai-based system for course
  management, portfolios, and field assessments
• Providing for independently-generated assessments
  of candidate performance
• Allowing for archiving of designated artifacts
  and ratings in a common database
• Developing tools for tagging artifacts ratings
  with one or more goals, from one or more goal
  sets reflecting program, course or personal
  objectives

4

• Examples from rSmart SU experience

5
(No Transcript)
6
RI K-12 Assessment Process

• Goal Management Tool provides standards
• Assignments tool links to standards and collects
  evidence
• Grad Portfolio Template selects evidence and
  reflection
• Reports aggregate evidence of learning for high
  school accreditation and state report cards

7
SU form linking assessments based on course
objectives and School proficiencies
Assessments linked to instructor-generated goals
From the global goal-set of SOE proficiencies
8
Using the data-point tool for on-line rating and
feedfack of SU candidate portfolios
9
So goal-aware tools are the solution, but what
are the (upstream) problems they help solve?

• Securing national accreditation
• Disconnected systems complicating problems that
  cannot be solved at any one level
• Different actors having different stakes in
  different problems
• Reconciling the tensions between demands for
  accountability and our commitment to
  student-centered pedagogy

10
Challenges of implementing such obviously
attractive solutions

• Post-accreditation let down (weariness,
  diminished sense of urgency, less willingness to
  defer to top-down mandates)
• Satisfaction with ad hoc solutions
• Reluctance to abandon favored course management
  systems
• Resistance to solving other peoples problems
• Greater accountability to others implies less
  autonomy for ones self
• Integrating systems is an on-going, iterative
  process, with no clear stages or roadmaps
• Expecting people to abandon old systems (at least
  in principle) before the details and implications
  of new systems are clear

11
Goal-aware tools help address some of their own
implementation problems

• Theyre simple (understandable, demystifying,
  seemingly unthreatening)
• They offer a single solution (or at least, part
  of the solution) to multiple problems (a solution
  that everyone can own for different reasons)
• Data-point tool allows for full implementation of
  the general solution, gradual implementation of
  its parts
• They invite stakeholders to discover new uses,
  including ones they themselves can develop and
  implement
• They build acceptance, if not support, for
  on-going development
• They help redefine accountability, providing
  active roles for students and faculty members and
  broadening the base of assessment data

12
For further information

• Joe Shedd
• Teaching Leadership Programs
• Syracuse University
• jbshedd_at_syr.edu
• Janice Smith
• The rSmart Group
• janice.smith_at_rsmart.com