Developing an Assessment Tool Box Workshop

1
Developing an Assessment Tool Box Workshop

• Rosa Buxeda
• (r_buxeda_at_rumac.uprm.edu)
• Lueny Morell
• (lueny_at_ece.uprm.edu)

2
Workshop Objectives

• Familiarize faculty with program classroom
  assessment tools
• Establish program assessment strategy
• Select assessment tool box
• Agree on common tools for the CoE
• Establish assessment plan for Summer 2001

3
Agenda

• 730 am Breakfast
• 800 Workshop Objectives principles of
  assessment
• 815 Exercise 1
• 830 On Assessment
• 1030 Break
• 1045 Program Outcomes Assessment

4
Agenda

• 1130 Exercise 2
• 1200 Lunch
• 100 pm Exercise 3
• 200 Exercise 4
• 300 Break
• 330 Exercise 5
• 430 Closure workshop assessment

5
Principles of Good Practice for Assessing
Learning (AAHE)

• The assessment of student learning begins with
  educational values.
• Assessment is most effective when it reflects an
  understanding of learning as multidimensional,
  integrated, and revealed in performance over
  time.
• Assessment works best when the programs it seeks
  to improve have clear, explicitly stated
  purposes.
• Assessment requires attention to outcomes but
  also and equally to the experiences that lead to
  those outcomes.
• Assessment works best when it is ongoing, not
  episodic.

6
Principles of Good Practice for Assessing
Learning (AAHE)

• Assessment fosters wider improvement when
  representatives from across the educational
  community are involved.
• Assessment makes a difference when it begins with
  issues of use and illuminates questions that
  people really care about.
• Assessment is most likely to lead to improvement
  when it is part of a larger set of conditions
  that promote change.
• Through assessment, educators meet
  responsibilities to students and to the public.

7
Exercise 1

• Think about ABETs a-k outcomes name 2
  assessment tools you are currently using that
  could be used to provide evidence that the
  outcome(s) is (are) being achieved.

8
Levels of Assessment

• University
• At the university level, learning outcomes
  assessment may refer to students' understanding
  demonstrating good citizenship by  showing
  respect to each other, appreciating and realizing
  cultural diversity and globalization, and
  acquiring  the life-long learning skills, and so
  on.
• Part of these outcomes are related to General
  Education objectives and goals. Others do not
  belong to any specific academic programs but are
  important outcomes for an educational
  institution.
• Some of the UPRM goals address these outcomes,
  which have to be assessed at the university
  level. There are other outcomes that can be
  assessed at different levels.

9
Levels of Assessment

• Program (discipline) level
• At the discipline level, learning goals
  objectives are determined by the faculty in the
  discipline or program in strong collaboration
  with all stakeholders. These goals objectives
  typically refer to the intended knowledge,
  skills, attitudes and values students are
  supposed to learn from their studies in the
  discipline.
• It is clear that faculty are the individuals who
  understand the programs best, and therefore,
  should be the primary individuals who own and
  conduct the assessment for the primary purpose of
  improving student learning outcomes. 

10
Levels of Assessment

• Classroom level
• At the classroom level, goals objectives are
  specifically designed for the given course by the
  individual instructor. Often, instructors conduct
  many direct and useful assessment activities but
  do not realize that they are already doing
  outstanding assessment activities.
• The only extra step that individual instructors
  need to take is to summarize the assessment
  results in terms of various categories of
  knowledge, skills and/or attitudes and values for
  the entire class, rather than for individual
  students.

11
Outcomes Assessment for ABET

• Program
• Pre-engineering
• BS
• Classroom
• Post-graduation

12
Classroom Assessment

• Classroom assessment is a systematic continuous
  approach to formative evaluation
• Classroom assessment techniques (CATS) are
  simple tools for collecting data on student
  learning in order to improve it.
• CATS are feedback devices, instruments that
  faculty can use to find out how much, how well,
  even how students are learning.
• Reference Angelo and Cross (1993)

13
The Classroom Assessment Cycle(adapted from
Angelo Cross, 1993)

• Phase 1 Planning for Classroom Assessment
• Choosing a class to focus on
• Focusing on an assessable teaching goal or
  question
• Designing a classroom assessment tool
• Phase 2 Implementing the Assessment
• Teaching the course or target lesson
• Assessing student learning by collecting feedback
• Analyzing student feedback.

14
The Classroom Assessment Cycle

• Phase 3 Responding to the Results
• Interpreting the results and formulating an
  appropriate response.
• Communicating results
• Making decisions
• Documenting the process

15
Program assessmentStudent Outcomes Assessment
Matrix
16
Student Outcomes Assessment Matrix
17
Basic Criteria for selecting tools

• Is it context-sensitive?
• Is is flexible?
• Is it likely to make a difference?
• Is it mutually beneficial?
• Is it easy to administer?
• Is it easy to respond to?
• Is it educationally valid?

18
Assessment tool box what instruments to use?

• Qualitative quantitative
• Examples

19
Some Tools for Classroom AssessmentTools can be
applied in class for immediate feedback

• References
• ABET Faculty Workshop
• Classroom Assessment Techniques, Angelo Cross

20

• Alumni Survey
• CEEB Math Score
• Course specific assessments (CATs)
• Departmental GPA
• Design/other competitions
• Ethics module assessment form
• Exams, homework, special problems
• Exit interviews
• Focus group (satisfaction)
• General GPA
• Graduate School advisory questionnaire
• Graduate school placement data
• Graduates post-graduation satisfaction survey
• GRE
• Honors, recognitions and awards
• Industry Advisory Board evaluation
• Industry/employer satisfaction survey
• Internships evaluations

21

• Laboratory Reports
• Membership in professional societies
• Oral presentation assessment form
• Portfolio evaluation
• Professional Exam (Fundamental Discipline
  specific)
• Report of attendance to seminars
• Science Fair participation, awards
• Senior Project (capstone)
• Simulations (real world situations)
• Standardized examinations
• Student college entrance index (IGS)
• Student Projects
• Teamwork Assessment form
• Transcript
• Undergraduate research papers and presentations
• Video of presentations
• Written report assessment form
• Other

22
Exercise 2 Tools currently in use

• Please share tools

23
NASA PaSCoR Tool Box

• Oral presentations
• Written reports
• Undergraduate research
• Summer internships
• Teamwork
• Lectures

24
Exercise 3 Outcomes Assessment Matrix

• Complete Program Outcomes Assessment Matrix for
  student outcomes
• Program objectives, including ABET a-k
• Tools
• Assessment schedule
• Use assessment tool box (shopping list)
• Results column will be completed at the end of
  the assessment schedule (documentation)

25
Program assessment
26
Exercise 4 Share results

• By department
• Can we agree on common tools?

27
Exercise 5 The Summer 2001

• Courses and activities to be assessed Summer 2001
• Complete column 1 in Outcomes/Tools/Schedule
  Matrix

28
From assessment to results

• Once the data is gathered, results have to be
  presented disseminated in the most effective way

29
PROMISE Howard Hughes project
30
(No Transcript)
31
Acting on Assessment Results

• Enhance classroom teaching
• Deep approach to learning
• Empowering students
• New teaching/learning strategies
• Collaborative learning
• Assigning resources
• Faculty development
• Accountability
• Sharing with stakeholders
• Industry
• Benchmarking
• Longitudinal studies
• other