Assessment Workshop Presented by: The Office of Institutional Research and Assessment

1
Assessment WorkshopPresented byThe Office of
Institutional Research and Assessment
2
(No Transcript)
3
Assessment Workshop

• Why do assessment?
• What is assessment?
• When to assess?
• How to assess?

4
Why do assessment?

• Improvement
• Accountability
• Accreditation

5
Improvement

• Curriculum
• Instructional methodology and practice
• Student services

6
Nothing is so perfect that it cannot be improved
upon.

• Trudy Banta

7
Accountability

• State Board of Education/ Board of Regents
• Public accountability
• Competition for limited resources

8
Every publicly supported social services agency
now has an outcome-based agenda.

• Trudy Banta

9
Accreditation

• Northwest Association of Schools and Colleges
  (NASC)
• We are to demonstrate through regular and
  systematic assessment that students who complete
  their programs, no matter where or how they are
  offered, have achieved a specified set of learner
  goals established for each program.

10
There must be evidence that students are
mastering the course and curriculum objectives.
Given the ferocious competition developing among
learning organizations worldwide, these
assessments are necessary steps.

• Trudy Banta

11
What is assessment?

• Assessment is a systematic process of looking
  at student achievement within and across courses
  by gathering, interpreting and using information
  about student learning for educational
  improvement.
• American Association of Higher Education (AAHE)

12
Characteristics of good assessment

• comprehensive, ongoing and evolutionary
• broad involvement from faculty
• clear, assessable educational goals and
  objectives
• uses a variety of assessment and evaluation
  methodologies
• collects meaningful and accurate data
• primary emphasis is on improvement of teaching
  and learning
• 
  (adapted from Seybert, 1998)

13
Why arent grades enough?

• grading practices are not standard
• need different ways of structuring program
  assessment
• grades reflect many things other than course
  content and mastery
• objectives differ
• good assessment requires multiple ways of
  measuring goal achievement

14
When to AssessCan be Decided According to When
Critical Decision-Making and Communication Occurs

• Entry assessment helps determine who should be
  admitted and who is prepared to benefit from
  which programs and courses.
• Midpoint assessment occurs when students reach a
  crucial decision point or level of attainment in
  their program of studies.
• Exit assessment helps determine which students
  have attained the prerequisite knowledge, skills,
  and abilities associated with the program goals.
• Follow-up (Post Completion) assessment helps
  determine the effectiveness of the educational
  programs in preparing students for further
  education, transfer, entry or reentry into the
  workforce, or the students personal goals.
• NCTLA

15
How to assess?

• Identify each degree and certificate program to
  be assessed.
• Identify student learning goals and the
  educational criteria and experiences for each
  goal.
• Identify appropriate assessment methods and
  strategies.
• Collect, analyze and interpret data.
• Specify program improvements.

16
Identify each degree and certificate program to
be assessed.

• All certificate programs
• All undergraduate programs
• All graduate programs
• All off-campus programs

17
Identify student learning goals

• establish three to six goals for each program,
  both graduate and undergraduate
• identify education criteria and experiences for
  each goal
• What is to be learned?
• What is the level of learning?
• What is the learning applied to?

18
Identify Key Components of Each GoalIdentify Key
Skills for Each GoalGoal Students will
understand and apply logical and ethical
principles to personal and social situations.

• What logical and ethical principles are learned
  regardless of specific coursework taken?
• How do students show their understanding and
  ability to apply these principles?
• How do we see students apply principles to their
  personal lives and development?
• How do we see students applying principles to
  social settings and circumstances?
• NCTLA

19
What if you do not have assessable goals?

• 1. Examine the set of required courses.
• 2. Ask, What have we been trying to teach?
  (Outcomes)
• Content knowledge?
• Cognitive skills?
• Values and attitudes?
• 3. Ask, What should students know before they
  enter the curriculum in order to succeed?
  (Entrance Criteria)
• 4. Ask, What should students know when they
  complete the curriculum in order to graduate?
  (Exit Criteria)
• 5. Ask, At what points in the curriculum are
  students doing well or having difficulty?
  (Midpoint Criteria)
• 6. Ask, Are our alumni successful in the
  field? (Post Completion Criteria)
• Adapted from NCTLA

20
What if you have assessable goals, but no
specific curriculum to support them?

• 1. Ask, Do we really teach students (the
  goal)?
• 2. If Yes, then identify if the goal is
• embedded throughout coursework, or
• achieved through an identifiable pattern of
  coursework
• 3. State the specific coursework pattern
  required to attain the general education goal.
• 4. Identify entry ability required for students
  to succeed at the collegiate level.
• 5. Identify key midpoints in the development of
  student abilities along the general education
  goal.
• 6. Identify levels of attainment or performance
  required for graduation.
• 7. Identify levels of attainment or performance
  in an employment setting.
• NCTLA

21
What to Assess

• Knowledge outcomes - core of concepts and
  material knowledge
• Skills outcomes - what a student can do
• Attitudes and values outcomes - those faculty
  believe to be important
• Behavioral outcomes - behaviors crucial to the
  curriculums impact

22
Blooms Classification of Cognitive Skills

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

23
Knowledge Outcomes

• Describe the basic components of empirical
  research.
• Give examples of major themes or styles in Music,
  Art, or Theatre.
• Recognize in complex text logical, rhetorical,
  and metaphorical patterns.
• NCTLA

24
Comprehension Outcomes

• Correctly classify a variety of plant specimens.
• Explain the scientific method of inquiry.
• Summarize the important intellectual, historical,
  and cultural traditions in Music, Art, or Theatre
  from the Renaissance to Modern times.
• NCTLA

25
Application Outcomes

• Demonstrate in the laboratory a working knowledge
  of lab safety procedures.
• Apply oral communication principles in making a
  speech.
• Compute the area of a room.
• Use editing symbols and printers marks.
• NCTLA

26
Analysis Outcomes

• Distinguish between primary and secondary
  literature.
• Diagram a sentence.
• Listen to others and analyze their presentations.
• Differentiate between historical facts and
  trivia.
• NCTLA

27
Synthesis Outcomes

• Revise faculty copy for a news story.
• Formulate hypotheses to guide a research study.
• Create a poem, painting, design for a building.
• NCTLA

28
Evaluation Outcomes

• Compare art forms of two diverse cultures.
• Critically assess an oral presentation.
• State traditional and personal criteria for
  evaluating works of art.
• Draw conclusions from experimental results.
• NCTLA

29
Identify the educational experiences for each
goal. List actions intended to enable the
students to achieve these goals.
30
Identify appropriate assessment methods and
strategies

• Choose one or two goals from each program
• Identify appropriate measures of goal attainment
• Identify the appropriate point of measurement
• Use multiple methods of assessment

31
Characteristics of Effective Performance Measures

• Relate to goals
• Focus on the vital few elements to measure
• Foster improvement
• Are well communicated
• Are reviewed as often as appropriate
• Provide information on level, trend and
  comparative/ competitive data
• Focus on the long-term well-being of the student
  and the program
• Adapted from Engelkemeyer, 1998

32
Measurement Blockers

• Fuzzy goals or action strategies
• Unjustified trust in informal feedback systems
• Entrenched measurement systems
• Incorrect focus
• No agreement on priorities
• Adapted from Engelkemeyer 1998

33
Collect, analyze and interpret data

• What did you find?
• What were the programs strengths and weaknesses?
• How well are the students learning the concepts?
• Is the learning achieved appropriate for the
  level?
• Is the learning being appropriately applied?

34
National Center on Postsecondary Teaching,
Learning, Assessment
35
National Center on Postsecondary Teaching,
Learning, Assessment
36
Identify Strategies for Change

• What will you do to improve student learning?
• Which program elements should be reinforced?
• Which program elements should be maintained?
• Which program elements should be strengthened?
• Which program elements should be modified?
• at the undergraduate level?
• at the graduate level?
• off campus?

37

• Be flexible, adaptive and prepared to change.
• There will always be problems.
• Things always change (mandates, circumstance,
  personnel, priorities.)
• View assessment as an evolutionary process.
• (Seybert, 1989)

38
References

• The Assessment Institute Resource Book.
  Assessment Institute. National Center for
  Postsecondary Teaching, Learning and Assessment,
  Burlington, VT. October 15-17, 1998.
• Ball State University. Assessment Workbook.
  Munci Ball State University Offices of Academic
  Assessment, 1992.
• Black, Karen E., Trudy W. Banta, and Jane L.
  Lambert. Best Practices in Program Review.
  AAHE Conference on Assessment. American
  Association of Higher Education, Cincinnati, OH.
  June 14, 1998.
• Bloom, Benjamin, et al. (ed) Taxonomy of
  Educational Objectives Handbook I. Cognitive
  Domain. NY David McKay, 1956
• Ewell, Peter T., ed. Assessing Educational
  Outcomes. New Directions for Institutional
  Research 47 San Fransisco Jossey-Bass, Sept,
  1985.
• Halpern, Diane F., ed. Student Outcomes
  Assessment What Institutions Stand to Gain. New
  Directions for Higher Education 59 San Fransisco
  Jossey-Bass, Fall, 1987.

39
References Cont.

• Idaho State Board of Education. Governing
  Policies and Procedures. Boise Idaho State Board
  of Education, 1994.
• Engelkemeyer, Susan West. Key Measures in
  Organizational Performance. AAHE Conference on
  Assessment. American Association of Higher
  Education, Cincinnati, OH. June 13, 1998.
• Wright, Barbara D. Assessment for Beginners
  Getting Started. AAHE Conference on Assessment.
  American Association of Higher Education, Miami
  Beach, FL. June 11, 1997.
• Seybert, Jeffrey A. Community College Assessment
  Strategies. AAHE Conference on Assessment.
  American Association of Higher Education,
  Cincinnati, OH. June 14, 1998.
• Sims, Serbrenia J. Student Outcomes Assessment
  A Historical Review and Guide to Program
  Development. Connecticut Greenwood Press, 1992.

40
NEED MORE INFORMATION?

• Archie George Jane Baillargeon
• 885-7995 885-5828
• Archie_at_uidaho.edu jane_at_uidaho.edu