The Critical Need to Construct and Implement a TeachingLearning Assessment Model for Diverse Populat

1
The Critical Need to Construct and Implement a
Teaching/Learning Assessment Model for Diverse
Populations

• James A. Anderson, Ph.D.
• Vice Provost
• Division of Undergraduate Affairs
• North Carolina State University
• James_Anderson_at_ncsu.edu

2
Assessment

• Refers to more than gathering information or data
• Refers to gathering or collecting evidence of
• 1) student performance and achievement,
• 2) program strengths, weaknesses, and
• 3) activity/ component effectiveness

3
Evaluation

• Refers to how one uses assessment evidence to
  improve or change process or program
• Refers to the value judgements one makes about
  the usage of assessment information

4
Basic Questions that Help Define the Assessment
Process

• Why are we doing assessment?
• What will we assess?
• How will we assess?
• Who will assess?
• How will the results be analyzed?
• How will the results be communicated and to whom?

5
Retention Model Planning

• What are your current (cohort) retention and
  graduation rates?
• How do they compare to that of the overall
  institution?
• To that of similar populations at comparable
  institutions?
• Are you satisfied with your programs current
  performance? If not, how do you plan to alter the
  current situation?
• Indicate any new goals or program redesign(s)
  that you (and your staff) have or plan to
  initiate to
• Improve the retention, transfer, and/or
  graduation of program students, or
• To increase the enrollment of program students in
  majors where 1st-generation, low-income students
  are underrepresented

6
Retention Model Planning Continued

• What changes have been initiated to improve
  student performance?
• What other initiatives are currently underway (or
  being planned) at your institution that are
  targeted at improving student learning, retention
  and/or graduation outcomes?
• How are you and/or members of your staff
  involved?
• During the past three years have you (or your
  staff) conducted any evaluations of the program
  as a whole or of program components?
• If yes, what were the results of the evaluations?
  
• What are the most pressing questions that you
  (and your staff) have about improving student
  outcomes?

7
Retention Model Planning Continued

• How do you presently use information to
• Identify student needs and direct services to
  them
• Assist in program administration
• In assessment and evaluation
• How do you presently use technology to
• Identify student needs and direct services to
  them
• Assist in program administration,
• In assessment and evaluation

8
Retention-Related Research

• Identifying dropout predictors
• Student characteristics
• Cognitive
• Non-cognitive
• Campus
• Identifying critical points/transitions
• Monitoring academic progress
• Identifying gateway/gatekeeper courses
  materials
• Identifying best/successful practices
• Outcomes/ assessment/evaluation

9
Questions that Direct the Development of Synergy
between Academic Affairs and Student Affairs

• What is the thinking task, intellectual
  experience, and/or co-curricula experience that
  needs to be designed relative to the preparation
  level and diversity of the students at your
  institution?
• Can the interpersonal transactions that occur in
  the everyday life of the student and that reflect
  cultural orientations serve as a basis for
  potential new models of critical thinking? What
  curricular experiences will promote this skill
  development?

10
Questions that Direct the Development of Synergy
between Academic Affairs and Student Affairs
Continued

• What structures need to evolve to assure that
  students have the opportunity to enhance academic
  self-concept and understand their role in the
  culture of learning at your institution?

11
Model Programs

• Tied to institutional mission curriculum
• Have clearly defined mission, goals, objectives
• Program strategies activities have theoretical
  foundation
• Individualized
• Small group collaborative learning
• Build community
• Student assessment/evaluation are core activities
• Information management is a core activity
• Affective socio-cultural components linked to
  academics

12
Model Programs Continued

• Program participation in admissions process
• Pre-freshman-year academic social preparation
• Program role in course selection
• Intrusive advising process
• Academic support services linked to coursework
• Core of faculty involvement support
• Powerful message of success through conscientious
  effort

13
What Works - 1

• Strong program ( components) philosophy
• Information management - assessment and
  evaluation as core activities
• Centralized or highly coordinated instructional
  service delivery
• Mandatory student assessment placement
• High degree of structure (for students)
• Mastery learning techniques
• Multiple approaches methods in instruction
• Sound cognitive and affective theory
• Working responses to diversity

Source H. Boylan, L. Bliss, B Bonham, 1997,
Boylan 1999, Boylan D. Saxton, 2000
14
What Works - 2

• Counseling/support activities designed to support
  teaching/learning
• Consistency between basic skills standards
  entry standards for regular curriculum
• Learning communities
• Courses or workshops on strategic and critical
  thinking
• Integration of critical thinking/logic
• Continuous staff development training
  (especially tutors learning assistants)

Source H. Boylan, L. Bliss, B Bonham, 1997,
Boylan 1999, Boylan D. Saxton, 2000
15
College Student Motivation Framework

• Determinants
• of Needs
• Self-actualization
• Need to achieve
• Competence
• Self-worth
• Developmental level
• Goals

NEEDS
MOTIVATION

• Determinants
• of Expectations
• Self-efficacy
• Previous experience
• Success of others
• Feedback
• Attributions

EXPECTATIONS
16
Sedlaceks Non-Cognitive Scales

• Positive Self-Concept or Confidence
• Realistic Self Appraisal
• Understanding and Dealing with Racism
• Preference for Long-range Goals over Short-term
  or Immediate Needs
• Availability of Strong Support Person
• Successful Leadership Experience
• Demonstrated Community Service

17
Some Questions about Teaching/Learning

• What do you expect your students to know and be
  able to do by the end of their education at your
  institution?
• What do the curricula and other educational
  experiences add up to?
• What do you do in your classes or in your
  programs to promote the kinds of learning or
  development that the institution seeks?

From Peggy Maki, Ph.D.
18
Questions Continued

• Which students benefit from which classroom
  teaching strategies or educational experiences?
• What educational processes are responsible for
  the intended student outcomes the institution
  seeks?
• How can you help students make connections
  between classroom learning and experiences
  outside of the classroom?
• How do you intentionally build upon what each of
  you teaches or fosters to achieve institutional
  educational objectives?

From Peggy Maki, Ph.D.
19
Characteristics of Effective Assessment Methods

• are structured, systematic, and ongoing--not
  episodic.
• relate to other institutional strategic
  long-range plans and to planning and budgeting
  processes.
• emerge from and are sustained by a faculty and
  administrative commitment.
• provide explicit and public statements about
  institutional expectations for student learning.

From Peggy Maki, Ph.D.
20
Characteristics Continued

• enable the institution to determine the fit
  between expectations for student learning and the
  level of learning achieved.
• provide encouragement and the means to test
  changes that could improve learning.

From Peggy Maki, Ph.D.
21
Distinguishing between Objectives and Outcomes

• Objectives state overarching expectations such
  as--
• Students will develop effective oral
  communication skills.
• OR
• Students will understand different economic
  principles.

From Peggy Maki, Ph.D.
22
Outcomes

• Statements that specify the kinds of knowledge,
  abilities, habits of mind, behaviors that
  demonstrate achievement of an objective. An
  objective can have more than one outcome.
• These statements rely on active verbs, such as
  create, compose, calculate, develop, build,
  evaluate, translate, etc.
• These statements form the basis of assessment,
  provide direction for instructional activity,
  inform students about faculty intentions.

From Peggy Maki, Ph.D.
23
Compare

• Students will write
• effectively.

to

• Students will compose a range of
  professional documents designed to solve
  problems for different audiences and purposes.

From Peggy Maki, Ph.D.
24
Compare

• Students will write
• effectively.

to
Students will summarize recent articles on
economics and identify underlying economic
assumptions.
From Peggy Maki, Ph.D.
25
Venues for Assessment

• Course-embedded
• In-class
• Out-of-class
• Off-campus
• On-line

From Peggy Maki, Ph.D.
26
Improving Teaching, Learning, Institutional
Culture

• Revising teaching strategies to improve student
  achievement
• Revising course content to assure appropriate
  attention to areas that need increased attention
• Replacing or revising courses, programs, services
  
• Changing sequence of courses
• Adding a requirement or required course
• Enhancing the advising process
• Increasing focus throughout the co-curriculum
• Increasing support services

From Peggy Maki, Ph.D.
27
Characteristics of Good Pedagogy which Affect
Application of Technology in the Classroom

• Courses should focus on learning rather than
  teaching (student-centered vs. teacher-centered).
• Interaction with the material should be
  student-controlled rather than teacher-controlled.
  
• Content delivery should be based on student
  knowledge, driven by frequent formative feedback
  (classroom assessment).

28
Characteristics of Good Pedagogy which Affect
Application of Technology in the Classroom

• Courses should be structured so that students
  interact with material in a pedagogically sound
  way.
• Finally, this should all be accomplished in the
  most parsimonious way.

29
Indicators that the Online Virtual Community Has
Been Forming

• Active interaction involving both course content
  and personal communication.
• Collaborative learning evidenced by comments
  directed primarily student to student rather than
  student to instructor
• Socially constructed meaning evidenced by
  agreement or questioning, with the intent to
  achieve agreement on issues of meaning

R. Pallof K. Pratt, 1999
30
Indicators that the Online Virtual Community Has
Been Forming Continued

• Sharing of resources among students
• Expressions of support and encouragement
  exchanged between students, as well as
  willingness to critically evaluate the work of
  others

R. Pallof K. Pratt, 1999