Writing and Using Assessment Plans to Enhance Teaching, Learning,

1
Writing and Using Assessment Plans to Enhance
Teaching, Learning, Curriculum

• November 10, 2006

2
Todays Objectives

• Understand some reasons why we need to assess
  student learning
• Identify and discuss each component of an
  assessment plan
• Brainstorm about some ways to make assessment
  useful and meaningful
• Brainstorm about appropriate methods to use when
  assessing student learning
• Introduce ways that the assessment process can
  impact teaching, learning, curriculum, and other
  things important to us in the university community

3
External Pressure to Assess

• Federal government pressures to assessNCLB Act,
  Spellings Commission Reports, etc.
• State pressures will not dissipate, given budget
  constraints
• Regional accreditors (for us, the Middle States
  Association) require it
• Local constituencies want itthey want to know
  how well universities and colleges are adding
  value to a students education
• The public increasingly expects it
• Even grant funders want itassessment systems
  submitted with grant applications, assessment
  results included with grant reports, etc.

4
Example of Regional Accreditors Review Middle
States Association Suggestion for Binghamton
University
5
Despite External Pressures, There Are Real
Advantages to Program Assessment

• Enables programs to answer external requests for
  information
• Empowers faculty, not bureaucracies to make
  decisions about curriculum, instruction, and
  learning
• Enhances grant application process
• Empowers faculty to reflect on student teaching
  and learning in a way that is non-threatening
• Enhances facultys ability to publish
• Improves student learning

6
Why An Assessment Plan?

• Facilitates periodic, not episodic assessment of
  student learning and program outcomes
• Serves as template for conversations about
  student learning
• Leads to improvement in student learning
• Enables faculty to play central role in managing
  student learning
• Facilitates information gathering for program
  review process
• Makes it easier to do something about what
  assessments have to say

7
Whose Needs Might We Consider When We Develop
Assessment Plans?
8

Speaking Different Languages Assessment
Communities In Practice
University Community

• Federal Guidelines
• State Guidelines
• North-Central Association Guidelines
• Public Expectations and Norms

Regulatory Community

• Mission Statement
• General Education
• Objectives
• Constituent Expectations and Norms
• Faculty Expectations

• Dispositional Expectations
• Professional Standards
• Advisory Board
• Expectations
• Professional Organization Norms and Expectations
• Employer Expectations

Program Community
Assessment Plan
9
Reflect on some of the different audiences that
might be interested in or that might require
assessment information about student learning.
What are some of those audiences?

• University Community
• Example Faculty
• Regulatory Community
• Example Federal Government
• Program (or Discipline) Community
• Example National Council on Accrediting Teacher
  Education (program accreditor)

10
Faculty Involvement Is Crucial

• The focus of assessment is not measurement, but
  how assessment information is used by faculty
• Only faculty are qualified to make decisions
  about how to use assessment information for
  enhancing teaching, learning, curriculum, etc.
• This is an opportunity, not an obstacle

11
Assessment As A Faculty-Based Process
12
Components of An Assessment Plan

• Student Learning Objectives
• Assessments
• Plan for Faculty Discussion About Assessments and
  How they Lead to Recommendations to Further
  Enhance Student Learning Objectives
• Plan to Implement Recommendations in Respect to
  Curriculum, Instruction, etc.

13
Identifying Student Learning Objectives
14
Defining Learning Objectives

• Knowledgewhat do we want students to know when
  they graduate (i.e., content knowledge, etc.)
• Skillsability to perform specific tasks, think
  in certain ways, etc. what should a graduating
  student be able to do?
• Competenciesability to perform specific tasks
  in real time, or authentically knowledge
  skills ? competencies also, what are some
  values, attitudes, behaviors we feel are
  important for graduates to have?

15

• Listing Student Learning Outcomes By Knowledge,
  Skill, and Competency

1. Think about some of the knowledge, skills, and
   competencies, you would like a student from your
   program to graduate with
2. List them
3. Discuss with a partner at your table.

16
(No Transcript)
17
Assigning Relevant Assessments
18
Selecting Appropriate and Meaningful Assessments

• At least one should be a direct assessment of
  student learning, meaning that they should
  involve observations of actual student
  performance
• Indirect assessments those that include student
  opinions about a programs ability to deliver on
  the student learning objectives, etc. can also be
  very helpful
• Using a combination of thesetwo or threemight
  provide meaningful information triangulation
  is increasingly being required by evaluators
• It is rarely a surprise to find out that
  departments and programs initially feel that they
  never directly assess student learning, only to
  find out after discussion that they have been
  doing so, only informally

19
Not everything that can be counted counts, and
not everything that counts can be counted.

• -Albert Einstein

20
Data is the plural of anecdote

• -Ronald Coase

21
Helpful Tips

• The process of selecting appropriate assessments
  is usually dependent on a number of factors
  experience in process, resource limitations,
  etc.
• While the preference is to move toward a
  comprehensive assessment system, reality suggests
  that the development of an assessment system is
  very much developmental it takes time to develop
  such a system.
• Although the prior point is important to keep in
  mind, most program and regional accreditation
  organizations are beginning to expect 3-5 years
  of periodically collected assessment information
• Therefore, begin incrementally. Start with
  assessments that create a balance between need to
  conserve resources and need to maximize the
  meaning gleaned from the process
• The most important question to ask at this point
  How will this information provide faculty with
  legitimate information that will affect learning,
  teaching, and curriculum?
• A shameless plug Ask the Assistant Provost for
  Curriculum, Instruction, Assessment for
  assistance!

22
Outcomes Assessments from the Perspective of
Different Communities
Assessment Plan
University Community
Examples State Licensure Exams (Passage
Rates) General Education Syllabi Reviews
Regulatory Community
Examples Student Satisfaction Surveys Portfolio
Assessments Graduate Record Exam (GRE) Math
Scores Focus Group Interviews Unstructured
Student Interviews
Examples Capstone Course Case Study
Reviews Advisory Board Input GRE Subject Exam
Scores
Disciplinary Standards Community
23

• Encouraging Faculty Discussion

24
Defining When Faculty Will Discuss Information

• Faculty (or an assessment committee) should be
  given information in advance to consider
• Focus of meeting is to discuss what assessments
  say about student learning objectives and any
  recommendations that might stem from such
  discussion it should not be a rubber stamping
  of findings or solely an opportunity to
  complaininstead, the focus should be upon
  recommendations
• It is important to focus on communicating results
  to faculty, and tracking ways that information is
  used

25
Tip on Producing An Effective Meeting

• Try to Put Assessment Results on One Page, If
  Possible

26
  Assessment Graduate Alumni Surveys Assessment Graduate Employer Surveys Assessment Expert Panel
NBPTS 1 Teachers Are Committed to Students and Their Learning      
1.1 Teachers recognize individual differences in their students and adjust their practice accordingly q8 "Using a variety of methods, strategies, and materials to promote development, learning and cooperation" (3.29--satisfactory) q5 "Using a variety of methods, strategies, and materials to promote development, learning and cooperation" (4.14--good) Graduate students know about a wide range of research-based pedagogies--expert panel average was 3.7 (satisfactory/well prepared)
1.2 Teachers have an understanding of how students develop and learn q4 "Using knowledge of child/adolescent development and relationships with students and families to plan instruction based on abilities, interests and needs" (3.14--satisfactory) q1 "Using knowledge of child/adolescent development and relationships with students and families to plan instruction based on abilities, interests and needs" (4--well prepared) Graduate students know how to teach based upon what is developmentally appropriate--expert panel average was 3.8 (satisfactory/well prepared)
1.3 Teachers treat students equitably q6 "Establishing caring, inclusive, stimulating and safe environments that facilitate learning, collaboration, independence, and intellectual risk taking" (3.0--satisfactory) q3 "Establishing caring, inclusive, stimulating and safe environments that facilitate learning, collaboration, independence, and intellectual risk taking" (3.0--satisfactory) Graduate students have developed a recognition of professional ethics of their discipline--expert panel average was 4 (well prepared)
1.4 Teachers' mission extends beyond developing the cognitive capacity of their students q19 "Helping students explore important life issues, and world issues" (2.57--satisfactory) q16 "Helping students explore important life issues, and world issues" (3.43--satisfactory) Graduate students appreciate how knowledge is linked to other disciplines--expert panel average was 3.4 (satisfactory).
27

• Implementing Faculty Recommendations, and
  Affecting Learning and Teaching

28
Ideas for Using Assessment

• Use information to pursue department or
  program-level initiatives or projects on teaching
• Incorporate assessment information into
  curriculum discussions and recommendations
• Have periodic discussions or reflective
  discussions on teaching using assessment
  information
• Use for grant applications
• Use in budget requests
• Use in fundraising letters or alumni functions
• Use to gather further information about student
  learning from instructors on a periodic basis

29
Ways to Track Effect of Faculty
Discussions/Recommendations

• Encourage inclusion in annual report (section
  under teaching effectiveness)
• Encourage submissions of faculty narratives how
  were faculty discussions and recommendations used
  in course design, selection of courses, etc.?
• Track how recommendations were enacted
  curriculum process, department/program
  initiatives, course sequencing, equipment
  requests, etc.
• Not so important to make an academic study out of
  this, or to make this too huge of a process, but
  documentation is helpful when writing annual
  reports on assessment, curriculum, instruction

30
Future Directions and Vision

• Will ask for report on four questions at end of
  each academic year
• Focus is not upon reporting for reportings
  sake, but instead upon how we can assist
• Focus will be upon serving facultyin helping
  them in accomplishing the last two of the four
  core questions
• Focus primarily upon impacthow has process
  impacted learning, teaching, and curriculum
  processes

31
Conclusion

• Four foci
• Objectives
• Assessments
• Faculty Usage
• Impact

32
All assessment is a perpetual work in progress.
-- Linda Suske, Unknown , May 3, 2005
33
In assessment, "the perfect is the enemy of the
good." Let's keep striving for the good. -- Tom
Angelo, Unknown , Unknown
34
Questions? Comments?