Assessment of ServiceLearning: Principles and Techniques

1
Assessment of Service-Learning Principles and
Techniques

• Barbara A. Holland, Ph.D.
• Senior Scholar, Indiana University-Purdue
  University Indianapolis
• Director, National Service-Learning Clearinghouse

2
Activity

• What is motivating you to think about assessment?

3
Effective Assessment Plans

• Focus on core issues
• Reinforce common definitions/terms
• Are grounded in data
• Reflect all participant perspectives
• Document strengths and areas for improvement
• Anticipates the audiences for and applications of
  results
• What you measure is what you get!

4
Assessment as Improvement

• Mechanism to tell your story
• What have you learned that is useful
• For your own work
• For you to share with others
• Focus on documenting impact and outcomes
  barriers and facilitators
• Considers all perspectives SL is collaborative

5
Outcomes of Assessment

• Descriptive information
• Analytic information and comparisons
• Case studies
• Evidence of impacts
• Principles of good practice
• Ideas for program improvement

6
Good and Bad Assessments

• Good assessments will
• define strengths
• validate knowledge
• provide evidence for resource decisions
• identify opportunities for improvement
• Bad assessments will
• consume energy and resources
• Undermine program activities

7
Service-Learning is

• Integrated into courses or other learning
  activities to meet specific learning objectives
  in ways that also
• Enhance community
• Transform participants
• Are intentional and rigorous
• Experiential and Reflective

8
Planning for Service-Learning

• What are the expected learning goals?
• How does SL help students reach those goals?
• What will be the teaching role of community?
• What will be evidence of mastery of content?
• How will we measure impact on student learning
  and development?
• How will we measure impact on community capacity?

9
Learning Objectives

• Community Learning about
• A particular community or population
• A particular issue, challenge, opportunity
• The provision of services to community
• A particular organization or grass-root effort
• Relevant public policies historic perspectives
• The role of stakeholders

10
Learning Objectives (Continued)

• Inter and Intra-personal Learning
• Working collaboratively with others
• About other groups and cultures (diversity)
• Practicing effective communications
• Developing self-efficacy
• Developing empathy
• Learning to appreciate different views

11
Learning Objectives (Continued)

• Learning to be a Learner
• Active -Independent
• Extract meaning from experience
• Apply knowledge to real world
• Use evidence to articulate ideas
• Learn across subject areas
• Find and assess the quality of information
  resources

12
Activity

• Using a few key words, describe one or two goals
  for service-learning.what impacts do you expect
  service-learning will have on
• Students, or teachers/leaders/faculty, or
  community partners?

13
Planning for Assessment

• Before you Begin
• What is the aim of your assessment?
• Who wants/needs the assessment?
• What resources are available?
• Who will conduct the assessment?
• How can you ensure results are used?

14
The Matrix/Multiple Method Approach

• Using your service-learning goals as a framework
• Build an Assessment Matrix
• Core concepts
• Key indicators
• Multiple methods
• Sources of information
• Use the matrix for implementation, analysis and
  reporting

15
Assesment Design

• Project goals What do we want to know?
• Core Concepts What will we look for?
• Indicators What will be measured?
• Methods How will we measure?
• Then
• Analysis
• Improvement actions
• Dissemination

16
Key Concepts Students

• Awareness of community
• Commitment to service
• Career exploration
• Self-awareness
• Understanding course content
• Communications skill development
• Cross-cultural skills

17
Student Example

• Goal Service-learning helps students discover
  their potential role in community life.
• Concept Career exploration
• IndicatorsExpressed career interest
  Demonstrated career skills and attributes
  Knowledge of career requirements Understanding
  of career responsibility to public
• Methods Survey, interviews, journals

18
Another Student Example

• Goal Prepare students to be effective and active
  citizens in their communities
• Concept Awareness of community
• Indicators Knowledge of issues, ability to
  identify assets/needs, understanding of problems
  and policies
• Methods survey, interviews, observation

19
Key Concepts Faculty/Teacher

• Motivation and attraction to engagement
• Professional development
• Impact/influence on teaching strategies
• Impact/influence on scholarship (higher ed)
• Other personal/professional impact
• Identification of barriers and facilitators
• Satisfaction with experience

20
Faculty/Teacher Example

• Goal Service-learning will improve teaching
• Concepts philosophy of teaching, teaching and
  learning strategies
• Indicators teaching roles, class format,
  organization, environment, values
• Methods observations, interviews, lesson plans

21
Key Concepts Community

• Capacity to fulfill organizational mission
• Economic effects
• Social benefits
• Perception of mutuality
• Satisfaction
• Sustainability of the partnership

22
Community Example

• Goal SL activities are collaboratively designed
  to meet organizational needs of partner
  organizations
• Concept Capacity to fulfill mission
• Indicators
• Insights about organizational directions and
  operations staff impacts Number of clients
  served Changes in activities offered Insights
  into assets and needs Leveraged
  resources/funding
• Methods Interview, observation, reports/documents

23
The Role of Partnership Assessment

• Strengthen the partnership
• Build a foundation of mutual understanding, based
  on a clear philosophy and common goals
• Reinforce mutual learning and decision-making
• Focus on feedback and improvement
• Remember that attitudes and perceptions matter as
  measures of benefit and satisfaction with the
  partnership

24
Activity

• Think of your own SL goals
• Identify one key concept you want to assess (what
  will you look for?)
• For that concept, propose two measurable
  indicators (what will you measure?)
• Focus on concepts and indicators DO NOT think
  about data collection methods!

25
Selecting Assessment Methods

• What instrument(s) to use
• Why and when to use it/them
• Consideration of characteristics
• Types of questions
• Format/design
• Process of data collection
• What to do with the data

26
Effective Methods

• Focus groups Efficient, interactive
• Interviews time intensive deeper views of
  individual experiences
• Observation- time intensive multi-purpose
• Surveys time efficient, objective, anonymous,
  can be superficial
• Journals, syllabi, documents, site reports-
  useful for validation and cross-checking

27
Resources

• Gelmon, Holland et al., Assessment Handbook,
  Campus Compact, 2001(www.compact.org)
• Furco Institutionalization Assessment Rubric,
  Forthcoming from Anker Pub, 2004
  afurco_at_uclink4.berkeley.edu
• CART (Compendium of Assessment and Research
  Tools). RMC Research http//cart.rmcdenver.com/
  
• National Service-Learning Clearinghouse
• www.servicelearning.org

28
Contact

• Barbara A. Holland, Ph.D.
• Director, National Service-Learning Clearinghouse
• www.servicelearning.org
• Barbarah_at_etr.org
• Toll-free 866-245-7378 ext. 273