Prue Anderson

1
How measuring the non-academic outcomes of
schooling can help guide school practice

• Prue Anderson
• Julian Fraillon
• ACER Research Conference
• Perth 2009

2
non-academic outcomes

• Successful learners who are able to plan
  activities independently, collaborate, work in
  teams and communicate ideas,
• Confident and creative individuals who have a
  sense of optimism about their lives and the
  future
• Active and informed citizens who act with moral
  and ethical integrity
• Melbourne Declaration, 2008

3
non-academic outcomes

• Is there such a thing?

4
Research Somers School Camp

• 10-day camp for grades 5 6
• adventure activities, environmental education and
  outdoor recreation
• students live and work in hut groups with
  students they do not know
• aim of camp is to develop students independence,
  confidence and self-esteem and to promote
  co-operation, communication and tolerance

5
What does research tell us?

• Prevailing view
• outdoor education programs inherently build
  character
• Literature review findings
• A general evangelical complacence underlies much
  of the research into the outcomes of outdoor
  education programs
• Common criticism of poorly constructed,
  un-standardised measurement instruments

6

• Counter view
• Outdoor education programs provide situations
  that elicit behaviour, such as an unadventurous
  person willingly trying new activities, but this
  is a result of the context rather than indicative
  of any significant change in the person.
• for programs to have lasting impact, people need
  to be able to reflect on their experiences in the
  broader context of their lives
• Andrew Brooks LaTrobe University

7
This process of measuring non-academic outcomes
can help guide school practice

• Defining the outcome
• Creating opportunities for students to
  demonstrate the outcome
• Measuring and improving

8
Defining the outcome What are you measuring?
9
Somers Camp outcomes
10
Overlapping ideas confuse teaching assessment
11
Clearly defined outcomes lead to focussed
teaching and assessment
12
Defining the outcome Is more better?
13
How do students typically grow?

• Likert scales administered in WA to measure
  students self-esteem, motivation to achieve
  potential, positive emotional management,
  responsibility for self
• e.g. I usually know how to behave properly.
• Y3 highest mean score
• Y9 lowest mean score
• For self reports and teacher judgments.

14
Defining the outcome What do you want to
achieve?

• Different contexts may have different optimum
• points of balance

Co-operation in class task
Co-operation with friends in yard
15
Defining the outcome What do you want to
achieve?

• The same context may require different points of
  balance on different outcomes

Co-operation in class task
Optimism in class task
16
Defining the outcome What do you want to
achieve?

• We suggest
• Incremental growth occurs in the increasing
  sophistication and depth of understanding of self
  and context that allows students to make good
  choices about how to behave.
• Cognitive reflective skills are critical in
  cultivating non-academic outcomes.

17
Defining the outcome What do you want to
achieve?

• Which outcome do you want to cultivate?
• What does it mean for students to develop in
• this outcome?

18
Non-academic outcomes

• Defining the outcome
• Creating opportunities for students to
  demonstrate the outcome
• Measuring and improving the outcome

19
Outcome
Somers Camp example

• Striving to achieve
• motivation, setting high
• goals, persistence

• High ropes course
• physically challenging, students set own goals
• strong encouragement to have a go
• high level of teacher support
• high chance of success

20
Somers Camp example

• Self Striving to achieve
• Assumption that students underestimate what they
  can do
• Students need highly supported, challenges to
  raise their expectations of what they can achieve
• After Somers Camp students will be more confident
  and set higher goals

21
prediction
orientation
22
Non-academic outcomes

• Defining the outcome
• Creating opportunities for students to
  demonstrate the outcome
• Measuring and improving the outcome

23
reflect on prediction
reflect on strategies used to get through course
reflect on support provided
generalise about learning
24
apply learning to new situations
set higher goals
have strategies for managing new challenge
25
Student responses

• Reflection on the task
• mainly restatement of given strategies
• good understanding of support provided
• few students able link high ropes activity to
  personal goals
• students generally showed little understanding of
  how to set personal goals

26
Improving the outcome

• RETHINK what do we want students to learn?
• activity is WOW factor
• motivation is persisting when things are
  boring/hard/lonely not thrill seeking
• what are better opportunities for students to
  learn?
• how can students be helped to link camp
  experiences to their everyday lives?
• what do students need to be taught about
  motivational strategies and personal goal
  setting?

27
Creating opportunities

• Is the task really focussed on the outcomes you
• want to address?
• Is there enough challenge?
• Classroom group work
• low-bar for routine work
• high-bar challenge in a specialised context

28
Creating opportunities

• Specialised contexts may be third party
• films
• novels
• issues in English
• Complex social situations analysed so students
• interpret and evaluate how others behave
• imagine and explain how they might behave
• explore alternative resolutions

29

• How does April manage her feelings?
• She blames herself and thinks she is totally
  incompetent.
• She tries to blame others for putting her in a
  horrible position.
• She cannot bear to think about her feelings and
  wants them to go away.
• She is interested in learning to understand and
  manage her feelings better.

30
Measuring the outcomeWhat are you measuring?

• Social skills assessments
• self-affirming?
• students reiterating given values?
• insightful and revealing?
• Team work skills
• co-operation or task completion?
• honesty or tolerance?

31
Measuring the outcome What instruments?

• Our experience
• focussed, probing reflective questions work
• show if students are learning what you hope
• spread students
• show what students need to learn next

32
Non-academic outcomes

• Defining the outcome
• Creating opportunities for students to
  demonstrate the outcome
• Measuring and improving the outcome

33
Somers Camp example

• Outcome
• Tolerance of others
• Understanding, consideration
• Inclusion, making friends
• Opportunities to learn whole camp

34
Examples of reflective questions

• What was the nicest thing someone you did not
  know did for you?
• Why did you like them doing this for you?
• What did you mainly do to be friendly to people
  you did not know (select often, sometimes, or
  hardly ever)
• I noticed if someone was left out.
• I tried to meet new people.
• I asked people I did not know to do things with
  me.
• I helped shy people feel comfortable.
• I did things with different people.
• Hut teacher observation This is a fair
  self-evaluation yes/no

35
Examples of reflective questions

• What do you usually do to get along with other
  people?
• (select often, sometimes, or hardly ever)
• I listen carefully to what people say.
• I have good ideas.
• I make people laugh.
• I get everyone to agree about what to do.
• I fit in with everyone else.
• Are you pleased with how friendly you have been
  at Somers Camp or do you think you could have
  been more friendly? Explain
• What would you like to change about yourself so
  you get on better with other people?
• What would help you make this change?

36
Example of marking scheme

• Are you pleased with how friendly you have been
  at Somers Camp or
• do you think you could have been more friendly?
  Explain.
• 2pt Evaluates and justifies friendliness. (Level
  4)
• Yes, I tried really hard and I did meet lots
  of new people.
• No, I could have tried harder, but I was too
  shy.
• Yes, I have been friendly to other people
  because I help them.
• I could have been more friendly because I want
  to make new friends.
• Yes, because I have met new friends and they
  like me.
• 1pt Evaluates friendliness with no
  justification. (Level 3)
• Yes, I was friendly.
• I could be friendlier.
• Im pleased with how friendly I have been.
• 0pt vague or irrelevant Everyone was friendly.

37
(No Transcript)
38
Somers Camp findings

• 75 of students demonstrated a limited capacity
  to reflect on their behaviour in groups, the
  dynamics of their interactions with others or to
  empathise with others in any substantial way
• these students require extensive teacher support
  to reflect more deeply on the challenges the camp
  offered in building relationships with unknown
  others, the way they responded to these
  challenges and how they might continue to build
  these skills.

39
Conclusion

• experiential learning is not enough to develop
  non-academic outcomes
• Students need
• a solid foundation of knowledge, skills and
  language
• opportunities to make informed reflections
• support to transfer their understandings to new
  contexts

40
This process of measuring non-academic outcomes
can help guide school practice

• Defining the outcome
• Creating opportunities for students to
  demonstrate the outcome
• Measuring and improving the outcome