StoryStation

1
StoryStation

• Peter Wiemer-Hastings Judy RobertsonKath
  Glasswell
• DePaul University University of
  EdinburghUniversity of Illinois at Chicago

2
Motivation

• Learning to write (well) is hard
• Multiple constraints / voices (Flower, 1994)
• Reader What do I want R to know? What do I know
  about what R knows? What do I want from R?
• Situation What am I supposed to write about?
  What do I know about my topic? What do I want to
  say about it?
• Socio-linguistic context How can I say this in a
  way that my teacher will accept?

3
Metacognitive Double Whammy

• Students must learn how to think about meeting
  the demands of multiple, possibly conflicting
  constraints.
• Students ordinarily dont know what those
  constraints are.

4
AI to the rescue!?

• Appropriate feedback on compositions can point
  students in the right direction.
• Teachers dont have time to give the best
  feedback to every student all the time.
• NLP techniques can analyze compositions, generate
  feedback.
• Associating different types of feedback with
  different animated agents might help the students
  learn the various constraints.

5
Research goals

• To investigate the effect of animated pedagogical
  agents on 10-11 year old pupils
• motivation,
• satisfaction,
• learning (eventually).
• To develop an intelligent tutoring system to help
  children write stories

6
Agenda

• Previous work
• Design process
• StoryStation features
• Evaluation preliminary results
• Future directions

7
Previous research on agents

• Mayer and Wittrock, 1996
• With agents, students have more social and
  emotional investment, more motivated, try harder,
  problem solve better.
• Herman the Bug (Lester et. al, 1997)
• Students more motivated when using agents, and
  learn more.
• Johnson, Rickel, and Lester, 2000
• increased communication bandwidth and social
  interaction with agents
• Review of agent research (Dehn and van Mulken,
  2000)
• Some serious methodological problems. No control
  in Lester et al.
• Moreno, Mayer, and Lester, 2001
• Included control. Agents are social actors,
  increase motivation, learning, transfer. Voice
  is important, face not.

8
Previous writing research

• Previous writing environments were designed to
  support cognitive processes of writing (e.g.
  Sharples 1988), based on Flower and Hayes (1980)
  theory.
• Writing apprehension (Madigan, Linton and
  Johnson, 1996)
• More recently, interest in providing emotional
  support for writers (e.g. Triffic Tales project)
• StoryStation basic strategy Provide positive
  feedback to encourage pupils to take
  responsibility for their own learning

9
Design process

• Requirements gathering with 2 teachers and 37
  pupils
• Integration of National English Language
  Curriculum goals
• Design team of 8 pupils and 2 teachers, working
  on the project regularly for 18 months
• Observation of classroom practice
• 3 pilot studies

10
StoryStation features

• Spelling
• Word banks
• Dictionary
• Thesaurus
• Vocabulary
• Characterization
• Plot (in progress)

11
Story re-telling Task

• Students hear a story-teller present a story (or
  a video)
• Students are asked to create their own version
  with StoryStation.
• Scaffolds students writing.
• Facilitates text analysis.

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
Evaluation 18 teachers

• Positive feedback is a good feature, contrasts
  with programs which simply point out mistakes.
• Useful for independent learning because it
  teaches pupils how to self correct.
• Good for pupils to get instant feedback rather
  than waiting till the next day.
• Would like StoryStation to help them in assessing
  pupils progress
• Generally, teachers are surprisingly trusting of
  technology

18
Evaluation - pupils

• 60 pupils (10 12 year olds)
• Half used agents version, half text-only
• Pupils
• Listened to 10 minute recording of The Screaming
  Skulls of Calgarth
• Watched demo of StoryStation
• Wrote their own version of Screaming Skulls
  using StoryStation (40 minutes to 75 minutes,
  depending on timetabling constraints)
• Filled in questionnaire
• Answered structured interview questions

19
Childrens comments on StoryStation

• Id probably buy it for my home computer
• Roseanna, age 12
• If I had StoryStation on my computer Id
  definitely use it more than Microsoft Word
• Rosemary age 12
• Its really good and it helps you to write
  better
• Shenel, age 12
• It makes writing fun. You enjoy it more when
  youve got the helpers there
• Joanna, age 11

20
Questionnaire results
6. StoryStation makes writing stories easier 7. I
think I need someone to help me use
StoryStation 8. I think the StoryStation advice
was useful 9. A teacher is more helpful than
StoryStation 10. StoryStation is boring
1. I enjoyed using StoryStation2. I think
StoryStation made my writing worse 3. I think I
would like to use StoryStation again 4. Using
StoryStation help me to write better 5. I found
StoryStation confusing
21
(No Transcript)
22
Nice pictures
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Some issues

• Childrens thoughts about agents
• Mixed ideas about whether agents are real.
• Childrens comparisons of teachers to
  StoryStation
• A teacher can tell from your eyes what youre
  thinking, but StoryStation cant Saima, age 11
• Social relationships The teacher can shout at
  you, but you can shout at StoryStation
• Knowledge Take StoryStations advice more
  because it is a computer

27
Current Proposal

• NSF-ROLE grant proposal with Elliott and
  Glasswell
• Clean up the metacognitive hypotheses and
  evaluations
• Address some of the questions about agents.

28
Conclusions

• Children think StoryStation is useful for writing
  stories
• StoryStation is motivating to children
• Agent users more likely to find StoryStation
  advice helpful, and want to use it again
• Agent users less likely to find StoryStation
  confusing
• Interesting social / gender interactions

29
Challenges

• How to foster metacognition
• Appropriate feedback depends on genre, context,
  audience, etc
• Dialog about student choices?
• How to measure learning
• How/where to get funding to continue

30
Questions or Comments?