Natural Language Generation for Intelligent Tutoring Systems

1
Natural Language Generation for Intelligent
Tutoring Systems

• Barbara Di Eugenio, Michael Glass, Michael J.
  Trolio, Heena Raval
• Collaborators
• Johanna D. Moore (U. Edinburgh, UK)
• Carolyn P. Rose (U. of Pittsburgh)
• Susan Haller (U. Wisconsin, Parkside)
• Stellan Ohlsson (Psychology, UIC)
• Supported by Office of Naval Research

2
Focus

• Deploy NL interfaces to support
  computer-supported instruction and educational
  technology
• NL interfaces crucial to next generation of
  Intelligent Tutoring Systems (ITS) the 2 Sigma
  effect Bloom 82
• Question how to inform production of NL feedback
  by ITS

3
A Tutoring Dialogue Excerpt

• TUTOR Can you understand why you would need to
  remove a wire from the circuit before attaching
  the leads of the ammeter?
• STUDENT Yes you cannot work with the power on
  and the power needs to be off or disconnected
• TUTOR So if the wire was not removed, you think
  the power would be on in the circuit eventhough
  the switch is open?
• STUDENT The power would not be on but it is
  possible for energy to still be going through the
  wire
• TUTOR Actually, that is not the case.

4
Research Questions

• Which tutoring strategies should an ITS adopt,
  and how are they expressed in NL?
• Which features of conventional dialogue carry
  over to tutoring? (e.g., requests)
• How to generate reactively? Need ability to
  replan
• How to evaluate? Need to positively affect
  student learning

5
Research Methodology

• Mark-up. Annotate language data
• Mine. Extract information from annotated corpus,
  via statistical techniques or machine learning
• Develop computational model based on results in
  (2)
• Conduct systematic evaluation of implemented
  system

6
One illustrative project DIAG-NLP

• Goal Evaluate the added value of NL
  interfaces to DIAG Towne97, an ITS shell to
  teach students how to troubleshoot complex
  systems
• Two versions of the system. They differ in how
  language feedback to student query is generated,
• DIAG-orig uses simple templates. Resulting text
  is repetitive
• DIAG-NLP uses NL generation technology

7
Results so far

• Between-subject study shows cumulative effect in
  favor of DIAG-NLP (9 measures out of 11)
• Three types of measures knowledge of subject
  matter usability of system recollection of
  actions

8
Current work

• Completed Data collection.
• 25 dialogues three tutors who verbalize
  output DIAG-orig provides
• In progress Data Analysis.
• Focus Domain modeling Tutorss actions
  portion of feedback produced by DIAG-orig the
  tutor verbalizes
• Next Computational model. Data Analysis will
  inform NL interface