Semantic Parsing for Robot Commands

1
Semantic Parsing for Robot Commands

• Justin Driemeyer
• Jeremy Hoffman

2
Purpose

• STAIR (STanford AI Robot)
• intelligent office assistant
• Spoken dialogue, e.g. commands to move
• Robot, could you move forward two feet?
• Please go up to my office, room two-oh-four, and
  wait there.
• Oh, uh, wait, hey robot, stop.
• Goal robust language understanding
• Extract meaning of utterance from transcription
• Overcome unexpected words or phrasing,
  stuttering, errors of the speech recognition
  component

3
Dialogue System
Audio Input
Utterance/ Command Semantics
SEMANTIC PARSER
SPEECH RECOGNIZER
DIALOG MANAGER
Text
Robot action (e.g., speak, move)
4
Semantic PCFG Parser

• Idea PCFG parsing with semantic grammar
• Tag sentences with semantic labels instead of
  syntactic labels, e.g., MoveWord,
  Destination, ExtraWord
• Train our Assn 3 PCFG parser from a treebank of
  hand-labeled utterance transcriptions
• From a parsed sentence, can read off the
  important content directly
• Problem no robot-command corpus (yet)
• Instead, automatically generate a fake treebank
  of tagged sentences
• We hand-wrote a PCFG for commands to move or
  control movement
• Also serves as probability prior once we get
  some real data

5
Frames and Slots In Our Grammar

• We focused on the three commands encoding
  movement.

Frame MacroMove MicroMove Control
Slots Destination holds either a room number or a named place (e.g., front lobby) that can be converted to a room number from a building map Direction e.g., forward, back, left, right, around Distance holds a number and a unit of distance (e.g., feet) CommandWord e.g., stop, wait, cancel, go, ok, continue
6
Command Extraction

• Parse tree
• (ROOT
• (S
• (MacroMove
• (MacroMoveWords (MacroMoveWord go)
  (MacroMoveWord to))
• (Destination
• (NamedPlace (WordThe the) (NamedPlaceWord
  lab) (NamedPlaceWord room))))))
• Desired Command
• (MacroMove (Floor 0) (Room 100))

7
Command Extraction

• Two parse trees could be different, but encode
  the same command
• Gold Standard
• (ROOT
• (S
• (MacroMove
• (RobotAddress
• (ExtraWords (ExtraWord hey))
• (RobotName (RobotNameWord stair))
• (PoliteWords (PoliteWord could)
  (PoliteWord you)))
• (MacroMoveWords (MacroMoveWord follow)
  (MacroMoveWord me) (MacroMoveWord to))
• (Destination
• (NamedPlace (NamedPlaceWord my)
  (NamedPlaceWord office))))))
• Extracted
• (ROOT
• (S
• (MacroMove

• Both encode
• Command Tree
• (MacroMove (Floor 2) (Room 243))