A Generic Episodic Memory Module

1
A Generic Episodic Memory Module

• Dan Tecuci
• August 18, 2005

2
The Need for Memory

• Remembering characteristic of intelligence
• Dont remember the past, bound to repeat it
• Memories used for
• Problem solving
• Adapting previous solutions
• Anticipation of future events (likely outcome,
  failures)
• Recognition (goal opportunities)
• Complex systems need a Generic Memory

3
Problem Solving Applications

• Memory-based planning
• Purchasing assistant
• Memory-based diagnosis
• Fault diagnosis
• Memory-based recognition
• Plan recognition

4
Memory Requirements

• External (imposed by the overall application
  system)
• Ability to store great number of experiences
• Ability to retrieve experiences relevant for
  current situation
• Robustness in solving problems
• Internal
• Content-addressable
• Organized for efficient recall
• Flexible matching

5
Human Memory

• Types of memories
• Semantic general knowledge
• Episodic specific events
• Procedural skills
• Working temporary storage
• Episodic memory (Tulving 83)
• distinct subsystem (functionally structurally)
  - maintains a record of specific events (e.g.
  last trip to the mall)
• Semantic memory (Quillian 69)
• Abstract, timeless knowledge of the world (e.g.
  the color of the sky)

6
Episodic vs. Semantic - Similarities

• Knowledge acquisition (mostly through the senses)
• Knowledge retention (passive, automatic)
• Knowledge utilization (retrieval)
• triggered by stimuli
• highly selective
• subject only aware of retrieval results
• retrieved knowledge can be described verbally

7
Episodic vs. Semantic - Differences

• Information handled
• Episodic specific events, temporally organized
  and self-centered
• Semantic summaries and abstractions, organized
  conceptually, generally agreed upon
• Operation
• Episodic one-shot learning, limited inferential
  capabilities, context dependent, more vulnerable
• Semantic acquired gradually, rich inferential
  capabilities, less vulnerable.
• Episodic memory develops after Semantic

8
Episodic Memory in AI Systems

• Dynamic Memory (Schank 82)
• Model of human episodic memory
• Reminding uncovers memory structures (MOPs, TOPs)
• No explicit representation, organization and
  global retrieval algorithm
• Cyrus (Kolodner 81)
• Organize and retrieve events by indexing and
  context elaboration
• Chef (Hammond 86)
• Indexes plans by goals, failures
• Memory used for plan repair, anticipate problems
• Veres basic agent (Vere 90)
• Stand-alone Episodic memory (dialogue management,
  planner)
• ULTM (Lawton et. al. 99)
• General purpose long-term memory general
  knowledge and domain specific heuristics
• Retrieval directed search
• Soar (Nuxoll 02, Nuxoll and Laird 04)
• Episodes stored as rules, used for predicting
  results of actions, reflection
• Retrieval spreading activation, deliberate
• Worst-case retrieval is linear, no organization
  of stored episodes, no relation between semantic
  and episodic memory, no automatic retrieval

9
Previous Approaches - Summary

• Ideas
• Reminding uncovers memory structures (Schank)
• Expectation failure drives reminding (Schank)
• Retrieval as directed search (Lawton)
• Shortcomings
• No generic representation of episodes
• Limited domains
• No generic retrieval algorithm
• Only index feature-value pairs

10
Proposed Work

• Design a generic Episodic Memory Module
• Purpose
• To be used in a variety of systems and domains
• Generic Episodic Memory API
• Organize systems experiences so that they are
  available for
• Problem solving by adapting prior cases
• Anticipation and repair of plan failures
• Opportunistically suggesting goals
• Supporting role, not a complete solution for
  problem solving in these domains

11
Memory Functions

• Encoding
• When to store episodes already segmented
• What to store entire episode
• How to store differences from norm
• Storage
• Similar episodes grouped into memory structures -
  MOPs
• MOPs form similarity network
• Retrieval
• Memory traversal process
• Automatic deliberate
• Cue construction differences
• Matching semantic, depth bound

12
A Generic Episode

• Generic Episode
• Context general setting in which the episode
  happened (e.g. goal)
• Contents the ordered set of events that make up
  the episode (e.g. plan actions)
• Outcome evaluation of the impact of the episode
  (e.g. success/failure)
• Applications
• Memory-Based Planning Context ? Contents
  Outcome
• Memory-Based Recognition Contents ? Context
  Outcome
• Memory-Based Diagnosis Outcome ? Context
  Contents

13
Memory API

• store
• NewEpisode ltcontext, contents, outcomegt
• Modifies memory to accommodate NewEpisode
  (indexed by how it differs)
• Returns previous episodes used in processing
  (automatic retrieval)
• retrieve
• Query ltcontext, contents, outcomegt
• Most similar prior Episode is returned along with
  how it differs

14
Memory Structures

• MOPs
• Organize similar episodes
• Encode expectations
• Exemplar links to prototypical Episodes
• Linked to other MOPs by difference links
• Domain Knowledge
• Actions, Objects, Goals, States
• Specific Episodes
• Sequences of Actions with clear Goal
• Indexed under one/more MOPs by their differences

15
Evaluation Domain

• Memory-based planning task
• Logistics domain (AIPS 2000) package delivery
• Planning Problem
• Given initial state, final state
• Find Plan that change initial to final state
• EM Problem
• Given initial state, final state
• Find a prior Plan that had similar initial and
  final states
• EM performance evaluated, not EMplanner

16
Episodic Memory Processing

• Storage and Retrieval - same basic process
• Given NewEpisode, Find the most similar
  PriorEpisode
• Hill-climb to MOP that best represents
  NewEpisode, then to the most similar Episode
• Differences suggest MOPs to search
• Similarity
• computed on one/more of the three dimensions
  (context, contents, outcome)
• Semantic match depth bounded graph match (Yeh
  03)

17
MOPs Example
Different City Deliveryfrom Post Office to
AirportAirplane at the Airport of City of Origin
MOP2
Package-Deliver
object
Package
location
Post-Office
Airport
inside
inside
location
City
Airplane
Different City Deliveryfrom Post Office to
AirportAirplane not at the Airport of City of
Origin
MOP1
18
Example

19
Evaluation Dimensions

• Accuracy were the retrieved episodes relevant?
• Measured - Precision, Recall, F-measure
• Scalability can memory store/retrieve a large
  number of episodes without a decrease in
  performance?
• episodes explored/retrieval task vs. episodes
  stored
• Content-addressability is memory
  content-addressable?
• retrieval accuracy vs. match-score(query, correct
  memory)
• Matcher flexibility relevant items on partial
  match?
• 10-fold cross-validation, 230 randomly generated
  planning problems, 11 types of plans
• Compared EM retrieval with exhaustive search
  through all MOPs

20
Results - Accuracy
21
Results - Scalability
22
Results Contents Addressability

• For successful retrievals (F-measuregt70)
• Match score between a query and the correct
  memory average is 78.76 (stdev5.13)

23
Proposed Work Episodic Retrieval

• Extend retrieval algorithm to other memory-based
  applications (recognition, diagnosis)
• Use other dimensions in retrieval (contents,
  outcome)
• Based on more than one dimension
• Similarity assessment different for sequences
  of events (order, effects, etc.)

24
Proposed Work Memory Organization

• Extend memory organization to large/diverse
  knowledge bases
• Organize different MOPs difference links
• Search
• Choose starting MOP
• Better indexing
• Use previous searches
• Navigate memory structure

25
Proposed Work Evaluation

• On different datasets
• Involving different memory tasks
• Focus on the whole system operation, not only on
  the memory task
• Evaluate memory contribution
• Performance of problem solving task
• System design simpler?

26
Summary - Goals

• Design a Generic Episodic Memory module can be
  used
• with a variety of applications and domains
• For a variety of memory tasks
• Provide API
• Desired qualities
• Scalable
• Robust
• Accurate
• Content-addressable

27
Summary - Accomplishments

• Representation of a generic episode
• ltContext, Contents, Outcomegt
• Memory based tasks expressed as retrieval using
  these dimensions
• Memory organization based on similarity of
  Episodes
• Generic Episodic retrieval algorithm
• Evaluated Episodic retrieval algorithm (context)
• Same accuracy as serial search
• More scalable
• Content-addressable

28
Summary Proposed Work

• Extend retrieval algorithm - recognition,
  diagnosis
• Extend memory organization to large/diverse
  knowledge bases
• Scale up evaluation - different datasets, memory
  tasks
• Focus whole system operation

29
The End
30
Other Issues

• When to stop looking for a better match?
• Trivial cases perfect match, no more MOPs
• When all important aspects of NewEpisode are
  explained by current MOP. But important is
  defined in terms of current MOP!
• Exploration algorithm
• Now depth bounded graph walk inefficient
• Want Memory - guide exploration. NewEpisode
  interpreted in terms of current memory structure.
  On failure, reminding occurs, move to a memory
  structure that can account for this failure (or
  create one) more efficient, more cognitively
  plausible

31
F-measure