Curious Places

1
Curious Places

• A Room that Adapts using Curiosity and Supervised
  Learning

Kathryn Merrick, Mary Lou Maher Rob Saunders
October, 2007 Key Centre of Design Computing and
Cognition, University of Sydney
2
Overview

• Adaptable, Intelligent Environments
• Curious Supervised Learning
• A Curious, Virtual, Sentient Room
• Limitations and Future Work

3
Adaptable, Intelligent Environments

• The computer for the 21st century
• Hundreds of computers per room
• Computers come and go
• (Weiser, 1991)
• Adaptability is important at two levels
• The middleware level
• The behaviour level

4
Adaptable Middleware

• Resource management and communication
• Adaptability has been widely considered at this
  level
• Real time interaction
• Presence services
• Ad hoc networking

Intelligent Room Project
Gaia
BlipSystems
5
Adaptable Behaviour

• Adapting behaviour to human activities
• Supervised Learning
• The Neural Network House
• Data mining
• Considered in fixed domains
• How can we achieve adaptive behaviour in response
  to changing hardware or software?

6
Adaptability by Curiosity and Learning

• Curiosity adapts focus of attention to relevant
  learning goals
• Learning adapts behaviour to fulfil goals
• Curious reinforcement learning
• Curious supervised learning

MySQL Database
Agent
Projector
Rear projection screen
Curious Information Display
Curious Research Space
PC
Agent
Bluetooth blip nodes
7
Supervised Learning

• Learning from examples
• A supervised learning problem P can be
  represented formally by
• A set S of sensed states
• A set A of actions
• A set X of examples Xi (Si, Ai)
• A policy p S ? A

8
Complex, Dynamic Environments

• Contain multiple learning problems
• P P1, P2, P3
• Learning problems in P may change over time
• Addition of new problems
• Removal of obsolete problems

9
Modelling Curiosity for Supervised Learning

• Aim to focus attention on states, actions and
  examples from a subset of problems
• Works by filtering
• Identify potential tasks to
• learn or act upon
• Compute curiosity values
• Arbitrate on what to filter
• High curiosity may trigger
• learning or action
• Low curiosity does not

S(t), X(t)
Curiosity
Observations and events
Task Selection
Curiosity Value
Arbitration
S(t)
X(t)
Learning
Action
10
The Curious Supervised Learning Agent

• Past states, examples and actions are stored in
  an experience trajectory Y
• Experiences may influence curiosity

11
A Curious, Virtual, Sentient Room

• A university meeting room in Second Life
• Seminars and Meetings
• Tutorials
• Skype-conferencing

12
Virtual Sensors and Effectors

• Floor Sensors
• SmartBoard and Chairs

• Blip System
• Lights

13
Meta-Sensors and Meta-Effectors

• Agent does not communicate directly with sensors
  and effectors
• Agent has a sensor of sensors and an effector
  of effectors
• BlipSystem provides an up-to-date list of current
  sensors and effectors and acts as an intermediary
  for communication

14
The Curious Room Agent

• Computational model of novelty used for curiosity
• Table-based supervised learning using
  associations
• Learns accurately but
• Unable to generalise

15
Behaviour of the Curious Place

• Avatar enters ? Lights go on
• Avatar sits ? SmartBoard on ? Lights off

16
Preliminary Evaluation

• 6 repetitions by human controlled avatars
  required for learning
• Can adapt to new devices
• Can adapt simple behaviours to form more complex
  sequences

17
Limitations

• Current prototype is proof-of-concept only, no
  significant empirical results yet
• Issue of if/when/how to forget behaviours
• Is an interface required for manual editing or
  override of learned behaviours?

18
Future Work

• Further work on curiosity models
• Design a suite of experiments to test attention
  focus in
• Environments of increasing complexity
• Dynamic environments
• More complex tasks