Epistemic Structure An inquiry into how agents add structure to the environment for cognitive congen

1
Epistemic StructureAn inquiry into how agents
add structure to the environment for cognitive
congenialitySanjay Chandrasekharan
2

• Roadmap
• Epistemic structure Introduction
• Types, Properties
• How epistemic structures work a model
• Implications
• How epistemic structures are generated a model
• Pilot study

3

• Epistemic structure what is it?
• Structures in the environment generated by
  organisms, allowing them to minimize cognitive
  load (perception, search, inference).
• Examples
• Quorum sensing -- bacteria
• Pheromone trails -- ants, termites etc.
• Place/way markers -- Red foxes, wood mice
• Mate selection/warning markers -- bowers,
  warning smells
• Human markers page numbers, color codes, badges,
  shelf talkers, speed bugs, road signs etc.

4

• Two claims
• Premises
• 1) Such structures are generated by organisms
  across species.
• 2) There is a cost to generating such structures.
• 3) These structures are used by organisms while
  executing tasks.
• 4) Without the existence of such structures, the
  same tasks would be highly complex, and require
  significant cognitive resources.
• Given these premises
• Claim 1Such structures exist because they reduce
  agents' cognitive load.
• Claim 2 Because such structures are generated by
  organisms across species, adding structure to the
  environment to reduce cognitive load is a
  fundamental cognitive strategy.

5

• Application context digital markers in the
  environment
• Intel projects using RFID (radio-frequency
  identification) tags
• In Robotics -- Objects with RFID tags will
  announce their properties, affordances and
  constraints to robots
• In Assisted Cognition -- Objects with RFID tags
  will announce their properties, affordances and
  constraints to handheld devices used by people
  with cognitive disabilities like dementia.

6

• Types of epistemic structure
• 1) Generated for self Wood mice, Red fox
• 2) Generated for self and others pheromone
  trails
• 3) Generated exclusively for others bowers,
  warning smells
• Human equivalents
• 1)Page-markers
• 2)Color codes
• 3)Badges

7

• Properties of Epistemic Structure
• They dont exist a priori in the environment,
  they are generated by organisms.
• They are stable structures. i.e. they persist
  over time.
• They add complexity to the world sensed by
  agents. But at the same time, they reduce agents'
  cognitive load.
• They reduce agents' cognitive load by
  substituting for internal structures or
  processes.
• They allow agents to function in a
  reactive/almost-reactive mode.

8

• How epistemic structures work Example 1
• Paris soccer world cup -- routes to venues marked
  using giant soccer balls
• Finding venues became easier. But how?
• Imagine the task without the soccer balls.

Start, End states in the world
Intermediate points in the world
System state
World line
Iterated queries to the world
Mind Line
Internal processing
9

• Example 1
• Same task after the soccer balls are put up.

Start, End states
Intermediate points
System state
World line
Iterated queries to the world
Mind Line
Internal processing
Single query, and confirmation
10

• Example 2 Peacock's tail
• Advertises the good health/genetic quality of the
  peacock
• Helps peahens make a mating decision
• Scenario without the tail
• Step 1 Peahen will need to look for cues
• Possible cues lengthy chase of prey, long
  flights (peacocks fly short distances), big beak,
  thick claws etc.
• Step 2 Peahen ranks male on all or some of these
  cues. (good, bad, okay). Lengthy process,
  computationally intensive.
• Step 3 Compares the rankings with previously
  encountered males, or a minimum standard
• Step 4 Makes a decision

11

• Scenario with tail
• Quick perceptual judgement (Is the tail
  well-developed? Are the eyes well-formed?)
• Compares with previously encountered tails, or a
  minimum standard
• Makes decision

12

• Whats the big difference?
• Tail exists only for the peahen to make her
  mating decision.
• Function-specific structure fits the agents
  function/task directly.
• Lengthy-chase-of-prey, long-flights, big-beak
  etc. do not exist for the peahen to make a mating
  decision.
• Function-neutral structures These need to be put
  together, synthesized, by the agent to reach a
  decision.

13

• Same for giant soccer ball
• Street names and landmarks in Paris do not exist
  for the purpose of directing soccer fans to game
  venues. Function-neutral structures.
• Street names and landmarks need to be put
  together to find the game venue.
• Giant soccer ball is a function-specific
  structure, exists to mark routes to game venues.
• Function-specific structures existing in the
  world minimize agents cognitive load.

14

• Details -- A
• If the soccer balls were small, they would not
  work, being big is essential for the structure to
  work. Same with the tail of the peacock, being
  big is essential.
• Structures can be small and subtle, but then the
  agent should be actively looking for them to be
  useful.
• This means either
• a) Epistemic structures work by "hijacking"
  attention and perception (being big, colorful
  etc.)
• or
• b)They work because perception is directed, the
  organism is actively seeking such structures.
  (Shades of Active Vision).
• Either way, epistemic structures will not work if
  perception is just a passive input channel.

15

• Details -- B
• Task-specific structures allow agents to
  "short-circuit" internal processing and execute
  the task.
• But the information needed to execute the task is
  constant.
• This means the information to execute the task
  comes from perception. That is, perception is
  doing more work.
• With appropriate external structures, perception
  can perform task/function-level processing.
• Same claim also made by Distributed Cognition.
• Implications AB. Epistemic structures work by
  "overloading" perception. More information.
• Function-specificity Details (A B) Functional
  structure/ information exists in the world, and
  is picked up directly by the organism.
• Gibsons direct pickup!
• A requirement for storing functional info in the
  world?

16

• How are epistemic structures generated?
• Six cases
• Organisms For self -- For self and others --
  Exclusively for others
• Humans For self -- For self and others --
  Exclusively for others
• Constraint on explanation
• Human case treated as an extension of the
  organism case.
• So avoid detailed reasoning processes

17

• Generating structure for self and others --
  Organism case
• Tiredness model
• Phase 1 How a structure is discovered
• Assumption 1 All organisms have a feedback loop
  that monitors their energy level/activity load,
  i.e. they can track their tiredness.
• Assumption 2 All organisms have a preference for
  minimizing energy utilization. Avoiding tiredness
  acts like a reward.
• Step 1 Organisms generate random structure in
  the environment as part of their activity
  pheromones, leaf piles, urine etc.
• Step 2 Some structures generated in the
  environment help organism shorten paths in a task
  environment pheromone trails make trip to
  nest/food faster/easier, leaf piles improve
  foraging, urine smell minimizes cache search.

18

• Step 3 In repetitive tasks, more encounters with
  the structure (shorter paths in the task
  environment) reinforces the use of the
  structure/path. The drop in energy utilisation
  acts as a reward.
• Phase 2 How structure generation is reinforced
• Step 4 More structure generation leads to more
  shorter paths.
• Step 5 This reinforces structure generation.
  Once again, energy conservation acts as a reward.
• Phase 3 How others start using the structure
• Step 6 Conspecifics use the structure because
  they share the same system same reward and same
  reinforcement.
• Similar to the formation of paths in fields. One
  person cuts across the field to minimize effort,
  others use the same path because their systems
  are similar and they also want to minimize
  effort.

19

• For self and others -- Human case
• Scenic detour Bartender study -- 10 novices, 10
  experts
• Task speed drill -- four drink orders presented,
  bartender has to mix the drinks quickly and
  accurately
• First condition count backward from 40 by
  threes, to avoid verbal rehearsal of drink
  orders
• Result Novices made lots of mistakes, experts
  unaffected
• Second condition asked bartenders to use a set
  of identical opaque black glasses for preparing
  the drinks
• Novices unaffected, experts mistakes rose
  17-fold.
• Expert bartenders were using the glasses and
  their states (the environment structure) to store
  states and memory while doing the task.

20

• Moral experts reduce cognitive load by pushing
  out information to the world -- they make the
  world store more information.
• Adding task-specific structure to the world is a
  version of this expert pushing out of
  information
• Happens gradually for complex tasks, usually
  through iterated steps (Eg use of paper
  place-holders for planes by air-traffic
  controllers)
• For simple tasks, cookie cutter structures
  (like stickies, notes etc.)

21

• But how do we manage to generate task-specific
  structures, the right ones that reduce cognitive
  load?
• By generating different structures, and testing
  their efficiency by running them on our system.
  
• This can be done either
• 1)By being situated in the task environment and
  generating real world structures, or
• 2)For more familiar tasks, simulating the task
  environment and potential structures, and testing
  their efficiency by running them in simulation
  in our system.
• Either way, all generated structures need to be
  run on our system to ensure their
  task-specificity/efficiency.
• Others use our structures because they get the
  same efficiency, given similar systems.

22

• No deliberate reasoning process is required for
  the generation of structures. Like a list of
  available resources, a list of possible
  structures, and a mapping.
• Either generate and test for real, or simulate.
• Reasoning processes can be used to generate
  structures -- lengthy process.
• However, the structures generated would still
  need to be tested by running on our system.
• This is because reasoning does not provide an
  estimate of the cognitive load attached to the
  structure.
• Non-reasoning mode is mode parsimonious.

23
Generating structures exclusively for
others Scenic detour 2 Informal study -- to test
how people generated structures exclusively for
others Two scenarios presented to
students Condition 1 You live in a small town.
There is a family from Zambonia (a country far,
far away) in your town, and the head of the
family is celebrating his 75th birthday. Many
Zambonians land up in your town for the
festivities. The Zambonians are from a very
different culture, so they are not familiar with
your language, conventions, public places and
your artifacts. Now 1) Every Zambonian has a
cell phone. And the phones keep ringing wherever
Zambonians go. You want the Zambonians to shut
off the phone when they are in places like
libraries, hospitals, religious places etc. How
can you help them do it?
24

• 2) In Zambonia, you go to a coffee shop and a
  waiter comes to you. You ask for coffee, and you
  get the standard Zambonian coffee, readymade.
  Zambonians smell coffee at your neighborhood
  Starbucks and go in, but they cant figure out
  how to get coffee. How can you help them get
  coffee?
• Most people suggested putting up signs for both
  problems
• Condition 2 -- given to 12 master students in
  engineering
• Problem 1 How can you design a cell phone that
  understands context? The phone should switch from
  ring to vibration mode when the user is in places
  like libraries, hospitals etc. The phone should
  forward all calls to voicemail when the user is
  driving a car. When the user is a passenger, the
  phone should receive calls.
• Same as problem 1 in the Zambonia condition

25

• Problem 2 How can a robot bring a user coffee?
• Issues
• Object recognition How can the robot detect the
  cup, coffee-maker and user from among other
  objects?
• Navigation How can the robot find the locations
  of the cup, the coffee-maker and the user, and
  navigate to each of these locations?
• Action-selection How can the robot decide which
  action to execute on the cup and the
  coffee-maker? And how can it execute those
  actions correctly?
• Same as problem 2 in the Zambonia condition
• For problem 1, only 3 students suggested adding a
  structure to the environment, i.e. a digital
  announcing device.
• Problem 2, only 1 student suggested adding
  digital tags to cups and coffeemakers

26

• Part 2 Fr both problems, partial solutions were
  given to participants. To check the ability to
  generate task-specific structures.
• For problem 1 A policy- announcing device,
  which would be installed in buildings. The device
  would announce to cell phones the policy of the
  building concerning cell phones. What should be
  the content of the message announced by the
  device?
• Only 5 students suggested the message that fitted
  the cell phone's task/function best, i.e. "shut
  up". Others suggested under-specified messages
  like "this is a library", which don't lead to the
  cell phone switching off. Given this message, the
  phone can shut off only after doing some
  inference (what should I do in a library?).
• Task-specificity is in comparison with signs
  asking humans to switch off cell phones

27

• For problem 2, we told students that they could
  attach RFID tags on the coffee cups and
  coffeemaker. What would they put into the tags?
• Once again, only one person suggested using
  action-directed tags, for instance
  supported_actions (hold, grasp) constraints
  (this_side_up, put_cup_here) etc.
• Task-specificity in comparison with equivalent
  structures for humans

28

• Why this difference?
• Another scenic detour -- counterfactual reasoning
• Generation of alternatives to reality
• Kahneman experiment Subjects given a story
  describing a fatal road accident, where a truck
  driven by a drug-crazed teenager crashes into a
  passing car, killing Mr. Jones.
• Task Jones family and family often thought and
  often said If only. How did they continue the
  thought?
• Two conditions In one, Mr. Jones chose to take a
  different route from normal, in the other
  condition, Mr. Jones left office earlier than
  usual. Both exceptions to norm.
• Result 80 of participants altered the
  exceptional value and made it normal.
• Some features of reality are more mutable than
  others

29

• How is this relevant here?
• 1)Creation of epistemic structure requires
  counterfactual thinking, generate alternatives
  to reality.
• In counterfactual thinking, even though there are
  many alternatives, people tend to readily choose
  some more available ones.
• Reality is bent along its elbows (Kahneman).
• Reality may have elbows while solving design
  problems as well.
• 2) Kahneman and Tversky propose that people use
  the simulation heuristic to generate these
  counterfactual scenarios.
• They simulate the given situation and mutate some
  aspects of it to generate alternatives, to
  undo the situation
• Generating epistemic structure requires
  simulation, the mechanics of simulation may
  explain the difference in performance for
  artifacts.

30

• Hypothesis Epistemic structure for others
  generated by simulating the other agent and its
  task environment. The reason the engineering
  students do not readily add structures to the
  environment for the artifact condition (robots
  and cell phones) is because they do not simulate
  the artifacts, so the environment is not
  available to them.
• Pilot study -- Three problems, 5 conditions
• The Problems Cell phone, coffee, laundry
• The Conditions Varied the cognitive distance
  between the participant and the agent involved in
  the problem
• 1) People from other cultures who can only read
  English 2)They cannot even read English, 3) They
  have no English and they are blind, 4) They are
  Martians 5) They are artifacts (cell phones,
  robots)

31

• Hypothesis If epistemic structure generated
  decreases systematically as the cognitive
  distance between participant and the task-agent
  increases, then participants are simulating the
  task-agent.
• 5 participants to each condition, each
  participant got 3 problems involving same
  task-agent
• Participants had to think aloud while solving the
  problems, and this output was taped. Participants
  also had to write down their solutions.

32

• Part 2 All participants given a partial
  solution, and asked to complete it.
• Solution There is a special electronic tag that
  you can stick to objects (like a stickie). You
  can inscribe whatever you want inside the tag in
  English, and people wearing a special earphone
  can hear this inscription when they come near the
  tags, or touch the tags. Moreover, this
  inscription will be translated into whatever
  language you wish, so Zhanjovians can hear your
  inscription in their language, Dharlaquans in
  their language, and Jharawajans in their
  language.
• The artifact condition had RFID tags for the
  robots and a digital device for the cell phone .
• Task 1 On which objects/places will you stick
  these tags? Mention the objects/places for each
  of the three problems.
• Task 2 What would you inscribe into the tags you
  put on these objects/places? Mention the tag
  contents for each of the 3 problems.

33

• Participants also answered a questionnaire, where
  they were specifically asked whether they tried
  to think in the agents shoes.
• Participant responses analyzed to extract four
  variables
• 1)Were epistemic structures suggested?
• 2)Were the messages and objects/places suggested
  in part 2 function-specific?
• 3)Did they simulate?
• 4)How many solutions were suggested?
• Variables plotted against the conditions

34
Good news performance drops!
35

• Bad news
• 1)Complexity effect epistemic structures
  suggested more for cell phone problem. Task
  specificity of structures also high for cell
  phone problem. For the more complex problems
  (coffee and laundry) performance drops for both
  structure creation and task-specificity.
• 2) Participants claim to be simulating in all
  conditions except the last. Even there, some of
  them claim to simulate the robots
• But the structure creation drops significantly
  for both martians and robots
• Simulation is only a necessary condition to
  generate epistemic structure, not a sufficient
  condition.

36

• Problems
• 3) For structure generation, worst performance in
  the blind condition.
• Without vision, epistemic structure generation
  difficult. Major jump in continuum?
• Drop blind condition?
• 4) Function specificity blip Computer
  scientists!
• 5) Structure generation similar for both martians
  and robots. Because cognitive capacities are
  vague?
• 6) Still no explanation for the human-artifact
  difference.

37

• Application relevance humans are bad at
  generating task-specific structures for artifacts
  and agents not similar to themselves.
• RFID-based robotics and Assisted Cognition
  projects need to take this into account.
• Current work
• More subjects, tighter protocol, condition
  involving memory-impaired task-agent.
• Would making task-specificity and structure
  creation explicit make performance better?
  Experiment 2
• How effective is epistemic structure in a dynamic
  environment? Robocup experiment.
• Terrys ant.

38
Questions? Comments?