Asking Questions and Developing Trust

1
Asking Questions and Developing Trust

• Stephanie Rosenthal
• Joint Work with Anind K. Dey and Manuela Veloso
• Carnegie Mellon University

2
Overview
Questions
Agent/ Robot
Human(s)
Responses
3
Agents Asking Questions

• Agent should explicitly share its state
  information
• context, prediction, uncertainty, features
• Goal Vary which information the agent shares to
  maximize accuracy

Robot
Email Sorter
4
Questions - Robot Example

• Robot State
• Context - image of the structure, color, position
• Prediction - shape of block
• Uncertainty - probability the prediction is wrong
• Features - context that defines the block

5
Questions - Robot Example

• Robot State
• Context - image of the structure, color, position
• Prediction - shape of block
• Uncertainty - prob. prediction is wrong
• Features - context that defines the block
• Example Question
• Cannot determine the block shape. You are working
  with the red and purple blocks. What shape is the
  red block? I think it is a triangular prism.

6
Questions - Robot Example

• Robot State
• Context - image of the structure, color, position
• Prediction - shape of block
• Uncertainty - prob. prediction is wrong
• Features - context that defines the block
• Example Questions
• You are working with the red and purple blocks.
  What shape is the red block? What features define
  that shape?

7
Findings
Robot
Email
8
Overview
Agents can manipulate their questions to maximize
the accuracy of human responses
Questions
Agent/ Robot
Human(s)
Responses
9
Recommender System
N products p1 p2 pN
R reviews
r1 r2

rM
Rij
M reviewers
c1 c2 cK Categories

• Goal Provide personal predictions for each user

10
Advice Givers

• Domain-Independent
• Initialize wiu 1/M
• For all products pj that user u requests
  predictions for
• Make prediction
• argmaxv I(Rijv)?wu
• If user gives opinion oj
• Update weights wu

• Domain-Specific
• Initialize wiu,d 1/M
• For all products pj that user u requests
  predictions for
• d domain of pj
• Make prediction
• argmaxv I(Rijv)?wu,d
• If user gives opinion oj
• Update weights wu,d

Good Reduces sparsity Bad Single set of weights
Good Category-based weights Bad Less data in
each category
11
Which Advice Giver is Better?

• Tradeoff between data and precision is not
  uniform across users
• User-dependent selection algorithm to decide
  which advice giver is best for each user

DI
DS
Selection
12
Summary
Agents can manipulate their questions to maximize
the accuracy of human responses
Questions
Agent/ Robot
Human(s)
Genre and frequency of questions affects the way
that the agent should develop trust with reviewers
Responses
13
Future Work
Agents can manipulate their questions to maximize
the accuracy of human responses
Questions
Agent/ Robot
Human(s)
Genre and frequency of questions affects the way
that the agent should develop trust with reviewers
Responses
with Mike Licitra, Nick Armstrong-Crews, Joydeep
Biswas
14
Questions?
15
(No Transcript)
16
Recommender System

• Advice giver weighs each reviewer for each user
  wiu
• For all users, initialize wiu 1/M
• When user u provides an actual opinion oj about a
  product pj, update all weights wu
• wiu e(ln(wiu) - Rij - oj)/K
• Advice giver predicts value v for product pj and
  user u
• argmaxv I(Rijv)?wu

17
Advice Giver Algorithm

• Initialize wiu 1/M
• For all products pj that user u requests
  predictions for
• Make prediction argmaxv I(Rijv)?wu
• If user gives opinion oj
• Update weights wu

18
Category-Dependent Advice Giver Algorithm

• Initialize wiu,k 1/M
• For all products pj that user u requests
  predictions for
• k category of pj
• Make prediction argmaxv I(Rijv)?wu,k
• If user gives opinion oj
• Update weights wu,k only

19
Tradeoffs

• Category-Independent Advice Giver
• More data to evaluate the weights of each
  reviewer, coarser trust model
• Category-Dependent Advice Giver
• More fine-grained evaluation of which reviewers
  to trust, less data per category
• Amazon.com, Netflix.com, Yahoo! Music
• V 1,2,3,4,5, M gt 100K, N gt 50 per user
• C 10 per dataset, 20 test users per dataset

20
Overview

• Asking Questions of Novice Users
• Developing Trust in Large Sets of Online Users
• CoBot the Visitor Companion Robot

21
Asking Questions

• Agent should explicitly share its state
  information
• context, prediction, uncertainty, features
• Goal Vary which information the agent shares to
  maximize accuracy

22
Developing Trust in Humans

• Case Study - Recommender Systems

23
CoBot, Visitor Companion

• Escort a human visitor to their meetings
• Navigate indoor environments
• Share information relevant to the meetings
• Ask questions when it cannot perform a task