Ken Goldberg

1
Collaborative Teleoperation

• Ken Goldberg
• IEOR and EECS, UC Berkeley

2
Students and Colleagues Dezhen Song Frank van
der Stappen Vladlen Koltun Sariel Har-Peled Gopal
Gopalkrishnan Ron Alterovitz In Yong Song Judith
Donath David Pescovitz Eric Paulos  
3
Geometric Algorithms for Manufacturing
4
(No Transcript)
5
Outline

• Collaborative Teleoperation
• Cinematrix
• Co-opticon
• Tele-Twister

6
(No Transcript)
7
Nikola Tesla (1898)
8
Telerobotics Related Work

• Tesla, 1898
• Goertz, 54
• Mosher, 64
• Tomovic, 69
• Salisbury,Bejczy, 85
• Ballard, 86
• Sheridan, 92
• Sato, 94
• Presence Journal 92-
• O. Khatib, et al. 96

9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Collaborative Control
15

Taxonomy (Tanie, Matsuhira, Chong 00)
Single Operator, Single Robot (SOSR)
Single Operator, Multiple Robot (MOSR)
Multiple Operator, Single Robot (MOSR)
16
Outline

• Collaborative Teleoperation
• Cinematrix
• Co-opticon
• Tele-Twister

17
Cinematrix audience participation system R.
and L. Carpenter (1992)
18
A model of Cinematrix
Cursor on Shared Screen
Audience (2 groups)
dx

dy
(x,y)
19
Cinematrix Simulator
20
Ideal Response
x x fx(x), y fy(x) T
where x x, y T f Q(x,y) x sgn(g) ? -1,
0, 1 g(x,y) x2 y2 - r2
21
Performance Metric
Error ? local error / total area Performance
1 - Error
22

• Performance with Audience Diversity
• Ideal Players
• Drop outs
• Malicious Players
• Random Players
• Time Delayed Players

23
drop-outs,
Performance
100
63
0
50
100
0
malicious agents
24
random agents
Performance
100
63
0
50
100
0
25
Time delay (cycles)
Time Delayed Agents
26
Outline

• Collaborative Teleoperation
• Cinematrix
• Co-opticon
• Tele-Twister

27
n users
1 pan, tilt, zoom robotic camera
co-opticon
28
Example input 7 requested frames
29
One Optimal Frame
Co-opticon Problem Given n requests, find
optimal frame
30
Related Work

• Facilities Location Problems
• Megiddo and Supowit 84
• Eppstein 97
• Halperin et al. 02
• Rectangle Fitting
• Grossi and Italiano 99,00
• Agarwal and Erickson 99
• Mount et al 96
• Kapelio et al 95

31
Related Work

• Similarity Measures
• Kavraki 98
• Broder et al 98, 00
• Veltkamp and Hagedoorn 00
• Distributed robot algorithms
• Sagawa et al 01, Safaric01
• Parker02, Bulter et al. 01
• Mumolo et al 00, Hayes et al 01
• Agassounon et al 01, Chen 99

32
Problem Definition

• Requested frames ?ixi, yi, zi, i1,,n

33
Problem Definition

• Assumptions
• Camera has fixed aspect ratio 4 x 3
• Candidate frame ? x, y, z t
• (x, y) ? R2 (continuous set)
• z ? Z (discrete set)

4z
34
Problem Definition

• Satisfaction for user i 0 ? Si ? 1

? ? ? ?i
? ?i
Si 0
Si 1
35

Similarity Metrics

• Symmetric Difference
• Intersection-Over-Union

Nonlinear functions of (x,y)
36
Satisfaction Metrics

• Intersection over Maximum

Requested frame ?i , Area ai
Candidate frame ? Area a
pi
37
Intersection over Maximum si(? ,?i)
Requested frame ?i Candidate frame ?
si 0.20 0.21 0.53
38
(for fixed z)
Requested frame ?i
Candidate frame ?(x,y)
39

• Satisfaction Function
• si(x,y) is a plateau
• One top plane
• Four side planes
• Quadratic surfaces at corners
• Critical boundaries 4 horizontal, 4 vertical

40
Objective Function

• Global Satisfaction

for fixed z
ShareCam problem Find ? arg max S(?)
41
Properties of Global Satisfaction

• S(x,y) is non-differentiable, non-convex, but
• piecewise linear along axis-parallel lines.

42
ShareCam Algorithms

• Bruteforce Algorithm
• Compute S at each pixel (x,y)
• O(whmn)
• w, h width and height of panoramic image
• m number of zoom levels
• n users

43
Approximation Algorithm
Compute S(x,y) at lattice of sample points
d
44
Approximation Algorithm
? Optimal frame
Smallest frame at lattice that encloses ?
Optimal at lattice

• Run Time
• O(w h m n / d2)

45
Exact Algorithm

• Virtual corner Intersection between boundaries
• Self intersection
• Frame intersection

y
46
Exact Algorithm

• Claim An optimal point occurs at a virtual
  corner.
• Proof
• Along vertical boundary, S(y) is a 1D piecewise
  linear function extrema must occur at boundaries

47
Exact Algorithm

• Exact Algorithm
• Check all virtual corners
• ?(mn2) virtual corners
• ?(n) time to evaluate S for each
• ?(mn3) total runtime

48
Improved Exact Algorithm

• Sweep horizontally solve at each vertical
• Sort critical points along y axis O(n log n)
• 1D problem at each vertical boundary O(nm)
• O(n) 1D problems
• O(mn2) total runtime

O(n) 1D problems
49
Distributed Algorithm

• More users ? More computers available

50
Distributed Algorithm

• At the Server
• Sort horiz. boundaries
• O(n log n)
• At the Client
• Solve 1D problem
• for own
• vertical boundaries.
• O(nm)
• O(n(m log n)) Total

Four 1D problems
51
Examples
52
Examples
53
(No Transcript)
54
www.co-opticon.net
55
Future Work

• Continuous zoom (m?)
• Multiple outputs
• p cameras
• p views from one camera
• Temporal version fairness
• Integrate si over time minimize accumulated
  dissatisfaction for any user
• Network / Client Variability load balancing
• Obstacle Avoidance

56
Outline

• Collaborative Teleoperation
• Cinematrix
• Co-opticon
• Tele-Twister

57
robot
participants
remote environment
58
tele-actor
participants
remote environment
59
Spatial Dynamic Voting
60
Query Types
Navigational Yes/No Binary

61
Genetics Laboratory (LBL) November 2002
14 seniors from Galileo High, SF
62
(No Transcript)
63
(No Transcript)
64
Spatial Dynamic Voting
votel voting element v(x,y,t)
65
Automated Scoring (measuring user performance)

• Rewarding
• Attentiveness
• Engagement
• Responsiveness
• Collaboration
• Leadership
• How well you lead
• How well others follow
• A metric based on
• Votel position (x,y)
• Votel arrival time t

66
(No Transcript)
67
Majority cluster
68
Leadership metric
Defined in terms of membership in the majority
cluster, arrival time in that cluster, and
weighted sum of previous leadership scores (with
exponential decay).

where
69
Algorithm
Model each votel as a truncated spatial gaussian
distribution
70
Algorithm
Spatially sum all gaussian distributions.
71
Algorithm
Regions that intersect iso-density plane
define clusters
p 0.1
Grid Approximation 160 x 160 grid takes about
10 ms
72
(No Transcript)
73
tele-jenga, july 2003
74
(No Transcript)
75
(No Transcript)
76
(No Transcript)
77
(No Transcript)
78
(No Transcript)
79
(Audio)
80
(No Transcript)
81

• Measuring effect of
• hiding scores and/or other votels
• changing frame rate

82
Google tele-twister Live games Selected
Fridays, 12-1pm Pacific Time
83
Summary

• Collaborative Telepresence
• Cinematrix
• Co-opticon
• Tele-Twister
• goldberg_at_ieor.berkeley.edu

84
(No Transcript)
85
(No Transcript)