Yanfei Liu

1
Dynamic Workcell for Industrial Robots

• Yanfei Liu

Dept. of Engineering, IPFW Fort Wayne, IN
This work is done in Clemson University, SC
(07/2000-06/2005)
2
Outline

• Motivation for this research
• Current status of vision in industrial workcells
• A novel industrial workcell with continuous
  visual guidance
• Work that has been done
• Our prototype camera network based industrial
  workcell
• A new generic timing model for vision-based
  robotic systems
• Dynamic intercept and manipulation of objects
  under semi-structured motion
• Grasping research using a novel flexible
  pneumatic end-effector

3
Motivation for this research

• Current industrial workcells
• No vision or a single snapshot in certain
  locations

• Disadvantages
• Cannot deal with flexible parts
• Cannot deal with uncertainty

4
Motivation for this research

• Our novel dynamic workcell design
• Manipulation is integrated with visual sensing
• Applications ( reduce fixtures, handle objects on
  the ship)

5
System architecture

• A set of cameras embedded into the workcell
• An industrial manipulator with its conventional
  controller

6
Experimental platform

• Our prototype
• Staubli RX130 manipulator with its conventional
  controller
• Six cameras, wired to two PC-RGB framegrabbers
  mounted in a Compaq Proliant 8500 computer
• V Operating systems and language
• Alter command to accomplish real time motion

7
Tracking experiments
8
First part A new generic timing model for
vision-based robotic system
9
Introduction

• Classical visual servoing structure
• eye-in-hand systems
• Corke (1996), an eye-in-hand manipulator to
  fixate on a thrown ping-pong ball
• Gangloff (2002), a 6-DOF manipulator to follow
  unknown but structured 3-D profiles.
• part-in-hand systems
• Stanvnitzky (2000), align a metal part with
  another fixed part
• mobile robot systems
• Kim (2000), a mobile robot system to play soccer

10
Introduction

• Vision guided control structure
• Allen (1993), a PUMA-560 tracking and grasping a
  moving model train which moved around a circular
  railway.
• Nakai (1998), a robot system to play volleyball
  with human beings.
• Miyazaki (2002), a robot accomplished a ping pong
  task based on virtual targets

11
Introduction

• Three common problems in visual systems
• Maximum possible rate for complex visual sensing
  and processing is much slower than the minimum
  required rate for mechanical control.
• Complex visual processing introduces a
  significant lag (processing lag) between when
  reality is sensed and when the result from
  processing a measurement of the object state is
  available.
• A lag (motion lag) is produced when the
  mechanical system takes time to complete the
  desired motion.

12
Previous work

• the first two of the three problems have been
  addressed to some extent in previous works. All
  of these works neglect the motion time (motion
  lag) of the robot.
• Corke and Kim, presented timing diagram to
  describe time delay, used discrete time models to
  model the systems and simplified these
  asynchronous systems to single-rate systems.

13
Timing Model notation
14
Timing Model our prototype

• Inherent values (obtained by analysis/measurement)
  
• ?s 33ms ?u 193014 63ms
• ?wm 39ms ?wf (51627)/3 16ms ?w
  3916 55ms
• ?l ?s ?u ?w 151ms ?f 130ms
• User-variable values
• ?c 4ms ?q 40ms

15
Experiments

• Problem description
• The most recently measured position and velocity
  of the object is where the object was (?l?k) ms
  before, xt-?l- ?k, vt-?l- ?k
• The current position, xt
• N, ?d?

16
Experiments

• Solutions

Constraint
17
Experiments model validation

• Setup
• A small cylindrical object is dragged by a string
  tied to a belt moving at a constant velocity.
• The robot will lunge and cover the object on the
  table with a modified end-effector, a small
  plastic bowl.

18
Experiments (video)
19
Experiments

• Experiment description
• We set ?q to two different values, 40 and 80, in
  these two sets of experiments. We let the object
  move at three different velocities. For each
  velocity, we ran the experiment ten times.
• Results

20
Experiments (video)
21
Second part Dynamic Intercept and Manipulation
of Objects under Semi-Structured Motion
22
Scooping balls (video)
23
Scooping balls problem description
robot
xt , yt object position at time t vx , vy
object velocity at time t xr , yr initial robot
position xf , yf final impact position
x
Unknown variables yf , ?i
y
Open loop
Closed loop
Start tracking Make prediction (t)
Impact (t?i)
24
Scooping balls solution

• Solutions

• Object unsensed time
• Time between the last instant when reality is
  sensed and the final impact time
• Delay between visual sensing and manipulation

25
Timeline description object unsensed time
?t
processing lag(?l) ?k
synchronizing tracking
?q
?q

controlling
m
20
motion lag (?f)
finishing motion
closed loop
open loop
?t ?l ?k 4m ?f
m lt N 10, ?k lt 30 14 44ms
?t 151 ( 40 44 ) / 2 115 308ms
26
Impact point
z
y
27
Equations

• Solutions

• Implementation
• Predict the maximum acceleration of the object
  motion that the robot still can achieve a
  successful catch
• Calculate the size of the end-effector in order
  to overcome the maximum acceleration of the
  moving objects

28
Experimental Validation

• Setup
• Two types of end-effector (bowl, two scoopers
  with different width).
• Three types of interference (wind, bump, ramp)
• Results
• With wind interference

29
Experimental Validation

• with bump interference, weighted corner
• with bump interference, balanced

30
Experimental Validation

• with ramp interference, weighted corner
• with ramp interference, balanced

31
Third part A Novel Pneumatic Three-finger
Robot Hand
32
Related work

• Three different types of robot hands
• Electric motor powered hands, for example
• A. Ramos et. al. Goldfinger
• C. Lovchik et. al. The robonaut hand
• J. Butterfa? et. al. DLR-Hand
• Barrett hand
• Pneumatically driven hands
• S. Jacobsen et. al. UTAH/M.I.T. hand
• Hydraulically driven hands
• D. Schmidt et. al. Hydraulically actuated finger
• Vision-based robot hand research
• A. Morales et. al. presented a vision-based
  strategy for computing three-finger grasp on
  unknown planar objects
• A. Hauck et. al. Determine 3D grasps on unknown,
  non-polyhedral objects using a parallel jaw
  gripper

33
Novel pneumatic hand

• Disadvantages of current robot hands
• Most robot hands are heavy
• Even with visual guidance, the robot hand can
  only grasp stationary objects

• Novel hand architecture
• build-in pneumatic line in Staubli RX130
• Paper tube, music steel wire embedded inside
• Camera mount adjusting finger spread angle
• 120 degrees between each other

34
Novel pneumatic hand

• Close position Open position

• Our research here is to demonstrate that we use a
  novel idea to built a flexible end effector and
  it can grasp semi-randomly moving objects. This
  is not a new type of complex research tool-type
  robot hands.

35
Grasping research

• Problem statement

36
Grasping research

• Position prediction
• Same as the method in the second part work of
  this research
• Orientation adjustment
• Line fitting to get the final roll angle
• equations

37
Grasping experiments (video)
38
Conclusions timing model

• A generic timing model for a robotic system using
  visual sensing, where the camera provides the
  desired position to the robot controller.
• We demonstrate how to obtain the values of the
  parameters in the model, using our camera network
  workcell as an example.
• Implementation to let our industrial manipulator
  intercept a moving object.
• Experimental results indicate that our model is
  highly effective, and generalizable.

39
Conclusions dynamic manipulation

• Based on the timing model, we present a novel
  generic and simple theory to quantify the dynamic
  intercept ability of vision based robotic
  systems.
• We validate the theory by designing 15 sets of
  experiments (1050 runs), using two different end
  effectors under three different interference.
• The experimental results demonstrate that our
  theory is effective.

40
Conclusions novel pneumatic hand

• A novel pneumatic three-finger hand is designed
  and demonstrated.
• It is simple, light and effective.
• Experimental results demonstrate that this novel
  pneumatic hand can grasp semi-randomly moving
  objects.
• Advantages
• The compliance from pneumatics will allow the
  three-finger hand to manipulate more delicate and
  fragile objects.
• In the experiments of grasping moving objects,
  unlike the traditional gripper, the contact
  position for this continuous finger is not very
  critical, which leaves more room for sensing
  error.

41
Sponsors

• The South Carolina Commission on Higher Education
• The Staubli Corporation
• The U.S. Office of Naval Research

42
Thanks
43
Conclusions different manipulations
44
ramp interference
bump interference
The distribution of vi vavg in the balance
ramp and bump cases.
45
Determining the Values

• An external camera to observe operation
• A conveyor moving in a fixed path at a constant
  velocity
• A light bulb as a tracking object
• A laser mounted in the end effector of the robot