Humanoid robots help autistic children

1

• Humanoid robots help autistic children

2
Outline

• Motivation of the creators
• Autistic disorders
• A survey of the research
• Why robots might help
• The field of research
• Conclusions

3
Motivation

• Research in Human-Robot Interaction
• Looking for a killer application
• advertisement, receptionists, multi-media kiosks,
• theatre and installations,
• help elderly and disabled adults,
• normal child supervision
• toys
• Better How can we use robots to help people?

4
Autistic Disorders

• 1 of 300 children diagnosed with autism with
  rates rising
• To compare with other ilnesses
• 1 of 800 children diagnosed with Down syndrome
• 1 of 450 children diagnosed with juvenile
  diabetes
• 1 of 333 children will develop cancer by age 20
• Diagnosis currently made through behavioral
  observation
• no blood test or genetic screening is available
• there is evidence of a genetic link, so may be
  tests will arrive

5
Autistic Disorders What are their
Characteristics ?

• Inability to relate to other people
• Little use of eye contact with other people
• Difficulty understanding gestures and facial
  expressions
• Difficulties with verbal non-verbal
  communication
• Difficulty understanding others intentions,
  feelings, and mental states

6
Why Use Robots for children with autism?

• Most children, including children with autism,
  are attracted to robots.
• This natural affinity is exploited, and the robot
  is used as an interactive toy.
• Robots may provide a less threatening environment
  than interacting with people.
• Robots can provide a repetitive and more
  predictable environment.
• This safe environment can gently push a child
  with autism towards human interaction.

Here tell about my personal experiences with
autistic kids in our lab and the president of
Intel story
7
There is a connection of autism to imitation

• One theory Autism may be caused by early
  impairments in imitation and shared attention
  (Rogers Pennington, 1991) (Baron-Cohen, 1995)
• Imitation is a format of communication, a means
  to express interest and engage others in
  interaction (Nadel, 1999)
• Idea Use a doll-like robot to engage children
  with autism and teach basic imitative interaction
  skills
• From K. Dautenhahn, and A. Billard, Games
  Children with Autism Can Play With Robota, a
  Humanoid Robotic Doll, Proc. 1st Cambridge
  Workshop on Universal Access and Assistive
  Technology, 2002

8
Development of Robota robots for autistic
children
1. Research from Switzerland, Swiss Federal
Institute of Technology
A six-year old autistic boy playing with Robota.
He seemed curious about Robota's head movements
and so he touches the doll. From K. Dautenhahn,
and A. Billard, Games Children with Autism Can
Play With Robota, a Humanoid Robotic Doll, Proc.
1st Cambridge Workshop on Universal Access and
Assistive Technology, 2002
9
Imitation Using Robota

• Robota allows the child to understand that
  the dolls movement originates from his own
  movement (sense of agency)
• It helps to understand that the dolls movement
  is limited to a restricted category of movement
  (enhances intentional action)
• From J. Nadel, Early Imitation and a Sense of
  Agency, Proc. 4th Intl. Workshop on Epigenetic
  Robots, 2004

10
An autistic child playing chasing games with
the mobile robot From K. Dautenhahn, and A.
Billard, Games Children with Autism Can Play With
Robota, a Humanoid Robotic Doll, Proc. 1st
Cambridge Workshop on Universal Access and
Assistive Technology, 2002
11
Joint Attention Using Robota
Robota is controlled via teleoperation by the
investigator.
From B. Robins, P. Dickerson, and K. Dautenhahn,
Robots as Embodied Beings Interactionally
Sensitive Body Movements In Interactions Among
Autistic Children and a Robot, Proc. RO-MAN 2005
12
Investigator remotely encourages interaction and
immitation by children
Two autistic children Note Andys gaze at Jack.
The investigator encourages the children to show
each other how they can interact with the
robot. The robot will not move unless the
children show the same movement, i.e., they must
work together.
13
Andy and Jack touch each other to balance
themselves while each raising a leg.
14
Adam shows no interest in his classmates and
usually tries to avoid the rest of the children.
But Adam is interested in Robota.
Adam takes Robs hand to show him how to interact
with Robota.
15
The new prototype of Robota

• The current prototype of the doll-shaped humanoid
  robot Robota is presented here.
• The use of the robot Robota as part of studies
  with disabled children sets a number of
  constraints on its design.
• In particular, it requires that the robot bears a
  human likeness both in its body features and in
  the kinematics of its motions.
• The current design consists in a 23 degrees of
  freedom upper body, including a 3 DOFs spine, two
  7 DOFs arm, a 3 DOFs pair of eyes and a 3 DOFs
  neck.

16
The prototype of arm

• The current prototype of arm is a 6 DOFs arm with
  a 1 DOF gripper. It is 26 centimeter long for
  700gr.
• The motors were dimensioned so as to carry an
  external load of up to 200 gr.
• The different DOFs were designed to be
  reversible, in order to provide an interface for
  teaching the robot by demonstration. The first
  three degrees of freedom are placed in the
  shoulder. The three rotation axes cross at the
  same point.
• This implies that the elbow moves on a sphere. We
  have one DOF in the elbow and two DOFs in the
  wrist.
• The gripper is composed of 3 fingers actuated by
  one single DOF. To have an absolute measure of
  the position of each joint, we have placed
  potentiometers on each axis.
• We can, thus, measure the absolute position of
  the arm when the arm is switched on, and, hence,
  initialize the motor encoders without having to
  send the motors to the reset position.
• This ensures minimal risks when the robot
  interacts with children, by preventing any
  involuntary motion if the robot should reset
  itself.

17
(No Transcript)
18
(No Transcript)
19
The prototype of eyes

• A prototype of a 3 DOFs pair of eyes has been
  developped.
• One DOF drives the horizontal rotation of the two
  eyes and the two other DOFs drive the vertical
  rotation of each eye.
• Thus, the robot can wink but not squint!
• In each eye, we have placed one ''mobile phone''
  CMOS camera.
• The principal constraint in this project is the
  aesthetic of the robot.
• We use then real doll eyes that we modify to
  insert the cameras, by drilling a tiny hole
  through the pupil, making sure that the iris
  remains intact.
• The volume of the eyes is, however, too small to
  contain the electronic board (that proceeds to
  the digital conversion of the image).
• Thus, the sensor must be connected to the board
  through a flex cable in such a way that the cable
  is not impeding the movement of the eyes.

20
(No Transcript)
21
The prototype of neck

• For the prototype of Robota's neck, we have 3
  DOFs (lace, pitch and roll).
• These are placed in series, starting with lace,
  and, then pitch and roll.
• The system has been designed to support a load of
  400gr, so that it could still drive the head of
  the robot in any position.
• Pitch and Roll are controlled by transmission
  through a set of cables and pulleys, while Lace
  is controlled by a direct drive from the motor.
• Using direct drive and cable transmission
  minimizes the chance of encountering backlash
  problems.

22
(No Transcript)
23
The prototype of spine

• The current prototype of spine drives two DOFs
  for front-back and left-right bending
  respectively.
• The third DOF of the torso, supported by the
  spine, drives the horizontal rotation of the
  shoulders.
• The spine is about 200mm high for a diameter of
  90mm.
• It weights about 1Kg and supports a load of 2Kg
  located at 80mm on the spine.
• This corresponds to the estimated mass of the
  current prototypes and position of the mass
  center of the two arms and the head. To obtain
  a smooth curvature along the spine, we have used
  a low pressure hydraulic system.
• Four pistons are placed at each level of the
  spine (there are four levels), two for each DOF.
• To move the spine, we have two motors placed in
  its base.
• Using a reduction gear, each motor transmits the
  movement to an endless screw that moves two
  pistons working as a pump.

24
(No Transcript)
25
                                                                                                                             
                                                                                                                                                           
                                                                                                                                      
26
                                                                                                                                                                               
                                                                                                                                                             
                                                                                                                                             
27
(No Transcript)
28
Interacting with Keepon
2. Research from Japan
Keepon is controlled via teleoperation.
From H. Kozima, C. Nakagawa, and Y. Yasuda,
Interactive Robots for Communication-Care A
Case Study in Autism Therapy, Proc. RO-MAN 2005
29
Keepon is a robot that sees you and tracks you
Views from Keepons camera eyes
30
Attentive action
Emotive action

• Keepon's kinematic mechanism.
• Two gimbals are connected by four wires the
  lower gimbal is driven by two motors.
• Another motor rotates the whole inner-structure
  
• Yet another drives the skull downward for
  bobbing.

31
Enabling Interaction
Joint attention Sharing the perceptual
information
Eye-contact Referring to each other's mental
states
Enables people to exchange intention and emotion
toward a target.
32
Dyadic and triadic interactions
child
33
Scassellati Using Robots for Autism Diagnosis
3. Yale University
ESRA
Playtest
From B. Scassellati, Quantitative Metrics of
Social Response for Autism Diagnosis, Proc.
RO-MAN 2005
34
Autism Diagnosis Methods

• 1. Reaction to the ESRA robot with and without
  the face configuration

Can generate facial expressions using 5 servo
motors
Two motors added for horizontal eye movement
35
Autism Diagnosis Methods

• 2. Measure listening preferences to speech sounds

36
Autism Diagnosis Methods

• Vocal prosody, i.e., how something is said

Features F24 vs. F1 Mean pitch energy vs. mean
pitch
Separation of two features used in a Bayesian
classifier distinguishes low energy categories
(neutral and soothing) from high energy
categories (approval, attention, and prohibition).
37
Autism Diagnosis Methods

• 3. Position tracking relative to another person

38
Autism Diagnosis Methods

• 4. Gaze direction and focus of attention

Red adolescents with autism Blue typical
adolescents
39

• Linear discriminant analysis of autistic (au)
  and typical (nc) gaze patterns.
• Linear filters F(x) are trained to reproduce the
  gaze pattern G(x) of each individual x
• and then applied to predict the gaze patterns of
  any other individual.

• For example, F(au)G(self) indicates a filter
  trained on an individual with autism and tested
  on that same individual while F(nc)G(au)
  indicates a filter trained on a control
  individual and tested on an individual with
  autism.
• The mean performance of this data (y-axis) is a
  function of the response percentile of individual
  pairings.
• Significant differences (all plt0.01 for a
  two-tailed t-test) are seen between the following
  classes (1) F(nc)G(self), (2) F(au)G(self),
  (3) F(nc) G(other nc), and (4) the three other
  conditions.

40
University of Sherbrooke Project for students
with double meaning
4. University of Sherbrooke

• Project for engineering students
• Design a robotic toy for an autistic child
• Educational value
• Real world problem
• Students work together in a team
• Students must first investigate autistic disorders

41
University of Sherbrooke student projects for
autistic children
Pushing Jumbo around the play area.
Rolling game with Roball.
From Michaud, F., Théberge-Turmel, C. (2002),
"Mobile robotic toys and autism", Socially
Intelligent Agents - Creating Relationships with
Computers and Robots, Kluwer, pp. 125-132.
42
University of Sherbrooke
Autistic kids do some actual robot assembly
Girl showing signs of interest toward Bobus.
Assembling the arms and tail of C-Pac.
From Michaud, F., Théberge-Turmel, C. (2002),
"Mobile robotic toys and autism", Socially
Intelligent Agents - Creating Relationships with
Computers and Robots, Kluwer, pp. 125-132.
43
Kids design big stuffed robots
44
Other robots of this kind - the Tiger Kitty
5. The iCat by Philips Research
45
The Field of Researchers

• Francois Michaud
• University of Sherbrooke, Canada
• Kerstin Dautenhahn Ben Robbins
• University of Hertfordshire, UK
• Aude Billard
• Swiss Federal Institute of Technology (EPFL)
• Jacqueline Nadel
• French National Centre of Scientific Research

46
The Field of Researchers

• Brian Scassellati and Bob Schultz
• Yale University
• Javier Movellan
• University of California San Diego
• Hideki Kozima
• National Institute of ICT, Japan
• Michio Okada
• ATR, Kyoto, Japan

47
Conclusions

• The use of robots for autism therapy and
  diagnosis is just beginning.
• There is anecdotal evidence that robot therapy
  can help children with autism
• Can we do something with Robot Theatre?

48
Dolls, monsters, or may be something new?

• I believe that Halloween robot technology can be
  reused to build robot theatre with robots of
  natural size.
• Inexpensive, after Halloween.

49
More examples of Halloween robots
50
We changed Crazy Scientist to Professor Niels
Bohr for Teenage Robot Theatre
51
(No Transcript)
52
(No Transcript)
53
We converted Santa to Paekchong robot for Hahoe
Theatre
54
Sources of slides and ideas

• Brian Scassellati, Francois Michaud, Ben Robins,
  and Hideki Kozima Marjorie Skubic