Experimental Robot Musicians

1
Experimental Robot Musicians

• Tarek Sobh, Kurt Coble and Bei Wang
• School of Engineering
• University of Bridgeport

2
Table of Content

• Introduction
• Motivation music expressiveness
• Robot Musician Architecture
• Band overview
• Architecture overview
• Detailed module description
• A Robot Musician Example Jasche
• Mechanical changes in real-time performance
• Results
• Robot Musician Instrument
• Future Development
• Future Discussions
• Conclusion

3
Introduction

• Robot musicians perform on real instruments
  through the usage of mechanical devices, such as
  servomotors and solenoids
• Research innovations linking music, robotics and
  computer science

4
MotivationMusic Expressiveness

• Offer the audience live-experience very similar
  to listening to a human musician.
• Real instrument performance, such as the physical
  vibration of a violin string, provides a much
  stronger case in music expressiveness, versus
  electronic music synthesizers.
• Mozart - eine kleine nacht musik whole ensemble

5
MotivationMusic Expressiveness (cont.)

• Bypass several technical difficulties that are
  typically encountered by human musicians
• More degrees of freedom in real-time performances
  and reach a higher level of performance
  difficulty, flexibility and quality.
• As an example, a violin is played by a robot
  musician with hands that have 12 fingers.

6
Robot Musicians ArchitectureRobot Musicians Band
Overview

• Each member of the band is a robot musician,
  which specializes in either a string or a
  percussion instrument

7
Robot Musicians ArchitectureRobot Musicians Band
Overview (cont.)

• Robot musicians, the P.A.M. band, invented by
  Prof. Kurt Coble.

The moth features violin solo, composed by Prof.
Kurt Coble, companied by percussion ensemble,
electric base and electric guitar
8
Robot Musicians ArchitectureRobot Musicians Band
Overview (cont.)
Austin plays a Percussion Ensemble
Dusty plays a red electric guitar
9
Robot Musicians ArchitectureRobot Musician
Architecture Overview

• A three-module architecture

10
Robot Musicians ArchitectureRobot Musician
Architecture Overview(cont.)

• Software module interacts with users, provides
  the programming and composition environment and
  sends motion commands to the control module
  through a serial port.
• Control module involves rules that govern
  application processing and connects the software
  module with the motion module.
• Motion module - the hands of the robot musician
  powered mechanics such as servos and solenoids,
  which manage and provide access to the actual
  musical instrument.

11
Robot Musicians ArchitectureSoftware Module
Infrastructure

• Implemented as a set of Turbo C programs
• The software communicates with the control module
  through serial port I/O.
• Consists of two major components
• composition component
• performance component.

12
Robot Musicians Architecture Software Module
Composition Component

• Direct programming environment based on a set of
  pre-defined syntax, users can create C programs
  which control each detailed motion of the robot
  musician
• Compose music scores through the keyboard and
  mouse

13
Robot Musicians Architecture Software Module
Composition Component (Cont.)

• Recording function detects and records details
  of each key-press or mouse-movement event, and
  the time delay in between.

14
Robot Musicians Architecture Software Module
Performance Component

• Read and play the musical scores recorded by the
  composition component or directly composed by a
  user
• Single-line interpretation produce a musical
  experience that is almost identical to the
  original instrumental performance
• Tuning component adjust sound quality of the
  instruments before composition and performance

15
Robot Musicians ArchitectureControl Module

• An RSV Converter and a costume-manufactured
  motion control card
• The motion control card is built on an Aesthetika
  Robotika Servo Control Rev 4 board, powered by
  12V and 500 mA. It is custom manufactured for
  this project to support up to 4 servos (4 axes)
  and 12 solenoids
• The converter interfaces between the serial port
  and motion control card

16
Robot Musicians ArchitectureControl Module
(Cont.)

• Receives motion control commands from the
  software module and computes joint angles and
  servo motor velocities for each axis
• Deliver these values to the motion module,
  ensuring smooth servo motor movements.

17
Robot Musicians ArchitectureControl Module
(Cont.)
Control module with motion control card and
converter, linked with the motion module of an
incomplete keyboard robot musician hand
18
Robot Musicians ArchitectureMotion Module

• The Motion module is provided by servos and
  solenoids
• Two types of servos (Hitec Quarter Scale HS-700BB
  Servo and Hitec HS303 Servo) and three types of
  solenoids are used in its construction

19
Robot Musicians ArchitectureMotion Module (Cont.)

• A violin is set up with two bows controlled by
  two servos and twelve fingers (solenoids). Servos
  are attached to bows of the violin. They move the
  bows back and forth across the violin strings.
  Solenoids, act as magnets to depress the strings
  when charged
• Amazing grace traditional American folk song.

20
Robot Musicians ArchitectureMotion Module (Cont.)
Servo attached to one bow of Jasche
Solenoid (with holding power of 1.5 pounds)
attached to Jasche
21
Servomotor In Action
22
Robot Musicians ArchitectureMotion Module (Cont.)
A coffee containers plastic lid is connected
with a servo so it flutters against the body of a
drum when the servo receives control command from
the control module
Sample Motion Module Architecture drumstick
controlled by solenoid
23
Robot Musicians ArchitectureMotion Module (Cont.)
Sample Motion Module Architecture chimes wand
controlled by servo
24
Motion Module In Action

• Mozart - eine kleine nacht musik whole ensemble

25
Robot Musicians ArchitectureA Robot Musician
Example Jasche

• The Robot Musician Jasche plays a 2-bow 12-finger
  2-string violin tuned with 3 octave chromatic
  range (From F below middle C to two octaves above
  middle C), which produces 24 pitches.
• Happy Birthday

26
Robot Musicians ArchitectureA Robot Musician
Example Jasche (Cont.)
27
Jasche In Action
Amazing grace traditional American folk song
28
Robot Musicians ArchitectureA Robot Musician
Example Jasche (Cont.)

• Software module compose musical pieces through
  mouse movements
• Keys on the third line of the keyboard, symbols
  QWERTYUIOP correspond to 10 speed levels of the
  bow movement.
• Keys on the second line of the keyboard, symbols
  1234567890- correspond to 12 solenoids used to
  press the strings, producing different pitches

29
Robot Musicians ArchitectureThe Robot Musician
Band Architecture
30
Robot Musicians ArchitectureThe Robot Musician
Band Architecture (Cont.)

• 8 MCC boards are chained together, meaning, 8
  robot musicians can be controlled by one single
  computer terminal simultaneously

31
The Mechanical Changes in Real Time Performance

• The smooth transition between two musical notes
  (continuation of music score) the improvement of
  mobility, elasticity and sensitivity of the robot
  hands
• High speed communication between the PC and the
  motion control card becomes critical

32
Real Time PerformanceMechanical Issues
33
Results

• The recent work on robot musicians has some
  promising results. The existing system is found
  to be a robust method in controlling complex
  musical mechanics
• A musical piece collection will follow
• 7sg2 composed by Prof. Kurt Coble, the name of
  the musical piece stands for "7 solenoids guitar
  part 2", 6-string electric guitar, tuned in a
  parallel major 7th chord

34
Results drum set in action
35
Results whole ensemble

• Botman composed by Prof. Kurt Coble, featuring
• whole ensemble

36
Robot Musical InstrumentA New Generation and A
New Art Form

• Musical instruments that are more complex in
  nature for robot musicians
• A combinational drum set drums are specially
  positioned so that they are easily accessed by
  the robot hands.
• A violin can be played with two bows
  simultaneously

37
Robot Musical InstrumentA New Generation and A
New Art Form (Cont.)

• Digitally-Manipulated Analogue Signal Systems
  (D-MASS) is proposed in this work as a
  description of this new art form. The following
  Table makes a comparison between MIDI (Multi
  Instrument Digital Interface) and D-MASS.

38
Future Development

• Software/hardware tools read and play
  traditional scores.
• soft computing (fuzzy logic, neural networks,
  evolutionary and genetic algorithms) and
  artificial intelligence / expert systems
  techniques
• Improving the emotional quality of the music
  performance
• Having the robot musicians listen to various
  music pieces, recognize the tones, improve on
  them and then re-play them
• Wireless control

39
Future Discussions

• Appreciating the interaction of music, computer
  science, robotics and the new concepts that
  emerge from robot musicians.
• The analysis of the degree of intelligence and
  sensitivity achieved by robot musicians

40
Conclusions

• The experience of designing robot musicians is a
  breath-taking experiment.
• A fully functional robot orchestra can be
  established in the near future, which will play
  not only simple tunes, but also masterpieces.

41
Acknowledgements

• James Sedgwick
• Dr. Jeffrey Johnson
• Dung Hoang and Ying Guo
• Interdisciplinary Robotics, Intelligent Sensing,
  and Control (RISC) laboratory
• Arnold Bernhardt Center, Austin Mather Bubble
  Theatre

42
Current Project Status

• 2002 Sigma Xi Grant-in-Aid of Research recipient
• Paper titled Experimental Robot Musicians has
  been accepted by the Journal of Intelligent
  Robotic Systems

43
Thank You