Microsoft Research Summer School

1
Robots Unlimited PHD Summerschool July
2006 Alexander Brändle Marco Combetto
2
Agenda

• Who we are?
• Why robotics?
• Activity overview
• Future ideas

3
Intelligent Environments Team
Alexander Braendle
Marco Combetto
Pierre-Louis Xech
Andreas Heil (PHD Student)
4
Why robotics?

• How will Future Environments look like?
• Challenges/Needs
• Personal Devices, Embedded Devices, Sensors,
  Robots, Appliances?
• Enabling technologies,Programming
  paradigms,Interaction paradigms?
• How will future applications look like?
• Wide range of user types?
• Mixing real and virtual world?

Robots could form a part of it!
5
The Robotics Wave

• Service and consumer markets just emerging
• Remote assistance/presence
• Assistive
• Facilities maintenance
• Security
• Education
• Entertainment
• Academic Research
• Complex topic
• Moving from 8/16 to 32 bit
• Lots of hand-coded solutions
• Education and Hobbyist channel
• Still mostly 8 bit, starting to shift
• Strong governmental interest worldwide

EURON roadmap 2015 -gt Its all about software
6
Robots!

• Wide range of applications
• Commercial use
• Academic use
• Personal use
• How to program them?
• Challenges
• Complexity
• Reusability
• Reliability
• Resources
• Tools
• Technologies
• Choice
• Sharing
• Transference of skills/experience

7
Links

• European Robotics Network (EURON)
• European Robotics Platform (EUROP)
• IEEE RAS Technical Commitee on Programming
  Environments in Robotics and Automation

8
Robots, too?

• Software
• Internet Explorer
• Media Player
• MS Agent
• Mobile Phones

Definition A robot is a device, hard- or
software with the capability of sensing and
(re-)acting.
9
Robotics at Microsoft Research (Cambridge)
Robots in human environments

• End-user programming
• Developer Tools (Fischertechnik-Demo,
  Scatterweb-Demo)
• Visual Programming (.FUN-Demo)
• End-user debugging
• Microsoft Robotics Studio
• Coding4Fun
• Enabling new applications
• Emotional oriented computing
• Personal robotics
• Self-reconfigurable structures
• Learn from interaction with animals

10
The Fischertechnik ROBO Interface

• The in- and outputs
• 8 digital inputs
• 2 digital and analog distance sensors
• 4 analog sensors for resistance and voltage
• 4 motors with 8 different speeds

• The board
• Serial port, COM, RS232
• USB
• Infrared
• R/F module available
• Ethernet for the next hardware revision planned

TSSystemRoot\System32\mstsc.exe
11
Adding Control

• Now we want to control the robot
• Lets take another off the shelf product
• Ordinary Joystick

12
Coding4Fun

• http//msdn.microsoft.com/coding4fun/

13
Integrating Sensors

• Sensors are they eyes and ears of robots
• Increasing Demand for easy robust sensor platform
  from biology, ecology, civil engineering et al.

How to get easy access to Sensor Nodes systems

• A challenge on its own
• Where are sensor node platforms heading?
• Dealing with vast amounts of real-time data
  becoming available

14
Sensor Networks in the real world

• Robustness
• WSNs have to work 24/7
• WSNs have to offer uniform APIs
• Long-lived, autonomous networks
• Self-organizing and energy efficient
• Routing
• Data aggregation
• Deployment Setup
• Better development support
• Realistic simulation tools
• Heterogeneous testbeds
• Useful traces
• Tool integration (in well know platforms)
• Development for heterogeneous systems

ScatterWeb/.NET
15
Sensors (ScatterWeb) An open and flexible
platform for rapid protoyping implementation of
wireless sensor networks

• Nodes
• with/without sensors
• Sensors
• Luminosity, noise detection, vibration, PIR
  movement detection,
• Microphone/speaker
• IR sender/receiver
• stand-alone/modular
• Acceleration, humidity, temperature, luminosity,
  noise detection, vibration, PIR movement
  detection on demand
• Stand-by 7.6µA, 5 years life-time with AA
  battery and 1 duty-cycle
• Gateways
• WLAN, Ethernet, Bluetooth, GPS, GSM/GPRS, USB,
  RS485 , serial
• Software
• Management, flashing, routing, ns-2 simulation
  models
• TCP/IP, web server (http//193.10.67.150/),
  Contiki, TinyOS,
• Basic functions for energy management, routing

Jochen Schiller,Free University Berlin
16
.NET for ScatterWeb

• Extend the .NET tools and architecture to small
  devices

• Sensor world available to every developer
• Easy access to sensor values, events, and
  functions
• Understandable namespaces and interfaces
• Support for IntelliSense and dynamic help
• Well known programming model (events, methods,
  properties)
• Extensibility of the nodes logic (Tiny C) and
  instant IntelliSense-Update
• Development, deployment and debugging within
  Visual Studio

Jochen Schiller,Free University Berlin
17
A first step

• Attractive for developers
• End-users?

18
.FUN

• A compelling engaging programmable environment
  to play learn for children (introduce children
  to Computing in new ways)
• Make technologies of tomorrow accessible to non
  technical market (children, nurse, elderly,
  machine operator)
• Linking real and virtual world
• Wider market potential of Robotics (Industrial,
  Assistive technology, new consumer products,
  health)

Technical University Berlin
19
Next steps

• Keep graphical representation
• Using context data
• Domain specific language
• Changing the paradigm

20
Integrating a Context Server 1/2
Raw Sensor Data
Context Events
Context Server
Application(s)
Sensors
DB
Context Requests
Torben Weis, University of Stuttgart
21
Integrating a Context Server 2/2
Raw Sensor Data
Context Events
Context Server
Application(s)
Context Simulator
DB
Context Requests
3D Data
Torben Weis, University of Stuttgart
22
Torben Weis, University of Stuttgart
23
Domain-specific Languages

• Visual N
• Domain-specific
• Graphical language
• Extension of VRDK

Users can switch betweenboth notations

• N
• Domain-specific
• Textual language

Code translator

• C / VB.NET
• General purpose programming language

Torben Weis, University of Stuttgart
24
N - A Textual Notationfor Visual N

• ambient MyAmbient _at_ Person
• where filter (1.Age lt 40), filter(1.Location
  "Stuttgart")
• discover Lichter _at_ Light
• where distance(a),
• filter(1.Color "red")
• process OnLampAdd _at_ l Lichter.Added
• l.On()
• process OnLampRemove _at_ l Lichter.Removed
• l.Off()
• a 100

Torben Weis, University of Stuttgart
25
Microsoft Robotics Studio
A development platform for robotics community,
supporting a wide variety of users, hardware, and
application scenarios.
Microsoft Robotics Studio

• Simulation Tool
• Visual Programming Language

Authoring Tools

• Samples and tutorials
• Robot services
• Robot models
• Technology services

Runtime
Services and Samples

• Concurrency
• Services infrastructure

• Make it easy to manage asynchronous components
• Avoid need to understand manual threading,
  semaphores, etc.
• Provide a scalable programming model
• Make state observable, easily accessible
• Provide for reusability and failure
• Support component discovery and composition
• Support remote/distributed execution

26
Microsoft Robotics StudioKey Runtime Features

• Support standalone and distributed processing
  scenarios

Connected operation (remote execution on PC)
Disconnected autonomous operation (with optional
networked monitoring)
Distributed execution (execution across compute
units)
27
Microsoft Robotics Studio

• Extensible to a wide variety of hardware

28
Microsoft Robotics StudioServices and Samples

• Over 15 tutorials
• VB.Net, C, JScript
• Support for
• LEGO Mindstorms RCX
• LEGO Mindstorms NXT
• fischertechnik
• MobileRobots Pioneer P3

29
Microsoft Robotics StudioOther University Support

• Bryn Mawr College
• Carnegie Mellon University
• Cornell University
• Georgia Tech
• Massachusetts Institute of Technology
• Stanford University
• University of Pennsylvania
• University of Pisa
• University of Southern California
• University of Washington

30
Imagine a world

• where Paper is able to understand, what you are
  doing

31
Robotics at Microsoft Research (Cambridge)
Robots in human environments

• End-user programming
• Developer Tools (Fischertechnik-Demo,
  Scatterweb-Demo)
• Visual Programming (.FUN-Demo)
• End-user debugging
• Microsoft Robotics Studio
• Coding4Fun
• Enabling new applications
• Emotional oriented computing
• Personal robotics
• Self-reconfigurable structures
• Learn from interaction with animals

32
Emotion-Oriented Computing

• General Goal Make interaction between human and
  machine more natural for the humans
• Machine should be able to
• To register human emotions
• To convey and comunicate emotion
• To understand the emotional relevance of the
  event

11/6/2013
32
Microsoft Internal Only
33
Human Centred, Affects and Emotions

• Enhance communication through compelling, fun and
  emotional interactions with computing (Mobile
  devices, Robotics)
• Seeking to make the process more intuitive,
  interactive and appealing to a wider group of
  people. Experimenting, evolving, evaluate
  emotional models
• Support privacy, intimacy and different level of
  information sharing
• Extending the software as a new medium
• Affective Media and Mobile media (affective loop)
• Participatory creation, exchange, authoring and
  fruition
• Storing the intimacy, the private value of the
  things
• Fusion of different kind of data sensors,
  embodiments,

11/6/2013
Microsoft Internal Only
33
34
Human Centred, Affects and Emotions

• Enhance communication through compelling, fun and
  emotional interactions with computing. Seeking to
  make the process more intuitive, interactive and
  appealing to a wider group of people.
  Experimenting, evolving, evaluate emotional
  models
• Support privacy, intimacy and different level of
  information sharing
• Extending the software as a new medium
• Affective Media and Mobile media (affective loop)
• Participatory creation, exchange, authoring and
  fruition
• Storing the intimacy, the private value of the
  things
• Fusion of different kind of data sensors,
  embodiments,

11/6/2013
34
Microsoft Internal Only
35
SensingInteraction with Environments

• Some examples

36
Affective DiaryDesigning for bodily
expressiveness and self-reflection

• Collecting memories including body memorabilia
  mingled with mobile materials (SMS, MMS,
  photographs, music listened to, video,..
• Offering a diary medium in which those memories
  can be mirrored and organised
• Empowering the user to create meaning and alter
  those representations
• Prototype
• build on TabletPC and Smartphone
• Sensordata (movement, arousal)

11/6/2013
Microsoft Internal Only
36
37
BSP Interactive storytelling in vast location
based

• Pervasive computing Location based games provide
  navigational challenges e.g. Chasing
• Mobile Media A flexible media and framework for
  interaction
• Interactive storytelling Balance linearity with
  user control, believable characters
• The concept of believable environments, and our
  implementation, put a research focus on
  possibilities to
• Enrich pervasive games with location
• dependent narratives
• Design the stage set to improve
• interactive storytelling

11/6/2013
Microsoft Internal Only
37
38
SensingInteraction with Robots

• An example

39
Why Robotics?

• Interest about robots is spreading fast
• Reinventing Personal Computing
• New way of interact because the usual ways are
  not enough (keyboard, mouse, etc.)
• We look for new interaction modality? Does Human
  size matter? Social aspects..
• Robots as an embodiment
• How to discover contexts and adapting?
• Technology
• Technology seems to be mature enough to produce
  robots not only for industry
• Software for controlling social robots should be
  robust and should adapt to the changing
  environment
• It seems that control systems theory doesnt
  scale well in this setup

39
40
The mind and the body

• Neurophysiologists suggest that the body plays a
  crucial role in our cognitive process
• The brain experiences the world through the body
• The environment is well known to the brain
• Outside environment is a set of states of inside
• Agents with body (embodied agents) try to mimic
  this schema to produce intelligent behavior
• Dautenhan and Jakobi show that bodys presence
  influences the behavior of software (anything
  already heard?)

40
41
Component structure
Vision Roblet
Knowledge Base
Body Map
Arm Roblet
WiFi Roblet
MotionRoblet
Sensors
Internal perception
Network perception
41
42
Robotics4.net

• Realization of a platform for define the body of
  a robot as a set of agents (Roblets) that acts as
  intermediaries between the devices and the
  reasoning software
• The robot provides a physical body and a set of
  core services to support the development of
  highly autonomous agent situated in both
  cyberspace (e.g Internet, virtual entities) and
  real world
• Definition of an Object Model, an API, a set of
  abstraction that allows
• Portability (on different type of platforms)
• Scalability from small (e.g. Lego) to complex
  systems
• Integration with the MS Development Platform and
  MS OS and Application suites

42
43
Functions implemented

• The platform provide the following base services
  to support the software implementing the roblets
• General purpose functions (I/O)
• Basic motion abilities
• Collision avoidance software
• Vision software for face recognition
• Speech and gesturing recognition
• Positioning and navigation aided by video input
• Network perception
• IM interface
• The Architecture is available as a simple
  Software Development Kit freely downloadable from
  http//www.robotics4.net

43
44
Perception
11/6/2013
Microsoft Internal Only
44
45
Evolving the framework

• Reduce the programming model of Robotics4.NET to
  the one adopted by the Robotics Studio CTP
• Extending the Mind/Body model with
  self-developing tools, model and methods to
  verify the behavior of systems imposing very high
  level constrains and how that can be both
  formally and programmatically verified
• Testing interaction in real scenarios

45
46
Questions