Second Life: Bots and Motion Planning

1
Second Life Bots and Motion Planning

• Russell Gayle
• Comp 790-058 Robot Motion Planning
• Fall 2007
• October 1, 2007

2
Last time

• SL Overview
• The Grid
• Simulators / Regions
• Avatars, Primitives
• LSL Crash Course

3
Aside Path Tracer

• Results of the LSL portion (Part 0) should look
  something like this
• http//www.youtube.com/watch?vbcemyUuzCdseurl
• Update
• As of this morning, the UNC Sandbox has turned
  into an access restricted area.
• Ive emailed UNC about a place where I can put
  it.
• If I dont hear back soon, Ill send it out to
  everyone.

4
Introduction SL Bots

• What are bots?
• In Second Life?
• Two varieties
• LSL based (prims)
• Come in many shapes and sizes
• Wide range of functions
• Primarily server side code
• LibSL based (avatars)
• Function by manipulating avatars
• Mixed computation from client and server
• Network delay becomes an issue

5
LSL Bots

• Prims with (motion) control
• Control is defined through LSL
• Integrated sensing (LSL sensors)
• Can interface with remote resources
• http//www.youtube.com/watch?v824TFPerXsQ
• Some of the remaining content for this section is
  adapted from
• Alon Shalita
• Tel-Aviv University
• Motion Planning in Virtual Environments Workshop

6
LSL Bots LSL Limitations

• List limitations
• No native arrays
• Lists are sequential access
• Limited length
• Limited by memory script has available (16KB
  total)
• Compile time lists are limited to 72
• No nested lists
• Type conversion
• More of an annoyance, automatic conversion may
  not work
• Cannot incorporate existing libraries
• SL is moving to Mono (Open Source C)
• Not sure when this will happen though

7
LSL Bots Framework

• Limitations make it difficult to develop large,
  complex applications (like motion planning)
• Track open issues at
• https//jira.secondlife.com/browse/SVC/component/1
  0043
• Though, predefined motion like the roomba example
  should not be too hard
• One solution Use remote resources
• Hybrid planner
• Motion, physics, controls stay in SL
• Communicate with remote resources to get planning
  solutions

8
LSL Bots Communications

• LSL has the ability to send and receive
  communications
• XML-RPC (inbound)
• HTTP (outbound)
• XML-RPC
• Data limited to about 255 bytes
• Long latency (about 3 seconds)
• HTTP
• Data in request limited to scripts free memory
• Data in response limited to 2049 bytes
• Responses is asynchronous
• Do not send more than 1 per second

9
LSL Bots Comm. Example

• HTTP makes the most sense
• Example
• PHP Number Adder
• HTTP POST allows more data to be sent
• HTTP GET is limited to call length

Uses HTTP POST request
lt?phpfirst_POST"first"second_POST"seco
nd"if (!ereg("0-9", first) or
!ereg("0-9", second))  die ("first and
second must be numbers")echo
firstsecond?gt
10
LSL Bots Comm. Example

• HTTP makes the most sense
• Example
• Number Adder LSL Script

key requestiddefault    state_entry()   
        llSay(0, "Hello, Avatar!")       
touch_start(integer total_number)          
 llSay(0, "Touched.")         requestid
llHTTPRequest("http//www.somehostname.com/add.php
",        HTTP_METHOD, "POST", HTTP_MIMETYPE,
"application/x-www-form-urlencoded",      
 "first9second20")            
http_response(key request_id, integer status,
list metadata, string body)            if
(request_id requestid)                  
 llSay(0, body)         else          
 llSay(0,(string)status" error")   
Initiate HTTP request
Process response
11
LSL Bots Sensors

• Functionality for scripts to scan for items
• Scan by name/id, type
• Within a range and arc radians from viewing
  direction
• llSensor( string name, key id, integer type,
  float range, float arc )
• llSensor
• Run a sensor once
• llSensorRepeat
• Repeat the same sensor over some interval
• llSensorRemove
• Remove a repeating sensor

12
LSL Bots Sensor example

• Finding the start of a maze (Alon Shalita)

default    state_entry()          
 llSay(0, "Touch me to scan for the maze")   
    touch_start(integer total_number)     
      llSay(0, "Touched.")      
 llSensor("Maze Start", NULL_KEY, ACTIVE
PASSIVE, 90, PI)            sensor(integer
total_number)            if (llDetectedName(0)
"Maze Start")                    llSay(0,
"Maze start is at " (string)llDetectedPos(0)) 
              no_sensor()          
 llSay(0, "Error Sensor failed")   
Find maze start
Start sensor
With any id
Either moving or not
Within 90 meters
Within a sphere around the rim
13
LSL Bots Sensor example

• Finding the start of a maze (Alon Shalita)

default    state_entry()          
 llSay(0, "Touch me to scan for the maze")   
    touch_start(integer total_number)     
      llSay(0, "Touched.")      
 llSensor("Maze Start", NULL_KEY, ACTIVE
PASSIVE, 90, PI)            sensor(integer
total_number)            if (llDetectedName(0)
"Maze Start")                    llSay(0,
"Maze start is at " (string)llDetectedPos(0)) 
              no_sensor()          
 llSay(0, "Error Sensor failed")   
Sensor returned an item
Nothing returned
14
LSL Bots Motion Planning

• There are several possible solutions
• Using CGAL
• Proposed by Danny Halperin and Alon Shalita

Stop
Start
Obstacles
15
LSL Bots Motion Planning

• There are several possible solutions
• Using CGAL
• Proposed by Danny Halperin and Alon Shalita
• Write a script to sense obstacles

Stop
Start
Obstacles
16
LSL Bots Motion Planning

• There are several possible solutions
• Using CGAL
• Proposed by Danny Halperin and Alon Shalita
• Write a script to sense obstacles
• Append robot and obstacles to a file

Stop
Start
Obstacles
17
LSL Bots Motion Planning

• There are several possible solutions
• Using CGAL
• Proposed by Danny Halperin and Alon Shalita
• Write a script to sense obstacles
• Append robot and obstacles to a file
• Run CGAL on the file and send the response

Stop
Start
Obstacles
18
LSL Bots Motion Planning

• There are several possible solutions
• Using CGAL
• Proposed by Danny Halperin and Alon Shalita
• Write a script to sense obstacles
• Append robot and obstacles to a file
• Run CGAL on the file and send the response
• Process the response and run the path

Stop
Start
Obstacles
19
LSL Bots Considerations

• Identifying obstacles
• Sensors only return nearest 16 objects
• This may be fixed soon
• Much easier for self-built environments
• Know exactly which prims to look for
• Limited script memory, HTTP response size
• Can only send finite amounts at a time
• Send in groups
• Process results in groups
• Note CGAL is not necessary
• Any command-line planner which can return text
  information to SL should suffice

20
LSL Bots Results Movies
21
LSL Bots Some other ideas

• None of these have been implemented (to my
  knowledge)
• Bug algorithm
• Online mapping
• Have the robot move until it collides, record
  this information
• Use other prims as waypoints
• Difficult to maintain
• Ordering could be maintained and stored remotely
• Potential fields
• Keep track of a goal and move in that direction
  while moving away from obstacles

22
LibSecondLife (LibSL)

• Potential for complete avatar simulation
• Reverse engineering of SL network protocol
• Framework for using protocol with The Grid
• Theoretically, could implement new SL viewer
• Based on C
• All C features are available
• Much more flexible programming environment
• Avatar control
• Could have an avatar control a prim, in theory
• Client-server interface
• Extend range of SL capabilities
• Actively developed
• Much of this section is just ideas so far

23
LibSL Getting Started

• Prerequisites
• C compiler
• Mono (all platforms except Windows)
• Visual Studio .NET 2005 (Windows)
• Visual Studio Express .NET (Windows, free)
• Source code via SubVersion
• svn//openmetaverse.org/libsl/trunk
• Build code
• Start a new project
• Add libsecondlife as a reference
• For more help
• http//www.libsecondlife.org/wiki/Use_libSL_to_log
  in_to_the_SL_grid

24
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
Include libsecondlife libraries
25
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
Define SecondLife client(s)
26
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
Add your bots name
27
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
Define a connected event
28
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
Try to log in to the Grid
29
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
When connected, send a message
30
LibSL Framework

• LibSL example

using System using System.Collections.Generic
using System.Text using libsecondlife  
namespace MyFirstBot class MyFirstBot
public static SecondLife client new
SecondLife()   private static string
first_name "First" private static string
last_name "Last" private static string
password "password"   public static void
Main() client.Network.OnConnected new
NetworkManager.ConnectedCallback(Network_
OnConnected) if (client.Network.Login(first_na
me, last_name, password, "My First Bot", "Your
name")) Console.WriteLine("I logged into
Second Life!") else Console.WriteLine(
"I couldn't log in, here is why "
client.Network.LoginMessage)  
static void Network_OnConnected(object sender)
Console.WriteLine("I'm connected to the
simulator, going to greet everyone around
me") client.Self.Chat("Hello World!", 0,
ChatType.Normal) Console.WriteLine("Now I
am going to logout of SL.. Goodbye!")
client.Network.Logout()  
After your message, logout
31
LibSL Framework

• Most actions are defined within callbacks
  (events)
• Network events
• OnConnected / OnDisconnected
• OnCurrentSimChanged
• Client (Avatar) events
• OnInstantMessage
• OnChat
• OnTeleport
• Object events
• OnNewAvatar / OnNewPrim
• OnObjectUpdated / OnObjectKilled

32
LibSL TestClient

• Application provided with the SVN code base
• Open source
• Standard LibSL development framework
• Most actions are included as commands
• Easy to define and use
• Invoked from console or by assigning a Master
  for your bot
• e.g. Wear, GiveAll, Stats, Location, Sit
• Supports multiple simultaneous avatar bots
• Organizes most incoming objects as the SL viewer
  would see them

33
LibSL Avatar Control

• For planning, we must be able to move the avatar
  to a desired location
• No built in command for avatars
• You may be able to hack something by attaching a
  prim that can move
• Use LibSL to build a simple motion controller
• LibSL can send data packets which instruct the
  avatar to move
• For SL planning purposes (thus far), the
  following should suffice
• Move to a position specified by a distance and
  direction
• Move to a specified position

34
LibSL Avatar Control

• Homework 1 Hints
• If you use TestClient, make use of existing
  commands
• It will be very valuable to allow your robot to
  move toward either
• A sequence of points
• A sequence of direction/distance pairs
• Make yourself a meter marker to test accuracy

35
LibSL Motion Planning

• Given a basic controller, we need to tell the
  avatar where to go
• Several standard approaches
• Cell Decomposition
• Potential Fields
• Roadmap
• For all approaches, sensing the environment is
  necessary
• LibSL comes with an ObjectManager class
• Keeps a Dictionary of all current Prims and
  Avatars
• In the same way the SL viewer does, as far as
  they understand

36
LibSL Local Mapping and Planning

• Bug-like approach to planning
• Move your avatar in a direction toward the goal
  as far as possible
• Have it follow the barrier in a given direction
  until you can make progress toward the goal again
• No need memorize the environment, just when
  youre colliding

37
LibSL Local Reactions

• Standard walking motion in SL signaled through
  AutoPilot packets
• This is essentially the straight-line path
• It is cancelled when an event occurs that causes
  the motion to end
• Collision
• Reached destination
• Thus, avatars cannot easily react
• Need a motion model that is modifiable
• A solution
• Move in small increments
• Requires some careful tuning of current velocity
  to keep motion smooth

38
LibSL Behavior Control

• Avatars come with default behavior
• However, they are capable of performing other
  actions / animations
• Solution Animation Override (AO) to the rescue
• Collect a set of animations youd like to use
• Get an AO gadget
• Francis Chung's Wet Ikon seems to be a free
  favorite
• When embedded in a worn prim, it will override
  the default
• Depending on the environment, use LibSL to change
  the various animations
• Video?

39
LibSL Demo

• Basics of LibSL and TestClient

40
LibSL Other Ideas

• A more kinodynamic approach
• Alter the motion controller to be like that of a
  car
• Hybrid approach
• What Im currently playing with
• Prims do a great job of finding nearby obstacles
• Prims are easy to move
• Use a sequence of prims as a roadmap
• Easy for an avatar to follow
• Difficult to maintain

41
Fin

• Motion Planning in SL
• Using LSL
• Using LibSL
• Any questions?