Using LEGO MindStorms in CS 1 and 2

1
Using LEGO MindStorms in CS 1 and 2

• LMICSE Workshop at SIGCSE 2006
• March 4, 2006

2
Who are We? The LMICSE Project

• LMICSE Lego MindStorms in Computer Science
  Education
• Goal Development of MindStorms based materials
  for use throughout the undergraduate CS
  curriculum
• Principal Investigators
• Frank Klassner (Villanova University)
• Pamela Lawhead (University of Mississippi)
• Myles McNally (Alma College)
• Website http//www.mcs.alma.edu/LMICSE/

3
Workshop Schedule

• Introductions
• Overview of the Current MindStorms Platform
• Using MindStorms in CS 1
• Using MindStorms in a Closed Lab Environment
• Overview of the Turtle Sequence
• Using the BlueJ IDE with MindStorms
• Using MindStorms in CS 2
• GridWalker Module Overview
• Random Walk Lab
• Wavefront Propagation Lab
• A Glimpse at Lego MindStorms NXT

4
The Current MindStorms Platform

• LMICSE Workshop at SIGCSE 2006
• March 4, 2006

5
The Lego MindStorms Kit

• The RCX (the brains)
• Two Geared Motors
• Two Touch Sensors
• One Light Sensor
• A USB IR Tower
• A CD-ROM
• The Constructopedia
• Lots of Lego Pieces!

RCX Brick and Standard Sensors and Motors
6
A Basic Robot Design

• The Constructopedia is a building guide.
• Shown at left is the basic Roverbot from the
  Constructopedia
• A robust design
• Allows attachment of any of
• double bumper (shown)
• wheel sets (shown)
• single bumper
• light sensor
• tank like treads
• A recommended design starting point

Basic Roverbot Configuration
7
The RCX Itself (1)

• Hitachi H8 series microcontroller
• 8 bit CPU at 16 MHz
• 32 KByte RAM
• 16 KByte ROM for the BIOS
• Serial I/O (input/output)
• ADC (analog digital converter)
• Built-in timers

RCX Brick
8
The RCX Itself (2)

• Liquid crystal display (LCD)
• Four push buttons
• One small internal speaker
• Three sensor inputs
• Three motor outputs
• One infrared (IR) interface with a proprietary
  protocol
• Requires 6 AA batteries

RCX Brick
9
Actuators

• With Kit
• Geared Motors
• Available From Lego
• High-speed Motors
• Mini-motor
• Light Brick
• Available third party
• Servo Motor/Servo Controller

Servo Controller w/ 2 Servo Motors
Ungeared and Geared Motors
10
Sensors Basics

• MindStorms sensors come in two types
• Active RCX supplies power
• Example Light sensor (supplied with kit)
• Passive RCX just detects changes in resistance
• Example Touch sensor (supplied with kit)

11
Sensors Available from Lego

• Touch
• Passive
• Supplied with Kit
• Light
• Active
• Supplied with Kit
• Rotation (Angle)
• Active
• Add-on

Touch, Light and Rotation Sensors
12
Third Party Sensors (1)

• The sensor interface is well documented
• Many sets of building instructions for sensors
  exist, for example see Michael Gasperis website
• http//www.plazaearth.com/usr/gasperi/lego.htm
• Two commercial suppliers we know of
• http//www.mindsensors.com
• http//www.hitechnic.com

13
Third Party Sensors (2)

• Sonar (Ultrasonic range)
• Available both from Mindsensors and Hitechnic
• Magnetic Compass Sensor
• Available from Hitechnic
• Active and Passive Multiplexer
• Available from Mindsensors
• And many others

Hitechnic Sonar
Mindsensors Active Multiplexer
Hitechnic Compass
14
Software Architecture
Typical Software Architecture Levels
15
Software Architecture

• System ROM Layer
• Low Level Control Software
• Allows a second piece of software - the firmware
• Can be bypassed for direct control of the hardware

16
Lego Firmware

• The Lego firmware provides an environment for the
  execution of programs on the RCX.
• Must be downloaded (just once) before programming
  the RCX.
• Other firmware solutions are available.
• Lego Firmware v. 2.0 permits
• 5 separate programs
• 10 threads
• 32 global variables
• 16 local variables per thread
• Broadcast IR protocol

17
Alternative Firmware

• Firmware replacements further push the limits of
  the RCX
• Popular replacements are tied to alternative
  programming environments
• Examples
• LejOS (Java)
• see http//lejos.sourceforge.net/
• BrickOS (C/C)
• see http//brickos.sourceforge.net
• RCXLisp (Lisp)
• Part of the LMICSE project
• See http//robotics.csc.villanova.edu/

18
Using MindStorms in CS 1

• LMICSE Workshop at SIGCSE 2006
• March 4, 2006

19
The Turtle Laboratory Sequence

• Designed for teaching programming fundamentals
• Based around the Turtle, a LOGO-like
  abstraction from the details of motors and
  sensors

A Turtle
20
The Turtle Laboratory Sequence

• Original work was done by Scott Anderson
  (Wellesley) with support from Frank Klassner
  (Villanova)
• Coded the Turtle software
• Created drafts of seven CS 1 oriented labs
• Current work by Myles McNally (Alma)
• Simplified somewhat the Turtle software
• Twelve labs oriented towards CS 1
• Each lab was used in a fall, 2004 CS 1 course
• Each lab has now been refined based on that
  experience and videos added

21
The Turtles Basic Abilities

• Motion
• Move forward or backward (timed or continuous)
• Turn left or right (timed or based on degrees)
• Can calibrate turning time required to turn n
  degrees
• Stop
• Events
• Can respond to events
• Simple event queue model
• NONE (queue is empty), RIGHT, LEFT, BOTH, VIEW,
  PGM, RUN
• Singing
• Can play a tone for a duration

22
The Physical Turtle (1)

• Many designs could be used
• Requirements are
• Differential drive
• Left motor in motor port A
• Right motor in motor port C
• Left and right front mounted bump sensors
• Left touch sensor in sensor port 1
• Right touch sensor in sensor port 3

23
The Physical Turtle (2)

• The design we recommend is based on designs from
  the Constructopedia
• The Roverbot Driving Base (p. 12)
• The Wheel Sets (p. 17)
• The Double Bumper (p. 30)

The Standard Roverbot
24
The Physical Turtle (3)

• Then make the following modifications
• Remove the front (smaller) set of wheels
• Add a slider to the bottom of the bot between
  where the front wheels had been
• Move the RCX itself further back on the base, so
  that its weight is more centered over the
  remaining wheels
• These changes improve the exactness of turns

The Turtle top and bottom
25
The Current Turtle Laboratories

• Java-based (in particular LeJOS)
• Covers topics found in a modern, object oriented
  CS 1 course
• Basic types and expressions
• Flow of control
• Classes and methods
• Arrays, stacks and lists
• Interfaces
• Inheritance and abstract classes
• Polymorphism
• Loosely follows the topic order in the Lewis and
  Loftus text, but could be used with almost any
  object oriented text.

26
Turtle Laboratory Topics (1)

• Sequential Control Use straight-line code and a
  "Turtle" robot to move in a few geometric
  figures.
• Variables and Expressions Use more advanced code
  and variables to create more interesting shapes.
• Methods Use methods to separate code into parts
  and also use the Random class.
• Methods with Parameters / Scope Use methods with
  parameters and returns, instance variables, and
  the Math class.
• Classes Define a class that allows musical notes
  and rests to be represented and played back by
  the RCX.

27
Lab 4, Task 5
Right Angle Random Patrol
28
Turtle Laboratory Topics (2)

• Event Driven Programming Work with the basics of
  event driven programming and focusing on using
  decision structures to respond to events.
• Loop Control Structures Work with each of the
  loop control structures in Java in the context of
  event processing.
• Using Interfaces Define interfaces then
  implement them to run races and to draw figures
  with the robots.
• Array Structures Use arrays to record inputs
  from the user and then traverse a course using
  the recorded values to know when to move and when
  to turn.

29
Lab 6, Task 3
Use Events to Drive the Turtle
30
Turtle Laboratory Topics (3)

• Navigation Implement a navigation interface
  which allow the Turtle to go to positions in its
  world, then input a series of positions and have
  the Turtle visit them.
• Inheritance Define several classes that handle
  notes and rests, and an abstract class that each
  is an extension of.
• Sorting and Polymorphism Use polymorphism with
  various sorting algorithms.

31
Lab 12, Task 2
Travel to Points Sorted by Horizontal Position
32
Programming the Turtle LeJOS

• The basic programming environment is LeJOS
• an open source project
• based on a subset of Java with extensions for the
  RCX
• missing Java features hard to predict (i.e., no
  switch statement)
• contains some advanced robotics libraries
• navigation (odometry)
• behaviors (subsumption architecture)
• image processing (vision)
• available at lejos.sourceforge.net

33
Where to Learn More about LeJOS

• The LeJOS Tutorial an online resource which is
  part of the LeJOS project (on their website).
• The LeJOS PowerPoint at the LMICSE site
• http//www.mcs.alma.edu/LMICSE/Workshops/alma2/pow
  erpoints/LejosProgramming.ppt
• Books such as
• Core Lego MindStorms Programming, by Brian
  Bagnall (Prentice-Hall, 2002),
• Programming Lego MindStorms in Java, by Giulio
  Ferrari, et al (Syngress, 2002),
• and many others.

34
Programming The Turtle Abstraction

• LeJOS provides the ability to directly control
  motors and poll sensors
• The Turtle abstracts from these low level
  routines, allowing users to direct the Turtle
  move forward, turn, and register events.
• This results in simpler, more readable programs.

35
An Example LeJOS Program

• Notice the use of low level control

import josx.platform.rcx. public class Patrol
public static void main(String args)
throws InterruptedException
Motor.A.forward() while (true)
Motor.C.forward() // go forward
Thread.sleep (5000)
Motor.C.reverse() // turn around
Thread.sleep (1000) Motor.A.stop()
Motor.C.stop()
Patrol back and forth (v. 1)
36
The Corresponding Turtle Solution

• Notice the simpler, more direct coding solution

• public class Patrol
• public static void main(String args)
  
• while (true)
• Turtle.forward(2000) // go forward
• Turtle.turn(1000) // turn around
• Turtle.stop() // stop all motors

Patrol back and forth (v. 2)
37
Using Calibration

• Once calibrated, the Turtle can do degree-based
  turns

• public class Patrol
• public static void main(String args)
  
• Turtle.calibrateTurn(360,3000)// calibrate
• while (true)
• Turtle.forward(2000) // go forward
• Turtle.left(180) // turn around
• Turtle.stop() // stop all motors

Patrol back and forth (v. 3)
38
Using the Turtle Event Model

• Use the nextEvent() method and the Turtle event
  constants

• public class MoveForward
• public static void main(String args)
  
• Turtle.forward() // move forward
• int event Turtle.nextEvent()
• while (event ! Turtle.RUN) // until RUN
  button
• event Turtle.nextEvent() // is pressed
• Turtle.stop() // stop all
  motors

Move forward until the RUN button is pressed
39
Using the BlueJ IDE with MindStorms

• LMICSE Workshop
• June 14 - 17, 2005
• Alma College

40
What is BlueJ?

• Freeware developed by Michael Kolling and others
• A popular integrated development environment for
  Java
• Often used in introductory CS courses
• Available at www.bluej.org, easy to install
• Go ahead and open BlueJ on your PC

41
Setting up BlueJ for MindStorms

• Two requirements
• Install LeJOS
• Available from lejos.sourceforge.com
• Just do the basic install
• Add the BMJT plugin to BlueJ
• Available from ddi.uni-paderborn.de/mindstormstool
  s/bjmt
• Good instructions available online
• These steps are already done for you on your PC!

42
Once the BMJT Plugin is installed

• You need to configure it
• This is already done for you on your PC!

43
Firmware

• Now you can download the firmware to the RCX
• Plug the tower into a usb port
• if on a PC you may need to install the usb
  drivers
• Place the RCX in front of the tower, turn it on
• Choose Firmware
• You will see a progress dialog on the PC
• On the RCX you will see an ascending series of
  numbers

44
Compiling an Existing File

• Right-click (Command-click) on the file icon
• Choose Compile from the pop-up menu

45
Downloading a Project

• Robot must be on and in front of tower
• Right-click (Command-click) on the file icon for
  the main class
• Choose Download from the pop-up menu

46
Try it Out!

• Place the robot in its play area.
• Press run.
• It should move forward until it hits an obstacle.

47
Hands-on Time!

• Exercise 1 Modify the GoForward program so that
  the Turtle moves forward, turning away from
  obstacles when necessary. That is, when the left
  bump sensor is pressed, back up and turn right,
  and likewise for the left bump sensor.
• Exercise 2 Craft a program in which you enter a
  number by pressing the right bumper that many
  times, and then have the Turtle travel around in
  a square shape that many times. Signal the end
  of your input by pressing the left bumper. The
  Turtle should then begin to move.
• Or do any of the lab exercises from the LMICSE CS
  1 area.

48
Using MindStorms in CS 2

• LMICSE Workshop at SIGCSE 2006
• March 4, 2006

49
The GridWalker Labs

• Introduce AI themes in CS 2
• robotic systems
• graph searching algorithms (without the graph)
• occupancy grids and path planning
• Use basic user defined data structures like
  stacks and queues
• Reinforce material learned in CS 1 such as
• classes and inheritance
• two dimensional arrays
• exceptions

GridWalker Robot with Sonar
50
The Projects
Random Walk (and Back)
Wavefront Propagation
Classical Search Algorithms
Adding Sonar
Project Dependencies
51
Occupancy Grids

• An occupancy grid is a discrete grid that
  represents the environment.
• Each cell in the grid is either
• occupied (part of an obstacle).
• empty (part of free space).
• If any part of an obstacle is in a grid cell,
  then the cell is considered to be occupied.
• Grids representing real world environments are
  often fine grained (cells representing 4 to 6
  square area).

52
Occupancy Grids
A 14 by 8 grid Black figures are the
obstacles Gray areas are the occupied cells
53
Basic GridWalker Environment
Wood Blocks Mark Occupied Cells
54
Project 1 A Random Walk (and Back!)

• The robot goes for a random walk on an occupancy
  grid containing no obstacles.
• Then retraces its steps back to the starting
  point.
• Basic strategy is to use a stack
• At each point the robot moves one cell north,
  east, south, or west (if legal).
• On the way out, push the directions traveled onto
  the stack.
• On the way back, pop the stack and travel in the
  opposite direction.

55
GridWalker in Action
56
Overall Solution Architecture
BlueJ Class Diagram for Project 1
57
The Grid Walker Class

• The heart of this project is the definition of
  the GridWalker class. It contains
• a constructor that specifies the grid size and
  the starting point of the robot
• a goto method that takes as a parameter a grid
  location and causes the robot to move from its
  current grid position to that location
• convenience methods north, east, south, and west
  that move the robot one cell in the respective
  direction.

58
Implementing the GridWalker

• Three possibilities
• Implement the GridWalker directly, using timing
  for distances and turns
• but timing depends on battery strength, so is
  inexact over time
• Use one of the two LeJOS navigation classes
• Timed Navigator same problem as above
• Rotation Navigator this is the ticket!
• Requires two rotation sensors
• Details in paper

59
The Rotation Sensor

• Thread an axle through the sensor
• One revolution of the axle is 16 clicks of the
  sensor
• So it can measure a changes in angle of 22.5
  degrees
• Which is why it is also known as an angle sensor

A Rotation Sensor
60
Mounting Rotation Sensors

• Here is a solution that is a simple modification
  of the basic Roverbot

Rear View of Roverbot Chassis
61
The LeJOS Rotation Navigator Class (1)

• Rotation Navigator implements the Navigator
  interface
• Important Navigator methods
• public void gotoPoint(float x, float y)
• Rotates the RCX robot towards the target point
  and moves the required distance
• public float getX()
• Returns the current x coordinate of the RCX.
• public float getY()
• Returns the current y coordinate of the RCX
• public float getAngle()
• Returns the current angle the RCX robot is facing

62
The LeJOS Rotation Navigator Class (2)

• assumes differential drive, with rotation sensors
  (encoders) for the left and right sides in sensor
  ports 1 and 3.
• the constructor

public RotationNavigator(float wheelDia, float
AxleWidth, float ratio)
the ratio of encoder revolutions to axle
revolutions
the distance from the center of the left wheel to
the center of the right wheel
the diameter of a wheel
63
Grid Walker Implementation (1)

• Define GridWalker to extend RotationNavigator
• Have instance variables for
• initial grid position (x and y)
• current grid position (x and y)
• the number of columns and rows in the grid
• the size of a grid cell (assume it is square)
• The constructor is passed this information, plus
  the information the RotationNavigator class needs

64
Grid Walker Implementation (2)
import josx.robotics. public class GridWalker
extends RotationNavigator int currentH,
currentV, initialH, initialV int dimensionH,
dimensionV int cellSize public
GridWalker(int cH, int cV, int dimH, int dimV,
int cSize,
float wDia, float wBase, float ratio)
super(wDia, wBase, ratio) initialH
currentH cH initialV currentV cV
dimensionH dimH dimensionV
dimV cellSize cSize
The beginning of the class definition and the
constructor
65
Grid Walker Implementation (3)

• In implementing the the goto method
• need to check that the new location is legal
  (i.e., in the grid)
• if not, throw an exception (optional, but a good
  opportunity to reinforce the use of programmer
  defined exceptions)
• need to convert grid locations to absolute
  locations
• RotationNavigator assumes the robot starts at 0,0
  with an orientation of 0 degrees
• GridWalker allows the programmer to specify any
  cell as the starting location, but still assumes
  orientation of 0 degrees

66
Grid Walker Implementation (4)
public void gotoCell(int h, int v) throws
OutOfBoundsException if (h gt 0 h lt
dimensionH v gt 0 v lt dimensionV)
gotoPoint(gridHToPoint(h), gridVToPoint(v))
currentH h currentV
v else throw new
OutOfBoundsException(h, v)
private float gridVToPoint(int n)
return (n - initialV) cellSize
private float gridHToPoint(int n)
return (n - initialH) cellSize
The goto method and helpers
67
Remaining Implementation Details

• Add the north, east, south, and west methods to
  the GridWalker class
• Implement a stack of ints (or Objects - Integers)
• Then in the main class
• First (random walk)
• Generate a series of random ints between 0 and 3
• Travel in that direction and push on stack
• Second (travel back)
• Pop the stack until empty, traveling in the
  appropriate direction

68
Project 2 Wavefront Propagation

• Design a program that will move the robot from a
  starting point to a goal point
• using the shortest possible path
• navigating around obstacles
• only moving north, east, south, or west
  (Manhattan movement)
• Obstacle locations are known in advance

69
Propagating the Wavefront

• Image a wave moving out from the goal cell.
• When the wave first reaches each of the other
  cells, it is labeled with the time it took to
  reach it.
• The wave moves Manhattan

70
Propagating the Wavefront (2)

• If there are occupied cells, the wavefront simply
  flows around them.

71
Propagating the Wavefront (3)

• The path from any cell to the goal is implicit in
  the grid labels
• Until at the goal, move to an adjacent cell with
  a smaller label
• There may be many different equal length paths

72
Overall Solution Architecture
BlueJ Class Diagram for Project 2
73
Propagating the Wavefront (3)

• Initialisation
• Allocate a new two dimensional array D the size
  of the occupancy grid.
• Set cells in D which correspond to occupied cells
  to high values and all others to -1.
• Set the value of the goal cell to 0.
• Enqueue the location of the goal cell (a point).
• While (queue is not empty)
• Dequeue a point L.
• Find the value V associated with point L in D.
• Find the neighbours of L with a -1 value.
• Set the values of these neighbours to V1.
• Enqueue each of these points.

74
Wavefront Propagation Implementation (2)
public void setValues (Queue q, int grid)
q.enqueue(new Point (goalX, goalY)) gridgoal
YgoalX 0 while ( !q.empty() ) Point
currentP (Point) q.dequeue() int x
currentP.x int y currentP.y int newX
x1 // go east if (newX lt gridy.length
gridynewX -1) gridynewX
gridyx 1 q.enqueue(new
Point(newX,y)) // also need cases
for the other three directions
The base method for setting the wavefront values
75
Wavefront Propagation Implementation (3)
public void travel () int curX startX int
curY startY while (curtX ! goalX curY
! goalY) int curValue gridcurYcurX
if (curX1 lt grid0.length
gridcurYcurX1 curValue - 1) curX
curX 1 // move to the east else //
cases for the other three directions robot.goto
(curY, curX)
Making the robot move to the goal location
76
Project 3 Adding Sonar

• Begin with an occupancy grid in which all cells
  are marked free
• Dynamically recompute the route as the robot
  encounters obstacles during its travel

40 khz sonar sensor from Mindsensors.com
77
Sonar Basics

• Sonar sensors are often used in robots for
  obstacle avoidance, navigation and map building
• Sonar sensors work by
• emitting a short burst of ultrasonic sound (often
  40 khz)
• sensing reflected signals (if any)
• computing object distance by using the elapsed
  time

78
Basic Solution Approach

• Initialisation
• Perform the wavefront labelling routine using an
  occupancy grid with all cells marked free.
• Do
• Determine a direction to travel.
• Orient robot in that direction.
• Use sonar to check if next cell in that direction
  is occupied.
• If it is, update occupancy grid and rerun
  labelling routine.
• If it is not, move to that cell.
• while (goal not reached and the cell label not
  -1)

79
(No Transcript)
80
Hands-on Time!

• Exercise 1 Try out the Random Walk program.
  Then modify it so during the random walk phase
  the robot never turns around and goes back to the
  spot it just occupied, e.g., never goes south
  after just having gone north.
• Exercise 2 Try out the Wavefront program. Then
  modify it so that the robot randomly chooses
  among the next legal locations during its travel.
  That way it should travel different routes each
  time it moves from the starting point to the
  finish.

81
A Glimpse at Lego MindStorms NXT

• LMICSE Workshop at SIGCSE 2006
• March 4, 2006

82
The Future of Lego MindStorms

• Lego MindStorms NXT
• Announced January, 2006
• Will start shipping August, 2006

83
The NXT Intelligent Brick

• Dual processor unit with more memory
• 32 bit processor
• 256 Kbytes FLASH
• 64 Kbytes RAM
• 8 bit processor
• 4 Kbytes FLASH
• 512 Byte RAM
• 4 input ports / 3 output ports
• Matrix display
• Real sound speaker
• USB 2.0 and Bluetooth

84
Motors and Sensors

• 3 Interactive Servo Motors
• Rumored to have 1 degree sensitivity
• Ultrasonic Sensor
• Sound Sensor
• Improved Touch Sensor
• Improved Light Sensor
• 6-wire cords

85
Supplied Programming Software

• Icon-based drag and drop program development
  environment
• PC and Mac compatible
• Based on LabVIEW from National Instruments

86
Need to Know More about NXT?

• Check out the Lego site
• http//mindstorms.lego.com
• Read what the bloggers are saying
• http//www.bnxt.com/community/news/

87
Thank you for Attending Our Workshop!

• Acknowledgements
• This work was partially supported by a United
  States National Science Foundation CCLI grant
  DUE-0126494
• Thanks to the numerous students have worked on
  the LMICSE project
• Please consider attending our upcoming 3-4 day
  workshops
• This summer at Villanova University
• This fall at the University of Mississippi