Multi-Agent Systems:

1
Multi-Agent Systems

• The Tao of Soccer

A tutorial presented at SFU Surrey Yu
Zhang March 8th, 2005
2
Overview

• Multi-Agent Systems.
• Intelligent Agents, agent designs
• multi-agent systems.
• The Tao of Soccer.
• Background
• Client-Server Architecture
• GUI client and Java3D view
• TOS AI programming guide
• TOS Demo
• The End
• Questions Answers

3
What is an agent?

• An agent is an entity that perceives its
  environment through sensors and acts upon that
  environment through effectors.1

• For each possible percept sequence, an ideal
  rational agent should do whatever action is
  expected to maximize its performance measure, on
  the basis of the evidence provided by the percept
  sequence and whatever built-in knowledge the
  agent has.1

4
Design Intelligent Agents

• The job of AI is to design the agent program that
  implements the agent mapping from percepts to
  actions.1
• In order to design an agent program, we need to
  know the possible percepts and actions, what
  goals or performance measure that agent is
  supposed to achieve, and what sort of environment
  it will operate in.
• We will consider three basic types of intelligent
  agents Simple reflex agents, agents that keep
  track of the world and goal-based agents.

5
Simple reflex agents

• A simple reflex agent1 stores a set of
  condition-action rules. For a TOS soccer player,
  we can have rules like
• If have-ball and close-to-opponent-goal then
  shoot
• The simple reflex agent works well only if the
  correct decision can be made on the basis of the
  current percept.
• Problem But certain states can not be determined
  only by the current percept.
• A soccer player is dribbling the ball.

6
Agents that keep track of the world

• This type of agents maintains some internal state
  information in order to better determine world
  states and conditions.1
• The agent program also maintains two more types
  of knowledge.
• How the world evolves independently of the agent.
• Hows the agents own actions affect the world.
• Problem Reflex agents are not flexible. The
  rules has to be re-written to adapt to a new goal
  or a new environment.

7
Goal-based agents

• Goal-based agents1 store goal information and
  the information about the results of possible
  actions in order to choose actions that achieve
  the goal.
• Goal-based agents reasons about the future.

• How to set goals?
• The designer manually set the goals for the
  agent.
• The agent sets its own goals by defining a
  utility function.
• Utility is a function that maps a state to a
  real number, which represents the associated
  degree of happiness.

8
Layered agent architecture

• A layered agent architecture is modeled as a
  hierarchy of interactive sub agent modules.3
• The higher layers are usually event-driven,
  goal-based.
• The lower layers are fixed-sample-time driven,
  simple-reflex-based.
• Each layer is a sub-agent with its own inputs and
  outputs.
• Inputs controls signals from the higher layer,
  the sensing signals from the environment and the
  states from the lower layer.
• Outputs controls signals to the lower layer and
  the states to the higher layer.
• Control commands flow down and the sensing
  information flow up.

9
Multi-agent systems

• A multi-agent system is a group of agents that
  interact to solve problems that are beyond the
  individual capabilities.2
• Multi-agent system design is more complicated
  than a single agent design. It presents many new
  cooperation issues.
• A multi-agent system solves the big problem by
  divide and conquer. This presents us
• Cooperation issue 1 - Task allocation how to
  assign responsibility to a single agent?
• Solution each agent is assigned a role within
  the team with its own behavior expectations and
  goals.
• Now agents may have conflicting knowledge,
  actions and goals. This presents us
• Cooperation issue 2 Resolving Conflicts How to
  resolve conflicting knowledge, actions and goals
  among agents.
• Solution assign authority levels to roles,
  communication and negotiation among agents.
• Now agents need to communicate, this presents us
• Cooperation issue 3 Communication How to
  understand each other?
• Solution develop a language and a communication
  system that can accept and interpret this
  language.

10
The Tao of Soccer
Tao
direction way method road path principle
truth reason skill method Tao (of Taoism) a
measure word to say to speak to talk
11
Background and History

• 1992 Prof. Alan Mackworth proposed robotic
  soccer as a test bed for multi-agent system
  research and started the world's first
  soccer-playing robots dynamite.
• 1997 The first RoboCup was held during the
  International Joint Conference on Artificial
  Intelligence (IJCAI), in Nagoya, Japan.
• Soccer Server is the official simulator for the
  RoboCup Simulation League.
• Written in C, complicated client/server
  protocol, no human playing ability, only runs on
  UNIX type machines.
• 2001 The Tao of Soccer was created.
• Written in pure Java, similar to RoboCup but with
  simplified client/server protocol, and it can be
  played by human as well.

12
TOS a soccer environment simulator

• For each simulation step, TOS calculates and
  gives each agent its percept, gets an action from
  each agent, and then updates the environment.
• TOS Simulator has five main classes.
• SoccerWorld represents the state of TOS
  environment. A soccer field, two teams of
  players, one group of viewers and a ball.
• Host a thread waits for connection requests and
  actions from agents.
• HeartOfWorld a thread calculates and gives each
  agent its percept, updates the SoccerWorld at
  every simulation step.
• SoccerRules simulates and calculates the soccer
  rules, such as off-side, goal scores.
• SoccerPhysics simulates the movement of player
  and the ball.

13
Soccer Physics simulation

• Soccer field is 2D rectangular, 100 x 65 meters.
  The center of the soccer field is set to (0,0), Y
  goes up, X goes right.
• The players and the ball are treated as circles.
  Movements of these objects are simulated stepwise
  for every 50 milliseconds. (24steps/sec)
• At each simulation step, movement of each player
  is calculated in the following manner
• P1 P0 V0
• V1 V0 A0
• A1 FORCE K1 - V0 K2
• movement of the ball is calculated as
• P1 P0 V0
• V1 V0 A0
• If (kicked by a player)
• A1 KICKFORCE K1 V1 0
• else A1 -FRICTIONFACTOR V0
• Collisions When more than two players overlap,
  all of them are moved back until they do not
  overlap. Then their velocities are multiplied by
  -0.1.

14
Soccer Rules simulation

• Once the server starts, it enters a 4-period
  match cycle repeatedly until it is turned off.
• Pre Game The referee is not activated. No score
  is recorded.
• First Half The game starts. The referee is
  activated to enforce the soccer rule.
• Half Time The referee is deactivated again.
• Second Half The game restarts. The referee is
  reactivated to enforce the soccer rule.
• A kick-off is a way of starting or restarting
  play at the start of the match, or after a goal
  has been scored or at the start of the second
  half of the match. When the kick-off mode is on,
  the opponents of the team taking the kick-off are
  at least 9 meters from the ball until the ball is
  touched by an opponent player.
• Other implemented soccer rules are
• Goal kick
• Corner kick
• Throw in
• Offside, the offside rule can be turned off by a
  command-line switch

15
TOS client-server architecture

• TOS runs on internet using the client-server
  architecture.
• TOS simulator runs as the server to provide
  percepts to the agents and receives actions from
  them on the internet.
• TOS agents run as the clients to receives
  percepts from the server and send actions to the
  server on the internet.
• Percepts and actions are encoded as LISP-like
  text messages, packed and transmitted as UDP/IP
  packets between the server and client.

• A client can be implemented in any language.
• A Java client can use built-in API
  soccer.common to communicate with the server.
• There are two types of TOS clients (agents),
  Player and Viewer.

16
Client to Server Connect

• First, a client has to join the game by sending a
  connection message to the server.
• (c ClientType SideType). For example (c p l)
• To do this in Java, we use API in soccer.common
• // step 0 setup network communication, usually
  only done once
• Transceiver transceiver new Transceiver( false
  )
• // step 1 fill the data
• ConnectData connect new ConnectData(ConnectData.
  PLAYER,ConnectData.LEFT)
• // step 2 create the UDP packet by providing
  data and server address
• Packet packet new Packet( Packet.CONNECT,
  connect, address, port)
• // step 3 send the packet
• transceiver.send( packet )
• What if the connection message got lost during
  the transmission? Or the team is full?
• Solution the server sends back an acknowledge
  packet.

17
Server to Client Initialize

• The sever responds the connection request from
  the client by sending an initialization message
  back.
• (i ClientType Number)
• ClientType can be one of l, r, v or f.l means
  left, r means right, v means viewer and f means
  full. In the case of left or right, Number is
  returned as the player's player number. full
  means connection fails because of the full team.
• To make the connection, the client can send a
  connection packet every 60 seconds until
• it receives an initialization packet from the
  server, which means a successful connection.
• or it reaches the maximum number of tries, and
  fails the connection.

18
Server to Client Player Percepts

• After the client-server connection is
  established, the server begins to send visual
  information to the client.
• If the client is a player, it gets see packet
• (s Time Side Id X Y Dir Status BallX BallY
  BallControllerType BallControllerId PlayerInfo
  PlayerInfo ...)
• PlayerInfo (PlayerSide PlayerId X Y Dir)
• If the client is a viewer, it gets view packet
• (v Time BallX BallY BallControllerType
  BallControllerId PlayerInfo PlayerInfo ...)
• The difference between see and view packet is
  view packet does not have visual data for viewer
  himself.
• We use soccer.common API to process visual data.
• packet transceiver.receive()
• if(packet.packetType Packet.SEE) see
  (SeeData) packet.data
• distance2Ball see.player.position.distanc
  e( see.ball.position )
• direction2Ball see.player.position.direction(
  see.ball.position )

19
Server to Client Player Percepts

• The server also sends referee information to the
  client whenever the situation on the field is
  changed.
• (r Time Period Mode LeftName LeftGoal RightName
  RightGoal)
• Time simulation step
• Period 0 1 2 3 0 means preGame, 1
  means firstHalf, 2 means halfTime, 3 means
  secondHalf.
• Mode 0 1 2 3 4 5 6 7 8 9
  10 110 means beforeKickOff, 1 means kickOffL,
  2 means kickOffR, 3 means throwInL, 4 means
  throwInR, 5 means cornerKickL, 6 means
  cornerKickR, 7 means goalKickL, 8 means
  goalKickR, 9 means offsideL, 10 means offsideR,
  11 means playOn.
• We use soccer.common API to process referee data.
• packet transceiver.receive()
• if(packet.packetType Packet.REFEREE)
  referee (RefereeData)packet.data
• gamemode referee.mode
• gamePeriod referee.period

20
Client to Server Player Actions

• A player can move around by sending the drive
  command to the server.
• (d Dir Force)
• -180 lt Dir lt 180
• -30 lt Force lt 100
• To do this in Java, we use API in soccer.common
• // step 0 setup network communication, usually
  only done once
• Transceiver transceiver new Transceiver( false
  )
• // step 1 fill the data
• DriveData driver new DriveData( direction,
  force )
• // step 2 create the UDP packet
• Packet packet new Packet( Packet.DRIVE, driver,
  address, port)
• // step 3 send the packet
• transceiver.send( packet )

21
Client to Server Player Actions

• If the player is close enough to the ball, he can
  kick it by sending the kick command to the
  server.
• (k Dir Force)
• -180 lt Dir lt 180
• -30 lt Force lt 100
• To do this in Java, we use API in soccer.common
• // step 0 setup network communication, usually
  only done once
• Transceiver transceiver new Transceiver( false
  )
• // step 1 fill the data
• KickData kicker new KickData( direction, force
  )
• // step 2 create the UDP packet
• Packet packet new Packet( Packet.KICK, kicker,
  address, port)
• // step 3 send the packet
• transceiver.send( packet )

22
Client to Server Player Actions

• A player can broadcast a message to everybody on
  the field by sending the message to the server.
• (t Message)
• The length of the Message lt 30
• At most one Message every 30 second
• To do this in Java, we use API in soccer.common
• // step 0 setup network communication, usually
  only done once
• Transceiver transceiver new Transceiver( false
  )
• // step 1 fill the data
• TalkData talker new TalkData( Attack! Attack!
  Attack!)
• // step 2 create the UDP packet
• Packet packet new Packet( Packet.TALK, talker,
  address, port)
• // step 3 send the packet
• transceiver.send( packet )

23
Server to Client Player Percepts

• After the server receives a broadcast message
  from a client , it sends audio information to all
  clients in the game.
• (h Time Side Id Message)
• We use soccer.common API to process audio data.
• packet transceiver.receive()
• if(packet.packetType Packet.HEAR)
• heard (HearData) packet.data
• String message heard.message
• int speakerID heard.id
• int speakerSide heard.side

24
SoccerMaster a TOS GUI client

• SoccerMaster can be run in three modes.
• In the VIEW mode, it connects to the server as a
  viewer client.
• The user observes two AI teams play against each
  other.
• In the PLAY mode, it connects to the server as a
  player client.
• The user plays the game himself.
• In the REPLAY mode, it reads a log file and
  redisplays the match.
• The user reexamines a recorded match.

• The user can hot-switch Java2D and Java3D view
  any time, no application restart is needed.

25
TOS Java 3D View

• The java 3D view is implemented in package
  soccer.client.view.j3d.
• FieldJ3D is the main class for displaying the 3D
  scene. In this class, we add content nodes to the
  Java3D scene graph tree such as soccer players,
  the ball, the soccer field, the goal poles, etc.
• Robot contains the manually coded player 3D model
  using spheres and cylinders.
• MouseRotateXZ and MouseZoomOnRightClick are used
  to control the camera view.

Its very easy to add existing 3D models to TOS
3D scene. For example, to load a balloon into the
3D scene using Xj3D loader library
VRML97Loader modLoader new VRML97Loader() URL
modURL getClass().getResource("/model/balloon.wr
l") Scene model modLoader.load(modURL) BranchG
roup balloon model.getSceneGroup() scene.addChi
ld(balloon)
26
SFU Default AI Soccer Team

• In the package edu.sfu.soccer.agent, there are 4
  classes. AIPlayers, Formation, Robot, and
  WorldModel.
• AIPlayers is the main class, it maintains one or
  two teams of threads, each thread controls an
  independent TOS player.
• Robot contains the logic for controlling the TOS
  player.
• WorldModel contains the internal states, actions
  and team formation.
• Formation contains the team formation data.

• Each Robot is a goal-based agent, uses three
  main methods, sense(), plan() and execute().

27
AIPlayers the main class

• AIPlayers maintains one or two teams of Robots.
  Each team can have 0-11 players. The size of each
  team is given at the command line. For example
• To control 11 players of the left team.
• java -cp soccer.jar edu.sfu.soccer.agent.AIPlayer
  s -l 11 -r 0
• To control 11 players of the right team.
• java -cp soccer.jar edu.sfu.soccer.agent.AIPlayer
  s -l 0 -r 11
• Each player is run as a thread. Why do we use
  threads?
• To soccer server, each separate UDP socket
  represents a unique soccer client. Theres no
  difference between a process client and a thread
  client.
• A thread uses less computing resources than a
  process.
• One Java process can control 22 AI players. Its
  easier for us to use and to maintain the soccer
  AI.

28
WorldModel and Formation

• WorldModel is the memory bank for our soccer
  agent, it stores
• How the world evolves.
• 1, The static information about the soccer field,
  such as length and width.
• 2, The static information about the soccer
  physics simulation, such as the maximum speed of
  players and the ball.
• Internal states.
• 1, The low level percepts such as the positions
  of moving objects on the field.
• 2, The high level knowledge such as my relative
  distance and direction to the ball, if Im
  interested in getting the ball, if Im stuck,
  etc.
• Multi-agent. My current role in my team
  formation.
• The high level actions Im going to execute, such
  as shoot, move, chase, avoid stuck and avoid
  offside.
• Formation stores the formation information of a
  soccer team. A soccer team can have different
  style of formation, such as 433, 523 and
  424.

29
Robot a goal-based soccer agent

• Robot implements a goal-based agent.
• Sense() receives UDP packets from the server
  about its current percepts, updates its world
  model.
• Plan() determines the high level actions, such
  as shoot, dribble, move and chase.
• Execute() translates the high level actions into
  a sequence of low level actions that can be
  accepted by the soccer server, such as kick and
  drive.
• Plan() method calls a list of decision-making
  methods to determine the next high level action.
• ShouldIScore() determines if I should shoot the
  goal.
• ShouldIDribble() determines if I should dribble.
• ShouldIPass() determines if I should pass the
  ball to a teammate.
• DetermineWhereToMove() DeterminePlayerPos()
  determines where should I move when I dont have
  the ball

30
Create your own soccer team

• Approach 1 Make changes to the default SFU
  soccer team.
• Change the Formation class, experiment with
  different team formations.
• Improve the decision-making methods in the Robot
  class. These are ShouldIPass(), ShouldIScore(),
  ShouldIDribble(), DetermineWhereToMove() and
  DeterminePlayerPos() .
• Approach 2 Create your own AI team from scratch.
• If you have a great idea that is hard to
  implement in the default SFU team.
• If you want to experiment with some other agent
  architecture. One example of such alternative is
  Dynamo98 that uses constraint nets robotic
  architecture.
• Dynamo98 is in package com.graviton.dynamo98. You
  may also build your team on this AI
  implementation as well.
• Approach 3 Add new AI methods into the existing
  AI implementations.
• Include a rule-based engine, such as a Prolog or
  Fuzzy logic, to handle high level planning tasks,
  instead of hard-coding the decision-making
  methods.
• Add learning capabilities, such as neural
  networks, decision trees, reinforcement learning
  and genetic algorithms.

31
TOS DEMO Play TOS as a game

• Start the TOS server. Double click
  soccerserver.bat
• Start the TOS GUI. Double click soccer.bat
• Play the game with the coach ability on.
• Move the player around with the left mouse click.
• Open a DOS window, type sfu_team.bat both to
  start AI.
• Click space to chase the ball.
• When you have the ball, right click to kick the
  ball, left kick to dribble, and space to shoot.
• Step forward the game, continue, step forward.
• Forward the game period, play again.
• Toggle the player ID display.
• Toggle music.
• Toggle 3D display.
• Left click to rotate the 3D view, right click to
  zoom in/out the 3D view.
• Reset the top-down 3D view.
• Stop the game.

32
TOS DEMO TOS as a simulation

• Start the TOS server. Double click
  soccerserver.bat
• Open a DOS window, type dynamo98.bat both to
  start AI.
• Start the TOS GUI. Double click soccer.bat
• View the game with the coach ability on.
• Step forward the game, continue, step forward.
• Forward the game period, play again.
• Toggle the player ID display.
• Toggle music.
• Toggle 3D display.
• Left click to rotate the 3D view, right click to
  zoom in/out the 3D view.
• Reset the top-down 3D view.

33
TOS DEMO TOS Replay

• Start the TOS GUI. Double click soccer.bat
• Load the log file.
• Play the log file.
• Fast forward, normal, fast forward.
• Pause, play, pause, forward, pause, play
• Backward play, play, backward play, fast
  backward.
• Toggle 3D display.
• Left click to rotate the 3D view, right click to
  zoom in/out the 3D view.
• Reset the top-down 3D view.

34
The End and the beginning. How to get help and to
help others?

• Visit TOS website at http//soccer.sourceforge.net
  /soccer/
• Join the mailing list at http//games.groups.yahoo
  .com/group/tao_of_soccer/
• Post your questions and answers to the TOS forums
  or to the TOS mailing list.
• Make your AI team publicly available on the TOS
  web site.
• Join the TOS development.
• Find and fix bugs.
• Add new functions to the server and the GUI.
• Improve the 3D view.
• Make 3D models.
• Improve the documentation.
• Help to maintain the TOS website.
• Thank you and Good Luck!

35
References

• Artificial Intelligence, A Modern Approach. 1995,
  By Stuart Russell and Peter Norvig
• Multiagent Systems. 1998, By Katia Sycara. AI
  Magazine 19(2)
• A Constraint-Based Robotic Soccer Team. 2002, By
  Yu Zhang and Alan Mackworth. Constraints, 7,
  7-28.

36
Questions Answers