Applications. Environments.

1
Applications.Environments.

• EEL 5937 Multi Agent Systems
• Lotzi Bölöni

2
Applications
3
Electronic commerce

• User interface agents on e-commerce sites
• Bidding agents
• Controversy
• Acting on behalf of the customer?
• On behalf of the seller?
• No mechanisms for guaranteeing loyalty of agents

4
Spacecraft control

• The ground crew is usually required to track the
  spacecrafts progress and decide how to deal with
  eventualities
• Remember the control room for the Apollo moon
  missions?
• Expensive
• Very long reaction time if the spacecraft is far
• NASA is investigating making the probes more
  autonomous
• Example Deep Space 1

5
Virtual communities

• Agents as placeholders for user
• Temporarily
• Permanently
• Avatars
• Virtual corporations
• Consumer management agents
• Economic recession (the burst of the internet
  bubble) slowed down innovation in 2001-2004
• It is picking up again, as companies see agents
  as an alternative to outsourcing.
• Massively multiplayer role playing games
• E.g. Everquest, Asherons Call etc.
• As many times, gaming community is in the
  technical forefront.
• We will see many of these ideas going into the
  virtual corporations

6
Grid computing

• New paradigm for distributed computing
• Similar to the electric grid
• Providers and consumers. Contracts, requests,
  resource management
• Popular in the scientific computing community
• Has the potential to become a widespread approach
• If we manage the complexity of use
• Managing a long running computation
• Reacting to events (e.g. hardware failure)
• Computation steering
• Reconfiguring systems for changing priorities

7
Sensor networks

• Sensors
• Small hardware devices with sensing and wireless
  capabilities (e.g. motes).
• Agents which collect data, communicate to pass
  them to a sink
• In-network processing (data fusion, data
  correlation possible)
• Negotiation
• Active vs. passive sensors
• Who covers which target
• Newest trend sensor/actuator networks
• Sensors limited power, capabilities, fixed
  mostly sensing
• Actuators more power, may be mobile act on the
  environment

8
And many others

• Military applications
• Health care
• Intelligent home, intelligent car
• Dynamic spectrum management
• Negotiate spectrum resource allocations
• maybe you will propose the next one?

9
Conclusion

• Many interesting research issues
• Many interesting business opportunities
• Agent technology image hurt by many marketing
  pushes without significant technology behind them
• But real advances were made

10
Environments, events, actions
11
Environments, events and actions

• The agents live in an environment
• The operating system
• The internet
• The world of Quake / the world of Everquest
• For an AIBO robot the home of the owner
• The battlefield
• The environment is usually not fixed. It is
  changing through events and actions.
• Events changes in the environment, for which we
  do not know the source.
• Actions changes in the environment whose source
  is the agent or another known entity
• We usually consider events and actions to be
  discrete in time and space.

12
Environments Accessible vs. inaccessible

• An accessible environment is one in which the
  agent can obtain complete, accurate, up-to-date
  information about the environments state.
• Most moderately complex environments (including
  for example, the everyday physical world and the
  Internet) are inaccessible.
• Accessible environments
• Allows a more formal treatment of the agents
• Sensing is not an issue
• Planning, combinatorical computations become
  predominant (e.g. chess!)
• Inaccessible environments
• Sensing should be a large part of the agents work
• Exact planning less important than short time
  reactions

13
Environments Deterministic vs. non-deterministic

• A deterministic environment is one in which any
  action has a single guaranteed effect. There is
  no uncertainty about the state that will result
  from performing an action.
• Operating in a non-deterministic environment
  means that we need to verify the result of our
  actions.
• Deterministic environment
• Single program computing environment with strong
  reservations and optimism
• Non-deterministic
• The physical world
• The internet
• Multi-agent systems

14
Environments Static vs. dynamic

• A static environment is one that can be assumed
  to remain unchanged except by the performance of
  actions by the agent
• A dynamic environment independent changes happen
  (events, actions of other agents).
• A physical world is a highly dynamic environment.
  
• Some computer environments can be made static,
  but the interesting ones are dynamic.

15
Environments Episodic vs. non-episodic

• In an episodic environment, the performance of an
  agent is dependent on a number of discrete
  episodes, with no link between the performance of
  the agent in different scenarios.
• Episodic environments are simpler, because the
  agent developer can ignore the long term history
  of the agent.
• Example
• A battery powered robot lives in a non-episodic
  environment.
• Everquest is non-episodic, Unreal Tournament is.

16
Environments Discrete vs. continuous

• An environment is discrete is there are a fixed
  finite number of entities and percepts in it
  (e.g. a chess game)
• Continuous no isolatable entities, events as
  analog change etc. (e.g. the ocean)
• Discussion
• Computers are usually perceived as discrete.
• Nature is usually perceived as continuous, at
  least at the macro level.
• We can usually isolate actions, because they are
  performed by the agent
• Isolating events is more difficult.

17
Physical Environment embodied agents

• Special case of the environment where the
  environment is the physical world or a simulation
  of it.
• Agents have a physical location (x,y) or (x,y,z).
  
• The environment has a geography.
• Actions and sensing are dependent of physical
  proximity.
• There is a continuity property you need to
  traverse locations to get from place to place.
• Communication can be helped or hindered by
  geography.
• The physical and virtual environment can coexist!

18
Virtual environment / Internet

• Locations are hosts. Multiple agents can occupy
  the same host.
• Mobility between hosts is possible the agents
  move directly from host to host.
• Communication capabilities are independent of the
  location.
• The actions of the agents are software actions.
• Communication with user interfaces the location
  of the user still matters!!!