SelfOrganisation in Agent Organisations

1
Self-Organisation in Agent Organisations

• Ramachandra Kota, Nicholas Gibbins
• and Nicholas R. Jennings
• School of Electronics and Computer Science
• University of Southampton, U.K.

2
Motivation

• Autonomic systems
• computing systems with self-management
• solution to the problem of maintaining large,
  complex computing systems? (Kephart and Chess,
  2003)?
• Self-organising multi-agent systems
• autonomous
• adaptive (continuous over time)
• decentralised and robust
• a paradigm to develop autonomic systems (Tesauro
  et al., 2004)?

3
Research Objective

• For problem-solving agent organisations with
• co-operative agents working together
• receive, allocate and execute dynamic stream of
  tasks
• representing a distributed computing system
• Develop a decentralised structural adaptation
  method
• local interactions gt used by every agent gt
  robust
• continuous gt keep improving gt optimising

4
Related Work

• Self-organisation by reactive agents
• Centralised mechanisms requiring specialised
  agents
• Organisation self-design (OSD) based spawning and
  merging agents
• Adaptation in agent networks (contd.)

5
More Related Work

• Adaptation in agent networks
• Does not work for models with multiple kinds of
  links (organisational relations)
• The links of an agent do not affect its own
  performance
• (no organisational load)
• Ignores computational load incurred for
  adaptation
• (meta-reasoning)
• Not designed to handle systems where agents join
  and leave over time (dynamic organisations)

6
Organisation Model

• A problem-solving agent ax
• provides a set of services Sx
• has a limited computational capacity Lx
• when assigned a service instance (SI) sij,
  either-
• allocates the SI to another agent, or
• executes the SI and then allocates the subsequent
  dependent SIs

si1 s1, p1
root SI
Example of a task
si3 s0, p3
si0 s0, p0
si2 s2, p2
si4 s3, p4
leaf SIs
7
Organisation Model Structure

• The relation between two agents determines-
• the knowledge held about each other
• the interactions permitted between them
• Three levels of relationships-
• Acquaintance no interaction
• Peer low frequency of interaction
• Authority (superior-subordinate) high frequency
  of interaction

aX
az
aY
aW
Example of an organisation structure
8
Organisation at work an example
aX
si1 s1, p1
root SI
si3 s0, p3
az
aY
si0 s0, p0
si2 s2, p2
si4 s3, p4
leaf SIs
aW
Task
Organisation
9
Organisation Model Performance Evaluation

• costORG depends on communication messages passed
• Agent capacity Lx consumed by load lx as-
• lx ? pi M. ? mj,x R.rx
• load execution allocation reorganisation
• load due to allocation (mj,x) depends on
  structure
• rewardORG depends on speed of task completion
• profitORG rewardORG - costORG

sii ? WxE
sij ? WxF
10
Adaptation example scenario
aX
si1 s1, p1
root SI
az
si3 s0, p3
aY
si0 s0, p0
si2 s2, p2
si4 s3, p4
leaf SIs
aW
Task
Organisation
11
Adaptation state transitions
12
Adaptation Method

• Formulated using the decision theoretic approach
• Changing the relation denoted as actions

Peers
Subordinate
Subordinate
Peers
Subordinate
Just acquaintances
Just acquaintances

• Pairs of agents jointly estimate the expected
  utility (to the organisation) of changing their
  relation
• Calculated for every possible action from the
  state and the action with best expected utility
  chosen

13
Expected Utility Calculation

• Expected utility calculation based on the history
  of interactions and delegations involving the two
  agents
• Using a Value function of the form
• Vx,y ?loadx ?loady ?loadOA ?costcomm
  ?costreorg

14
Attribute values form_subr(x,y) action

• ?Loadx - M Asgx,total filledx(ttotal) /
  ttotal
• M management coefficient
• Asgx,total total number of SIs allocated by ax
• ttotal total number of time-steps of existence
  of ax
• filledx(ttotal) number of time-steps within
  ttotal that axs capacity was filled with load

x
x
x
x
x
The attribute values are calculated on basis of
past interactions and delegations involving the
two agents
15
Meta-Reasoning

• Reasoning about reorganisation with another agent
  needs computation (lx ...... R.rx)
• R computation required for reasoning with
  another agent
• rx number of acquaintances that ax reasoned with
• ax should chose k acquaintances for reasoning
  in a time-step
• what is the value of k?
• which k acquaintances?

16
Meta-Reasoning contd.

• Reduced to coupon collectors problem
• Randomised approach
• E (X) n ln(n) O(n)
• can chose the k acquaintances randomly
• Value of k decided at a time-step t as
• kt max1, (Lx - lx)/R, acqtsxchangedx,t-1/kt-1
  

17
Dynamic Organisations

• Agents join and leave the organisation over time
• Forming relation with a new agent
• no history of interactions gt no data for
  expected utility
• explore versus exploit trade-off

18
Dynamic Organisations contd.

• Same principle as in WOLF (Win Or Learn Fast)
• agent with unused capacity gt winning gt ignores
  new agents
• agents with waiting queue of SIs gt losing gt
  actively seek subordinates
• searches amongst the new incoming agents for
  those providing in-demand services and forms a
  superior-subordinate relation

19
Empirical Evaluation

• Simulation Parameters
• Heterogeneity of the agents
• Similarity of the tasks
• Comparison Methods
• Central an omniscient external centralised
  allocator performs the allocations ? maximum
  profit obtained
• Random agents randomly reorganise their relations

20
Results Static Orgs

• free-Adapt no reorganisation load (R 0)
• all-Adapt k is always equal to total
  acquaintances

21
Results Static Orgs
Experiments by varying the reorganisation
coefficient R
22
Results Dynamic Orgs

• free-Adapt no reorganisation load (R 0)
• nowolf-Adapt no WOLF component in the algorithm

23
Results Dynamic Orgs
A sample scenario
24
Future Work

• Organisations where the agents internal
  characteristics change over time
• Environments where the kind of similarity present
  within tasks changes over time
• Change proactively when the distribution of
  future tasks is known
• Explore a principled Reinforcement Learning
  approach for adaptation

25
Thank you!!

• ??