Automated Negotiation for Grid Notification Services

1
Automated Negotiation for Grid Notification
Services

• Richard Lawley, Keith Decker, Michael Luck, Terry
  Payne and Luc Moreau
• Intelligence, Agents, Multimedia Group
• School of Electronics Computer Science
• University of Southampton
• ral01r_at_ecs.soton.ac.uk

2
Outline

• Introduction
• Notification Services
• Negotiation
• Negotiation Engine
• Evaluation of Negotiation Engine
• Future Work Conclusion

3
Introduction

• Providers need to inform clients of
• Changes to database content
• New releases of tools or services
• State changes in long-running jobs
• Message-Oriented Middleware layer
• Provider notifies middleware layer
• Middleware layer informs clients
• e.g. MQ Series, JMS, MSMQ

4
Publish-Subscribe Model

• Information is published on a topic
• Consumers subscribe to an information topic
• Publishers publish information on a topic
• Updates are sent to subscribers

S1
NS
P1
S2
Publisher
Notification Service
Subscriber
5
Notification Services

• Message-Oriented Middleware layer using
  Publish-Subscribe model
• Responsible for asynchronous delivery of messages
  between publishers and subscribers
• Can Filter Combine Notifications allowing
  selective notifications and regular digest
  delivery

6
Notification Service Examples

• Grid Monitoring Architecture
• (Tierney, Aydt, Gunter, Smith, Swany, Taylor and
  Wolski 2002)
• Grid Notification Framework
• (Gullapalli, Czajkowski and Kesselman 2001)
• OGSA Notification Framework
• (JaiPaul 2003)
• NaradaBrokering
• (Fox and Pallickara 2001)
• myGrid

7
myGrid

• e-Science project aimed at Bioinformatics
• Workflow-based in-silico experiments
• Workflow specified how an experiment should be
  composed
• Experiments can be long-running
• Notification Service communicates results
• NS also used between services

8
myGrid - Bioinformatics

• Bioinformatics is data-intensive
• SWISS-PROT Protein Knowledgebase
• Over past 14 months has grown by 20, with
  approximately 200 changes per day
• ? Constantly expanding
• Query to find new annotations or sequences that
  are similar to existing sequences

9
Notification Services - Problems

• Providers and Consumers can specify QoS
  constraints
• Notification frequency
• Price paid
• Mismatch in preferences of provider consumer
• Consumer wants updates immediately and for free
• Producer wants to publish regular updates and
  charge
• Notification Services do not address this problem
• Negotiation can find mutually acceptable values

10
Negotiation

• The process by which two or more parties
  exchange proposals in order to reach a mutually
  acceptable agreement on a particular matter
• Components of negotiation
• Item the object under negotiation
• Proposals set of values suggested by one party
• Preferences values acceptable to a party
• Utility functions a method of evaluating the
  utility of a particular proposal

11
Negotiation

• Negotiation Protocols
• Set of rules governing a negotiation
• Types of participants
• Valid negotiation states actions
• Negotiation Strategies
• How a participant will behave (within protocol)
• How to generate proposals
• When to accept proposals

12
Negotiation

• Cooperative Negotiation
• Preferences and utility functions shared
• Social Welfare (benefit to all parties) is
  primary objective
• Competitive Negotiation
• Private preference and utility functions
• Personal benefit is primary objective
• Competitive Negotiation chosen for Notification
  Service
• Hard to share utility functions preferences if
  they depend on locally-sensed environmental
  factors
• Notification Service may be used in competitive
  environment, e.g. service providers competing for
  business

13
Negotiation Engine Definitions

• Negotiations are between requester and requestee
• Conditions negotiated over are negotiation terms
• A proposal contains values for negotiation terms
• Conversation between requester requestee is a
  Negotiation Thread
• Number of proposals exchanged is the Negotiation
  Thread Length
• Preferences specify acceptable ranges
• Ideal value sensible ideal world value
• Reservation value limit of acceptability

14
Negotiation Process
15
Negotiation Model

• Bilateral Service-Oriented Negotiation Model
• Based on Negotiation Decision Functions
• (Faratin, Sierra Jennings 1998)
• Automated negotiation using external factors in
  proposal generation/evaluation
• Model only handles negotiation protocol
• External program provides inputs to proposal
  generation evaluation process

16
Negotiation Model

• Proposals generated by tactics and strategies
• Tactics function evaluating a single
  negotiation term
• Time-dependant
• Resource-dependant
• Imitative
• Strategies way of weighting different tactics
• Can vary over course of negotiation
• Favour time-dependant tactics towards deadline
• Proposal utility weighted summation of utility
  of each term in proposal
• Proposal accepted when U(Pincoming) gt
  U(Pgenerated)

17
Evaluation of Negotiation Engine

• Evaluate suitability for Notification Service
• Experiment setup based on original evaluation
• Variables grouped into environments
• Preferences and Deadlines
• Repeatedly running in same environment yield
  identical results
• Range of environments generated and used for all
  of the experiments
• Single tactic used for each party
• All tactics played against each other

18
Exp1 Variable Deadline

• What affect does varying the deadline have on
  utility and success rate?
• Deadlines measured in number of messages
• No communication overhead in test setup
• Vary deadline (1-100 messages). Measure utility
  and success rate.
• Hypothesis Utility and success rate will be
  poor with shorter deadlines.

19
Exp1 Variable Deadline Results

• Higher utilities achieved with longer deadlines
• Approaches optimal utility
• Higher success rates with longer deadlines
• Moves towards 100 success rate

20
Exp1 Variable Deadline - Results

• Plot time used against utility achieved
• Utility increases as time used decreases
• Explained by plotting overlap of prefs (F)
  against messages

21
Exp2 Multiple Terms

• Negotiations typically over many terms
• e.g. price, duration, frequency
• Is Utility affected by number of terms?
• Number of terms varied between 1 and 25
• All terms weighted equally
• Deadline varied between 30 and 60 (random)
• Record utilities and execution time (CPU time)

22
Multiple Negotiation Terms - Results

• As number of terms increases, utility remains
  fairly constant
• Only valid when all terms weighted equally
• Execution time is linearly related to the number
  of terms

23
Exp3 Execution Time

• Is execution time linearly related to number of
  messages?
• Deadlines varied between 30 and 60
• Minimum, average (over 100 runs) and maximum
  execution times measured for each deadline
• Communication delays are ignored as in test setup
  components are coupled directly together

24
Exp3 Execution Time - Results

• Minimum and mean execution times linearly related
  to number of messages
• Maximum execution time has unpredictable peaks
• Garbage collection?
• Only in minority of cases as mean is close to
  minimum

25
Notification Service Example

• Example from Bioinformatics field
• SWISS-PROT Protein Knowledgebase
• Over past 14 months has grown by 20, with
  approximately 200 changes per day
• ? Constantly expanding
• Query to find new annotations or sequences that
  are similar to existing sequences

26
Notification Service Example

• Assumptions
• 1000 Subscribers, each with 100 sequences they
  want matched
• Searches can be run iteratively to refine results
• Subscriber would like to be notified of changes
  every 5 hours

27
Notification Service Example

• By accepting subscriber requests, 5,000,000
  searches per day required
• Use negotiation over two terms
• Frequency (time between notifications)
• Provider preference 168-24 hours
• Subscriber preference 5-120 hours
• Number of iterations
• Provider preference 1-3
• Subscriber preference 5-1

28
Notification Service Example

• Provider preferences kept constant
• Consumer preferences vary by a small random
  amount
• Deadlines between 30 and 60 messages
• After running negotiations
• Average Frequency 67.6 hours
• Average Iterations 2.31
• ? 651,000 searches per day (87 reduction)

29
Notification Service Example

• Number of queries required for each outcome
  plotted against provider utility
• Shows how utility decreases as the number of
  queries increases

30
Notification Service Example

• Greater utility is represented by needing to do
  fewer queries for a provider
• Using Negotiation allows the provider to increase
  their utility and still satisfy needs of consumer
• Introducing Negotiation may reduce consumer
  utility, but would allow more consumers to be
  serviced

31
Future Work

• Peer-to-peer Notification Services
• Notification Services talking to Notification
  Services
• Consumers subscribing to single service,
  receiving updates from elsewhere

NS1
NS2
P2
S1
Publisher
Notification Service
S2
NS3
P1
Subscriber
NS4
32
Future Work

• Support negotiations for peer-to-peer
  notification service by supporting chained
  negotiation
• Negotiations with intermediaries
• Negotiations where intermediaries may be able to
  satisfy request with existing commitments
• Integrate with myGrid Notification Service

33
Conclusion

• Negotiation in a Notification Service allows
  differences in QoS preferences to be resolved
• Negotiating over QoS allows a provider to support
  more clients

34
Questions?