Trading Agent Competition

1
Trading Agent Competition
Lecture Series 18th May 2005
Presenter Minghua He
2
How the game is played?
http//www.sics.se/tac
3
Trading Agents Competition Classic Game

• Each game comprises 8 competing agents
• Each agent
• assembles travel packages
• Round trip flight
• Same hotel for all nights
• Some entertainment
• for 8 customers from 28 auctions
• Each customer has preferences on various aspects
  of trip
• score based on how well it satisfies requirements
  
• Game lasts 12 minutes
• auction server in (www.sics.se/tac)
• agents run on own local machines

4
(No Transcript)
5
The Auctions

• Flights
• 8 auctions
• (4 in, 4 out)
• Unlimited tickets
• Single seller auction
• Prices update randomly
• Clears continuously
• Closes at end of game

• Hotels
• 8 auctions
• (4 TT, 4 SS)
• 16 rooms per hotel per night
• 16th price multi-unit English auction
• Ask price is 16th highest price
• Updated immediately in response to new bids
• One closes at end of every minute after 4th

Entertainment 12 auctions Per entertainment,
per night Continuous double auction 12
tickets per agent initially Closes at end of
game
6
SouthamptonTAC Design
Environment Sensor

• Game features
• Inter-related bidding
• Uncertainty
• Trade-off in bidding
• Flight auctions
• Hotel auctions
• No optimal strategy for all situations

TAC
asks/ bids
Hotel price predictor
Bids preprocessor
Allocator
Flight categoriser
Hotel bid adjustor
Entertainment bid processor
bids
7
Classifying TAC Environments

• Three environment types
• Competitive High hotel prices 5 (6), 238
  (557), 155 (102), 40 (11)
• Non-competitive Low prices 7 (12), 92 (27), 70
  (53) 62 (7)
• Semi-competitive Medium prices 5(2), 128(71),
  128(60), 116(3)
• Set strategy according to assessment of
  environment
• Before game starts
• During game

8
TAC Environment Recognition
Non-Competitive Semi-Competitive
Competitive
Similarity degree
1
0 50 80 100
150 200 Price

• Reference price
• Before game Maximum of average price history of
  previous 10 games
• During game Current maximum price

9
Setting the Bidding Strategy
Competitive
Non-competitive

• Risk Seeking
• Buy large number of flights
• tickets at beginning
• Seldom change customers
• travel plans

• Risk Averse
• Buy some flight tickets
• at beginning
• Bidding based on
• games progress

SouthamptonTAC varies according to its perception
of the environment
10
Strategy per Auction Flights

• Risk seeking
• buy all flights once game starts
• Risk averse
• buy some flights at start of game
• buy remainder based on categorisation
• Fast changing immediately
• Slow changing in last minute

11
Strategy per Auction Entertainment
12
Strategy per Auction Hotels

• Off-line reasoning about demand
• Higher demand higher prices
• TT2 and TT3 in most demand
• Fuzzy reasoning to predict clearing prices
• ask price rate of change of that hotel auction
• counterpart hotel ask price rate of change of
  price

• Predicting clearing price of TT2 auction
• IF current ask price of TT2 is low and current
  ask price of SS2 is high
• THEN likely price increase is medium
• IF current ask price of TT2 is high and ask price
  increase of TT2 in quick
• THEN likely price increase is big

13
Evaluation By Competition

• TAC-01 (27 entrants)
• Third in final
• Highest average score and lowest standard
  deviation
• TAC-02 (26 entrants)
• Second in final (lt 0.8 difference with top
  agent)
• Both competitions (12 entrants)
• Most successful among four agents that qualified
• for final rounds of both tournaments

14
Evaluation By Controlled Experiments

• Two competitor agents
• Risk-seeking agent (RS-agent)
• based on behaviour of livingagents, UMBCTAC, and
  Walverine
• Risk-averse agent (RA-agent)
• based on SouthamptonTAC-01, Retsina, and sics
• Conjecture
• More RS agents in a game, more competitive it
  will be
• More RA agents in a game, less competitive it
  will be

15
Experiment Set-up
Number of RA-agents
Number of RS-agents
Non-Competitive (average clearing price 67)
Competitive (average clearing price 240)
16
Performance of SouthamptonTAC
OK in non-competitive games
Best in competitive games
Worst when many like itself
17
Performance of RS-agents
Best in non-competitive games
Poor in competitive games
18
Performance of RA-agents
Best in competitive games
OK in non-competitive games
19
Overall Findings

• Profits for all agent types increase with more
  RA-agents
• Since hotel prices kept low
• When few SouthamptonTAC agents (lt 4),
• SouthamptonTAC always outperform both RS-agents
  and RA-agents
• When more SouthamptonTAC agents,
• market less efficient
• Similar behaviour of switching between hotels
  cause moderate hotel prices and unused rooms

20
Value of Strategy Adaptation

• Experimental set-up
• One SouthamptonTAC
• One non-adaptive SouthamptonTAC
• cannot change its strategy once a game has
  started
• Remaining agents drawn randomly from pool of
  RS-agents and RA-agents
• Average score after 164 games
• SouthamptonTAC (3138)
• Non-adaptive SouthamptonTAC (2937)
• Being adaptive does improve performance!

21
Other Successful Agents
22
Rule changes from 2004

• Games are 9 minutes instead of 12
• First hotel auction close after 1 minute
• Changed flight price perturbation

23
Trading Agent Competition Supply Chain
Management Game

• Todays supply chains are mostly static, relying
  on fixed, long-term trading partners. Not optimal
• Dynamic management allows for finding better
  matches between suppliers and customers as market
  conditions change
• Planning and coordinating the activities of
  organisations across the supply chain
• Procurement of raw materials to finished goods
  deliver

24
(No Transcript)
25
Game overview

• Six personal computer assembly agents compete for
  customer orders and for procurement various
  components of a period of several months.
• At the end, the agent with the highest sum of
  money in the bank is the winner
• Spending buying components paying storage,
  paying penalties, paying overdraft penalties if
  in debt to the bank
• Earning selling PCs, receiving interest from
  bank if balance is positive

26
Components

• PCs are built from 4 component
• CPUs, motherboards, memory, and hard disks

27
Bank

• All agents begin with a zero bank balance
• Money is added when an order is shipped to a
  customer and deducted when components are
  received from suppliers or a delivery penalty is
  incurred
• At the start of each game a fixed interest rate
  is chosen uniformly in the interval 10-20
• Interest rate is per year (365 days) and is
  applied at the end of each game day

28
Supplier Model

• Each supplier collects all RFQs from the agents
  and processes them at the end of the day
• RFQ response comes in the form of an offer
• ltOffer-id, RFQ-id, Quantity, DueDate, Pricegt
• If the order cannot be filled in its entirety
  then two amended offers are sent partial offer
  and earliest complete order
• The agent then decided which offer, if any, to
  accept
• Price offered by a supplier is based on
• Available production capacity
• Quantity agents ask for

Price increases as capacity decreases or quantity
required increases
29
Customers

• Request PCs of different types to be delivered on
  a certain due date, e.g., the following RFQ
• ltPC, Quantity, ReservePrice, Penalty,
  DueDategt
• The quantity of each order is chosen uniformly
  between 1, 20
• Agents must bid to satisfy the entire order (both
  quantity and due date) for the bid to be accepted
• Customer collects all the bids and selects the
  bid with the lowest prices as the winning bid

30
Agent

• What do agents do each day?
• Negotiate supply contracts
• Bid for customer orders
• Menage daily assembly lines
• Each agent get
• RFQs and orders won from customers
• Quotes / offers and delivery of parts from
  component suppliers
• Bank account statement
• PC and component inventory report from the factory

31
Agent

• Each day (lasting 15 seconds) agent plans
• Which customer RFQs to bid on
• Which components to request
• Which supplier offers to accept
• What PCs to assemble in the factory (limited
  capacity of the assembly lines 2000 cycles per
  day)
• Which assembled PCs to ship to which customers

32
(No Transcript)
33
SouthamptonSCM Overview

• Component agent which RFQs and orders to send
  to which supplier
• Customer agent receives RFQs from customers and
  decides what offers to respond with
• Factory agent receives supplies delivered from
  the suppliers and schedules the production
  and delivery

34
Component Agent

• Challenge buy what it needs in time at a low
  price
• Solution
• Day-0 procurement all the suppliers have their
  full capacity available, thus at its lowest price
• Make a tradeoff between placing a big order on
  Day 0 and buying gradually during the rest of the
  game
• The quantity of Day-0 procurement should be the
  quantity of components needed in a low demand
  game
• During the game, predict how many more components
  needed. If need buy more, keep checking component
  price and buy components if the price is low

35
Customer Agent

• Challenge to determine which customer RFQs to
  bid for and at what price?
• Our solution
• Inventory driven methods to choose RFQs
• Soft computing techniques to calculate the price
• Heuristic method to adapt offer prices

36
Choosing RFQs and setting prices
Pricing strategy on day d

• List RFQs in decreasing order of (pres-cpenalty
  /q)
• Update the production capacity Ck of day k
• offeredCycles 0
• reservedCyclesk 0
• Calculate the reference price for each kind of PC
  piref
• For each RFQ in the list
• Poffer max piref ? (1f (ddue)), pibase
• if PC inventory q then
• offer q PCs at Poffer
• decrease PC inventory by q
• else if component inventory q and
• reservedCyclesddue-2q? ci
  Cddue-2 ? ? then
• offer q PCs at Poffer
• increase offeredCycles by q ? ci
• decrease reservedcyclesddue-2 by q ? ci
• decrease component inventory accordingly
• else do not offer PCs to this customer

37
Calculating Reference price

• Reference price a reasonable current market
  price for PC type i
• Piref Pilow (Pihigh - Pilow ) r
• Pilow is the lowest transaction price of type i
  on the previous day
• Pihigh is the highest transaction price of type i
  on the previous day
• r ? 0.4, 1.2 is an adjustment factor which
  determines how far away the reference price is
  from the lowest price
• r is set through the fuzzy mechanism and is
  adapted according to the quantity of orders
  received and expected

38
Fuzzy reasoning

• Sugeno controller
• Factors considered in the rules
• Demand in the market
• Component inventory level
• How far into the game
• Example
• if demand is high and inventory is high and
  end-of-game is far then r is big

39
Adaptation of offer prices

• Basic idea
• Agent can only use 2000 production cycles each
  day
• To maximise throughput, the number of production
  cycles to produce the received customer orders
  should also be 2000
• However, we cannot just base on 2000, some of
  the production cycles may be reserved by earlier
  customer orders

Price adaptation strategy

• Calculate receivedCycles
• expectedCycles min2000, offeredCycles ? ?
• if receivedCycles lt expectedCycles then r r-?
• else receivedCycles gt expectedCycles then r r?

40
Factory agent

• Challenge scheduling what to produce and when to
  produce it?
• Our solution
• Manufacturing PCs according to customer orders
• Satisfying orders with a earlier delivery date
• Deliver customer order as soon as it is filled
• If there is factory assembling cycles left and
  number of finished PCs is within a threshold,
  produce additional PC even without customer
  orders
• (very useful at the end stage of the game)

41
Production schedule for day d

• List the orders with due date d2 in list 1
• List late orders (but still valid) in the
  decreasing order of the due date into list 2
• List the future orders in the increasing order of
  the due date into list 3
• Append list 2 to list 1 and list 3 to list 2
• For each order in the combined list
• If computers in the inventory can fill the order
  then deliver the computers
• Else if components are available and factory
  capacity is not full then produce more PCs to
  fill the order
• If there is extra factory capacity left and
  enough components, then check whether additional
  PCs should be produced

42
Evaluation By Competition

• TAC / SCM -04 (29 entrants)
• Seeding round third
• Quarter-final second
• Semi-final first
• Final sixth (our agent was adversely affected by
  the fact that several agents sent RFQs on Day 0
  for huge quantities of components)

43
Evaluation by competition game analysis

• Evaluate the pricing model
• Compare the offering price for the RFQs that the
  agents respond to in a same game
• Interested in the games with good agents (pick
  games from semi-finals)

44
Comparison of daily offer prices
45
Comparison of daily order quantity and revenue
46
Value of Pricing Strategy

• SouthamptonSCM adaptively adjusts the price
  offered to the customers to ensure that the
  factory maintains as close to full production
• SouthamptonSCM (76 5405)
• FreeAgent (58 4011)
• Mr.UMBC (61 4300)
• More customer orders generally means more revenue

47
Evaluation by controlled experiments

• Devise two competitor agents that adopt identical
  strategies to SouthamptonSCM except for the
  pricing strategy
• Two competitor agents
• Risk-seeking agent (RS-agent)
• Bid aggressively at high offer prices to obtain a
  higher profit margin in selling the PCs
• Risk-averse agent (RA-agent)
• Bid cautiously and only seeks to attain a
  reasonable profit margin
• Other agents in the game are dummy agents

48
Revenue of each kind of agent
Experiment A 1 SouthamptonSCM, 1 RS and 1
RA Experiment B 1 SouthamptonSCM, 2 RS and 1
RA Experiment C 1 SouthamptonSCM, 3 RS and 1
RA
49
Comparison of daily offer prices in controlled
experiments
50
Comparison of daily order quantity and revenue in
controlled experiments
51
General Observations

• Day-0 procurement strategy is more effective than
  the build-to-order procurement
• As more agents use the same broad Day-0
  procurement, it is more likely that there will be
  a delivery date
• As more agents use the risk-seeking strategy, the
  performance of the RS-agents is more negatively
  affected, while that of RA-agents is positively
  affected
• The agent that can best adapt its offer price to
  the changing environment will thrive best in the
  game

52
Changes from the 2004

• Some changes are made
• to reduce the lottery effect in supplier
  interactions
• to reduce the incentive for making large
  procurement commitments on day 0
• To reduce the incentive to manipulate demand
  strategically (e.g., by placing large requests
  with suppliers without intending to order)

53
References

• M. He and N. R. Jennings (2004) "Designing a
  successful trading agent using fuzzy techniques"
  IEEE Trans on Fuzzy Systems 12 (3) 389-410.
• M. He and N. R. Jennings (2003) "SouthamptonTAC
  An adaptive autonomous trading agent" ACM Trans
  on Internet Technology 3 (3) 218-235.
• M. He and N. R. Jennings (2002) "SouthamptonTAC
  Designing a successful trading agent" Proc 15th
  European Conf. on AI (ECAI-2002), Lyon, France,
  8-12.
• M. He, A. Rogers, E. David and N. R. Jennings
  (2005) Designing and evaluating an adaptive
  trading agent for supply chain management
  applications Proc. IJCAI Workshop on Trading
  Agent Design and Analysis, Edinburgh, Scotland.