Exchanges for Complex Commodities: Search for Optimal Matches

1
Exchanges for Complex Commodities Search for
Optimal Matches
Jianli Gong
Committee Eugene Fink Dmitry B. Goldgof Sudeep
Sarkar
2
Motivation

• Build an automated
• exchange for trading
• goods and services

3
Motivation

• Build an automated
• exchange for trading
• goods and services

• Combine the speed
• and liquidity of the
• stock exchange

4
Motivation

• Build an automated
• exchange for trading
• goods and services

• Combine the speed
• and liquidity of the
• stock exchange with
• the flexibility of eBay

5
Outline

• Previous work

6
Outline

• Previous work
• Complex exchange
• Data structures

7
Outline

• Previous work
• Complex exchange
• Data structures

Josh, Jenny
8
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

Josh, Jenny
9
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

Josh, Jenny
Jianli
10
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

11
Previous Work

• Combinatorial auctions
• Standardized exchanges

12
Combinatorial Auctions

• Complex commodities
• Asymmetry between
• buyers and sellers
• Illiquid
• Examples
• OptiBid, SBIDS, eMediator

13
Standardized Exchanges

• Simple commodities
• Symmetry between
• buyers and sellers
• Liquid
• Examples
• Stocks, Futures

14
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

15
Complex Exchange

• Market
• Orders
• Fills

16
Market
All items that can be traded form a market.
Example All conceivable vehicles compose a car
market.
17
Orders
An order is a subset of the market along with a
price function.
18
Orders
An order is a subset of the market along with a
price function.
Example Any Mustang or Corvette.
19
Orders
An order is a subset of the market along with a
price function.
Example Any Mustang or Corvette Mustang for
38,000 or Corvette for 40,000.
20
Orders
An order is a subset of the market along with a
price function.
Example Any Mustang or Corvette Mustang for
38,000 or Corvette for 40,000, 1 for every
ten miles.
21
Buy and Sell Orders
Price
Buy order
22
Buy and Sell Orders
Price
Buy order
23
Fills
Buy order Any Color Sports Car 36,000
Sell order Silver Limo 50,000 or Red Mustang
32,000
24
Implemented Exchange
Specific sell orders.
Good Sell a red Mustang, made in 1999.
Bad Sell any color Mustang, made before 1999.
25
Implemented Exchange
Price
Price
Buy order
Sell order
26
Implemented Exchange
Price
Buy order
27
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

28
Data Structures

• Tree of sell orders
• List of buy orders

29
Tree of Sell Orders
Model
Corvette
Mustang
30
Tree of Sell Orders
Model
Corvette
Mustang
Year
Year
1996
1997
1988
1985
1986
31
Tree of Sell Orders
Model
Corvette
Mustang
Year
Year
1996
1997
1988
1985
1986
Mileage
Mileage
Mileage
Mileage
Mileage
50K
1996 Mustang, 50K miles, 7,000
32
Tree of Sell Orders
Model
Corvette
Mustang
Year
Year
1996
1997
1986
Mileage
Mileage
Mileage
50K
1996 Mustang, 50K miles, 5,000
33
Trading Cycle
Process all new orders
Re-match all old orders
Stop trading?
No
Yes
34
Processing
Process
Re-match
Stop?
For every new order If it is a buy
order, search for matches if not filled, add
it to the list. If it is a sell order, add it
to the tree.
35
Re-Matching
Process
Re-match
Stop?
For every buy order search for matches among
the sell orders.
36
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

37
Optimal Matches
The system searches for the best-price matches in
the indexing structure.
38
Optimal Matches

• The system searches for the best-price matches in
  the indexing structure.
• Depth-first search (Josh)

39
Optimal Matches

• The system searches for the best-price matches in
  the indexing structure.
• Depth-first search (Josh)
• Best-first search (Jianli)

40
Depth-First Search
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
41
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
42
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
43
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
44
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
45
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
46
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
Mileage
Mileage
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
47
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
Mileage
Mileage
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
48
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
Mileage
Mileage
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
49
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
50
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
51
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
52
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
Best price
53
Depth-First Search
Main drawback If there are many matching leaves,
the search takes a long time.
54
Depth-First Search
With Limited Number of Matches
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
55
Depth-First Search
Buy Order Any car made after 1990
With Limited Number of Matches
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
56
Depth-First Search
Buy Order Any car made after 1990
With Limited Number of Matches
Model
Limit 2 matches
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
57
Depth-First Search
Buy Order Any car made after 1990
With Limited Number of Matches
Model
Limit 2 matches
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
58
Depth-First Search
Buy Order Any car made after 1990
With Limited Number of Matches
Model
Limit 2 matches
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
Mileage
Mileage
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
59
Depth-First Search
Buy Order Any car made after 1990
With Limited Number of Matches
Model
Limit 2 matches
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
Best price among found matches
60
Depth-First Search
Buy Order Any car made after 1990
With Limited Number of Matches
Model
Limit 2 matches
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
Actual best price
Best price among found matches
61
Depth-First Search
With Limited Number of Matches
Main drawback The search may not find the
best-price match.
62
Best-First Search

• Store the best price for each subtree
• Use these prices to guide the search

63
Best Prices for Subtrees
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
64
Best Prices for Subtrees
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
65
Best Prices for Subtrees
Model
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
66
Best Prices for Subtrees
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
67
Search for Best Price
Buy Order Any car made after 1990
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
68
Search for Best Price
Buy Order Any car made after 1990
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
69
Search for Best Price
Buy Order Any car made after 1990
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
70
Search for Best Price
Buy Order Any car made after 1990
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
71
Search for Best Price
Buy Order Any car made after 1990
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
72
Search for Best Price
Buy Order Any car made after 1990
Model
4,000
Corvette
Mustang
Camry
Year
Year
Year
7,000
4,000
5,000
1996
1999
2000
Mileage
Mileage
Mileage
7,000
18,000
5,000
4,000
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
Best price
73
Quality Function
A buyer can specify a quality function that ranks
the matching sell orders.
74
Quality Function
A buyer can specify a quality function that ranks
the matching sell orders. The system searches for
the matches with the highest quality.
75
Quality Function
Example Quality of a car 4 for each year after
1970
76
Quality Function
Example Quality of a car 4 for each year after
1970 5 for each 10K miles
77
Quality Function
Example Quality of a car 4 for each year after
1970 5 for each 10K miles 6 for each 1,000
78
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
79
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
80
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
Mileage
Mileage
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
81
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
82
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
83
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
Quality 37
Quality 3
Quality 0
84
Depth-First Search
Buy Order Any car made after 1990
Model
Corvette
Mustang
Camry
Year
Year
Year
1996
1999
2000
Mileage
Mileage
Mileage
50K
10K
180K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
2000 Camry, 180K miles, 5,000
Highest quality 37
Quality 3
Quality 0
85
Monotonic Attributes

• The quality monotonically decreases with the
  price

86
Monotonic Attributes

• The quality monotonically decreases with the
  price
• Usually, it is also monotonic on several
  other attributes

87
Monotonic Attributes

• The quality monotonically decreases with the
  price
• Usually, it is also monotonic on several
  other attributes
• Car quality
• Increases with the year
• Decreases with the mileage

88
Best-First Search

• For every subtree, store the best value of
  each monotonic attribute
• Use these values to estimate the quality of
  the best match in every subtree

89
Best Values for Subtrees
Model
Corvette
Mustang
Year
Year
1996
Mileage
50K
1996 Mustang, 50K miles, 7,000
90
Best Values for Subtrees
Model
Corvette
Mustang
Year
Year
1996
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
1996 Mustang, 50K miles, 7,000
91
Best Values for Subtrees
Model
Corvette
Mustang
1988, 20K, 4,000
2000, 10K, 5,000
1996, 50K, 7,000
Year
Year
1996
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
1996 Mustang, 50K miles, 7,000
92
Best Values for Subtrees
2000, 10K, 4,000
Model
Corvette
Mustang
1988, 20K, 4,000
2000, 10K, 5,000
1996, 50K, 7,000
Year
Year
1996
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
1996 Mustang, 50K miles, 7,000
93
Search for Best Match
Buy Order Any car made after 1990
2000, 10K, 4,000
Model
Corvette
Mustang
1988, 20K, 4,000
2000, 10K, 5,000
1996, 50K, 7,000
Year
Year
1996
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
1996 Mustang, 50K miles, 7,000
94
Search for Best Match
Buy Order Any car made after 1990
Model
91
Corvette
Mustang
1988, 20K, 4,000
2000, 10K, 5,000
1996, 50K, 7,000
Year
Year
1996
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
1996 Mustang, 50K miles, 7,000
95
Search for Best Match
Buy Order Any car made after 1990
Model
91
Corvette
Mustang
Camry
1996, 50K, 7,000
1988, 20K, 4,000
Year
Year
Year
85
1996
1999
Mileage
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
96
Search for Best Match
Buy Order Any car made after 1990
Model
91
Corvette
Mustang
Camry
1996, 50K, 7,000
Year
Year
Year
38
85
1996
1999
Mileage
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
97
Search for Best Match
Buy Order Any car made after 1990
Model
91
Corvette
Mustang
Camry
Year
Year
Year
38
85
37
1996
1999
Mileage
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
50K, 7,000
50K
10K
1996 Mustang, 50K miles, 7,000
1999 Camry, 10K miles, 18,000
98
Search for Best Match
Buy Order Any car made after 1990
Model
91
Corvette
Mustang
Camry
Year
Year
Year
38
85
37
1996
1999
Mileage
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
37
50K
10K
1999 Camry, 10K miles, 18,000
1996 Mustang, 50K miles, 7,000
99
Search for Best Match
Buy Order Any car made after 1990
Model
91
Corvette
Mustang
Camry
Year
Year
Year
38
85
37
1996
1999
Mileage
Mileage
20K, 4,000
180K, 5,000
10K, 18,000
37
50K
10K
1999 Camry, 10K miles, 18,000
1996 Mustang, 50K miles, 7,000
Highest quality 37
100
Outline

• Previous work
• Complex exchange
• Data structures
• Optimal matches
• Performance

101
Performance
Empirical evaluation

• 1,500 MHz CPU
• 1,024 Mbyte memory
• 166 MHz bus

102
Control Variables

• Search strategy
• Number of orders
• Length of item description

103
Measurements

• Processing time
• Re-matching time
• Throughput

104
Processing Time
Process
Re-match
Stop?
Logarithmic Scale
Linear Scale
1,000
400
10
300
time (sec)
200
0.1
100
0.001
0
1 10 100 1,000 10,000 100,000
50K 100K 150K 200K 250K
number of orders
number of orders
Best-First Search, Unlimited DFS, and Limited DFS
105
Re-Matching Time
Process
Re-match
Stop?
Logarithmic Scale
Linear Scale
1,000
400
10
300
time (sec)
200
0.1
100
0.001
0
1 10 100 1,000 10,000 100,000
50K 100K 150K 200K 250K
number of orders
number of orders
Best-First Search, Unlimited DFS, and Limited DFS
106
Total Time
Process
Re-match
Stop?
Logarithmic Scale
Linear Scale
1,000
400
10
300
time (sec)
200
0.1
100
0.001
0
1 10 100 1,000 10,000 100,000
50K 100K 150K 200K 250K
number of orders
number of orders
Best-First Search, Unlimited DFS, and Limited DFS
107
Throughput
Number of attributes Orders per second
1 3 10 5,800 1,500 20
108
Throughput
Number of attributes Orders per second
1 3 10 5,800 1,500 20
1 3
10
number of attributes
Best-First Search, Unlimited DFS, and Limited DFS
109
Main Results

• Use of quality functions
• Search for optimal matches
• Performance evaluation

110
Future Work

• Extended sell orders
• Advanced indexing structures, such as
  kd-trees and quadtrees
• Search for approximate matches
• Trading different commodity types in a common
  exchange