Stream Applications

1
Stream Applications

• By
• Michael Stonebraker
• Adjunct Professor, MIT

2
Purpose

• Are we whistling Dixie
• Anything that cant be done by a conventional app
  server and a DBMS
• Reality check on functionality
• Anybody need QoS?

3
Who

• Me
• Dennis Shasha
• Ted Johnson

4
Categorization

• The real time enterprise
• No sensors
• Current sensor apps
• There currently exists an interested user
• Future sensor apps
• Wishful thinking

5
Traditional View of the World
Actions
Transactions
Operational system
Bulk data mover
Data mining
Periodic data movement
Data Warehouse
Bulk data mover
Operational system
6
The Traditional View

• Oriented toward identifying pearls of wisdom
  that will change strategic long-term business
  practices
• Not toward tactical (real time) business analysis
  and tuning
• To do that, you need real time stream coming out
  of operational system

7
Examples

• Schwab
• Has a system to compute the commission on each
  trade in real time
• 300K of rule-based code (home brew)
• No aggregates presumably legacy DBMS lookup

8
Examples

• Fidelity
• Trades originate in lots of systems
• Get executed in several system
• Has a system to route trades in real time
• Complains it is too slow (not quite sure why)
• Wants prioritization of trades

9
Current Sensor-based Applications

• US military (Mitre)
• Batallion monitoring
• 20 queries on web site
• Mix of history and CQ
• Some queries are pretty hard (tell me when 3 of
  my 4 tanks cross the front line)

10
Current Sensor-based Applications

• US military (Mitre)
• QoS crucial (drop everything to work on incoming
  missile)
• Main bottleneck network bandwidth
• But military apps are not politically correct

11
Current Sensor-based Applications

• Environment monitoring
• Fish pills
• To monitor mercury level in water
• Whistling Dixie

12
Future Sensor-based Applications electronic
property stickers

• MIT library
• Mis-shelved books
• And stolen books
• Whistling Dixie

13
Future Sensor-based Applications electronic
property stickers

• MIL/LCS
• Lost projectors
• Stolen laptops
• Whistling Dixie

14
Future Sensor-based Applications supply chain
management

• Gilette
• Bought 10m dimes
• Presumably to put on retail bundles
• For theft?
• For supply chain replenishment
• May be whistling Dixie