Query Processing over Data Streams

1
Query Processing over Data Streams

• Stream-query processing arises naturally in
  Network Management

R1
R2
R3
2
Query Processing over Data Streams

• One-time queries and Continuous queries
• One-time queries
• Evaluated once over a point-in-time snapshot of
  data set
• Includes traditional DB queries.
• Continuous queries
• Evaluated continuously as data streams continue
  to arrive
• Usual type of queries over data stream
• The answer is produced over time

3
Query Processing over Data Streams (contd)

• Predefined and Ad hoc queries
• Predefined
• Supplied to data stream management system before
  any relevant data has arrived
• Usually continuous queries
• Ad hoc
• Issued online after the data streams has already
  begun
• Can be either one-time or continuous queries
• Complicates design of data stream management
  system (DSMS)

4
Query Processing over Data Streams General
Architecture
User/Application
SDMS (Stream Data Management System)
Results
Multiple streams
Stream Query Processor
Stream Synopses (Main memory and/or Disk)
5
Query Processing over Data Streams

• Join Operation
• Using sketches for self-joins, binary joins, and
  complex joins over data streams estimating
  correlated aggregates
• Approximate-query processing
• Using histogram and wavelet synopses

6
Advanced Topics and Future Work

• Advanced Techniques
• 1Distinct Value Estimation
• 2 Sliding Windows
• - At every time t, a data record arrives
• The record expires at time t N (N is the window
  Length)
• When is it useful?
• - Make decisions based on recently observed
  data.
• Stock data
• Sensor network
• Future Work
• - Stream Algebra, Query languages and
  Approximations.