A Query Language for a Data Refinery Cell Martin Kersten, Erietta Liarou, Romulo Goncalves CWI, Amst

1
A Query Language for a Data Refinery
CellMartin Kersten, Erietta Liarou, Romulo
GoncalvesCWI, Amsterdam, The Netherlands
2
Philips Research World Perspective
lt2007
2007
gt2007
sensor cluster ESP
Semantic Sensors
mobile sensor cluster
mobile
integrated management
stationary
distributed management
distributed
PC-less sensor net
AmbientDB
sensor net
3
Philips Research World Perspective
Semantic Sensors
Rich applications Energy-rich environment Ad-hoc
networking Long-term archival Skewed event streams
distributed management
AmbientDB
4
The MonetDB Software Stack
XQuery
SQL 03
SQL/XML
Optimizers
SOAP
Open-GIS
GIS
MonetDB 4
MonetDB 5
compile
MonetDB kernel
An advanced column-oriented DBMS
5
The MonetDB Software Stack
DataCell
Orthogonal extension of SQL03 Clear
computational semantics Minimal extension to
MonetDB
Extensions
SQL 03
Optimizers
MonetDB 5
MonetDB kernel
An advanced column-oriented DBMS
6
DataCell Components
Receptor Basket Query Table Emitter
Pickup events from the environment Temporary
store for events Continuous SQL
query Persistent tables Delivery of events to
the environment
7
Presence DataCell
Global temporary No transaction logs No a priori
order
create basket sensite ( sid integer, tag
timestamp default now(), enter boolean )
8
Presence DataCell
create basket moves ( sid integer, tag
timestamp default now(), x int, y int, h int, v
int )
9
Presence DataCell
Adapter components Tuple or XML protocol TCP or
UDP
copy into sensite (sid, enter) from
sensite-port 2391, 1 2391, 0 2391,
0
10
Presence DataCell
copy into moves (sid,x,y,h,v) from
movement-port 2392, 0,0,5,5 2392,
5,5,2,3
11
Presence DataCell
create basket presence ( tag timestamp, enter
boolean )
12
Presence DataCell
insert into presence select s.tag,enter
from sensite as s, moves as m where
s.tag gt m.tag - delta() and
s.tag lt m.tagdelta()
13
Querying Streams

• Sliding Windows in SQL03
• They assume a sequence
• They are limited to expressions that aggregate
• They carry specific first/last window behaviour
• They are read-only queries
• They rely on predicate evaluation strictly before
  or after the window is fixed

• Basket expression
• They can be bound using a ordered-by constraint
• They can be explicitly defined by predicates over
  their content
• They can be based on predicates referring to
  objects elsewhere in the database

14
Presence DataCell
create basket warning ( cnt integer )
15
Presence DataCell
insert into warnings select count() from
select p. from metronome(1 hour)
as m, presence as p
16
Presence DataCell
copy from warning into led-port
17
Presence DataCell
Open issue All baskets may become polluted with
un-used tuples delete from basket where tag lt
now() 1 day.
18
Presence DataCell

• create basket sensite (
• sid integer,
• tag timestamp default now(),
• enter boolean
• )
• create basket moves (
• sid integer,
• tag timestamp default now(),
• x int, y int, h int, v int
• )
• create basket presence (
• tag timestamp,
• enter boolean
• )
• create basket warning (
• cnt integer
• )

• copy into sensite (sid, enter) from
  sensite-port
• copy into moves (sid,x,y,h,v) from
  movement-port
• insert into presence
• select distinct tag, enter
• from select s.tag,enter from sensite as s, moves
  as m
• where s.tag gt m.tag - delta() and s.tag lt
  m.tagdelta()
• insert into warnings
• select count()
• from select p. from metronome(1 hour) as m,
  presence as p
• delete from basket where tag lt now() 1 day.

19
The MonetDB Software Stack
SQL 03
MonetDB Assemblee Language
Optimizers
MonetDB 5
MonetDB kernel
An advanced mature column-oriented DBMS
20
SQL to MAL

• MonetDB Assemblee Language
• (t1,..,t32) module.fcn(a1,..,a32)
• t1 module.fcn(a1,..,a32)
• t1 fcn(a1,..,a32)
• t1 v1 operator v2
• t1 constant
• (t1,..,tn) (c1,..,cn)

21
SQL to MAL
22
Select from tables
23
DataCell Architecture
factory stepOne()bit xbatlng,int
basket.bind("X","X_pl") zbatlng,int
basket.bind("Z","Z_pl") barrier go true
basket.lock("X") basket.lock("Z") be
algebra.select(x,0,nil) bat.insert(z,be)
bat.delete(x,be) basket.unlock("Z")
basket.unlock("X") yield qry00true redo
go true exit go end stepOne

• continuous queries are translated into factories,
  MonetDBs notion of co-routine.

24
DataCell Architecture
25
DataCell Architecture
Select from select from X as A, select
from select from Y as B as C

• The computational model underlying continuous
  query processing is based on Petri-nets, or
  predicate nets in particular.

select from select from Y as B as C
select from X as A
select from Y as B
26
Performance outlook
Actuators
Sensors
Actuators
Sensors
DataCell
27
Inter-process communication
28
Inter-process communication
29
Inter-process communication
30
Ongoing activities

• Performance evaluation on the Conveyor and
  Diamond experiments
• Wrap the MAL templates in the SQL compiler
• A (web-) GUI for development and debugging would
  be great.

31
(No Transcript)