XL: A Platform for Web Services

1
XL A Platform for Web Services

• Daniela Florescu
• BEA Systems
• Andreas Grünhagen
• University of Heidelberg
• Donald Kossmann
• University of Heidelberg

2
Agenda

• Motivation for XL
• XL Programming Model
• XL Platform
• Conclusion

3
Overall Project Goals

• Easier Development of Web Services
• faster, cheaper
• better maintenance, evolution of software
• application programmers vs. technical gurus
• Easier Administration of Web Services
• Performance, Robustness, Security
• Plan of Attack
• higher level of abstraction (from WS standards)
• DB Technology

4
State of the Art

• Web Services are great for integration
• Many good standards
• XML, XQuery, SOAP, WSDL, UDDI,
• Many good tools emerging
• Hype
• everybody does it good marketing
• students are interested in this technology
• self-fulfilling prophecy

5
But

• Programmers have a hard time
• jungle of technologies
• impedance mismatch
• simple things are complicated/redundant (e.g.,
  logging)
• Performance, Management
• too many software layers
• data marshalling
• optimization is difficult
• administrators are expensive

6
Web Services conceptual layers
XML Protocol (SOAP) XML Schema validation XSLT/
Xquery evaluation
application logic data validation error
handling caching, replication and distribution
queries and updates integrity constraints triggers
transactions
7
Good old impedance mismatch

• Lack of productivity
• Programmers deal with at least three different
  programming paradigms (XML, Java/C, SQL)
• Lack of performance
• Run-time data marshalling through at least three
  different forms (XML ? business objects ? SQL
  tables)
• Data should be represented in a single format in
  the entire stack

8
Good old impedance mismatch

• Lack of productivity
• Programmers deal with at least three different
  programming paradigms (XML, Java/C, SQL)
• Lack of performance
• Run-time data marshalling through at least three
  different forms (XML ? business objects ? SQL
  tables)
• Data should be represented in a single format in
  the entire stack
• The abstract format should be
• . XML

9
Unsatisfactory support for Web Services in
Current Architectures

• Web Services specific requirements
• message handling
• asynchronicity
• parallelism
• non-determinism
• event-based programming
• Hard-coded, often using external libraries
• Problems
• too low level
• hard to optimize automatically if not in the
  language
• Web services need a programming language that
  natively handles all of the above

10
What is XL?

• A new programming model
• A platform (implementation)
• virtual machine
• database engine
• a few tools debugger, console
• (Regression Test Tools and Protocols)
• (Methodology)

11
Agenda

• Motivation for XL
• XL Programming Model
• XL Platform
• Conclusion

12
Requirements for a new programming language

• 1. Unique data model and type system XML
• 2. Gracefully co-exist with existing W3C
  standards (XML, XML Schema, XQuery, Xforms, SOAP,
  WSDL, etc.)
• 3. Should be an extension of XQuery
• 4. Declarative as much as possible
• 5. Imperative constructs
• 6. Native support for Web services specific
  features
• 7. Forbid manual optimizations and low level
  specifications
• 8. Allow automatic compilation (what ? how)
• 9. Simple.

13
Design Principles

• Use W3C Standards as building blocks
• XML, XML Schema, XQuery, SOAP, WSDL,
• Imperative Statements a la Java and C
• Assignment, Loops, Error handling,
• (Persistent) State is kept in variables
• PI Calculus Distributed Computing, Composition
• Syntactic Sugar
• Logging, conversations, retry,

14
Web Services in XL

• Local Definitions
• Schema, Namespaces, Functions (XML Schema,
  XQuery)
• Local Operations (not seen in WSDL)
• Local Declarations (State)
• Global Variables (Datenbase)
• Persists with Web service one instance per Web
  Service
• Conversation Variables (session data)
• Persists with conversation one instance per
  conversation
• Declarative Clauses
• integrity constraints, behavior of Web Service,
• Public Operation Specifications
• public operations (seen in WSDL)

15
Web Services in XL

• Syntax
• web service ltURIgt
• ltlocal definitionsgt
• ltlocal declarationsgt
• ltdeclarative clausesgt
• ltpublic operation specificationgt
• end web service
• WSDL is automatically generated

16
Example Web Service
web service http//www.auction.com import
schema nswww.OnlineAuctionSchemas.xml
global variable nsCustomers customersDB
conversational
variable nsBid maxBid invariant
every x in customersDB satisfies x/nsbalance
gt 0 invariant every x in customersDB
satisfies x/nsagegt 18 conversation
pattern mandatory conversation timeout
duration(P10M) ... !! Public
Operations end web service
17
XL Operation

• Syntax
• operation ltnamegt Type -gt Type
• ltdeclarative clausesgt
• body
• ltstatementgt
• Declarative Clauses
• Preconditions, postconditions, conversation
  pattern, error handling, logging
• input, output variables for input and output

18
operation login conversation pattern
mandatory precondition input instanceof
LoginInfo body if (not (some x in
customersDB satisfies x/name
input/name and x/pwd input/pwd)) throw
lterrorgt Wrong login information! lt/errorgt
else let output
ltresultgtWelcome dear
input/name!lt/resultgt
19
State of a Web Services

• Global variables (Database)
• One instance per Web Service
• Persists with life time of Web Services
• Example customer database
• Conversation variables (Session data)
• One instance per conversation
• Persists with life time of conversation
• Established from Context when operation is
  called
• Example highest bid of an auction

20
State of an Auction

• web service http//www.auction.com
• !! define Types for Customer, Address, Bids,
  etc.
• import schema nswww.OnlineAuctionSchemas.xml
  
• !! declare Customer Database
• global variable nsCustomer customersDB
• !! declare best bid for each auction
• conversational variable nsBid maxBid
• ... !! operations of the Web services
• end web service

21
operation login conversation pattern
mandatory precondition input instanceof
LoginInfo body if (not (some x in
customersDB satisfies x/name
input/name and x/pwd input/pwd)) throw
lterrorgt Wrong login information! lt/errorgt
else let output
ltresultgtWelcome dear
input/name!lt/resultgt
22
operation takeBid conversation pattern
mandatory precondition input instanceof
Bid body if (input/amount leq
maxBid) throw lterrorgt Bid is too low!
lt/errorgt else let
maxBid input let output ltok/gt

23
XL Statements

• Assignment statement
• let type ltvargt ltexprgt
• Type is optional XML Schema Type (e.g.
  xsinteger)
• Expr is any XQuery expression (e.g. y 5)
• Var is a (global, conversation, local) variable
  (e.g. x)
• Variables contain instance of XQuery data model
  (seq. of items)
• Update statements (XUpdate)
• insert x/book_at_isbn 4711 into y/bib

24
Example Assignment
let x ltpogt for i
in product where
i/price gt 500 return
ltlineitemgt
ltproductgt i/name lt/productgt
...
lt/lineitemgt
lt/pogt
25
XQuery expressions

• Constants
• 1.34, Doe
• Variables
• x, y
• Functions and operators
• 12, concat(x, y), F(x)
• Path expressions
• x/person/name/last
• FLWR expressions
• for x in //person
• where x/name/lastDoe
• return x/age

• Sort-by
• sort //person by name/last
• Quantifiers
• some x in //person satisfies
• x/agegt60
• every x in //person satisfies
• x/agegt10
• XML node constructors
• ltintroductiongt
• Hi. My name is x/name and I work for
  x/employer.
• lt/introductiongt

26
XL Statements Send (Invoke)

• Synchronous Message (RPC-style)
• ltexprgt --gt uri operation --gt ltvargt
• evaluate expr
• send SOAP message to Web Service specified by
  uri(can be any Web Service Java, .NET, BPEL,
  XL, )
• Wait for result returned by Web Service (SOAP
  message)
• Copy content (body) of returned SOAP message into
  var
• Asynchonous Message
• ltexprgt gt uri operation gt uri
  operation
• Do not wait for answer
• Optional specify operation that handles answer
  (if any)

27
XL Statements Overview

• Imperative Statements (a la Java and C)
• Conditional IF-THEN-ELSE, SWITCH
• Loops FOR, WHILE, DO-WHILE, FLWD
• Error Handling THROW, CATCH
• Web-Service-specific Statements
• logpoint
• wait
• retry

28
XL Statements Errors

• Exceptions
• throw ltexprgt
• Assertions
• asssert ltboolean exprgt throw ltexprgt
• Error Handling
• try ltstatementgt
• catch (ltvargt)
• ltstatementgt

29
XL Statements Conditionals

• If Then Else Statement (same as Java)
• if (ltboolean exprgt) ltstatementgt
• else
• ltstatementgt
• Switch Statement (different than Java)
• switch
• case (ltboolean exprgt) ltstatementgt
• case (ltboolean exprgt) ltstatementgt
• default statement enddefault

30
XL Statements Loops

• while (ltboolean exprgt)
• ltstatementgt
• or
• do ltstatementgt while (ltboolean exprgt)
• or
• (for ltvargt in ltexprgt)
• (let ltvargt ltexprgt)
• (where ltboolean exprgt)?
• do ltstatementgt

31
XL Statements WS-specific

• Logging (in predefined history variable)
• logpoint ltexpr1gt as ltname1gt ... ltexprNgt as
  ltnameNgt do ltstatementgt
• RETURN, HALT, NOTHING, SLEEP
• WAIT ON EVENT, CHANGE
• wait on event ltboolean exprgt
• wait on change ltvargt
• RETRY
• retry ltstatementgt maximum ltint exprgt times throw
  ltexprgt
• retry ltstatementgt timeout after ltduration exprgt
  throw ltexprgt

32
Declarative Clauses

• HISTORY
• log all calls
• DEFAULT, UNKNOWN OPERATION
• operations to call by default
• INIT, CLOSE
• operations called when service is started and
  terminated
• INVARIANT
• (DB-style) integrity constraints

33
Declarative Clauses

• ON EVENT, ON CHANGE
• trigger, periodic executions of operations (cron)
• ON ERROR INVOKE ltOPERATIONgt
• error handling (if not handled locally)
• CONVERSATIONPATTERN
• default pattern (operations can overrite)
• CONVERSATIONTIMEOUT
• terminate conversation automatically after
  timeout

34
Conversation Pattern

• Must an operation be called as part of an
  operation?
• Must an outgoing message be part of the same (or
  other or no) conversation?
• Example
• login operation is part of a conversation
• Web page request is not part of a conversation
• Auction All bids part of the same conversation
• Privacy / Routing Each hop new conversation

35
Predefined Patterns (J2EE inspired)
36
Statement combinators

• Sequencing
• statement1 statement2
• Parallelism
• statement1 statement2
• (Guarded) choice
• statement1 statement2

• Data flow
• statement1 statement2
• Failure
• statement1 ? statement2

see CSP (Hoare) and Web Services calculus
(Cardelli)
37
Advantages of XL

• An all XML world
• eliminate the inherent complexity of a
  mix-and-match system
• High level (declarative) scripting language
• programmers can concentrate on the application
  logic, not on caching, distribution,
  replications, queues, networking and low level
  protocols
• allow automatic compilation and optimization
• Extension of existing XML and Web services
  standards

38
State of the Art

• Theory
• e.g., CSP, Web services calculus
• Dataflow languages/systems
• e.g., Ptolemy
• Workflow languages and paradigms
• e.g., WSFL, Vortex
• Database programming languages
• e.g., O2C, persistent SmallTalk
• Web scripting languages
• e.g., WebL, XLang, eFlow, eSpeak, Curl, Water,
  AppleScript
• Recent Web languages
• e.g., WSCI, BPEL4WS

39
XL vs. BPEL

• XL and BPELare compatible (WS standards)
• XL is older and non-committee work
• more homogeneous, less comprimises (???)
• XL supports XQuery (and up-coming XUpdate)
• XL has Java-like imperative statements
• BPEL will become a standard (XL not)
• But
• BPEL might adopt XL ideas (???)
• XL might find some fans (Siemens, Airbus,
  students ???)
• XL platform / components usable in broader
  context (???)

40
Agenda

• Motivation for XL
• XL Programming Model
• XL Platform
• Conclusion

41
XL Platform

• Goals
• make optimization automatic
• distributed execution in a cluster (availability,
  parallelism)
• Approach
• Execute XL program like a database query
• Compile Time
• Algebra of basic statements
• translate program into statement graph
• automatic optimization of statement graph
• Run Time
• streaming execution of statements (Iterator
  Model)
• dispatcher for load balancing
• caching data and query shipping with
  centralized data store

42
Compile-Time-System
XL Program
Metadata
Compiler
DB
XL Algebra
43
XL Algebra

• 8 logical statements
• Assignment (let)
• Send SOAP message
• Receive SOAP message
• Wait
• Sync
• Updates Insert, Delete, Replace
• Implemented by physical statements
• local calls, multicasts, materialization, event
  indexes, unroll loops, ...

44
Statement Graph

• Send product information to all customers
• let a customers/email
• for e in a do product/info gt e

let a customer/email
not p
let p empty(a)
p
let e a1
...
send(a, product/info)
let a a except e
45
Optimization

• Mix of various techniques
• dragon-book techniques (compiler construction)
• database optimization (SIGMOD, VLDB)
• new techniques, combinations
• Examples
• Common sub-expressions, Reorder statements
• Multi-query optimization, Batched updates
• Identify local, side-effect free invocations
• switch, if-then-else optimizations
• parallelize loops (e.g., for do loops)
• data vs. query shipping
• ...

46
Run-Time-System
XML Message
Dispatcher
XML Message
XML Message
Virtual Machine
Virtual Machine

XQuery
Context
SOAP
XQuery
Context
SOAP
Cache
Cache
DB Access
DB Access
Data Base (Code, Data, Metadata)
47
Virtual Machine

• Interprets statements of the statement graph
• Binds Messages
• Supports stream-based processing
• Multi-Threading
• (Response time guarantees, Re-scheduling)

48
Virtual Machine

• Library of (physical) statements
• for each logical statement, at least one impl.
• Context Management
• statement is executed in a context
• variables, schemas, conversation ids,
• Scheduler
• determines next statement to execute

49
Processing a Message

• Create and initialize a context
• Bind body of message to context (input)
• Load and clone statement graph of operation
• Push first statement into queue of scheduler
• Interpret the statement

50
Scheduler
Context Msg1
Context Msg2
S1
S1
S2
S3
S2
S3
S3

S1
S2
Queue
51
Stream-based Processing

• Goals
• lazy evaluation of programs
• reduce response time and memory requirements
• bursty arrival of data, pipelined parallelism
• Approach
• statements as iterators (open, next, close)
• normalization of XL statement graphs
• some subtleties graph, not tree. loops.

52
Statement Graph

• Send product information to all customers
• let a customers/email
• for e in a do product/info gt e

let a customer/email
not p
let p empty(a)
p
let e a1
...
send(a, product/info)
let a a except e
53
Normalisation

• Send product information to all customers
• let a customers/email
• for e in a do product/info gt e

let a customer/email
not p
let p empty(a)
p
p
let e a1
send(a, product/info)
...
let a a except e
54
Clustering

• Nodes run instances of XL virtual machine
• Nodes interact with central data store local
  cache
• Dispatcher allocates nodes for incoming messages
• load balancing
• cache awareness based on conversation
• Process migration possible(all context
  information is serializable)

55
Differences and Commonalities to Conventional
Engines

• XL vs. Java VM and Application Servers
• JVM does not support (Data) Pipelining
• JVM much weaker optimization
• JVM needs IPC and marshalling for persistence
• JVM has no intra-program parallelism
• AppServers also operate on Cluster(load
  balancing, asynchronous communication, persistent
  state)
• XL vs. Database Query Engines
• XL is Turing-complete
• XL has more global optimization
• XL asynchronous, inpredictable communication
  thus, dynamic reoptimization (dispatching) a
  must!
• Data Pipeling, Optimization based on
  Algebra(potentially cost based optimization,
  automatic parallelism)

56
Initial Experiments

• Application Business processes
• RosettaNet
• Online-Auction
• TPC-W
• Benchmarks
• measure response time, throughput, memory
  footprint
• vary size of messages, database sizes, value
  distributions
• Alternative implementations
• Java
• XL without streaming
• XL with data streaming
• (BPEL - in progress)

57
RosettaNet (1)
58
RosettaNet (2)
59
Agenda

• Motivation for XL
• XL Programming Model
• XL Platform
• Conclusion

60
Summary

• Web Services are good!
• Web Service technology not mature
• bottom-up approach by standard commities
• XL simplifies use of WS technology
• top-down approach for usability
• XL is not mature
• need use cases, feedback from real users
• need more time/resources for the platform

61
Implementation Status

• First full prototype in Java operational(demo
  http//xl.informatik.uni-heidelberg.de)
• currently, re-designing system
• SOAP via Tomcat (Apache)
• Materialized and pipelined execution
• Simple optimization, clustering
• Use BEA XQuery engine as black box
• Use MySQL as persistent store
• No serious benchmarking yet