How to Design and Generate Web Programs

1
(No Transcript)
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
(No Transcript)
12
(No Transcript)
13
(No Transcript)
14
Programming theInteractive Web

• Shriram Krishnamurthi
• Brown University

15
The Interactive Web

• The Web is increasingly dark matter
• Numerous Web APIs
• The Common Gateway Interface (CGI)
• Java Servlets
• Active Server Pages, Java Server Pages
• Scripting languages (Perl, PHP, etc)
• Microsofts Web Services

16
Where You See This

• URLs become simple
• https//onepass.continental.com/asp/statement.asp
  
• URLs become complex
• http//maps.yahoo.com/py/ddResults.py?PytTmapta
  rnametardescnewnamenewdescnewHashnewTHas
  hnewStsnewTStstlttlnsltslnnewFLUse
  AddressBelownewaddr3007SantaMonicaBoulevard
  newcszsantamonica,canewcountryusnewTFLUseA
  ddressBelownewtaddr2815SantaMonicaBoulevard
  newtcszSantaMonica,CA904042409newtcountryus
  SubmitGetDirections

17
Why Dynamic Content?

• Server maintains large database
• Continuous upgrades (software and data)
• Platform independence for clients
• Sometimes, just a Web interface to an existing
  program (eg, airline reservations)

18
Red Herring Says

• No software? No problem. You should be moving
  all your business processes onto the Web anyway.
  (The Angler, Anthony B. Perkins, October 2002)
• Discusses successful online subscription-based
  service
• No CD to install, no maintenance, no backup,
  and no need to upgrade!

19
The Orbitz Problem

• Not limited to punk script monkeys!
• Also found on Web sites of
• Microsoft
• Apple
• the National Science Foundation

20
Programming InteractiveWeb Scripts
21
Printing a Message(Console)

• print
• Hello, World\n
• exit

22
Printing a Message(Web)

• print
• lthtmlgt
• ltheadgtlttitlegtTestlt/titlegt
• lt/headgt
• ltbodygt
• ltpgtHello, World!lt/pgt
• lt/bodygt
• lt/htmlgt
• exit

23
Printing Uptime(Console)

• print
• Uptime s\n
• system (uptime)
• exit

24
Printing Uptime(Web)

• print
• lthtmlgt
• ltheadgtlttitlegtUptimelt/titlegt
• lt/headgt
• ltbodygt
• ltpgtsystem (uptime)lt/pgt
• lt/bodygt
• lt/htmlgt
• exit

25
Area of Circle(Console)

• r read Enter radius
• print
• area is d\n
• (3.14rr)
• exit

26
Area of Circle(Web)
Enter radius
r get_binding radius
bindings ltpgtarea is (3.14rr)lt/pgt
27
Adding Two Numbers(Console)

• n1 read Enter first
• n2 read Enter second
• print
• sum d\n
• (n1 n2)
• exit

28
Two User Interfaces
Enter first
Enter second
Enter first
Enter second
29
Interacting with Web Scripts
30
Interacting with Web Scripts
31
Interacting with Web Scripts
32
Interacting with Web Scripts
33
Interacting with Web Scripts
34
Interacting with Web Scripts
35
Adding Two Numbers(Web)
Enter first
n1 get_binding n1
bindings ltformgtlt/formgt
36
A Central Problem

• Web scripts write a page, then terminate
• When the user replies, another script reads the
  forms bindings and performs the next step

37
Adding Two Numbers (Web)
Enter first
n1 get_binding n1
bindings ltformgtlt/formgt
Enter second
38
Adding Two Numbers(Web)
Enter first
n1 get_binding n1
bindings ltformgtlt/formgt
n2 get_binding n2
bindings ltpgtsum (n1 n2)lt/pgt
Enter second
free variable
39
In Practice

• System signals an error
• The user doesnt get a useful answer
• The user may not understand the error
• User expended a lot of effort and time
• Program captures variable by accident (i.e., it
  implements dynamic scope!), or
• internal server error

40
Adding Two Numbers(Web)
Enter first
n1 get_binding n1
bindings ltformgtlt/formgt
n2 get_binding n2
bindings ltpgtsum (n1 n2)lt/pgt
Enter second
41
Adding Two Numbers (Web)
Enter first
n1 get_binding n1
bindings ltformgtlt/formgt
n1 get_binding n1 bindings n2
get_binding n2 bindings ltpgtsum (n1
n2)lt/pgt
Enter second
n1
Enter second
42
The Actual Form

• lthtmlgt
• ltheadgt
• lttitlegtThe Addition Pagelt/titlegt
• ltbodygt
• ltpgtEnter the second numberlt/pgt
• ltform method"get"
• action"http//www. .../cgi-second.ss"gt
• ltinput type"hidden" namen1" value1729"gt
• ltinput type"text" namen2" value"0"gt
• lt/formgt
• lt/htmlgt

43
Problems

• Generating forms is a pain
• Programmer must manually track these hidden
  fields
• Mistakes can have painful consequences
• (Worst, silently induce dynamic scope)

44
Bad News

• Thats the easy part!

45
Whats in a URL?

• Lets go back to this URL
• http//maps.yahoo.com/py/ddResults.py?PytTmapta
  rnametardescnewnamenewdescnewHashnewTHas
  hnewStsnewTStstlttlnsltslnnewFLUse
  AddressBelownewaddr3007SantaMonicaBoulevard
  newcszsantamonica,canewcountryusnewTFLUseA
  ddressBelownewtaddr2815SantaMonicaBoulevard
  newtcszSantaMonica,CA904042409newtcountryus
  SubmitGetDirections

46
Whats in a URL?

• Lets go back to this URL
• http//maps.yahoo.com/py/ddResults.py?PytTmapta
  rnametardescnewnamenewdescnewHashnewTHas
  hnewStsnewTStstlttlnsltslnnewFLUse
  AddressBelownewaddr3007SantaMonicaBoulevard
  newcszsantamonica,canewcountryusnewTFLUseA
  ddressBelownewtaddr2815SantaMonicaBoulevard
  newtcszSantaMonica,CA904042409newtcountryus
  SubmitGetDirections

47
Breaking it Down

• Write it differently
• http//maps.yahoo.com/py/ddResults.py?
  newaddr3007SantaMonicaBoulevard
• newcszsantamonica,ca
• newcountryus
• newtaddr2815SantaMonicaBoulevard
• newtcszSantaMonica,CA904042409
• newtcountryus
• SubmitGetDirections

48
Breaking it Down

• Or
• http//maps.yahoo.com/py/ddResults.py?
• newaddr 3007SantaMonicaBoulevard
• newcsz santamonica,ca
• newcountry us
• newtaddr 2815SantaMonicaBoulevard
• newtcsz SantaMonica,CA904042409
• newtcountry us
• Submit GetDirections

It looks an awful lot like a function call!
49
The Real Picture
The script and the user are coroutines!
Event lines
script
user
50
Control Flow Back Button
A silent action!
51
Control Flow Cloning
script
user
52
Control Flow Bookmarks
script
user
53
What Programmers Need

• Multiply-resumable and restartable coroutines
• No language has exactly this the new control
  operator for the Web
• How do we implement it?

54
How to Reengineer Programsfor the Web
55
What we Want to Write

• n1 read
• Enter first
• n2 read
• Enter second
• print
• sum d\n
• (n1 n2)
• exit

56
What we are Forced to Write1 of 3

• Main () print
• ltform actionf1gt
• Enter first
• ltinput namen1gt
• lt/formgt

57
What we are Forced to Write2 of 3

• f1 (form) print
• ltform actionf2gt
• ltinput hidden namen1
• valueform.n1gt
• Enter second
• ltinput namen2gt
• lt/formgt

58
What we are Forced to Write3 of 3

• f2 (form) print
• The sum is
• form.n1 form.n2

59
Sensitive to Interaction
60
Why Does this Work?
61
Program Structure Destroyed

• n1 read
• Enter first
• n2 read
• Enter second
• print
• sum d\n
• (n1 n2)
• exit

• Main () print
• ltform actionf1gt
• Enter first
• ltinput namen1gt
• lt/formgt
• f1 (form) print
• ltform actionf2gt
• ltinput hidden namen1
• valueform.n1gt
• Enter second
• ltinput namen2gt
• lt/formgt
• f2 (form) print
• The sum is
• form.n1 form.n2

62
The Reengineering Challenge

• Web interfaces have grown up
• from scripts to programs (or services)
• Need debugging, maintenance, evolution,
• We would like a Web compiler that
• Automatically splits programs by form
• Automatically propagates fields
• Preserves behavior in the face of bizarre control
  flow

63
The Key Insight

• The manual conversion
• simply implements the
• continuation-passing style
• transformation!

64
Change I/O to HTML

• n1 read
• Enter first
• n2 read
• Enter second
• print
• sum d\n
• (n1 n2)
• exit

• n1 read
• ltformgtEnter first lt/formgt
• n2 read
• ltformgtEnter second lt/formgt
• print
• ltpgtsum
• (n1 n2)lt/pgt
• exit

65
Change the Input Operator

• n1 read
• ltformgtEnter first lt/formgt
• n2 read
• ltformgtEnter second lt/formgt
• print
• ltpgtsum
• (n1 n2)lt/pgt
• exit

• n1 read/web
• ltformgtEnter first lt/formgt
• n2 read/web
• ltformgtEnter second lt/formgt
• print
• ltpgtsum
• (n1 n2)lt/pgt
• exit

66
CPS Create Function for theRest of the
Computation

• n1 read/web
• ltformgtEnter first lt/formgt
• n2

read/web/k ltformgtEnter first lt/formgt
function (n1) n2
67
The Result

• read/web/k
• ltformgtEnter first lt/formgt
• function (n1)
• read/web/k
• ltformgtEnter second lt/formgt
• function (n2)
• print ltpgtsum
• (n1 n2)lt/pgt

68
Lift Functions

• Main ()
• read/web/k
• ltformgtEnter first lt/formgt f1
• f1 (n1)
• read/web/k
• ltformgtEnter second lt/formgt f2
• f2 (n2)
• print ltpgtsum
• (n1 n2)lt/pgt

69
Propagate Free Variables

• Main ()
• read/web/k
• ltformgtEnter first lt/formgt f1
• f1 (n1)
• read/web/k/args n1
• ltformgtEnter second lt/formgt f2 n1
• f2 (n1, n2)
• print ltpgtsum
• (n1 n2)lt/pgt

70
Convert to Web API

• f1 (n1)
• read/web/k/args n1
• ltformgtEnter second lt/formgt f2 n1

f1 (form) print ltform actionf2gt ltinput
hidden namen1 valueform.n1gt
Enter second ltinput namen2gt lt/formgt
71
Resulting Web Application

• Main () print
• ltform actionf1gt
• Enter first
• ltinput namen1gt
• lt/formgt

f1 (form) print ltform actionf2gt ltinput
hidden namen1 valueform.n1gt
Enter second ltinput namen2gt lt/formgt
f2 (form) print ltpgtsum form.n1
form.n2lt/pgt
72
Summary

• Three transformations
• Make all value receivers explicit functions
• Make all functions top-level, replace free
  variables with explicit parameters
• Replace first-class functions with first-order
  representations

73
A Remarkable Coincidence

• These are known as
• Conversion to continuation-passing style
• Lambda lifting
• Closure conversion
• All are standard compiler transformations

74
The Payoff

• Implementations available for many languages
• Not too hard to implement
• Correctness preservation can be put on a formal
  foundation
• Enables next level of theorems (robustness in the
  presence of unreported events)

75
Moral

• Thinking about Web programming as a programming
  languages question helps
• better explain the nature of the Web
• understand common problems
• identify and implement solutions
• point to shortcomings in language semantics also!

76
Broader Payoff

• The same results apply to
• GUIs (especially wizards)
• network i/o (upcalls)
• blocking to non-blocking i/o
• event-driven server construction
• and a host of other interactive programs!

77
Why Cant Languages do Better?
78
Program Structure Reinstatement

• n1 read
• Enter first
• n2 read
• Enter second
• print
• sum d\n
• (n1 n2)
• exit

• n1 read/web
• ltformgtEnter first
• lt/formgt
• n2 read/web
• ltformgtEnter second
• lt/formgt
• print
• ltpgtsum
• (n1 n2)lt/pgt
• exit

79
API Implication

• Build a richer API!
• Standard APIs offer get_binding, maybe also
  cookie store and access
• Demand read/web as a primitive
• Programmers Stand up for your rights
• make server implementors work harder!

80
The PLT Scheme Web Server
81
The Real Primitive

• read/web lies slightly
• n1 read/web
• ltformgtEnter firstlt/formgt
• We provide send/suspend
• n1 send/suspend
• function (k)
• ltform actionkgtEnter firstlt/formgt

82
Addition Servlet

• n1 send/suspend
• function (k)
• ltform actionkgtEnter firstlt/formgt
• n2 send/suspend
• function (k)
• ltform actionkgtEnter secondlt/formgt
• print
• ltpgtsum
• (n1 n2)lt/pgt
• exit

83
In Scheme

• Parenthetical syntax aside, this is a functioning
  PLT Scheme servlet
• But we do more than just add numbers

84
Scenario
85
GDB Servlet
i/o_page (cmd) ltpgtltstronggtcmdlt/stronggtltbrgt
read( send (cmd, gdb_process))lt/pgt

• driver_loop (prev_cmd)
• let next_cmd
• send/suspend (i/o_page prev_cmd)
• driver_loop (next_cmd)

86
Broader Applicability

• Can generalize over other shell-like programs
  (just provide a parser)
• Can allow tree-shaped exploration with about ten
  more lines of code (try doing this in GDB!)
• Retrofitting Web interfaces
• easy for databases
• tricky for already interactive programs

87
The CONTINUE Server

• Conference paper manager
• Name inspired by START server (UMd)
• Handles submission and review phases
• Used by several conferences PADL 2002, PADL
  2003, FDPE 2003, Scheme 2003,
• Two interesting scenarios

88
Scenario
89
Email Confirmation Servlet

• addr send/suspend
• function (k)
• ltform actionkgtEmaillt/formgt

send/suspend / ignore response / function
(k) send_mail (addr, k) ltpgtWe sent mail
to addrlt/pgt
add_to_database (addr)
90
Scenario
91
Scenario
92
One-Shot Guest URLs

• guest_review
• send/finish
• function (k)
• ltform actionkgt
• Overall Rating ltinput gt
• Expertise ltinput gt
• lt/formgt

93
Embedded Server

• Embedding the server in the DrScheme programming
  environment enables
• sophisticated servlet debugging
• implement all documentation features through the
  Web
• indexing
• searching
• user-sensitive customization

94
Safety Errors
95
Safety Errors
96
Stepping Through Code
97
Stepping Through Code
98
Other Research Issues

• Type systems for forms (general problem for
  scripting applications)
• Soundness/safety lifted to level of Web
  interactions
• Formal verification (theorem proving, model
  checking) impossible at present

99
Summary
100
Interactive Web Programs

• Web interactions are more complex than we might
  suppose
• Programming language semantics offers analysis
  and solutions and theorems for formal software
  engineering!
• A custom server with a better API enables more
  natural and more complex programs

101
Future Work

• Some languages inhibit the transformations
• send/suspend and send/finish only scratch the
  surface
• Lots of work to be done on type systems
• Many other kinds of interaction may be amenable
  to a similar analysis
• Formally verify the server and CONTINUE
• Keep eating our own dogfood!

102
Collaborators

• Matthias Felleisen (Northeastern)
• Robert Bruce Findler (Chicago)
• Matthew Flatt (Utah)
• Paul T. Graunke (Northeastern)

103
Potential Problem

• Problem Transmitted Web pages contain the values
  of free variables, maybe including the current
  account balance.
• Question so what?
• Solution Encrypt data when sending, decrypt when
  it returns.

104
Potential Problem

• Problem Where does the state of mutable
  variables and mutable records go?
• Solution 1 Use store-passing transformation to
  make store explicit. Pass it along to the client
  (as hidden field in Web form).

105
Potential Problem

• Problem Users can clone browser windows, copy
  pages, and with it the store.
• Solution 2 Register cells (store) and retain on
  the server or move over as a cookie.