HTTP and Apache

1
HTTP and Apache

• Roy T. Fielding
• eBuilt, Inc.
• The Apache Software Foundation
• fielding_at_apache.org

2
Overview

• HTTP Basics
• Apache httpd architecture
• Connection management (listening)
• Request Reading
• Finding a representation
• Conditional request evaluation
• Response Writing
• Connection management (closing)
• Future Improvements
• waka/2.0

3
HTTP Basics

• Web architecture
• Layered-client-server
• Uniform resource interface
• Simple, but powerful
• Semantics are visible to intermediaries
• HTTP extends interface across network

4
HTTP Message Syntax

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n
• HTTP/1.1 200 OK\r\n
• Date Thu, 09 Mar 2000 154009 GMT\r\n
• Server Apache/1.3.12\r\n
• Content-Type text/html\r\n
• Content-Language en\r\n
• Transfer-Encoding chunked\r\n
• Etag a797cd-465af\r\n
• Cache-control max-age3600\r\n
• Vary Accept-Language\r\n
• \r\n
• 4090\r\n
• ltHTMLgtltHEADgt

5
Request Envelope

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n

6
Request Control Data

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n

7
Request Metadata

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n

8
Request Metadata

• POST /nasdaq/trade.php HTTP/1.1\r\n
• Host www.datek.com\r\n
• User-Agent poseur/1.0 libwww/3.1\r\n
• Content-type www/x-urlencoded\r\n
• Content-length 58\r\n
• \r\n
• TickerLNUXquantity140actionsell
• typelimitamt298.00

9
Request Data

• POST /nasdaq/trade.php HTTP/1.1\r\n
• Host www.datek.com\r\n
• User-Agent poseur/1.0 libwww/3.1\r\n
• Content-type www/x-urlencoded\r\n
• Content-length 58\r\n
• \r\n
• TickerLNUXquantity140actionsell
• typelimitamt298.00

10
Request Data

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n

11
Response Envelope

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n
• HTTP/1.1 200 OK\r\n
• Date Thu, 09 Mar 2000 154009 GMT\r\n
• Server Apache/1.3.12\r\n
• Content-Type text/html\r\n
• Content-Language en\r\n
• Transfer-Encoding chunked\r\n
• Etag a797cd-465af\r\n
• Cache-control max-age3600\r\n
• Vary Accept-Language\r\n
• \r\n
• 4090\r\n
• ltHTMLgtltHEADgt

12
Response Control Data

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n
• HTTP/1.1 200 OK\r\n
• Date Thu, 09 Mar 2000 154009 GMT\r\n
• Server Apache/1.3.12\r\n
• Content-Type text/html\r\n
• Content-Language en\r\n
• Transfer-Encoding chunked\r\n
• Etag a797cd-465af\r\n
• Cache-control max-age3600\r\n
• Vary Accept-Language\r\n
• \r\n
• 4090\r\n
• ltHTMLgtltHEADgt

13
Response Metadata

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n
• HTTP/1.1 200 OK\r\n
• Date Thu, 09 Mar 2000 154009 GMT\r\n
• Server Apache/1.3.12\r\n
• Content-Type text/html\r\n
• Content-Language en\r\n
• Transfer-Encoding chunked\r\n
• Etag a797cd-465af\r\n
• Cache-control max-age3600\r\n
• Vary Accept-Language\r\n
• \r\n
• 4090\r\n
• ltHTMLgtltHEADgt

14
Response Data

• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n
• HTTP/1.1 200 OK\r\n
• Date Thu, 09 Mar 2000 154009 GMT\r\n
• Server Apache/1.3.12\r\n
• Content-Type text/html\r\n
• Content-Language en\r\n
• Transfer-Encoding chunked\r\n
• Etag a797cd-465af\r\n
• Cache-control max-age3600\r\n
• Vary Accept-Language\r\n
• \r\n
• 4090\r\n
• ltHTMLgtltHEADgt

15
Overview

• HTTP Basics
• Apache httpd architecture
• Connection management (listening)
• Request Reading
• Finding a representation
• Conditional request evaluation
• Response Writing
• Connection management (closing)
• Future Improvements
• waka/2.0

16
httpd Initialization

• httpd core
• apache-2.0/src/main/.c
• Initiate program
• http_main.c
• Read configuration files
• http_config.c, http_core.c
• Start logging
• http_log.c
• Start server listeners (MPM)
• apache-2.0/src/modules/mpm/

17
Connection management

• Listening and Accept
• mpm/prefork/prefork.c
• mpm/mpmt_pthread.c
• Connection set-up
• src/main/http_connection.c
• Modular hooks for protocol independence
• ap_process_http_connection()
• Calls ap_read_request()and we are finally at the
  level of HTTP

18
Request Reading

• src/main/http_protocol.c
• GET /Test/hello.html HTTP/1.1\r\n
• Host kiwi.ics.uci.edu8080\r\n
• User-Agent GET/7 libwww-perl/5.40\r\n
• \r\n
• Request Line
• read_request_line(r)
• Header fields
• get_mime_headers(r)
• Host determination
• http_vhost.c

19
Request Processing

• src/main/http_request.c
• Access control
• Directory walk
• Location walk
• File walk
• Handler selection
• Field pre-processing
• Content Handler

20
Request Body

• src/main/http_protocol.c
• Declaration
• ap_setup_client_block(r, policy)
• Test
• ap_should_client_block(r)
• Read until done
• ap_get_client_block(r, )
• Shortcut
• ap_discard_request_body(r)

21
Finding a representation

• Each content handler is responsible for
  associating the URI with a resource and applying
  the method, returning a representation as the
  response
• Redirection
• Negotiation
• Subrequests
• Errors

22
Conditional request evaluation

• Conditional requests modify method processing
  depending on the current state of the resource
  representation
• Last-Modified
• Etag
• Cache validation
• If-Modified-Since
• If-Nonematch
• Prerequisites
• If-Match
• If-Unmodified-Since
• Range-If
• If

23
Response Writing

• HTTP/1.1 200 OK\r\n
• Date Thu, 09 Mar 2000 154009 GMT\r\n
• Server Apache/1.3.12\r\n
• Content-Type text/html\r\n
• Content-Language en\r\n
• Transfer-Encoding chunked\r\n
• Etag a797cd-465af\r\n
• Cache-control max-age3600\r\n
• Vary Accept-Language\r\n
• \r\n
• 4090\r\n
• ltHTMLgtltHEADgt
• Status code and reason phrase
• Header fields
• Body

24
Response Writing

• Status code and reason phrase
• r-gtstatus
• r-gtstatus_line status_linesidx
• Header fields
• r-gtheaders_out
• r-gterr_headers_out
• Headers set by modules
• mod_expires.c
• mod_headers.c
• Header set Author John Q. Public

25
Send Body

• Body
• File
• Buffer
• IO-layer
• Transfer-encoding
• Length delimited
• Chunked
• Connection terminated

26
Connection management

• Persistent connections
• The big hairy if
• ap_set_keepalive(r)
• Connection tear-down
• Lingering close

27
Overview

• HTTP Basics
• Apache httpd architecture
• Connection management (listening)
• Request Reading
• Finding a representation
• Conditional request evaluation
• Response Writing
• Connection management (closing)
• Future Improvements
• waka/2.0

28
Future Improvements

• Configurable protocol elements
• Methods, status codes, field names
• Tokenized field manipulation
• Field name -gt token
• Field value -gt linked list
• Streaming IO Layers
• Stacked protocol manipulation
• Separate streams for data and metadata
• Token streams
• Bucket brigades
• Strings, Files, Cache handles

29
waka/2.0

• A replacement for HTTP/1.x
• Emphasizes the REST architectural style
• Ideas from Speros HTTP-ng and W3C HTTP-NG
• Packet streams
• Control, meta-metadata, metadata, data
• Prioritized bandwidth usage
• Self-descriptive
• All syntax an extensible standard
• All cache rules in response
• Simple quality of service indicators
• Transport independent
• Tokenized
• Optimized for memory transfer

30
Questions?

• Where to go for more information
• http//www.ics.uci.edu/pub/ietf/http/
• http//www.ics.uci.edu/fielding/talks/
• Places to see
• Front Door www.apache.org
• Developer Notes dev.apache.org
• PR Database bugs.apache.org
• Apache Week www.apacheweek.com
• OReilly Net www.oreilly.net/apache/