The TEK System: Browsing the Web in Low Connectivity Communities

1
The TEK SystemBrowsing the Web in Low-
Connectivity Communities

• Bill Thies, Libby Levison, Saman Amarasinghe
• MIT Laboratory for Computer Science
• http//cag.lcs.mit.edu/tek

2
Web Browsing Current Method
GOOGLE
user
2. Send query to search engine, wait for reply
3. Read through results
5. If results good, click on site, wait for reply
6. Read through site
3
Web Browsing Current Method
GOOGLE
ISP
user
What are the barriers to access?

• High telephone fees
• High ISP fees
• Low-Bandwidth
• Low-Connectivity

4
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
user
SITE 1
SITE 2
Solution has two components
5
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
6
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
user
SITE 1
SITE 2
Solution has two components
7
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
8
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
9
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
10
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
11
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
12
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
13
Web Browsing TEK Approach
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
1. TEK Server returns low-bandwidth results
SITE 2
Solution has two components
TEK Time Equals Knowledge
14
Outline

• Protocol Details
• Rationale
• Server Details
• Current Status / Demo

15
Outline

• Protocol Details
• Rationale
• Server Details
• Current Status / Demo

16
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
17
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
18
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
19
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
20
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
21
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
22
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
23
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
24
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
25
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
26
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
27
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
28
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
29
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
30
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
6. Administrator finds results in email and opens
them
31
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
6. Administrator finds results in email and opens
them
32
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
SITE 1
user
SITE 2
SITE 2
1. Users start web browser and login to TEK proxy
2. Browse local web pages as if they were
connected
3. If page is not local, enqueue a search query
in proxy
4. When convenient, proxy connects to ISP and
sends queries
5. Server performs search, filters results and
replies
6. Administrator finds results in email and opens
them
7. Users can view results on future logins
33
Protocol Details
GOOGLE
GOOGLE
TEK SERVER
ISP
TEK PROXY
SITE 1
user
Additional features

• Accumulates digital reference library on client
• Reliable email-based communication protocol

34
Outline

• Protocol Details
• Rationale
• Server Details
• Current Status / Demo

35
Rationale I Decreased Cost

• Email accounts cheaper than web access
• Phone lines are cheaper, clearer, and more stable
  during off-peak hours
  
• Connection time is shorter
• User reads pages offline, not while connected
• Content is direct from ISP, not a distant server
• Results are more compact due to TEK Server
• Local web cache can prevent some searches

36
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

37
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

38
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

39
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

40
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

41
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

42
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

43
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

44
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

45
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

46
Rationale II Improved Reliability
GOOGLE
ISP
GOOGLE
TEK SERVER
user
SITE 1
SITE 2

• Email reduces dependence on network
• Never need continuous path from client to server

47
Rationale III Improved Convenience

• Sending email at night
• Frees telephone for other daytime uses
• Avoids daytime traffic in connecting to ISP
• Offline viewing of results is quick and reliable
• More people can use computer during daytime
• More relevant results thanks to TEK server

48
Outline

• Protocol Details
• Rationale
• Server Details
• Current Status / Demo

49
Server Details

• Extensive server-side processing
• Optimized for bandwidth, not speed
• Gathers pages from other search engines
• Filtering
• Removes duplicate or similar pages
• Looks for paragraph text, not just links
• Dither or remove images
• Compresses the result set
• Keeps track of each user
• Avoids sending a page twice to a given machine
• Enables more intelligent page ranking

50
Outline

• Protocol Details
• Rationale
• Server Details
• Current Status / Demo

51
Status

• Fully-functional prototype of TEK system
• Implemented in Java
• Hope to deploy by summer 2002
• Looking for a community of developers, beta
  testers, and people who can help us deploy!
  
• Future Work
• More intelligent query-builder on client
• Consider multi-language interfaces

52
The TEK Team

• Professor Saman Amarasinghe
• Post-doctoral fellow
• Dr. Libby Levison
• Masters Students
• Bill Thies
• Janelle Prevost
• Yuliya Litvak

• Undergraduates
• Tazeen Mahtab
• Genevieve Cuevas
• Saad Shakhshir
• Alexandro Artola
• Binh Vo
• Sheldon Chan
• Sid Henderson
• Mark Halsey