Lessons Learned from SETIhome

1
Lessons Learned fromSETI_at_home

• David P. Anderson
• Director, SETI_at_home
• Spaces Sciences Laboratory
• U.C. Berkeley
• April 2, 2002

2
SETI_at_home Operations
data recorder
3
Radio SETI projects

4
History and statistics

• Conceived 1995, launched April 1999
• Funding TPS, DiMI, numerous companies
• 3.6M users (.5M active), 226 countries
• 40 TB data recorded, processed
• 25 TeraFLOPs average over last year
• Almost 1 million years CPU time
• No ET signals yet, but other results

5
Public-resource computing

• Original GIMPS, distributed.net
• Commercial United Devices, Entropia, Porivo,
  Popular Power
• Academic, open-source
• Cosm, folding_at_home, SETI_at_home II
• The peer-to-peer paradigm

6
Characterizing SETI_at_home

• Fixed-rate data processing task
• Low bandwidth/computation ratio
• Independent parallelism
• Error tolerance

7
Millions and millions of computers

• Server scalability
• Dealing with excess CPU time
• Redundant computing
• Deals with cheating, malfunctions
• Control by changing computation
• Moores Law is true (causes same problems)

8
Network bandwidth costs money

• SSL to campus 100 Mbps, free, unloaded
• Campus to ISP 70 Mbps, not free
• First load limiting at 25 Mbps
• Now no limit, zero priority
• How to adapt load to capacity?
• Whats the break-even point (1GB per CPU day)

9
How to get and retain users

• Graphics are important
• But monitors do burn in
• Teams users recruit other users
• Keep users informed
• Science news
• System management news
• Periodic project emails

10
Reward users

• PDF certificates
• Milestone pages and emails
• Leader boards (overall, country, )
• Class pages
• Personal signal page

11
Let users express themselves

• User profiles
• Message boards
• Newsgroup (sci.astro.seti)
• Learn about users
• Online poll

12
Users are competitive

• Patched clients, benchmark wars
• Results with no computation
• Intentionally bad results
• Team recruitment by spam
• Sale of accounts on eBay
• Accounting is tricky

13
Anything can be reverse engineered

• Patched version of client
• efforts at self-checksumming
• Replacement of FFT routine
• Bad results
• Digital signing doesnt work
• Techniques for verifying work

14
Users will help if you let them

• Web-site translations
• Add-ons
• Server proxies
• Statistics DB and display
• Beta testers
• Porting
• Open-source development
• (will use in SETI_at_home II)

15
Client mechanism, not policy

• Error handling, versioning
• Load regulation
• Let server decide
• Reasonable default if no server
• Put in a level of indirection
• Separate control and data

16
Cross-platform is manageable

• Windows, Mac are harder
• GNU tools and POSIX rule

17
Server reliability/performance

• Hardware
• Air conditioning, RAID controller
• Software
• Database server
• Architect for failure
• Develop diagnostic tools

18
Whats next for public computing?

• Better handling of large data
• Network scheduling
• Reliable multicast
• Expand computation model
• Multi-application, multi-project platform
• BOINC (Berkeley Open Infrastructure for Network
  Computing)