terasaur A Hybrid Cloud Approach to Publishing Large Files

1
terasaurA Hybrid Cloud Approach to Publishing
Large Files

• John Reuning, Paul Jones
• IBM Cloud Academy Conference 2012

2
Gigabytes to Terabytes

• Data sets
• Open source software
• Virtual machine images
• Archival-quality music and video

3
Large File Transfer Challenges

• Reliability
• Throughput
• Ease of use

4
Gigabytes to Terabytes
5
BitTorrent

• Fault tolerance
• Built-in checksums
• Start-stop
• Wealth of clients

6
BitTorrent ? terasaur

• Discovery
• Authority
• Persistence

7
Architecture
8
Seed Bank Network
9
Open Source Technologies

• Grails, Groovy
• PostgreSQL
• Python
• MongoDB
• RabbitMQ
• Apache Solr
• libtorrent (Rasterbar)

10
Architecture
11

• http//terasaur.org
• http//www.ibiblio.org/terasaur