An introduction to Apache Mesos

1
Apache Mesos

• What is it ?
• Beyond Hadoop
• Resource Sharing
• Mesos Intentions
• Architecture
• Users

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
2
Apache Mesos What is it ?

• A cluster manager
• Provides resource isolation / sharing
• Run distributed systems on the same pool of
  nodes
• Hadoop, Spark, Storm, Jenkins, K ...
• Cluster monitoring via web UI
• Tasks isolated via Linux Containers
• Scalable 10k's
• Written in C
• C/Java/Python API's for new app dev

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
3
Apache Mesos What is it ?

• Its fault tolerant
• Leader election via Zoo Keeper
• Multi platform
• Linux , OSX , Open Solaris
• It is now a full Apache project
• A Master / Slave based system

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
4
Apache Mesos Beyond Hadoop

• What requirements are there beyond Hadoop ?
• Cluster sharing / multi tenancy
• Higher resource utilization
• Faster response times
• Resource sharing
• A wider range of functionality
• More out of the box functionality
• Graphical Cluster monitoring
• Graphical drag and drop cluster manipulation

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
5
Apache Mesos Resource Sharing
www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
6
Apache Mesos Intentions

• Avoid multiple specialised clusters
• Avoid the need for silo'ed clusters
• Avoid the lower utilization that this brings
• Offer the ability for multi tenancy
• Which offers
• Lower hardware / maintenance costs
• Higher utilisation
• Scalability
• Higher fault tolerance
• Program for the Data Center

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
7
Apache Mesos Architecture
www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
8
Apache Mesos Users

• Companies using Mesos
• Air bnb
• Cloud Physics
• HubSpot
• Shopify
• Twitter
• Vimeo
• Among many others

www.semtech-solutions.co.nz info_at_semtech-solutions
.co.nz
9
Contact Us

• Feel free to contact us at
• www.semtech-solutions.co.nz
• info_at_semtech-solutions.co.nz
• We offer IT project consultancy
• We are happy to hear about your problems
• You can just pay for those hours that you need
• To solve your problems