Cloud versus physical cluster

1
Cloud Versus Cluster

• When creating distributed systems
• How do you decide
• cloud versus physical cluster ?
• Or a hybrid system ?
• What factors need to be considered ?
• What open source systems might be used ?

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
2
Cloud Versus Cluster - Factors

• How do you decide cloud versus cluster ?
• What factors need to be considered ?
• Cost
• Security
• Data volumes / velocity
• Data peak / scaling
• Other ?

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
3
Cloud Versus Cluster Costs

• What Costs do you need to consider ?
• For public cloud
• For private cluster
• For hybrid
• Are there tax implications ?
• Are there costs benefits due to location ?
• What about about end of life for your systems ?
• Moving data
• Mothballing systems
• Any vendor lock in ?

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
4
Cloud Versus Cluster Costs - Cluster

• What are the costs / issues for a physical
  cluster ?
• Hardware
• Hardware failures
• Energy rental / location costs
• Personnel
• End of life costs
• Data transmission
• Networking
• Tax and tax benefits
• Other ?

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
5
Cloud Versus Cluster Costs - Cloud

• What are the costs / issues for a cloud based
  cluster ?
• Depends on type used .. i.e. IaaS
• Depends on data volumes
• Consider data transmission
• Personnel some but less than cluster
• End of life costs Important !
• Costs to move data !!
• Vendor lockin
• Tax and tax benefits
• Other ?

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
6
Cloud Versus Cluster Security

• How important is security to you ?
• Is your public cloud secure ?
• Can you guarantee that ?
• Perhaps consider a private cloud
• Or a hybrid public private cloud ?
• How resilient / available is the resource ?

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
7
Cloud Versus Cluster Data Volumes

• The next two slides consider data volumes
• What about resource scaling ?
• Especially for the internet of things ( IoT )
• Potentially expect large demand peaks
• Need to scale up as well as down
• Also, scaling needs to follow resource demand
• Dont want to be over scaled
• Dont want to be under scaled

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
8
Peak Data Demand

• The diagram below expresses the idea that
• Periodic demand can be many times average ( i.e.
  IoT )
• This places pressure on the software stack for
• Queueing, processing, storage

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
9
Project Resource Allocation Over Time

• The diagram below expresses the idea that
• Resource supply to a project will not meet demand
  exactly
• There will be under and over supply
• Unused or unavailable resources waste money

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
10
Cloud Versus Cluster Architecture

• What open sourced systems might be used ?
• Mesos will help with
• Resource sharing and management
• Brooklyn will help with
• Deployment modelling / monitoring
• Will allow multiple cloud deploys
• Cloud Stack will help with
• Cloud creation / virtualisation
• See other presentations here ..

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
11
Cloud Versus Cluster Summary

• Decide what factors are important to you i.e.
• Is security more important than cost ?
• Does IoT mean that auto scaling is required ?
• Consider how you will deploy to cloud / cluster
• Consider end of life
• Create a cost model at start of project
• To consider all cost factors
• To justify your decision
• Because every project instance will differ !

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
12
Available Books

• See Big Data Made Easy
• Apress / Springer Jan 2015
• See Mastering Apache Spark
• Packt Oct 2015
• Find the author on Amazon
• www.amazon.com/Michael-Frampton/e/B00NIQDOOM/
• Connect on LinkedIn
• nz.linkedin.com/pub/mike-frampton/20/630/385

www.semtech-solutions.co.nz info_at_semtech-solutio
ns.co.nz
13
Contact Us

• Feel free to contact us at
• www.semtech-solutions.co.nz
• info_at_semtech-solutions.co.nz
• Or connect on LinkedIn
• We are always interested in
• New technology
• Opportunities
• Technology based issues
• Big data integration