Delivering High Performance Plone

1
Delivering High Performance Plone

• Internet Development Group (ILRT)
• Dominic Hiles

2
Outline

• Measuring performance and requirements
• Profiling code
• Native caching in Zope and ZEO
• Using Zopes RAMCache Manager
• Caching externally e.g. Squid
• Scaling it all up -gt load balancing and
  redundancy
• Whats on our horizon?

3
Measuring performance

• Decide how much performance you actually need
• Easier when migrating an existing system, just
  analyse the web logs and/or use monitoring tools
• Allow plenty of headroom
• Use benchmarking tools, e.g. Apache Bench to
  assess performance before and after changes
• Test different content types in isolation e.g.
  web pages, file objects, images, CSS
• Consider authenticated vs. anonymous access
• Use varying numbers of requests and request
  concurrency
• Use a reasonable sample size (for example all Web
  pages on the site)

4
Profiling code

• Identify the slow content types with
  benchmarking
• Profile the code using appropriate tools, e.g.
  ZopeProfiler, PTProfiler
• Analyse the slowest parts of the code in more
  detail and optimise, but dont necessarily aim
  for perfection!
• Re-test when youre done to confirm your diagnosis

5
Hassle-free caching

• Zope provides caching out-the-box
• The ZODB cache size controls the number of
  objects loaded from the ZODB and held in a
  RAM-based cache
• If youve got lots of memory, make this bigger
  (e.g. 5,000 20,000 objects)
• ZEO also provides a disk-based ZEO cache, which
  holds objects loaded from the ZEO server
• Youre likely to want to make this bigger than
  the default, e.g. 100MB
• Performance of both can be assessed using
  standard Zope tools

6
More complex caching

• Zope provides a RAMCacheManager
• Stores the results of e.g. a Python Script or
  Zope Page Template in RAM. Results should
  therefore be lightweight and/or not too numerous.
• Useful for
• objects that change rarely e.g. a Not Found
  (404) response
• objects that change according to a limited number
  of parameters e.g. month of year
• other specialised objects that you may have
  written to take advantage of it
• In a ZEO setup, the cache exists (and may be
  different) on each ZEO client
• This can complicate the process of
  programmatically removing objects from the cache
  when stale

7
HTTP Caching

• Zope/Plone can set a series of headers on the
  content it serves. These determine how long the
  content should be considered fresh for
• Upstream caches (either your own or external
  ones) and users browsers inspect these headers
  and hold the content for the specified time,
  checking back for updated content when necessary
• Cache strategies are complex to implement and
  depend on your specific configuration
• CacheFu (a Plone product) provides some common
  strategies and some good documentation. The
  Cacheability Engine is useful for testing and
  also provides some good documentation.
• Once you have the correct headers, just implement
  your own cache

8
Implementing an HTTP Cache

• Caches serve content much more quickly than Plone
  can, as they dont need to generate the content,
  just serve it
• Say, 2 requests/second vs. 100s 1000s
  requests/second
• A good caching strategy should see most of your
  content (90 in our case) being served from this
  cache, minimising the load on your Zope servers
• Use whatever suits your environment e.g. Squid,
  Apache 2.2 mod_cache

9
Scalability and redundancy

• ZEO
• Client/server architecture
• ZEO Clients do the hard processing work and can
  be spread across different servers
• This provides redundancy and scalability
• Bottleneck may just move to ZEO server instead!
• Keeping a warm copy of the ZEO server as well,
  gives a fully redundant Zope system
• Multiple ZEO clients need something to distribute
  the traffic between them -gt a load balancer e.g.
  Pound
• Ensure that this redundancy is carried through
  the rest of the architecture

10
Bristol CMS Architecture
11
(No Transcript)
12
Summary

• Measure performance before you start
• Profile code and fix any bottlenecks
• Use the free Zope caching
• Use Zopes RAMCache Manager
• Implement an HTTP cache
• Use ZEO and load balancing to create a redundant
  and scalable system

13
Other ideas were looking at

• memcached a generic cache that could be shared
  across all the ZEO clients (even on different
  servers)
• May use this in preference to Zopes
  RAMCacheManager
• Would allow cached SQL queries to be shared
  across all servers
• Splitting out the Data.fs, mounting different
  sites and/or the portal_catalog separately
• Increasing RAM
• Using multiple throw away servers for the ZEO
  clients
• Improving the specification of the ZEO server