Experience Predicting Application Server Memory Usage

1
Experience Predicting Application Server Memory
Usage

• Thomas Reidemeister
• Mohammad A. Munawar
• Paul A.S.Ward
• University of Waterloo
• April 2005

2
Outline

• Introduction
• Difficulty of predicting in dynamic systems
• Case Study Java-based Application server
• A simple approach to predicting memory-usage
• Some experiments
• Work in progress

3
Introduction

• Need for monitoring system health
• Performance
• Security
• Availability
• Correctness, etc
• Sources of information
• Events (logs, notifications, etc)
• System status (resources, parameters, etc)
• Purpose
• Anomaly detection and diagnosis
• Dynamic reconfiguration
• Planning

4
Building a Model
WorkloadGenerator
Response Monitor
System Under Observation
Model Builder
Model
5
Using the Model
Workload
Response Monitor
System Under Operation
System Model
Problem Determination Tool
6
Dynamic Environments

• Changing nature
• System configurations can change
• System components (software/hardware) can be
  updated/added/removed
• Examples
• Operating systems, Applications using Virtual
  Machines
• Java-based applications
• Java Virtual Machine (JVM)
• Automatic memory management
• Dynamic class-loading
• Just-in-time compilation
• Online byte-code instrumentation, etc

7
Uncertainty

• Why is prediction is difficult?
• Beside lack of knowledge of the future, we have
  to deal with
• Availability of information
• Quality of available information
• Performance impact
• Dynamic environments (adaptive behaviour)
• Complexity of the underlying process and
  dependencies

8
Case Study

• Predicting memory-usage of a Java-based
  application server
• Memory leaks and resource exhaustion is a severe
  problem
• Automatic memory management makes it interesting
• Prediction can help with
• Load-balancing
• Capacity planning
• Possibly leak-detection
• What can we achieve with the simplest approach
  and information that is readily available?

9
Approach
10
Setup

• Software
• IBM WebSphere Application Server 5.1 running
  Trade3
• IBM DB2 UDB 8.1
• httperf and httpunit-based custom workload
  generators
• Hardware
• P4 2.0 GHz, 1024MB machine with Windows Server
  2003 running WebSphere
• P4 2.0 GHz, 512MB machine with Redhat Linux
  running DB2
• Dual-processor Xeon 2.8GHz with Redhat Linux for
  load generation
• Connected with a Gbit-network

11
Profiling

• Obtain low-level memory allocation information
  from the JVM
• Use timing of request execution to match
  associated memory allocation
• Execute requests sequentially and at reasonably
  distant time-points to better isolate their
  effects
• Repeat requests a sufficient number of times and
  with different parameter values to obtain good
  representative statistics
• Build histograms for each request-type and select
  mode as the representative value

12
Profiling (cont.)

• Examples

Memory bytes allocated by an idle server
Histogram of a request-type
13
Profiling (cont.)

• Issues
• Typically, only millisecond time-resolution is
  available. How much memory does a sub-millisecond
  request use? When requests overlap in a
  millisecond-interval, we apportion equally
• Should we isolate unusual, irregular
  memory-allocation due to internal house-keeping?
• Profiles can change. A search request may require
  more memory depending how many add/updates have
  taken place since the last profiling time
• Profiling requires a dedicated and isolated server

14
Monitoring

• Precise memory-usage information is not readily
  available
• For example, Websphere interface to query
  memory-usage has a KB resolution.
• Reported memory-usage usually includes objects no
  more referenced and waiting to be freed
• Relatively precise information can be obtained
  from the JVM, if we profile GC events. However,
  the profiling interface does not provide a
  complete picture
• Let us monitor at regular interval, but at some
  reasonable granularity. We thus have an
  approximate actual usage.

15
Monitoring (cont.)
16
Prediction

• Predict number of requests of each type that we
  expect to see
• Use the above together with the profiles created
  earlier to estimate memory-usage
• Any technique to project future requests can be
  used. e.g. simple Exponential Smoothing
• What prediction period to use?
• Fixed-duration memory usage information is
  approximate
• Variable-duration (e.g. time to next GC event)
  need for estimating interval length

17
Prediction (cont.)

• Experiments
• How close do we get to actual memory usage, if we
  knew exactly the number of each request-type
  received during an interval?
• Let us use variable-duration intervals, which
  correspond to GC events observed to estimate
  actual memory-usage
• The prediction model is instantiated with request
  counts and request-type profile (mode-values)
• No model fine-tuning

18
Prediction (cont.)
19
Results

• We slightly under-estimate in general, but
  closely follow the trend when a sufficient number
  of requests are received during the prediction
  interval
• With some calibration, we should be able to
  achieve better results
• Caching is an important factor that needs to be
  incorporated in our prediction model

20
Improving Prediction

• Build better profiles
• Isolate object allocation due to server
  house-keeping
• Deal with intervals with small number of requests
• Use a different model?
• Take caching into account
• Model effect of present memory allocations on
  future requests

21
Fine-grained Monitoring

• Our approach can be used for coarse-grained
  monitoring
• What if we wanted closer monitoring of the
  absolute memory used at any time?
• Can we develop a general technique that can, for
  example, help detect memory-leaks?
• Again, we would need to model the behaviour of
  the JVM in details
• E.g. take into account the effect of requests,
  occurrence of GC events, caching, etc
• Exploring Dynamic Bayesian Networks

22
Summary

• Described a simple approach to predicting
  memory-usage of an application server
• Mentioned difficulties and issues
• Showed that this approach is able to closely
  follow memory-usage trend
• Depending on purpose, such simple prediction
  models can be very effective
• Prediction in dynamic systems is tricky and
  require that we resort to approximations
• But, we need to be intelligent about how to
  approximate

23
Our Research

• Resource monitoring for anomaly-detection in
  dynamic environments
• Associating status information (e.g. resource
  usage) with discrete system events for problem
  detection and diagnosis
• Adaptive monitoring

24
End

• Thank you.
• Questions are welcome.