Performance Management on Linux : The HP OpenView way

1
Performance Management on Linux The HP OpenView
way

• Dr. R. Krishnan
• OpenView Performance
• HP, Bangalore

2
agenda

• the challenge of performance management on Linux
• the openview solution

3
performance fundamentals
                                                 
             
 

• performance (response time) is a function of
  workload vs. available resources (capacity)
  characterized by
• fixed parameters
• disk architecture
• variable parameters
• memory and cpu
• application
• partitioning

trend
4
understanding response time
                                                 
             
 

• multiplicity of workloads (application dependent)
• multiplicity of processing capacities
• global and component response times

5
some common performance problems
database server
hub
users/ customers
application servers
web servers
hub
legacy system
routers
routers
firewall
load balancer
ISP1
ISP2
End-user connection issues
Internet Bottlenecks
ISP Performance and Peering
Load Balancer Problems
Errors
Faulty Web Server within a cluster
Application Server Issues
Devices Issues
Server HW OS Issues
Content Issues

• General
• insufficient memory
• incompatible service packs and DLLs
• excessive queuing requests
• too many secure HTTPS connections

• Network
• inadequate internet pipe
• hidden bottlenecks between website and ISP
  (peering)
• faulty hops (misdirected traffic and lost
  packets
• mis-configured software and incompatible hardware

• Web Server
• insufficient memory
• poor web server design
• high CPU usage

• App Server
• poor cache management and high CPU usage
• lack of memory
• poor session management
• poor database tuning

• DB Server
• inefficient indexing
• fragmentation
• out-of-date statistics
• faulty application design

6
the problem of IT silos
I knew we shouldnt have upgraded the database
last weekend
I hope that memory leak hasnt reappeared
So the application is crawling to its knees and
you are all telling me everythings fine?!!
Can you bring up those reports that show that
the network is ok.
7
disciplines required
                                                 
             
 
service management commit to service levels
monitor compliance communicate to business
operational monitoring service level objectives
customer experience infrastructure elements
resource optimization utilization
trends/baselines hot spot analysis
forecasting and planning
performance problem resolution bottleneck
location problem diagnosis
8
processes required

• Performance management is the art of resolving
  current performance bottlenecks.
• A reactive process that tends to correct design
  flaws or infrastructure faults and failures. It
  requires a quasi-real-time analysis of problems.
• Capacity management is the art of predicting
  future performance bottlenecks.
• A proactive process that relies on past
  experience to forecast potential problems. It is
  founded on statistical analysis.

9
roles required
                                                 
             
 
Darn that problem is tough to reproduce. I wish I
had put in better debugging
When do we need to migrate to that T-1 based
Frame Relay? Can our servers handle the Christmas
rush?
Application Developer
Are we reaching the service levels we set for my
customers?
I dont think the problem is in the network. Its
probably in the database.
There are a lot of performance degradations
today. Ill have to escalate these to Bob.
Performance Expert
10
The OpenView solution for Linux
11
OV architecture for a typical data center
12
OpenView Architecture

• collect - SNMP traps, application and system
  logfiles, customer variables, etc
• process - filter, prioritise, group and correlate
  events
• act buffer messages, forward messages or
  perform automatic actions

• collect messages sent from agents
• process central storage of events and config
  data
• presentation of events for problem resolution and
  forward to trouble ticketing systems
• act operator or automated actions
• SMART Plug-ins application mgnt
• service view

• actively monitor common internet services and
  protocols through simulation
• actively monitor custom web-based transactions
  e.g. shopping cart through recorder/playback
• passively monitors popular web servers, web
  application servers, e-commerce servers, firewalls

• call management
• incident management
• problem management
• change management
• work management
• service level management

• complete and live view of network
• proactive problem isolation
• event correlation (connector down, scheduled
  maintenance, repeated events and pairwise
  correlation)
• clear reporting
• manage global networks

13
where performance management fits
adaptive management
performance
assurance
cc
service management
customer experience
customer portal reporting
transactionanalysis
SLAs and service models
IT Process definition and automation
service management
Infrastructure management
provisioning
usage
performance
fault
application web services
data storage
network services
systems servers
14
operational monitoring with openview
                                                 
     
your customer or end-user
top-down
 
customer experience monitoring
 
event management resource management
infrastructure monitoring
your computing environment
bottom-up
15
diagnosing network problems

• network node manager intelligent diagnostics
• openview problem diagnosis
• openview performance insight for networks

16
diagnosing system problems

• openview performance manager and agent
• hp openview glanceplus
• top tools and compaq insight manager

17
diagnosing application problems

• hp openview transaction analyzer
• application response measurement (ARM)
• hp openview performance manager
• hp openview reporter
• 3rd party integrations (e.g. Oracle Enterprise
  Manager Diagnostics and Tuning Packs)

18
OpenView Performance management stack
web-based analysis, planning and resource
management
central management system
system performance and transaction management
repository and data provider
OpenView GlancePlus
managed node
provides the flexibility and depth required for
comprehensive infrastructure management
19
HP OpenView GlancePlus

• 1000 metrics
• 1 sec interval
• Global, App, Process data
• Application Response Measurements (ARM)
• System tables

20
OV Performance Manager 4.0

• Easy to use web GUI to provides distributed near
  real-time access to multiple data sources from
  heterogeneous systems

21
diagnosing slo alarms
e2e response
OVIS
internet
database
app server
web server
legacy app
network segment
Transaction Analyzer
desktop
ISP
firewall
web page
DNS DHCP..
O/S
EJB COM
JSP ASP
SQL
path
main frame
Performance Manager
3rd party
Keynote
OVIS
Transaction Analyzer
Transaction Analyzer
Transaction Analyzer
3rd party
OVTA
process
switch
router
Performance Manager GlancePlus
Performance Insight Problem Diagnosis
22
drilling down to the source
                                                 
     
OVIS

• what is the customer experience?

23
application response measurement

• industry standard interface
• method for easily monitoring service levels in
  standalone or distributed applications
• simple API to instrument source code to track
  transactions
• SDK that ships with OpenView Performance Agent
  and Glance products
• facility used by several other OpenView products

START
what is ARM?
ARM
business application
STOP
management agents applications
network
business application
ARM
COMPLETED
response time transaction status measurements dril
ldown client to server
24
thank you!