IT Performance Management

1
IT Performance Management
2
What is Performance Measurement?

• The ongoing process of assessing progress
  toward achieving pre-determined goals and
  objectives, including
• information on the efficiency and effectiveness
  with which resources are transformed into goods
  and services (outputs)
• the quality of those outputs (i.e., how well they
  are delivered to the clients and the extent to
  which clients are satisfied)

3
Performance Measurement (cont.)

• the effectiveness of operations in terms of their
  specific contributions to the strategic goals and
  business objectives
• Performance measurement includes
• self-assessments
• Inspections
• performance audits/reviews

4
What is Performance Management?

• In general, performance management refers to
  the use of performance measurement information to
  help
• define organizational goals and objectives in
  clear, tangible, and quantified terms
• develop project / activity plans designed to
  attain the goals and objectives
• routinely monitor actual performance vs. plans

5
Performance Management (cont.)

• analyze significant performance deviations
• advise (via routine performance reports) key
  managers of situations requiring attention
• formulate corrective action plans
• implement corrective actions to remedy
  performance deviations or modify plans

6
The Goal of Performance Management

• Assure that organizations link plans to strategic
  goals and business objectives
• Make funding decisions in light of project /
  activity benefits and outcomes that support those
  goals and objectives
• Actively manage projects and activities to assure
  that the planned benefits are realized

7
To Summarize

• Performance management is actively and
  proactively managing an organization to assure
  that it achieves pre-determined levels of
  performance.

8
Performance Management is Not

• Management that takes action only after a
  performance problem significantly affects the
  business. This is crisis management and damage
  control, not performance management.

9
What are the performance management issues for IT
organizations?

• Many senior managers believe that IT
  organizations are
• wasteful and inefficient (as evidenced by chronic
  project cost overruns, late deliverables, etc.)
• not aligned with strategic goals and business
  objectives

10
Performance Management Issues (cont.)

• Senior management is often handicapped in its
  decision-making for IT because it has difficulty
  determining
• how well IT is supporting the organizations most
  important strategic and business needs
• if they have been presented with the best IT
  options for decision-making

11
Performance Management Issues (cont.)

• the appropriate funding levels for IT projects /
  activities (how much to budget)
• whether the IT organization is likely to deliver
  the promised benefits and results on time and
  within budgetTypically, these are
  characteristics of organizations that have paid
  insufficient attention to measuring and managing
  IT performance and results.

12
Dont IT organizations already measure and manage
performance?

• When asked about the performance management
  systems and processes, CIOs and IT managers will
  typically speak at length in terms, such as
• the polling function of the topology map provides
  a snapshot of overall network vitality, and it
  can issue alerts when nodes become unavailable or
  are slow to respond. When a node is failing, it
  automatically alerts network management. To get
  more information on a node, we drill down for
  critical station statistics

13
Dont IT organizations already measure and manage
performance (cont.)?

• the protocol distribution function tracks total
  packets, octets, percent utilization, packet
  rate, and average packet size
• The above provides examples of what are
  referred to as technical performance measures
  (e.g., through "configuration management" and
  "fault management").

14
Dont IT organizations already measure and manage
performance (cont.)?

• While management of technical performance is an
  important part of an IT organizations job, IT
  managers are responsible for assuring that they
  are meeting their organizations highest
  priorities and business needs to the extent
  possible with the available resources.

15
Dont IT organizations already measure and manage
performance (cont.)?

• The performance information that most IT
  organizations lack relates to the non-technical
  aspects of the job. Can IT and senior management
  tell whether
• IT projects / activities are progressing as
  planned?
• there are telltale signs of future performance
  problems?

16
Dont IT organizations already measure and manage
performance (cont.)?

• there are troubled projects / activities that
  require higher managements intervention?
• actions already taken to correct problems
  actually worked?

17
A few key terms and concepts

• Three concepts underlying performance
  management, which often confuse novices, are an
  organizations "management system", performance
  measures, and performance metrics
• A management system is an interconnected set of
  processes, and a process is a set of activities
  that produce products or services (results). For
  performance management and measurement purposes,
  products and services are treated alike. That is,
  the output of a process might be a product (such
  as a computer or a project deliverable) or a
  service (such as PC support or help desk
  assistance). Products and services are tangible,
  measurable, and susceptible to analysis.

18
A few key terms and concepts (cont.)

• Performance measures are indicators that can be
  systematically tracked to assess progress made in
  achieving predetermined objectives (e.g., the
  number of NT workstations deployed to replace
  Unix workstations) or service levels (e.g., help
  desk call resolution time).
• Performance metrics are standards of measurement
  (such as minimum acceptable elapsed time to
  resolve help desk calls). A metric establishes a
  benchmark target that is compared to actual
  performance - the difference between the
  benchmark (the plan) and actual performance
  provides insight into what is working as planned
  and what is not.

19
How does non-technical IT performance management
work?

• Performance management involves the routine, and
  sometimes non routine, measurement of key aspects
  of IT project / activity performance and making
  this information available to decision-makers.
• The goal is to assure that the benefits of IT are
  realized as planned.

20
How does non-technical IT performance management
work (cont.)?

• Preparatory steps for performance management
  initiatives include
• definition of the overall analytic framework
• formulation of implementation plans,
• assignment of analytic responsibility,
• staff and management training
• conceptual design of the necessary supporting
  systems applications

21
How does non-technical IT performance management
work (cont.)?

• Once the foundation is established, performance
  management involves a number of recurring steps
• Gather performance data on those variables of
  interest to IT and senior management
• Analyze the data to determine normal (baseline)
  levels
• Determine appropriate performance thresholds for
  each important variable exceeding a threshold
  indicates a problem worthy of attention (e.g.,
  achievement of a major project milestone is 30
  days, or more, behind schedule) and possibly
  action when necessary

22
How does non-technical IT performance management
work (cont.)?

• Periodically (often monthly) monitor performance
  variables for deviations from established
  thresholds (a.k.a., variances). When a
  performance threshold is exceeded (e.g., vacant
  PC support staff positions increases to 10 when
  the acceptable vacancy threshold is 3), analysts
  identify a variance and determine its
  significance. Depending on a number of factors
  (e.g., severity or risk), analysts may take one
  of several actions, including

23
How does non-technical IT performance management
work (cont.)?

• identifying the project / activity as "at risk"
  and more closely analyzing performance during the
  next reporting period
• contacting the project / functional manager for a
  verbal variance explanation
• investigate the cause of the variance and
  formulate corrective action recommendations
• Report on performance to senior management (at
  least annually), and functional management (at
  least quarterly)
• Initiate corrective action when necessary

24
How does non-technical IT performance management
work (cont.)?

• The foregoing steps are part of an ongoing
  process established to support a performance
  planning, monitoring, and control system
• The intent is to provide management with a
  stable, recurring "early warning" mechanism to
  detect and correct incipient performance problems
  before they actually become problems.

25
How does non-technical IT performance management
work (cont.)?

• Performance management also employs ad hoc
  methods to assess and predict performance. These
  include techniques such as simulations, trend
  analysis, and in-depth performance reviews.

26
Simulation (e.g., "what if" analysis)

• Simulation can be used to project how changes
  in certain activities might affect othersE.g.
  What if the PC support group is deploying NT
  workstations to replace Unix workstations ahead
  of schedule, how will this affect the software
  development group that must convert Unix-based
  software for use on NT?
• Such simulation can effectively alert management
  to impending problems (e.g., the software
  development group cant keep up with the
  accelerated NT conversion schedule as a result,
  business users would lose mission-critical
  functionality).
• This knowledge would enable management to
  implement corrective measures (e.g., either put
  NT deployment back on its original schedule or
  accelerate the software development process by
  increasing development staff).

27
Trend Analysis

• Trend analysis can be used to reveal
  recurring difficulties that are not detected by
  ongoing performance monitoring. E.g. Senior
  management has imposed a standard
  employee-to-consultant ratio for all functional
  areas it is the same for accounts payable as it
  is for the IT organization. A trend analysis,
  possibly spanning several years, might reveal
  that the realities of the IT business
  consistently force the IT organization to either
  violate the standard or shut-down important
  activities because sufficient numbers of
  technical personnel cant be hired.
• Trend analysis would tell management that the
  problems of the past are likely to recur unless
  the standard for IT is changed.

28
Performance Reviews (a.k.a., performance audits)

• Performance reviews can provide IT and senior
  management with objective, often comprehensive,
  assessments of the performance of a major
  function, activity, system, or process.
• Performance reviews (audits) are not financial
  audits, although they may include financial
  elements (e.g., a functional performance review
  might include evaluation of the adequacy of
  budget resources to accomplish business
  objectives).

29
Getting Started

• Measuring performance isn't easy too often,
  performance measurement initiatives falter
  because
• Those charged with planning and implementing the
  initiative had little or no prior experience with
  performance management and measurement (often,
  with the best of intentions, they repeat common,
  predictable, and sometimes fatal mistakes)

30
Getting Started (cont.)

• Managements information needs were not
  identified in advance, and a well-thought-out
  conceptual framework and implementation plan was
  not developed (for example, after a years work
  the first performance report is submitted to
  senior management two months late. Senior
  management then returns the report saying "this
  is not the information we need to see")
• Too much performance information materialized too
  soon (the organizations capacity to assimilate,
  interpret, and react to it was overwhelmed)

31
To help assure the success of a performance
management initiative

• Understand managements information needs
• Have a clear picture of what you are trying to
  achieve and how you will do it
• Involve people who have actually established and
  managed performance management systems
• Study what other organizations are doing - learn
  what works and what doesnt
• Manage expectations - make sure all key
  participants understand and agree on what will be
  accomplished and when
• Use a limited pilot to gain experience (in other
  words, confine your mistakes to a small area that
  you can fix quickly), and demonstrate early
  results