Welcome to ORS, Inc.

1
Results from the Performance Management
Survey By Janice Rouiller and Laura Stouffer,
SAIC and Amy Culbertson, OQM 1 April
2004 Note All data analyses were conducted
by SAIC.
2
Table of Contents

• Purpose and Methodology.3
• Respondent Information.6
• Perceptions of Cycle 2 PM Implementation
  ..14
• Perceptions of Needs for Cycle 3
  47
• Organizational Climate Perceptions
  63
• Customer Scorecard Ratings83
• Summary of Comments....96
• Summary....102
• Appendix - Individual Comments from
  Survey..107

3
Purpose

• Administer a customized version of the ORS
  Customer Scorecard to
• Gather feedback on past Performance Management
  (PM) implementation efforts
• Collect needs assessment information to tailor
  the on-going implementation of PM
• Gather data on the required Customer Scorecard
  dimensions
• Collect organizational culture change information
  for the Evaluation Study

4
Methodology

• Administer the survey at the completion of the
  second cycle of Performance Management
  implementation on an organization-wide basis
• Which occurred in January 2004
• Designed and tested web survey during February
  2004
• Survey went live on 2 March 2004
• Email requests sent to all PM Team Leaders,
  Members, and additional senior ORS/ORF managers
• First reminder email sent 8 March
• Second reminder email sent 15 March
• Closed survey field period 22 March
• Data file sent to contractor for analysis

5
Survey Distribution
Number of email recipients 196 Numb
er of respondents 70
Response Rate 36
6
Respondent Information
7
Respondents by Role
N 69
Number of Respondents
Note Multiple responses allowed. One
respondent skipped this question.
8
Respondents by Organization
N 69
Number of Respondents
Note One respondent skipped this question.
9
Respondents by Program Area
N 68
Number of Respondents
Note Multiple responses allowed. Two
respondents skipped this question.
10
Experience with the PM Process
N 66
N 65
N 63
N 65
N 65
Percent Yes
11
Approximately what percent of time did you spend
on PM-related activities during Jan 03 - Jan 04?
N 69
Number of Respondents
Percent of time Spent
12
On average, how often did your PM team meet
during the time period Jan 03 - Jan 04?
N 69
Number of Respondents
How Often PM Team Met
13
To what extent did your team members work between
meetings to implement PM?
N 69 Mean 6.17 Median 6
To No Extent
To a Great Extent
14
Perceptions of Cycle 2 PM Implementation (January
2003 - January 2004)
15
How helpful did you find the following in
implementing PM?
Not at all Helpful
Extremely Helpful
Mean Ratings
16
How helpful did you find the following tools in
implementing PM?
N 68
N 67
N 69
N 67
N 68
Not at all Helpful
Extremely Helpful
Mean Ratings
17
How helpful did you find the following tools in
implementing PM? Listings of the Services
Hierarchy
N 68 Mean 5.39 Median 6
Not at all Helpful
Extremely Helpful
18
How helpful did you find the following tools in
implementing PM? Performance Management Plan
(PMP) Template
N 67 Mean 6.91 Median 8
Extremely Helpful
Not at all Helpful
19
How helpful did you find the following tools in
implementing PM? Strategy Mapping
N 69 Mean 6.28 Median 7
Not at all Helpful
Extremely Helpful
20
How helpful did you find the following tools in
implementing PM? Measures Roadmap Template
N 67 Mean 5.82 Median 7
Not at all Helpful
Extremely Helpful
21
How helpful did you find the following tools in
implementing PM? PM Presentation Template for
Conference
N 68 Mean 6.94 Median 8
Not at all Helpful
Extremely Helpful
22
How helpful did you find the following data
analysis services in implementing PM?
N 67
N 68
N 68
N 68
N 67
Extremely Helpful
Not at all Helpful
Mean Ratings
23
How helpful did you find the following data
analysis services in implementing PM Design of
Data Collection Forms
N 67 Mean 5.46 Median 6
Not at all Helpful
Extremely Helpful
24
How helpful did you find the following data
analysis services in implementing PM
Establishing Data Collection Methodologies
N 68 Mean 5.97 Median 7
Not at all Helpful
Extremely Helpful
25
How helpful did you find the following data
analysis services in implementing PM Graphing
Data
N 68 Mean 6.34 Median 7
Not at all Helpful
Extremely Helpful
26
How helpful did you find the following data
analysis services in implementing PM Process
Behavior Chart Analysis
N 68 Mean 5.54 Median 6
Not at all Helpful
Extremely Helpful
27
How helpful did you find the following data
analysis services in implementing PM ORS
Customer Scorecards
N 67 Mean 6.00 Median 7
Not at all Helpful
Extremely Helpful
28
How helpful did you find the following
communication vehicles in implementing PM?
N 68
N 68
N 68
Not at all Helpful
Extremely Helpful
Mean Ratings
29
How helpful did you find the following
communication vehicles in implementing
PM Performance Management Website
N 68 Mean 5.26 Median 5
Not at all Helpful
Extremely Helpful
30
How helpful did you find the following
communication vehicles in implementing PM PM
Related Emails from OQM
N 68 Mean 5.93 Median 6
Not at all Helpful
Extremely Helpful
31
How helpful did you find the following
communication vehicles in implementing PM PM
Conference
N 68 Mean 5.03 Median 5
Not at all Helpful
Extremely Helpful
32
How helpful did you find assistance from the
following groups in implementing PM?
N 69
N 69
N 68
N 68
Not at all Helpful
Extremely Helpful
Mean Ratings
33
How helpful did you find assistance from the
following groups in implementing PM? OQM Staff
N 69 Mean 7.07 Median 8
Not at all Helpful
Extremely Helpful
34
How helpful did you find assistance from the
following groups in implementing PM? PM
Consultants
N 69 Mean 8.13 Median 9
Not at all Helpful
Extremely Helpful
35
How helpful did you find assistance from the
following groups in implementing PM? Other PM
Teams
N 68 Mean 4.17 Median 4
Not at all Helpful
Extremely Helpful
36
How helpful did you find assistance from the
following groups in implementing PM? Your
Supervisor
N 68 Mean 6.88 Median 8
Not at all Helpful
Extremely Helpful
37
Associate Director Involvement in PM
Mean Ratings
N 69
N 62
To a Great Extent
To no Extent
N 69
N 63
Extremely Helpful
Not at all Helpful
38
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle?
N 69
N 69
N 69
N 69
N 69
N 69
N 69
N 68
To No Extent
To a Great Extent
Mean Ratings
39
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Balanced Scorecard Approach to PM
N 69 Mean 6.37 Median 7
To No Extent
To a Great Extent
40
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Methods to Gather Data
N 69 Mean 6.16 Median 7
To No Extent
To a Great Extent
41
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Methods to Analyze Data
N 69 Mean 6.22 Median 7
To No Extent
To a Great Extent
42
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Presentation Preparation and
Delivery
N 69 Mean 6.02 Median 7
To No Extent
To a Great Extent
43
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Working in a Team
N 69 Mean 6.24 Median 7
To No Extent
To a Great Extent
44
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Using Excel
N 69 Mean 5.32 Median 5
To No Extent
To a Great Extent
45
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Using PowerPoint
N 69 Mean 5.27 Median 6
To No Extent
To a Great Extent
46
To what extent has your knowledge/skills/abilities
(KSAs) improved in the following areas during
this PM cycle? Using Visio
N 68 Mean 4.04 Median 3
To No Extent
To a Great Extent
47
Perceptions of Needs for Cycle 3
48
In planning for the next PM cycle, how helpful
would each of the following be to your PM Team?
Not at all Helpful
Extremely Helpful
Mean Ratings
49
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team?
N 69
N 69
N 69
N 68
N 68
N 67
Not at all Helpful
Extremely Helpful
Mean Ratings
50
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team PM Consultants Provided by OQM
N 69 Mean 8.31 Median 9
Not at all Helpful
Extremely Helpful
51
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team OQM Staff
N 69 Mean 7.40 Median 8
Not at all Helpful
Extremely Helpful
52
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team Regular Meetings with your
Management
N 69 Mean 6.75 Median 7
Not at all Helpful
Extremely Helpful
53
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team Rotating Quarterly Presentation
Meetings
N 68 Mean 5.61 Median 6
Not at all Helpful
Extremely Helpful
54
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team Yearly Performance Management
Conference
N 68 Mean 5.40 Median 6
Not at all Helpful
Extremely Helpful
55
In planning for the next PM cycle, how helpful
would each of the following tools/resources be to
your PM Team IT Support to Establish Data
Collection Systems
N 67 Mean 7.35 Median 8
Not at all Helpful
Extremely Helpful
56
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team?
N 67
N 68
N 68
N 68
N 67
N 67
Not at all Helpful
Extremely Helpful
Mean Ratings
57
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team? Balanced Scorecard Methodology
N 67 Mean 5.63 Median 6
Not at all Helpful
Extremely Helpful
58
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team? Setting up Data Collection
Systems and Forms
N 68 Mean 6.02 Median 7
Not at all Helpful
Extremely Helpful
59
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team? Analyzing Data Through Graphs
N 68 Mean 6.05 Median 7
Not at all Helpful
Extremely Helpful
60
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team? Analyzing Processes with
Process Behavior Charts
N 68 Mean 6.18 Median 7
Not at all Helpful
Extremely Helpful
61
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team? Customer Assessment through
Interviews, Focus Groups and Surveys
N 67 Mean 6.27 Median 7
Not at all Helpful
Extremely Helpful
62
In planning for the next PM cycle, how helpful
would each of the following training sessions be
to your PM Team? ORS ABC Cost Accounting Model
N 67 Mean 6.08 Median 7
Not at all Helpful
Extremely Helpful
63
Organizational Climate Perceptions
64
Organizational Climate PerceptionsSummary
N 68
N 68
N 68
N 68
N 68
N 68
N 67
N 68
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
65
I see the value of performance management for
improving business operations in my area.
N 68 Mean 3.72 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
66
My organization (ORS or ORF) is committed to the
performance management effort.
N 68 Mean 4.21 Median 5
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
67
I understand what the Services Hierarchy is and
its purpose.
N 68 Mean 3.73 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
68
The culture of my organization (ORS or ORF) is
changing to be more results-oriented.
N 68 Mean 3.83 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
69
Managers in my organization (ORS or ORF) believe
accountability is an important organizational
value.
N 68 Mean 4.13 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
70
Performance management has contributed to
improvements in my area.
N 68 Mean 3.46 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
71
Performance management will assist my
organization if we have to go through an A-76
competition.
N 67 Mean 4.01 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
72
My PM Team (or the groups I am responsible for)
are actively involved in data collection and
analysis.
N 68 Mean 4.04 Median 4
Strongly Disagree
Strongly Agree
Neither Agree nor Disagree
73
Organizational Climate Perceptions by Program Area
74
Program and Employee Services Climate Perceptions
Percent Agree or Strongly Agree
N 16
N 16
N 16
N 16
N 16
N 16
N 16
N 16
75
Scientific Resources Climate PerceptionsPositive
Responses
Percent Agree or Strongly Agree
N 15
N 14
N 15
N 15
N 15
N 15
N 15
N 15
76
Security Services Climate Perceptions
Percent Agree or Strongly Agree
N 5
N 5
N 5
N 5
N 5
N 5
N 5
N 5
77
Management Services Climate Perceptions
Percent Agree or Strongly Agree
N 11
N 11
N 11
N 11
N 11
N 11
N 11
N 11
78
Real Estate and Facilities Climate Perceptions
Percent Agree or Strongly Agree
N 12
N 12
N 12
N 12
N 12
N 12
N 12
N 12
79
Organizational Climate Perceptions by Role
80
Division Directors/Office Directors Climate
Perceptions
Percent Agree or Strongly Agree
N 7
N 7
N 7
N 7
N 7
N 7
N 7
N 7
81
PM Team Leaders Climate Perceptions
Percent Agree or Strongly Agree
N 18
N 18
N 18
N 18
N 18
N 18
N 18
N 18
82
PM Team Members Perceptions
Percent Agree or Strongly Agree
N 43
N 42
N 43
N 43
N 43
N 43
N 43
N 43
83
Customer Scorecard Ratings
84
Customer Scorecard Ratings for PMOverview for
Cycle 2
N 68
N 68
N 67
N 68
N 68
N 68
N 68
N 68
N 65
Unsatisfactory
Outstanding
Mean Ratings
85
Cost Satisfaction Ratings
N 68 Mean 6.12 Median 7
Unsatisfactory
Outstanding
86
Quality Satisfaction Ratings
N 68 Mean 6.82 Median 8
Unsatisfactory
Outstanding
87
Timeliness Satisfaction Ratings
N 67 Mean 6.58 Median 8
Unsatisfactory
Outstanding
88
Reliability Satisfaction Ratings
N 68 Mean 6.83 Median 8
Unsatisfactory
Outstanding
89
Availability Satisfaction Ratings
N 68 Mean 7.03 Median 8
Unsatisfactory
Outstanding
90
Responsiveness Satisfaction Ratings
N 68 Mean 7.14 Median 8
Unsatisfactory
Outstanding
91
Convenience Satisfaction Ratings
N 68 Mean 6.78 Median 8
Unsatisfactory
Outstanding
92
Competence Satisfaction Ratings
N 68 Mean 7.06 Median 8
Unsatisfactory
Outstanding
93
Handling of Problems Satisfaction Ratings
N 65 Mean 6.33 Median 7
Unsatisfactory
Outstanding
94
Customer Scorecard Ratings for PMTrend Analysis
Mean Ratings
N 68
N 37
N 68
N 70
N 67
N 67
N 68
N 67
Unsatisfactory
Outstanding
95
Customer Scorecard Ratings for PMTrend Analysis
(cont.)
Mean Ratings
N 68
N 69
N 68
N 68
N 68
N 66
N 68
N 69
N 65
N 65
Unsatisfactory
Outstanding
96
Summary of Comments
Note 34 out of 70 respondents made comments
(49).
97
What did you value most about this cycle of
performance management?

• Assistance provided by consultants and OQM staff
  (9)
• Professional guidance and support from OQM staff.
• Knowledgeable and helpful consultants.
• Increased involvement of OQM staff in team
  meetings.
• Implementing measurement systems (6)
• Developing systems to gather data.
• Using data to improve processes.
• Knowledge and Experience Gained (4)
• Experience has led to fuller appreciation of the
  Performance Management Process.
• Continued experience is leading to greater
  comfort with the process.

Note Comments were sorted into categories. Most
frequently mentioned categories listed first.
98
What did you value most about this cycle of
performance management? (cont.)

• Understanding our processes (4)
• Greater understanding of the processes.
• Actually seeing improvements in processes.
• Training Offered (2)
• Measures Workshop and Balanced Scorecard
  Orientation.
• Just in Time training.

Note Comments were sorted into categories. Most
frequently mentioned categories listed first.
99
What obstacles were most challenging to you
during this cycle of performance management?

• Lack of time/resources (10)
• Office is short staffed and workloads at all-time
  high
• Difficulty scheduling OQM staff for timely
  meetings
• Office identified late and struggled to catch
  up with process
• Data Collection and Measurement (9)
• Manual data collection
• Definition of measures difficult
• Analysis of measures difficult
• Applying Balanced Scorecard Approach (6)
• Lack of flexibility in application
• Use of resulting data to make changes is absent

Note Comments were sorted into categories. Most
frequently mentioned categories listed first.
100
What obstacles were most challenging to you
during this cycle of performance management?
(cont.)

• Lack of management involvement (3)
• PMP not reviewed
• Close communication missing
• Conference (2)
• Lack of management presence
• Full presentation too time-consuming focus
  should be on presentation of a few key indicators

Note Comments were sorted into categories. Most
frequently mentioned categories listed first.
101
How can we help integrate PM into your daily
business activities?

• Foster use of data. (7)
• Show how data is being used by Service Groups and
  by ORS/ORF Managers.
• Follow up with Service Groups on implementation
  of objectives.
• Continue support currently provided. (6)
• Maintain communication throughout the year.
• PMP is very helpful in its present form.
• Increase management involvement in process. (2)

102
SummarySurvey Respondents

• Respondents were primarily from ORS (3/4th)
• About half of the PM team leaders answered the
  survey
• Smallest response from the security program area
• Respondents were experienced with PM (3/4th) and
  had received training (3/4th)
• About 74 of respondents spent 30 or less of
  their time during the past year on PM
• Team meetings typically occurred biweekly (40)
  or once a month (36)

103
Summary (cont.)Feedback on Past Cycle

• The five most helpful tools/resources were the
  Consultants, OQM Staff, PM Presentation Template,
  PMP template, their supervisors
• Respondents reported just above mid-scale
  improvements to their KSAs for most topics listed
• Respondents valued assistance provided,
  implementing measurement systems, knowledge and
  experience gained, understanding of processes
• Obstacles included lack of time/resources, issues
  with data collection/measurement, applying the
  BSC approach

104
Summary (cont.)Planning for Next PM Cycle

• Most helpful to teams for the future would be PM
  consultants, OQM staff, IT Support for data
  collection systems
• To foster the integration of PM into daily
  business operations, respondents said we need to
  help foster the use of data, continue support
  provided, increase management involvement in
  process

105
Summary (cont.)Organizational Climate

• Majority of respondents either agree or strongly
  agree that
• Their organization is committed to PM (76)
• They see the value of PM for improving business
  operations (68)
• Managers in their organization believe
  accountability is important (68)
• The culture of their organization is changing to
  be more results-oriented (63)
• Senior managers more strongly endorse these items
  compared to team leaders and team members

106
Summary (cont.)Customer Scorecard Trend Analysis

• Ratings on all dimensions are lower in Cycle 2
  compared to Cycle 1
• Current ratings
• Highest on responsiveness, competence,
  availability
• Lowest on cost, handling of problems, timeliness

107
Appendix

• Individual Comments From Survey

108
What did you value most about this cycle of
performance management? Individual Comments

• Really customizing the strategic plan to our
  particular area . Implementing a system to gather
  data to understand our service delivery.
• Measuring customer service performance is
  valuable, however, I am not sure much was gained
  by this exercise over the past several years
  other than taking people away from assigned
  duties.
• Understanding the process.
• Nothing.
• News I can use!
• Results from data collection - mostly trend
  reporting.
• Nothing - I waste of time and tax payers money.
• Amy is a wealth of knowledge and always eager to
  assist when needed.
• The increased involvement of the OQD staff in the
  team meetings.

109
What did you value most about this cycle of
performance management? (cont.) Individual
Comments

• Our OQM representative and PM consultant listened
  to our concerns about this method of measuring
  our productivity. They convinced us that this
  type of measurement and representation could
  represent our points of view and culture better
  than we imagined.
• Continuity from prior year. Beginning to provide
  a context, esp. in using strategy mapping.
• Our consultant was great. This is the first year
  I understood what we were doing.
• It spurred us to initiate and develop a new call
  center that we know use to keep track of our work
  and help keep more statistics on the work we do.
  The concept of coordinated, consistent
  performance mgt techniques and metrics is a good
  one.
• I support the formalized training for ABC,
  Balance Scorecard etc. However, it was extremely
  beneficial to have just in time training provided
  by Pat Patterson as we progressed through our PMP
  process.

110
What did you value most about this cycle of
performance management? (cont.) Individual
Comments

• Learning about data collection and how to use
  that data to develop improvement plans.
• OQM supported a program review and a root cause
  analysis as part of this years PMP that was
  extremely useful. Really appreciate the
  expertise and responsiveness of both OQM and
  contract staff. Our contract staff is a real
  standout.
• The Measures Workshop and Balanced Scorecard
  Orientation.
• Our consultant was extremely helpful - Pat
  Patterson. Carmen, Amy and Joe were also
  extremely helpful even though they were not our
  consultant. They answered our questions before
  we got to Josh (who was not our OQM consultant)
  but he was helpful.
• I gained experience in and fuller appreciation
  for the importance of collecting and analyzing
  data about our Division's efficiency and
  effectiveness. The PMP has provided insights
  that are guiding us in improving both of these
  areas. The data we collected and analyzed had
  use beyond informing us about our own
  operation--when SDs external to our program
  recently requested detailed information about our
  operations, we were able to respond quickly and
  completely.

111
What did you value most about this cycle of
performance management? (cont.) Individual
Comments

• It's become so familiar that I spend more time on
  the data versus the process.
• OQM and the consultant's support throughout the
  process.
• Experience under my belt in all areas this cycle.
• I valued the professional guidance provided by
  OQM. Ms. Amy instilled the knowledge to our group
  on why we perform this task which gave more
  definition and purpose for doing it. She was
  outstanding and I would like to commend her for a
  job well done. From depositing her guidance and
  understanding into our group, I have learned the
  reasons for the PMP which will reveal to the
  manager how to effectively administer the
  business.
• Having a chance to make suggestions as to what
  can be improved in the process. Actually seeing
  some improvements in the different layers of the
  process - this is invaluable!

112
What obstacles were most challenging to you
during this cycle of performance
management?Individual Comments

• The OQM staff member for my team, was not
  committed, only attended less then half of the
  mtgs.
• Too many man hours are required for the PM
  Initiative during a time when FTEs are reduced
  and workloads are at an all-time high
  Question the cost effectiveness of the PM
  Initiative. Do we have any data the supports the
  cost (i.e., man-hours, OQM salary, consultants)
  of the PM Initiative?
• THE BSC is not being implemented right, nor are
  the results being used for anything. I think the
  current system is too prescriptive, it's not
  flexible enough for organizations to incorporate
  their own modifications or metrics. It has to be
  balanced scorecard, which is ok, but the not only
  way.
• The Associate Director never reviewed our
  template though given to him in the beginning
  months. Management in our office is in process
  of changing and I am apprehensive to new
  management.

113
What obstacles were most challenging to you
during this cycle of performance management?
(cont.) Individual Comments

• A culture that doesn't support accountability
  Resistance to change by people in the
  organization.
• Trying to identify our division and its
  measurable values.
• I do not think we need the conference at the end.
  It was a farce to have the ORSAC list who they
  wanted to present and then not one of them show
  up. It makes no sense and just takes up valuable
  time. I think that the Management Council should
  be consulted on what to do this year. Let them
  vote on what should be done, instead of us being
  told what to do, so there is buy in.
• The lack of time to fully define, measure, and
  implement results.
• Defining and measuring unit cost.
• Too much time spend on this project by team
  members.
• Having to manually collect data.
• Analyzing the data through graphs and making
  trend projections.

114
What obstacles were most challenging to you
during this cycle of performance management?
(cont.) Individual Comments

• Move from Discrete Service to Service Area
  review continues to prove problematic in applying
  Internal Business perspective to improve business
  performance.
• Trying to define meaningful measurements of
  productivity and progress within this model of
  activity.
• financial perspective. Hard to relate to
  effectiveness. Is a measure for the sake of
  measuring only. insistence on full up
  presentation preparation each year. Why not some
  key indicators only except for a select few
  chosen each year on rotating basis.
• Director of our organization did not find time to
  meet with our team even once, although we had a
  number of issues to discuss. Success of our
  service group is greatly dependent on close
  communication with upper management in ORF and
  with other NIH stakeholders. This is missing.
  We can only correct what we control our
  suggestions for changing what we don't control
  are always ignored.

115
What obstacles were most challenging to you
during this cycle of performance management?
(cont.) Individual Comments

• I think the ASA/PMP is pushed down to too low a
  level. My five person office really shouldn't be
  doing it's own ASA/PMP. It's a time consuming
  dis-economy of scale. Our parent office should
  incorporate it into their ASA/PMP, we are just
  too small in numbers and dollars to get any
  significant value out of the time we spend on the
  ASA/PMP. We collected stats and performance
  metrics on our work even before the ASA/PMP,
  because that's what good mgt does, which was
  sufficient enough for us to track and improve our
  service. But it was less intensive than the
  current PMP.
• The lingo. If you do this on a daily basis it's
  easy to remember. The entire process of balanced
  scorecard, service hierarchy, etc. is hard to
  remember if you do not do it every day.
• Getting information to resolve issues.
• OQM Staff member hardly ever attended any of our
  meetings. Miscommunication regarding what was
  needed.

116
What obstacles were most challenging to you
during this cycle of performance management?
(cont.) Individual Comments

• Due to multiple layers of management involvement
  we did some re-tracking over materials that
  produced some levels of frustration. Some
  tracking forms proved less than useful.
• Our Team was not initially convened when others
  in our Division started so we had to play
  catch-up. The service for which I was designated
  Team Leader passed to the MEO, so it was
  determined late in the cycle that we would not
  present.
• Time -our office was short staffed (and still
  is). Being able to get to have any quality focus
  was difficult -- This is where our consultant was
  extremely helpful. Because of the systems we
  used for counting (e.g. ADB) we had to do a lot
  of manual counting.
• Gathering data for entire discreet service.
• After three years of ASA/PMP, I feel that our
  division has addressed everything that was/could
  be of concern. I see PMP as now being in a
  maintenance mode for us since our process is
  fairly straightforward.

117
What obstacles were most challenging to you
during this cycle of performance management?
(cont.) Individual Comments

• Team member participation.
• Last minute changes.
• Reading the graphs.
• Data collection - I waited too late before
  collecting it. Obtaining the correct measures
  and objectives.
• Trying to make the experience as interesting as
  possible since the process could be boring at
  times.
• Internal disagreements in OQM staff led to
  confusion on how proceed with the process.
  Delay in proceeding forward due to schedule
  in-availability of OQM staff to meet.

118
How can we help integrate PM into your daily
business activities? Individual Comments

• Managers realizing the importance of data for
  decision making.
• Make the reasons of why we are doing this more
  concrete - more specific examples of how it has
  helped others.
• Understand what our division does and applying
  the PMP to the function.
• There must be something else that can be done
  instead of the conference. We should be able to
  learn from each other on how to collect data.
• I have used the provided control charts to
  forecast and project overtime needs.
• Have upper management do this project.
• Follow up on documentation of implementation of
  objectives, i.e., training records and
  frequencies of meetings/actions.
• Insist on development of standard and specialized
  business tools and services to support the
  service activity.

119
How can we help integrate PM into your daily
business activities? (cont.) Individual Comments

• As far as I can tell the ASA/PMP is a waste of
  time, nothing is ever done with any of the
  information collected. The data is collected and
  analyzed and we look at it and say "oh that's
  nice" or "that's interesting", but no changes or
  new initiatives have ever started because of the
  data, not our level or at the ORS/ORF level. If
  there have been changes at any level I would be
  interested in knowing about them. Particularly
  at the ORS/ORF level.
• I already do by collecting data and putting it
  into spreadsheets almost monthly to track trends.
  I did not ever do this until the PMP program so
  it has helped me significantly. The data really
  does help in showing the value of my programs and
  where there are weaknesses and strengths!
• By being focused on true value added.
• Maintain communication throughout the year.

120
How can we help integrate PM into your daily
business activities? (cont.) Individual Comments

• The PMP is very helpful in its present form.
  Like virtually any good thing, however, it has
  the potential to grow to a point of diminishing
  returns, with gradual increases in requirements
  to collect, analyze and present data detracting
  from a Division's
• Unfortunately, the only way of assuring it gets
  tied in is to tie it to performance reviews.
• You already have.
• Continue support you are currently providing.
• Data review and collection.
• You have helped.
• We are integrated and know the cost of the
  activities of our service group.

121
OtherIndividual Comments

• This is too long and I don't have time to fill it
  all out.