California Independent System Operator

1
California Independent System Operator

• 1998 Client Satisfaction Survey
• October 1998

Conducted By
Energy Insights 566 Dalewood Drive Orinda, CA
94563 [phone]
Energy Insights
2
Background

• Research Objectives
• Determine the relative importance of service
  attributes to clients
• Measure performance for each service attribute
• Assess overall satisfaction
• Establish baseline and improvement trends for
  each service attribute
• Enhance findings with qualitative research
• Focus on areas where improvement is needed

3
Background - Survey Design

• Three sections, each with 22 questions
• Section I Importance of specific factors
• Section II Performance on specific factors
  (April 1998)
• Section III Current performance on specific
  factors
• Overall rating of client satisfaction in April,
  1998 and today
• Two open-ended questions to conclude survey

There may be inherent bias in all improvement
trends due to survey methodology. The survey was
not administered in April and therefore, actual
April performance was not assessed. In addition,
when participants were asked to rate current
performance, their April performance ratings
could be easily recalled for comparison.
4
Survey Participants

• Total number of participants 64
• Total number of organizations 41
• Participation rate 92

5
Survey Results

• Survey addresses key client satisfiers
• Average importance 8.9
• Range between 7.0 and 9.6
• Current performance scores primarily in the 5-6
  range
• Significant improvement in performance since
  April however, gaps remain large
• Gaps over 2 on fourteen of the twenty-two factors

There may be inherent bias in all improvement
trends due to survey methodology. The survey was
not administered in April and therefore, actual
April performance was not assessed. In addition,
when participants were asked to rate current
performance, their April performance ratings
could be easily recalled for comparison.
Gap equals the difference between the
importance rating and the performance rating.
This indicates how closely the ISO is meeting
clients expectations. The bigger the gap,the
greater the difference between client
expectations and perceived service delivery.
Gaps of 2.0 or greater generally indicate an
area that requires further investigation for
improvement.
6
Reasons Why Scores Increased

• Clients noted several improvements
• improvement in communication/information
• effort and dedication of ISO staff in various
  departments
• improvements made in ISO systems
• improvement in level of knowledge

7
Response by Major Area
8
Client Representative
Question 1. Your Client Representative provides
prompt and accurate responses to your questions
and requests. Question 2. Your Client
Representative is accessible when
needed. Question 3. Your Client Representative
keeps you well-briefed on issues and events that
impact your business. Question 4. Your Client
Representative knows the policies and procedures
of the ISO and applies that knowledge to help
solve your business problems. Question 5. Your
Client Representative understands the needs and
challenges of your business. Question 6. Your
Client Representative is your advocate within the
ISO.
Gap equals the difference between the
importance rating and the performance rating.
This indicates how closely the ISO is meeting
clients expectations. The bigger the gap the
greater the difference between client
expectations and perceived service delivery.
Gaps of 2.0 or greater generally indicate an
area that requires further investigation for
improvement.
9
Scatter Diagram - Concept
10
Client Representative
Scatter Diagram April 1998 - Current
Question 1. Your Client Representative provides
prompt and accurate responses to your questions
and requests. Question 2. Your Client
Representative is accessible when
needed. Question 3. Your Client Representative
keeps you well-briefed on issues and events that
impact your business. Question 4. Your Client
Representative knows the policies and procedures
of the ISO and applies that knowledge to help
solve your business problems. Question 5. Your
Client Representative understands the needs and
challenges of your business. Question 6. Your
Client Representative is your advocate within the
ISO.
11
Client Representative

• Findings
• Client Representative viewed as strength of the
  ISO
• Knowledge level and responsiveness show vast
  improvements
• Significant improvement since April
• Areas of client concern
• Client Representatives job perceived as having
  too large a scope
• Accessibility and responsiveness of Client
  Representative
• Client Representatives understanding of client
  needs, the industry, and ISO procedures.

12
Relationships with Clients
Question 7. The ISO provides an effective forum
to discuss policies and procedures. Question
8. The ISO is fair and consistent in applying
its policies and procedures. Question 9. The ISO
incorporates your feedback into its
decision-making process. Question 10. The ISO
effectively and efficiently resolves
disputes. Question 11. The ISO provides timely
and accurate communication on the changing market
environment (i.e. emergency situations,
operations, and off-normal conditions.)
13
Relationships with Clients
Scatter Diagram April 1998 - Current
Question 7. The ISO provides an effective forum
to discuss policies and procedures. Question
8. The ISO is fair and consistent in applying
its policies and procedures. Question 9. The ISO
incorporates your feedback into its
decision-making process. Question 10. The ISO
effectively and efficiently resolves
disputes. Question 11. The ISO provides timely
and accurate communication on the changing market
environment (i.e. emergency situations,
operations, and off-normal conditions.)
14
Relationships with Clients

• Findings
• Low performance except in communication on
  market environment
• Huge gaps, especially in resolving disputes and
  fairness in applying policies and procedures
• Significant improvement since April
• Areas of client concern
• Forums are provided, but effectiveness questioned
• Settlements and dispute process needs improvement
• Lack of emphasis on client concerns within ISO
  (Wayne does not believe this is the same point as
  on the original slide).

15
ISO Electronic Systems
Question 12. The ISO systems are always up and
running. Question 13. The scheduling process is
efficient and easy to use. Question 14. The
subsystems, such as ASM, CONG and BEEP,
effectively answer your questions about how the
markets are operating.
16
ISO Electronic Systems
Scatter Diagram April 1998 - Current
Question 12. The ISO systems are always up and
running. Question 13. The scheduling process is
efficient and easy to use. Question 14. The
subsystems, such as ASM, CONG and BEEP,
effectively answer your questions about how the
markets are operating.
17
ISO Electronic Systems

• Findings
• High performance on system reliability
• Huge improvements since April
• Subsystems need improvement, but are much
  improved
• Areas of client concern
• Scheduling system viewed as cumbersome, though
  clients have become accustomed to it
• Subsystems (e.g., BEEP,CONG) are viewed as
  ineffective

18
ISO Flow of Information
Question 15. The ISO clearly communicates with
you in a timely manner on tariff changes and
settlement statements. Question 16. The ISO web
site provides you with the information you need
to effectively conduct your business. Question
17. The ISO provides preliminary statements,
final statements, and invoices that meet your
business needs.
19
ISO Flow of Information
Scatter Diagram April 1998 - Current
Question 15. The ISO clearly communicates with
you in a timely manner on tariff changes and
settlement statements. Question 16. The ISO web
site provides you with the information you need
to effectively conduct your business. Question
17. The ISO provides preliminary statements,
final statements, and invoices that meet your
business needs.
20
ISO Flow of Information

• Findings
• All factors viewed as very important
• Huge gap on statements and invoices meeting needs
• Areas of client concern
• More public market information wanted, especially
  on generation
• Statements and invoices are not meeting
  individual client needssome want less
  information and some want more

21
ISO Operational Support
Question 18. The operation instructions from the
ISO are clear and concise. Question 19. On
operational questions, the ISO provides accurate
and timely responses. Question 20. The ISO
provides timely and useful assistance when
entering or validating schedules.
22
ISO Operational Support
Scatter Diagram April 1998 - Current
Question 18. The operation instructions from the
ISO are clear and concise. Question 19. On
operational questions, the ISO provides accurate
and timely responses. Question 20. The ISO
provides timely and useful assistance when
entering or validating schedules.
23
ISO Operational Support

• Findings
• Performance below average on 2 of the 3 factors
• Scheduling assistance viewed as strength
• Large gaps exist for clarity of operation
  instructions and timely real-time responses
• Area of client concern
• Real-time issue resolutioncommunication and
  execution

24
ISO Role
Question 21. Where appropriate, the ISO is
effective in helping its clients reach consensus
on key issues. Question 22. The ISO
appropriately balances the forces of the free
market and the operational control of the
electric transmission system.
25
ISO Role
Scatter Diagram April 1998 - Current
Question 21. Where appropriate, the ISO is
effective in helping its clients reach consensus
on key issues. Question 22. The ISO
appropriately balances the forces of the free
market and the operational control of the
electric transmission system.
26
ISO Role

• Findings
• Consensus building viewed as less important role
• Large gap on balancing free market and
  operational control of electric system
• Areas of client concern
• Monopolistic attitudes
• Effort used to build consensus may not be
  worthwhile

27
Question 23

• Overall, the ISO provides exceptional service

There may be inherent bias in all improvement
trends due to survey methodology. The survey was
not administered in April and therefore, actual
April performance was not assessed. In addition,
when participants were asked to rate current
performance, their April performance ratings
could be easily recalled for comparison.
28
1998 Client Satisfaction Survey

• Detailed Analysis of Results

29
Detailed Analysis of Results

• Priority Diagram
• Priority Importance x (10 - Performance)

The priority score gives one indication of
factors requiring further investigation. The
higher the priority score, the more critical the
need for attention. Those factors that are
highlighted have significantly higher priority
scores than the average..
30
Detailed Analysis of Results

• Comparison of Performance Ratings April 1998 to
  Current

There may be inherent bias in all improvement
trends due to survey methodology. The survey was
not administered in April and therefore, actual
April performance was not assessed. In addition,
when participants were asked to rate current
performance, their April performance ratings
could be easily recalled for comparison.