NASPI July 19 Architecture Meeting with PNNL

1
NASPI July 19 Architecture Meeting with PNNL

• August 2007
• David Chassin
• Jeff Dagle
• Henry Huang
• Ian Gorton
• Ranata Johnson
• Kris Koellner
• Paul Myrda

2
Caveats

• This presentation is based on initial results
  from a design charette
• Some major issues are still being discussed
• We need your input early and often
• Comments need to be noted and send to Ranata
  Johnson _at_ PNNL(ranata.johnson_at_pnl.gov)

3
Architecture Discussion

• Key Assumptions
• Initial Architecture Concepts
• Application Use Cases
• Phasor Application Classes
• Cost Considerations

4
Key Assumptions

• Access control required
• Classification of PMUs differentiated services
• Reference bus data must be made available
• Spoof detection needed for data integrity
  purposes
• Flexible data access schemes depending on usage
• PMUs might be p2p connected to an application as
  a work-around
• Physical interconnect could be Internet, NERCnet,
  others (MPLS)

5
Basic Architecture - Proposed
Utility A
Monitoring Center X
Archive
PMU
PMU
PMU
PMU
Apps
Historian
PGW
Apps
PGW
Utility B
Monitoring Center Y
Historian
PGW
Apps
PGW
Archive
Apps
PMU
PMU
PMU
PMU
6
High-Level Architecture View
PhasorNET High level Architecture
7
Single Utility View Architecture
8
Purpose of PhasorNET Gateways

• Principal access point for inter-organizational
  phasor traffic
• Access/admin rights enforcement
• Disseminates access rights
• Maintains data integrity
• Handles format compatibility issues
• Manages traffic class priority

9
Benefit of publish-subscribe

• Avoid many-to-many traffic congestion
• Publishers announce what data is available
• Subscribers announce what data they want
• Gateways relay only what is needed and only once,
  no matter how many clients each channel has

10
Application Use Cases
11
Small Signal Stability (Feedback Control)

• Low latency, as fast as possible
• Data integrity
• no gaps
• If gaps, application needs to handle
• Operating on lt12 PMUs that may not be
  geographically close (hence longer latency)

Class A Example
12
State Estimator Enhancement State Measurement

• Time alignment of data based on time stamps
• Ability to select a range of PMUs for state
  estimation
• Latency up to 5 seconds or so
• Data loss OK
• Error estimates generated by PMUs
• Configuration utility widely disseminated (e.g.
  topology)

Class B Example
13
Post Event Analysis

• Based on archived records
• Requires completeness and accuracy
• buffering at all level of architecture
• Tolerates data being delivered to archives up to
  one hour late
• Recovery protocols that move data to archive
  after connectivity failures
• Keep data on demand for a specified period (e.g.
  1 week).
• Ability to mark some data
• Ability to handle chain of custody
• PMU configuration data availability (will be in
  asset management system for a utility)
• Data rate

Class C Example
14
PMU Visualization (Like RTDMS)

• PMU selection can be a few or many
• Lower latencies, seconds is fine
• Can be minutes late, but must be aligned to
  specific period
• Some data loss and late delivery is acceptable
• Wider tolerance for accuracy (loosely
  compression)
• Required data rate

Class D Example
15
Phasor Application Classification

• Legend
• Not very important
• Somewhat important
• Fairly important
• Critically important

16
Cost Considerations

• Centralized NERC retains control
• Pros easy to explain
• Cons higher cost, less robust, new model
• Decentralized NERC delegates to utilities
• Pros lower cost, more robust, existing model
• Cons hard to understand

17
Additional Slides

• These remaining slides can be used for the
  detailed discussions

18
Vulnerabilities

• Reliance on external time sources for sync
• Commercial PMUs not password protected to
  reconfigure and no audit of changes
• How robust are PDCs to a cyber attack involving
  spoofed, bad data?
• Can PDCs be comprised by attacks using their
  control or generic networking protocol?
• Limited situational awareness
• Implicit trust
• c37.118 protocol
• PDC device has no mechanism for establishing
  trust
• PDC does not authentication/authorization
• Data encrypted should be optional
• VPNs can be attacked
• Single points of failures
• How reliable is PDC equipment?
• Quality of data could be poor due to incorrect
  calibration of equipment

19
Issues

• Handling delays in PMU data delivery within the
  architecture
• Data Distribution
• Currently from 100 PMUs, get 5GB per day
• Protocols are not robust to disruptions that will
  occur
• Current systems based on UDP
• Architecture must accommodate diversity of
  equipment from multiple vendors including
  different data formats
• Classes of PMUs (based on importance of data
  delivery), which may require more high reliable
  infrastructure (see matrix in early slide)

20
Issues (Cont.)

• Trade offs between UDP and alternative protocols
• How is configuration information transmitted?
• Channel design has to be done carefully so one
  PMU down doesnt bring down adjacent ones
• Manage multiple interconnects from different
  utilities on same port
• Handle multiple protocols from different
  interconnects simultaneously
• PDCs might be layered
• Substation
• Regional
• Division
• corporate

21
Application issues

• Connect to and view each phasor data stream
• Diagnostics from source to destination QoS,
  config
• Most applications only access small number of
  phasors
• Need to be able to get all phasor data from an
  interconnect, or individual channels within that
  interconnect
• Need an addressing scheme (possibly based on
  pub-sub)
• Different PMU formats can be converted to a
  common data representation to facilitate
  translation from one to other
• CDR Standardization
• Level of granularity needed for access control
  lists for PMUs/Channels
• Diagnostics and QoS that detect channel problems
• GPS, PMU, PDS, relay switch outages
• PMU outage
• UDP saturation
• Can be diagnosed from missing data, and exploited

22
Commissioning

• Centralized Tools and Rules for PMU Addressing
• Configuration Information for PMU entered into
  repository
• PMU Naming Conventions
• Channels
• Owner
• Station
• Units
• Location (lat/long)

23
Diagnostics Requirements

• Latency Calculation
• Similar to SNMP diagnostics architecture for
  plugging in diagnostics tools for health
  monitoring, management.
• Useful metrics/measurements definitions
  (including performance)

24
General Scalability Requirements

• 100,000 to 1 million PMUs total
• 1000-10,000 per utility, or area in a big utility
• 1,000 utilities
• Up to 1000 PMU readings per application
• maximum of 100 or less is more typical

25
Performance Requirements

• As fast as possible
• Prioritized requests

26
Reliability/AvailabilityRequirements

• Feedback control should have high
  reliability/availability
• Visualization is lower than above
• Post-event analysis needs data and data can be
  old
• State estimator should have reasonable
  reliability/availability needs
• Data loss should be avoided by buffering at every
  level
• queues at each for async messaging

27
Security Requirements

• Support access control lists on data resources
• Support encryption on demand
• perhaps as specified by data producer
• Secure configuration management
• Audit/logging
• Key Access (future)

28
Data Needs

• Data owner would typically be a utility
• Data is held for a short period (e.g. 1 week).
• Ownership defined logically and discoverable
  through a directory
• A longer term archive or even thrown away