Power Grid Research at PNNL

1
Power Grid Research at Pacific Northwest National
Laboratory
Moe Khaleel Laboratory Fellow and
Director Computational Sciences and Mathematics
Division September 10, 2009 HPC User Forum
2
Overview

• Introduction
• Problems on the electrical grid
• Need for HPC on the electrical grid
• Operations
• Integration of renewables
• Cyber security
• Need for HPC at vision level
• State estimation
• Multithreaded platforms for contingency analysis
• Looking forward

3
Transform the way the U.S. generates, transmits,
distributes and uses electricity

• Current U.S. electricity infrastructure is
  inadequate for national energy priorities in the
  21st century. Three main areas need to be
  addressed
• Capacity
• Grid management (wide area, real-time)
• Vulnerability, resiliency, reliability
• The future state of the grid must be able to
• Substantially increase the integration of
  renewables
• Reduce carbon emissions
• Provide flexibility to enable electrification of
  transportation and reduce dependence on oil
  imports (substitute electricity for oil)
• Respond to increased demand
• Reality The current grid infrastructure/operation
  is limited
• Currently manage/engage grid at sub-optimal
  (service territory) level cant efficiently move
  electrons across large enough spaces
• No ability to see performance across grid (lacks
  transparency)
• Inability to integrate renewables need storage,
  ability to offset, transmission across service
  territories where generated, load/dispatch
  renewables
• System communication has been one-directional
  supply to demand
• Fragmented authority, control, market, function,
  regulation
• Need to build new functionality and
  infrastructure into the grid

3
4
Transform the way the U.S. generates, transmits,
distributes and uses electricity

• Capacity
• Transmission infrastructure cannot meet future
  load growth and large-scale renewable
  connectivity to grid
• Utilities not incentivized to build physical
  infrastructure
• Difficult to site permit new transmission
  infrastructure
• Renewable resource physically isolated from high
  grid transmission infrastructure
• Grid Management
• Unable to manage grid at national, interconnect
  scale
• Large scale models that allow examination and
  optimization of future national grid do not exist
• Integrated wide area models (variable renewable
  generation, energy storage, distributed
  generation, demand management) that describe
  real-time power flow and predict reliability do
  not exist
• Ability to see and understand the grid at
  interconnections scale are limited wide area
  grid performance is not accessible, transparent
  so cant optimize supply and demand across
  limited service areas
• Transparent real-time monitoring and operation
  currently not in place
• Large-scale wind generation introduces
  significant variability
• Large scale electric energy storage capability is
  limited pumped hydro, flywheels,
  electro-chemical systems connected to and
  supporting the power grid
• Vulnerability and Resiliency
• Susceptible to cyber, other threats can we
  prevent, respond to threats?
• Resilient to catastrophic events can we rapidly
  recover?

4
5
Power System Elements
6
Problems on the electrical grid

• Inadequacy of current control center functions
• Slow, not able to keep up with the change of the
  grid
• Static, no dynamic information for real-time
  operations
• Computational Issues with todays grid operations
• Real-time grid view static
• Not able to capture grid dynamics
• Low computational efficiency not keep up with
  system changing
• No use of high-performance computing architectures

7
Problems on the electrical grid
8
The need for HPC on the electrical grid

• Major HPC Architectures
• Shared-memory architecture for extensive data
  sharing and un-uniform data access
• Distributed-memory architecture for less data
  sharing and uniform data access
• Hybrid architecture - re-configurable
  architecture e.g. FPGA shared-memory
• Performance of Parallel Algorithms/Programs
• Speedup/Scalability
• Reliability

9
The need for HPC on the electrical grid

• Parallel Computing is essential
• Only explicitly parallelized algorithms can take
  advantage of multi-core parallel computers
• Parallel Computing is an art
• Parallelization approaches are problem-dependent
• Computing implementation needs to consider good
  match of computing architecture and the problems

10
Todays Electrical Grid Operations Paradigm
11
Trends Impacting Control System Security

• Open Protocols
• Open industry standard protocols are replacing
  vendor-specific proprietary communication
  protocols
• General Purpose Computing Equipment and Software
• Standardized computational platforms increasingly
  used to support control system applications
• Interconnected to Other Systems
• Connections with enterprise networksto obtain
  productivity improvementsand information sharing
• Reliance on External Communications
• Increasing use of public telecommunicationsystems
  , the Internet, and wireless for controlsystem
  communications
• Increased Capability of Field Equipment
• Smart sensors and controls with
  enhancedcapability and functionality

12
The Emerging Cyber Threat

• Industry has long history of planning for and
  coping with natural disasters and other
  reliability events
• Through industry standard operating procedures,
  there is much effort expended to reduce
  likelihood of cascading outages leading to
  widespread blackouts
• Historically, cyber security focused on
  countering unstructured adversaries
• e.g., individuals, untargeted malicious software,
  human error
• Very little protection against structured
  adversaries intent on exploiting vulnerabilities
  to maximize consequences
• e.g., terrorist groups, organized crime, nation
  states
• Insider threat remains very challenging, can be
  used as part of structured threat vector
• New possibilities for widespread sustained
  outages resulting from cyber attack are now being
  contemplated
• But industry still not ready to cope with this
  threat

13
The need for HPC State Estimation

• Power system State Estimation (PSE)
• Given power grid topological information,
  telemetry on line flows, bus injections or bus
  voltages
• Compute a reliable estimate of the system state
  (bus voltages), validate model structure and
  parameter values
• Calculated using Weighted Least-Squares (WLS)
  method
• WLS minimize
• Where r z - h(x), and r is the residual vector,
  x is the system state, z is a vector of measured
  quantities, h is a vector function, wi is the
  weight for residual ri and W is as diagonal
  matrix.
• This is a non-linear problem, which is solved
  using the Newton-Raphson iterative procedure

14
The need for HPC State Estimation

• PSE
• Every iteration of the method requires solving a
  large set of sparse linear equations
• Sparse matrices are derived from the topology of
  the power grid being analyzed
• The number of non-zeros per row varies greatly
  and the matrix is badly conditioned
• The set of linear equations can be solved using
  direct solvers such as sparse LU factorization or
  iterative solvers such as sparse Conjugate
  Gradient (CG)
• PSE is a critical element of the software used by
  power grid control centers
• Under real-time constraints (lt 10 seconds)
• Commercial PSE solvers are not commonly parallel

15
Power System State Estimation
16
Model Validation Need and Challenges
Recorded system dynamics vs. simulation results
California and Oregon Intertie (COI) real power
flow during the August 10, 1996 event
17
The need for HPC Multithreaded Platforms for
Contingency Analysis

• Growing class of scientific applications is
  becoming memory-bound
• Many scientific applications exhibit irregular
  memory access patterns
• CPU and memory technology trends indicate that
  the situation will not improve anytime soon
• Multithreaded architectures offer an appealing
  alternative for irregular applications
• Processors tolerate memory access latencies by
  switching execution context between multiple
  hardware threads
• Examples of such architectures are the Cray MTA-2
  and XMT systems and the Sun Niagara
• Latency tolerance mechanisms should improve the
  performance of irregular, data-intensive
  applications with abundant fine-grained
  parallelism

18
Role of Contingency Analysis

• From N-1 to N-x
• To improve situational awareness
• From Balancing Authorities to a Wide Area
• Example 35 BAs in west
• Further require N-x CA
• To better understand cascading failures
• N-x Contingency Analysis
• Result in a large number of cases. N-5 ? 1020
  cases for the west 1020 seconds lots of data
• Needs better contingency selection and
  post-processing

19
Looking forward PNNLs vision of advanced
contingency analysis
Industry Need
Multi-failure (N-x) massive contingency analysis
Issue
Massive number of cases
Massive amount of data
Need
Smart case selection
Operator-oriented data processing
Our Solution
Contingency Analysis
Data
10Z cases
10Y cases
10X cases
Prediction
Major Tasks
past now future
Trending
Time
Graph Theory
Contingency Index Sorting
Visual Analytics Graph Trending
Expected Outcomes
Interactive Graphing

• An advanced tool for contingency analysis to
  support the operation of the nations critical
  infrastructures, e.g. power grids
• Analysis capabilities through generic
  implementation of algorithms on Cray XMT
  ThreadStorm processors, applicable to other
  problems

20
Tool Suites for Advanced Power System Simulation
SCADA Measurements
Phasor Measurements
Dynamic state estimation
Parallel state estimation
Parallel contingency analysis
Look-ahead dynamic simulation
On-line voltage stability
Market Monitoring
Dyn. contingency analysis
Visualization
? National Level monitoring, planning, design
? Utility Level
? Applicable to current grid operations
? Next-generation grid operation tools
21
Looking forward PNNLs long-term vision for
electrical grid operations
22
Electricity Infrastructure Operational Center at
PNNL

• Use HPC to analyze sensor data for real-time grid
  monitoring, prediction, and operation
  data-driven models, stochastic simulations,
  visual analytic tools, secure sensor network
  infrastructure, data provenance

23
QUESTIONS?