Complex Systems Science and CSIRO: Into the Future

1
NEMSIM Practical Challenges for Agent-based
Simulation of Energy Markets
George Grozev and David Batten CSIRO
Manufacturing and Infrastructure Technology

• Complex Systems Science and CSIRO Into the
  Future
• Rydges Hotel, Melbourne
• 10-12 August 2005

2
Presentation Overview

• History and concepts (NEM as a CAS)
• NEMSIM overview and key features
• Practical challenges for agent-based simulation

3
Brief Historical Review of the Project

• July 2002 A postdoc position awarded. Dr. Xinmin
  Hu started in Nov. 2002.
• January 2003 Commenced as a CSS project
  Top-up funding from CSIROs Centre for Complex
  Systems Science
• October 2003 Commenced as a Theme 1 project in
  CSIROs Energy Transformed Flagship Program
• April 2004 Flagship Science e-Seminar Series 2
  Energy Transformed (John Wright, David Batten)
• April 2005 NEMSIM Industry Focus Group Meeting
  (Mercure Hotel, Melbourne)

4
NEMSIM National Electricity Market Simulator

• Agents in NEMSIM
• 27 Scheduled Generator Companies
• 12 Non-scheduled Generator Companies
• 20 Network Service Providers
• 29 Market Customers
• 9 Traders
• An Independent System Operator (NEMMCO)
• Potential Clients
• Regulators (ACCC, AER, AEMC)
• Government (DITR, SA, Tasmania)
• TNSPs (Powerlink, Transgrid)
• Customers (EUAA, ERAA, Origin, AGL)

5
The NEM is a Complex Adaptive System

• Evolving markets on an interconnected grid
• about 100 interacting, autonomous agents (firms)
  others
• about 300 grid-connected, generating units.
• Agents are intelligent, adaptive behave
  differently
• pursue goals unique to their firms interests
• make decisions on the basis of their own
  knowledge/beliefs
• change strategies in the light of their and
  others experiences.
• No agent knows what all the other agents are
  doing
• each agent has access to only a limited amount of
  information.
• Some act more conservatively than others
• e.g. they are more constrained (e.g. by debt)
  than others.

6
Our Science Agent-based Simulation

• Equations-based models
• too static, aggregate or stylized to handle this
  complexity
• Agent-based simulation
• computational experiments
• software agents, environments and rules
• agents learn and adapt strategies over simulated
  time
• evolutionary computation and equations-based
  methods
• can explore impacts of rule changes before their
  introduction
• can evolve adaptive responses of competitors
• collective outcomes can be unexpected, even
  undesirable

7
Presentation Overview

• History and concepts (NEM as a CAS)
• NEMSIM overview and key features
• Practical challenges for agent-based simulation

8
NEMSIM Overview
9
NEMSIM Overview - continued
10
NEMSIM Generating Units Displays
Bid Stacks
Dispatch
Revenue
GHG Emissions
11
Key Features

• Includes all key players in the NEM
• Models individual agents behaviour
• Weather model and data from 100 years
• Wholesale market model and extending to other
  markets, e.g. contract market
• Potential effect of distributed generation
• Transmission modelling
• Bid strategies e.g. lookaheads
• Scenario investigations new plants,
  maintenance, emergency shutdown, blackouts, new
  rules
• Scenario comparisons
• Reports dispatch, revenue, CO2, by regions, by
  companies, by plants, weekly, monthly, yearly
• Environmental markets, e.g. carbon trading

12
Other Important Features

• XML editor
• Simulation time control
• Lookaheads
• Scenario comparison
• Distributed generation
• New plants
• Maintenance shutdown
• Reports

13
Area of Applications

• Short-term trading
• analyse market bidding data
• analyse what-if bidding scenarios
• Medium-term hedging and contract markets
  (retailers, generators)
• Long-term investment (new generators,
  transmission lines, distributed generation,
  renewables)
• Greenhouse gas emissions estimates
• Carbon trading (when rules are proposed)
• Explore the impact of new technologies, new
  market rules, new grid structures, new
  participants

14
Presentation Overview

• History and concepts (NEM as a CAS)
• NEMSIM overview and key features
• Practical challenges for agent-based simulation

15
Selection of Agent-based Simulation Platform

• Develop our own platform
• EMCAS - Argonne National Lab
• DIAS/FACET Argonne National Lab
• RePast
• Swinburnes simulation framework agent
  implementation of the Victorian Gas Market

16
Other Practical Challenges for NEMSIM

• Adequately reflecting all the subtleties inherent
  in market-to-network interdependencies (DITR)
• Developing efficient heuristic algorithms for
  interactive decision-making
• e.g. adaptive learning procedures
• e.g. multi-criteria decision-making
• Distinguishing between counterintuitive results
  and programming errors
• Keeping running times reasonable while adding
  more dynamic features
• Developing confidence and trust among potential
  users and the market operator (NEMMCO)

17
Challenges of Learning in the NEM

• Depending on their own competitive position, each
  generator behaves differently
• Bidding strategies differ between states, but
  even more so between generators within states
• Although strategies differ, we may be able to
  develop a generic bid function for all of them
  (just varying parameters/markups)
• Most generators change bid capacities,
  occasionally changing bid prices (or price
  increments)
• Thus each firm that owns generating units will
  need to be examined, if we wish to approximate
  reality

18
Potential Learning Algorithms

• Genetic algorithms (see e.g. Goldberg, 1989,
  Mitchell, 1998, Chattoe, 1998, Dawid, 1999)
• Genetic programming (see e.g. Koza, 1992)
• Reinforcement learning algorithms (see e.g. Erev
  and Roth, 1998 Sutton and Barto, 1998)
• Q-learning (see e.g. Watkins, 1989 Tesauro and
  Kephart, 2002)
• Classifier systems (see. e.g. Holland, 1992)
• Learning algorithms for automated markets (see
  e.g. Gjerstad and Dickhaut, 1998 Tesauro and
  Kephart, 1998)

19
NEMSIM agents can look ahead, sideways and back
LOOK AHEAD (Strategy evaluation)

• Own unit availability
• Price trends/peak loads
• Hedging strategy
• Weather
• Load forecasts

• Competing unit availability
• Competing bids
• Market rules

TIME
Agent

• Bid acceptances/rejections
• Unit utilization
• Unit profitability
• Market price vs. bid price
• Weather and Load

LOOK BACK (Short and Long Term Memory)
20
Look-aheads in NEMSIM

• Agents have look-ahead capabilities
• Run the simulation forward for various periods
• Test compare a range of available strategies
  and plans
• Agent adopts strategy showing best possible
  outcome
• Strategies retested at start of each new period
• Plans/strategies changed to counter changes of
  others
• Does a look-ahead capability add value to the
  existing (comparative static) approaches?

21
Value of a Look-ahead Capability
22
FG Meeting Challenges for NEMSIM

• Focus more, refine agents adaptive behaviour
• How agents think and interact, not just bid
• Explore demand-side management options
• Locational issues
• Customers as agents
• Differentiate between short and long-term
• Treat GHG/carbon tax/emissions trading
• Explore DG/wind/green power
• Talk to appropriate potential users
• Regulators
• Government policy makers
• Network companies

23
Practical Advantages of NEMSIM

• Practical application of a CSS methodology
• To a real world complex adaptive system (the NEM)
• Socio-economic/physical/environmental
  interactions
• Each and every agents adaptive behaviour can be
  represented and modified
• Different collective outcomes can be generated
  and performances compared in advance
  (look-aheads)
• Conditions when unattractive outcomes occur (like
  price volatility market power) can be
  identified
• This kind of simulation goes beyond the classical
  simulation models in energy economics
• User-friendly human-machine interface

24
Acknowledgments

• Research and Development Group
• Energy Transformed Flagship
• Swinburne University of Technology

• CMIT
• George Grozev
• David Batten
• John Mo
• Miles Anderson
• Geoff Lewis
• Mario Sammut

• CMAR
• Jack Katzfey
• Marcus Thatcher

• UNSW
• Xinmin Hu

• Paul Graham Theme Leader Energy Futures
• Terry Jones - Theme Leader Low Emission
  Distributed Energy

• Prof. Myles Harding
• Neale Taylor

25
Thank you