Optimal Electricity Supply Bidding by Markov Decision Process

1
Optimal Electricity Supply Bidding by Markov
Decision Process

• Authors Haili Song, Chen-Ching Liu, Jacques
  Lawarree, Robert Dahlgren

Presentation Review By Feng Gao, Esteban Gil,
Kory Hedman IE 513 Analysis of Stochastic
Systems Professor Sarah Ryan February 14, 2005
2
Outline

• Introduction
• Purpose
• Problem Formulation
• Model Overview
• Summary

3
Introduction

• Electric Industry has Transitioned from Regulated
  to Deregulated
• Regulated Vertically Integrated, Monopolistic
  Market
• Deregulated Ideally, Perfect Competition Market
• Decision Analysis Based on Profit Competition

4
Introduction Contd

• Traditional Power System
• Generation, Transmission, Distribution,
  Consumption

5
Introduction Contd

• Structure of Power Market
• Optimize Resources With Competition

6
Introduction Contd

• Day Ahead Market Considered
• Inelastic Demand
• Generation Companies (GenCos) are Risk-Neutral
• GenCos Bid a Price (P) Quantity (Q)
• Bids are Chosen from Cheapest to Most Expensive
• Market Clearing Price P from the Highest Chosen
  Q
• (Similar to the New Zealand Great Britain
  Electricity Markets)

7
Purpose

• Overall Objective Maximize Expected Profit over
  a Planning Horizon of 7 Days
• For all States, Determine Optimal Bidding
  Strategy. Depends on
• Competitors Bidding
• Load Forecasting
• Remaining Time Horizon (given day i, 7 i)
• Production Limit (Max Available Supply Remaining
  over Planning Horizon)
• Accumulated Data over Time (Past Load Price
  Data)

8
Problem Formulation

• States are Defined by 7 Variables
• Peak Load Peak Price (2)
• Off-Peak Load Off-Peak Price (2)
• Current Production Limit for the Remaining
  Planning Horizon (1)
• Load Forecast for the Following Day (2)
• Aggregation Limits Number of States
• P D Broken into High, Medium, Low
• Illogical States Ignored (High P Low Q, etc.)

9
Problem Formulation Contd

• Transition Probability Depends on
• Current State i, Subsequent State j, Decision a
• Pr (i, j, a)
• Decision Maker Receives Reward
• R (i, j, a)
• State of the Market Defines the Competitors Bids
  Decision Makers Bidding Options
• Bid Prices are Determined Using a Staircase
  Supply Fn for Varying MW

10
Problem Formulation Contd

• MDP Algorithm Considers
• Rewards Based on Load Forecast
• Decisions of a State
• Competitors Bidding Characteristics
• Decision Options Affect Transition Probabilities
  Rewards
• Competitors Bids are Independent
• Scenarios (s) are exclusive

11
Model Overview

• Probability of Scenario s
• Pr (i, n, k) Probability that Supplier n (n
  m) Chooses Option k in State i
• m is decision maker
• Competitors Bids are Independent
• Remaining Production Limit
• q (i, s, t) is the Q used in period t for
  scenario s.
• Spot Price for Scenario s SP(i, s, t)

12
Model Overview Contd

• Probability to Move from State i to j
• Pr (i, LF (j, t) Probability that Load Forecast
  for Day After Tomorrow is LF (j, t) Given Present
  State i
• Reward for Decision Maker, r (i, s)

13
Model Overview Contd

• Reward for Transition from i to j Decision A is
  Sum of Rewards Weighted by Conditional
  Probabilities
• V (i, T1) Total Expected Reward in T1
  Remaining Stages from State i
• Solved by Value Iteration

14
Summary

• Introduction
• Electric Market is now Competitive
• GenCos Bid on Demand
• Purpose
• MDP Used to Determine Optimal Bidding Strategy
• Problem Formulation
• Transition Probability Determined by Current
  State, Subsequent State, Decision Made
• 7 Variables to Define a State
• Aggregation Used to Limit Dimensionality Problems
• Model Overview
• 7 Day Planning Horizon
• Objective is to Maximize Summation of Expected
  Reward
• Value Iteration

15
Questions???
16
References

• Song, H. Liu, C.-C. Lawarree, J. Dahlgren,
  R.W, Optimal Electricity Supply Bidding by
  Markov Decision Process, IEEE Transactions.
  Power Systems, Vol. 15, no. 2, pp 618-624, May
  2000.
• http//www.acclaimimages.com/_gallery/_pages/0037-
  0409-0607-4216.html