Modeling Electricity Demand: A Neural Network Approach

1
Modeling Electricity Demand A Neural Network
Approach
INFORMS Meeting October 26, 2004, Denver, CO
Christian Crowley GWU Department of Economics
2
Support
This study is part of a larger joint effort
supported by a EPA STAR grant Implications of
Climate Change for Regional Air Pollution and
Health Effects and Energy Consumption Behavior

• Co-researchers in the project are
• Frederick Joutz from the George Washington
  University Department of Economics
• Benjamin Hobbs and Yihsu Chen from the Johns
  Hopkins University Department of Geography and
  Environmental Engineering
• Hugh Ellis from the Johns Hopkins University
  School of Public Health

3
Outline

1. Context of the Research
2. Introduction to ANN Modeling
3. Basics of Electricity Demand
4. Developing the Demand Model
5. Results

4

• I. Context of the Research

5
Context

• The modeling efforts of the STAR grant are
• Electricity load modeling and forecasting

6
Context

• The modeling efforts of the STAR grant are
• Electricity load modeling and forecasting
• Electricity generation and dispatch modeling

7
Context

• The modeling efforts of the STAR grant are
• Electricity load modeling and forecasting
• Electricity generation and dispatch modeling
• Regional air pollution modeling

8
Context

• The modeling efforts of the STAR grant are
• Electricity load modeling and forecasting
• Electricity generation and dispatch modeling
• Regional air pollution modeling
• Health effects characterization

9

• II. Intro to ANN Modeling

10
ANN Model Architecture
11
ANN Model Architecture
12
ANN Model Architecture
13
ANN Model Architecture

• Elements of a neuron
• Multiplicative Weight (w)

14
ANN Model Architecture

• Elements of a neuron
• Multiplicative Weight (w)
• Additive Bias (b)

15
ANN Model Architecture

• Elements of a neuron
• Multiplicative Weight (w)
• Additive Bias (b)
• Transfer Function ( f )

16
ANN Model Architecture

• Elements of a neuron
• Multiplicative Weight (w)
• Additive Bias (b)
• Transfer Function ( f )
• The Neuron uses these elements in its operation,
  when it receives an input (p) and produces a
  result (a)

17
ANN Model Architecture

• Once a neuron receives an input (p), the neurons
  operation consists of two parts

18
ANN Model Architecture

• Once a neuron receives an input (p), the neurons
  operation consists of two parts
• Linear Transformation
• using the weight (w) and bias (b)

19
ANN Model Architecture

• Once a neuron receives an input (p), the neurons
  operation consists of two parts
• Linear Transformation
• using the weight (w) and bias (b)
• Application of the Transfer Function ( f )

20
ANN Model Architecture

• Once a neuron receives an input (p), the neurons
  operation consists of two parts
• Linear Transformation
• using the weight (w) and bias (b)
• Application of the Transfer Function ( f )
• The neuron then passes the result (a) on to the
  next part of the ANN.

21
ANN Model Architecture

• Linear Transformation
• Multiply the input (p) by a weight (w) and add
  the bias (b) to get the intermediate result (n)

22
ANN Model Architecture

• Linear Transformation
• Multiply the input (p) by a weight (w) and add
  the bias (b) to get the intermediate result (n)
• n w x p b

23
ANN Model Architecture

• Linear Transformation
• Multiply the input (p) by a weight (w) and add
  the bias (b) to get the intermediate result (n)
• n w x p b
• Pass the intermediate result (n) to the transfer
  function

24
ANN Model Architecture

• Transfer Function
• Apply the transfer function ( f ) to the
  intermediate result (n) to create the neurons
  result (a)

25
ANN Model Architecture

• Transfer Function
• Apply the transfer function ( f ) to the
  intermediate result (n) to create the neurons
  result (a)
• For neurons in the input layer, the transfer
  function is typically a hard-limiting switch at
  some threshold, say n 0
• f(n) (0 for n 0 1 for n gt 0)

26
ANN Model Architecture

• Transfer Function
• Apply the transfer function ( f ) to the
  intermediate result (n) to create the neurons
  result (a)
• For neurons in the input layer, the transfer
  function is typically a hard-limiting switch at
  some threshold, say n 0
• f(n) (0 for n 0 1 for n gt 0)
• For neurons in the output layer, the pure
  linear transfer function typically has no
  effect
• f(n) n

27
ANN Model Architecture
28
ANN Model Architecture
29
ANN Model Architecture
30
ANN Model Architecture
31
ANN Model Architecture

• Example A Neurons Operation
• Say a neuron has a weight of 10, a bias of 100,
  and a hard-limiting transfer function with a
  threshold of n 0
• w 10 b 100
• f(n) (0 for n 0 1 for n gt 0)

32
ANN Model Architecture

• Example A Neurons Operation
• Say a neuron has a weight of 10, a bias of 100,
  and a hard-limiting transfer function with a
  threshold of n 0
• w 10 b 100
• f(n) (0 for n 0 1 for n gt 0)
• Say the neuron receives an input of 5
• p 5

33
ANN Model Architecture

• Example A Neurons Operation
• Say a neuron has a weight of 10, a bias of 100,
  and a hard-limiting transfer function with a
  threshold of n 0
• w 10 b 100
• f(n) (0 for n 0 1 for n gt 0)
• Say the neuron receives an input of 5
• p 5
• The neurons linear transformation produces an
  intermediate result of 150
• n 10 x 5 100 150

34
ANN Model Architecture

• Example A Neurons Operation
• Say a neuron has a weight of 10, a bias of 100,
  and a hard-limiting transfer function with a
  threshold of n 0
• w 10 b 100
• f(n) (0 for n 0 1 for n gt 0)
• Say the neuron receives an input of 5
• p 5
• The neurons linear transformation produces an
  intermediate result of 150
• n 10 x 5 100 150
• The neurons transfer function produces the
  neurons result of 1
• a f(n) 1

35
ANN Model Architecture
36
ANN Model Selection

• The weights and biases are determined by the ANN
  during training

37
ANN Model Selection

• The weights and biases are determined by the ANN
  during training
• Training involves presenting the ANN with
  input-output examples

38
ANN Model Selection

• The weights and biases are determined by the ANN
  during training
• Training involves presenting the ANN with
  input-output examples
• The ANN finds the weights and biases that
  minimize the performance function

39
ANN Model Selection

• The weights and biases are determined by the ANN
  during training
• Training involves presenting the ANN with
  input-output examples
• The ANN finds the weights and biases that
  minimize the performance function
• The performance function is typically some
  measure of model error, such as MSE

40
ANN Model Selection

• The researcher specifies
• the number of layers
• Input
• Output
• Hidden (if any)

41
ANN Model Selection

• The researcher specifies
• the number of layers
• Input
• Output
• Hidden (if any)
• the number of neurons in each layer
• typically 1-5 input neurons
• typically 1 output neuron

42
ANN Model Selection

• The researcher specifies
• the number of layers
• Input
• Output
• Hidden (if any)
• the number of neurons in each layer
• typically 1-5 input neurons
• typically 1 output neuron
• the type of transfer function for each neuron
• typically hard-limiting in the input layer
• typically pure-linear in the output layer

43
ANN Model Selection

• The researcher specifies
• the number of layers
• Input
• Output
• Hidden (if any)
• the number of neurons in each layer
• typically 1-5 input neurons
• typically 1 output neuron
• the type of transfer function for each neuron
• typically hard-limiting in the input layer
• typically pure-linear in the output layer
• the performance function for the network (MSE)

44

• III. Basics of Electricity Demand

45
Electricity Demand

• Utilities are interested in forecasting peak
  demand

46
Electricity Demand

• Utilities are interested in forecasting peak
  demand
• Peak demand determines how many generators the
  utility must bring on line

47
Electricity Demand

• Utilities are interested in forecasting peak
  demand
• Peak demand determines how many generators the
  utility must bring on line
• Overestimating demand is costly

48
Electricity Demand

• Utilities are interested in forecasting peak
  demand
• Peak demand determines how many generators the
  utility must bring on line
• Overestimating demand is costly
• Underestimating peak leads to electricity
  shortfalls

49
Electricity Demand

• Hourly peak electricity demand depends on
• Weather

50
Electricity Demand

• Hourly peak electricity demand depends on
• Weather
• Time of the Year

51
Electricity Demand

• Hourly peak electricity demand depends on
• Weather
• Time of the Year
• Day of the Week

52
Electricity Demand

• Hourly peak electricity demand depends on
• Weather
• Time of the Year
• Day of the Week
• Holidays

53
Electricity Demand

• Hourly peak electricity demand depends on
• Weather
• Time of the Year
• Day of the Week
• Holidays
• Recent Electricity Demand

54

• IV. Developing the Demand Model

55
General Model

• Independent Variable
• where
• h 124
• is the hour being modeled

56
General Model

• Independent Variable
• where
• h 124
• is the hour being modeled
• d January 1, 1995 September 30, 2003
• is the date of the observation

57
General Model

• Independent Variable
• where
• h 124
• is the hour being modeled
• d January 1, 1995 September 30, 2003
• is the date of the observation
• (Load is given in MWh)

58
General Model

• Dependent Variables
• 1. Weather
• where
• h 124
• is the hour being modeled
• d January 1, 1995 September 30, 2003
• is the date of the observation
• (Temp is given in Fahrenheit degrees)

59
General Model

• Dependent Variables
• 2. Time of the Year
• where
• i 112
• is the month of the observation
• (Month1 1 for January, 0 otherwise,
• Month2 1 for February, 0 otherwise, etc.)

60
General Model

• Dependent Variables
• 3. Day of the Week
• Dayj
• where
• j 17
• is the weekday of the observation
• (Day1 1 for Monday, 0 otherwise,
• Day2 1 for Tuesday, 0 otherwise, etc.)

61
General Model

• Dependent Variables
• 4. Holidays
• Holiday
• a dummy variable for recording days with
  different demand profiles due to businesses
  closing
• (Holiday 1 for Thanksgiving, New Years Day,
  etc. and 0 otherwise)

62
General Model

• Dependent Variables
• 5. Recent Electricity Demand
• electricity demand for the previous three hours

63
General Model

• Dependent Variables
• 5. Recent Electricity Demand
• electricity demand for the previous three hours
• electricity demand for the previous day at this
  hour

64
General Model

65

• V. Model Selection

66
Data

• Electricity demand data were obtained from PJM, a
  large Independent System Operator (ISO) in the
  mid-Atlantic U.S.

67
Data

• Electricity demand data were obtained from PJM, a
  large Independent System Operator (ISO) in the
  mid-Atlantic U.S.
• Temperature data were obtained from the National
  Climatic Data Center (NCDC) for the dates and
  areas of interest

68
Model Selection

• I chose to build an ANN with two layers
• Input Layer with 1-20 hard-limiting neurons
• Output Layer with 1 pure-linear neuron

69
Model Selection

• I chose to build an ANN with two layers
• Input Layer with 1-20 hard-limiting neurons
• Output Layer with 1 pure-linear neuron
• I trained the ANN using the MSE as the
  performance function

70
Model Selection

• I chose to build an ANN with two layers
• Input Layer with 1-20 hard-limiting neurons
• Output Layer with 1 pure-linear neuron
• I trained the ANN using the MSE as the
  performance function
• The optimal number of input neurons was to be
  determined by evaluating their effect on the ANN

71
Model Selection

• Adding neurons allows a model to be more flexible

72
Model Selection

• Adding neurons allows a model to be more flexible
• But each additional neuron requires estimating
  several more parameters (weight and bias vectors)

73
Model Selection

• Adding neurons allows a model to be more flexible
• But each additional neuron requires estimating
  several more parameters (weight and bias vectors)
• McMenamin and Monforte (1998) suggest observing
  the Schwartz Information Criterion (SIC) as
  additional neurons are added to an ANN

74
Model Selection

• The SIC is a measure of model performance, based
  on the MSE, penalized for degrees of freedom.
• (NB the statistic reported is often the natural
  log of the SIC)

75
Model Selection

• The SIC is a measure of model performance, based
  on the MSE, penalized for degrees of freedom.
• (NB the statistic reported is often the natural
  log of the SIC)
• If a neurons benefit outweighs its cost, SIC
  falls

76
Model Selection

• The SIC is a measure of model performance, based
  on the MSE, penalized for degrees of freedom.
• (NB the statistic reported is often the natural
  log of the SIC)
• If a neurons benefit outweighs its cost, SIC
  falls
• When the SIC starts to rise, the optimal number
  of neurons has been surpassed

77
Model Selection
SIC as a Function of Neurons in the Input
LayerHour 13 Hour 18
78
Model Selection
SIC as a Function of Neurons in the Input
LayerAverage over all 24 Hours
79

• V. Model Results

80
Model Results

• For hourly models estimated for the Summer
  months
• two models use 3 input neurons
• five models use 4 input neurons
• thirteen models use 5 input neurons

81
Model Results

• ANN models typically perform well with abundant
  data from non-linear relationships. The hourly
  models account for about 99 of the variation in
  the test data, with MAPE around 0.8.

82
Model Results

• ANN models typically perform well with abundant
  data from non-linear relationships. The hourly
  models account for about 99 of the variation in
  the test data, with MAPE around 0.8.
• The three poor performers were
• Hour 14 R2 0.88, MAPE 2.8
• Hour 18 R2 0.76, MAPE 7.2
• Hour 19 R2 0.68, MAPE 8.0

83
Model Results

• The models appear to be biased towards low values
  in the test period
• As the test period is composed of the most recent
  data, this bias may be due to a growth trend
  (data were not de-trended)
• or to unusually high demand in the test period

84
Model Results
Typical Hour Hour 12
85
Model Results
Poor Performer Hour 14