Market data and SDDP modelling

1
Market data and SDDP modelling

• Brian Bull (brian.bull_at_electricitycommission.govt
  .nz)

2
Three apparently unrelated topics

• The joys of modern dentistry
• Half-hourly data in the ECs Centralised Dataset
  now published as an SQL database
• SDDP modelling at the EC (as used in the Auckland
  400 kV draft decision)

3
The Centralised Dataset

• The EC is tasked to publish the Centralised
  Dataset periodically and distribute it to
  participants The purpose of the Centralised
  Dataset is to support efficient planning
  processes by ensuring the collection and
  maintenance of historical information required to
  make decisions on transmission and transmission
  alternatives.
• First release was published in July 2005 second
  release in July 2006
• Three main sections
• Network configuration data system load flow
  models (PSS/E, DigSilent), line diagrams, circuit
  breaker information
• Hydrological data weekly and monthly inflows at
  various sites, 1931-2005
• Half-hourly data
• Meter data (HVDC flows, active power by GXP,
  reactive power by GXP, generation by grid
  injection point)
• Bids and offers
• Prices
• Binding constraints

4
Gnash

• Program for accessing half-hourly data,
  distributed with the CDS
• Purpose-built program coded in Fortran
• Loads half-hourly data from a bewildering variety
  of source formats and enables you to view or dump
  the results
• Appearance can be forbidding (see next slide)
• Slightly disturbing oral fixation
• Very useful for reading and collating data, but
  not really set up for doing analysis in the
  past we have preferred to dump Gnash output to a
  flat file and analyse it in another program such
  as Matlab

5
Gnash (screenshot)

6
MySQL Halfhourly Database

• The 2006 Centralised Dataset now includes an
  MySQL database containing the half-hourly data,
  as well as Gnash
• Gnash remains the most authoritative and
  official version, but MySQL can often be more
  convenient for analysis
• MySQL is a free, very popular, multiplatform SQL
  system
• Often used in websites, but used for many other
  purposes as well
• No special server hardware necessary, it will run
  on your desktop machine
• GUI front-end is available for Windows (see next
  slide)
• Or you can use Microsoft Access as a front-end if
  you prefer
• Allows users to run queries like this
• select from offers
• where bidder'GENE'
• and grid_injection_point'HLY2201'
• and record_date'2005-03-04'

7
MySQL (screenshot)

• text

8
What is in the SQL database?

• Data include
• Half-hourly prices by node, extending back to
  late 1996
• Half-hourly metering data, extending back to late
  1996
• Half-hourly offer data by node, extending back to
  mid 2002
• Half-hourly binding constraint data, mid 1998
  through mid 2005.
• Some summary tables added for convenience
• Regional offtake totals (Auckland, North Isthmus,
  Waikato, etc, etc)
• Daily and monthly (unweighted) mean prices by
  node
• Additional error checking and cleaning
• Dummy prices indicating missing or invalid data
  removed (e.g. -9999)
• Initial and final runs of zeros in price data
  removed
• Records with all values missing removed
• Etc, etc

9
What kind of queries are supported?

• Here are some examples
• show final prices at the Pakuranga node in /MWh
  for each trading period on 4 March 2005
• show the maximum price at each node in the course
  of a given day
• show a list of pricing nodes in the North Island,
  excluding generation nodes
• show Genesis's offers for all 48 trading periods
  at Huntly on 4 March 2005
• show the offtake in kW at Woodville in each
  trading period of 4 March 2005
• show the offtake (expressed as an average power
  in kW) for each region of NZ in trading period 25
  on 4 March 2005
• show load by trading period on 4 March 2005 for
  all nodes where the price rose above 100/MWh
• show the most commonly binding constraints over
  the period covered

10
Installing the database

• If you are already working with MySQL, it is very
  easy
• Only issue is that you will need plenty of free
  hard drive space I
  recommend 20 GB at time of installation
• If not, you would just need to install MySQL
  first
• Can run on your desktop PC no need for a
  dedicated server machine
• Installation package can be downloaded from the
  internet instructions included in CDS
• Plenty of good online help available
• One small glitch appeared after release. The
  installer for the MySQL database is zipped for
  some reason the UltimateZip program included with
  the CDS will not unzip it. Use WinZip (I have
  only tried v10)
• If you want the Centralised Dataset and dont
  have it, contact me or Brian Kirtlan
  (brian.kirtlan_at_electricitycommission.govt.nz) for
  your free copy

11
Next steps

• EC is working on an add-on to the database,
  providing the ability to produce various graphs
  of half-hourly data (offer stacks, nodal price
  traces, etc).
• Temporarily on back burner hoping to include in
  next CDS release if people express interest
• Much work still to be done on naming and
  classifying metering data
• Many series from many sources some as yet
  unnamed, others may be named wrongly and need to
  be corrected
• Improving the metadata around metering is an
  important work programme for us
• Want to have consistent labelling of nodes
  throughout MySQL database (would facilitate
  queries joining multiple tables)
• NWC Ltd are busily producing data products based
  on metering and pricing information

12
And now for something completely different
13
A Brazilian model sitting on my desk
14
SDDP

• SDDP is a widely-used model produced by PSR (a
  software company based in Rio de Janeiro)
• a hydrothermal dispatch model with
  representation of the transmission network used
  for short, medium and long term operation
  studies
• SDDP is a least-cost model (like Spectra) rather
  than a market simulation
• The operating policy for each hydro plant is
  determined using an optimisation technique called
  stochastic dual dynamic programming (hence SDDP)
• The criterion is least expected cost, including
  thermal fuel costs, variable OM costs, penalties
  for breaching defined constraints, etc, etc
• The EC uses SDDP to model New Zealands
  electricity system
• Calculates the pattern of dispatch to meet
  predicted loads, on a time step varying from
  hourly to monthly
• Some ability to model transmission, though not to
  the level of power systems analysis (PSA)

15
SDDP at the Electricity Commission

• First used for Waitaki water rights submission
• Model was then further developed for use in
  analysing Transpowers Auckland 400 kV grid
  investment proposal
• SDDP model was used for one specific part of the
  analysis only the
  estimation of loss costs
• SDDP was used to prepare a dispatch pattern for
  each generation scenario, which was then run
  through the ECs purpose-built Loadflow software
  to estimate loss costs under each transmission
  scenario
• The Initial Statement of Opportunities (SOO),
  published in 2005, used Spectra for generation
  modelling. The new SOO will use SDDP.
• SDDP will be used to evaluate generation
  scenarios, in terms of unserved energy (USE) and
  net present value of costs incurred
• SDDP has also been found useful for various
  one-off analyses (for example, analysing the
  economic viability of the proposed
  Onslow-Manorburn pumped storage scheme)

16
How does SDDP compare to Spectra?

• Some big advantages
• SDDP is more flexible (our perception is that
  Spectra has some inconvenient hard-coded
  limitations for example, restrictions on new
  thermal plants in the South Island)
• SDDP can be used to model the AC grid at various
  levels of detail (will discuss the
  options later if time permits), whereas Spectra
  does not model transmission, apart from the HVDC
  link
• SDDP can be run with an hourly time step
• PSR provide good support for SDDP
• But some disadvantages too
• In the New Zealand context, Spectra has been used
  for some time and the model is well understood
  and validated whereas our SDDP model is still
  in development
• SDDP executes much slower than Spectra
• Im not kidding, it really is a lot slower
• Only 5 load blocks are modelled (Spectra had 14)

17
Hows your Portuguese?

• SDDP manuals are in (usually) good English
• Some datafiles still use Portuguese
• Wouldnt it be nice if the abbreviation for
  spilled outflow was, say, spill, rather
  than qverti

18
Time frame in our SDDP models

• Three options monthly time step, weekly time
  step, or hourly time step
• EC needs to run models over a 30-year period,
  which means using the monthly time step (weekly
  time step is limited to about 7 years)
• note, test runs comparing weekly and monthly time
  steps have produced quite similar results so
  this may not be a big issue
• In the Auckland 400 kV analysis, we used Tom
  Halliburtons SDDP Looper program
  to have our cake and eat it too
• firstly do a monthly run over a long period (25 /
  30 years)
• the Looper then invokes SDDP to do an hourly run
  for the first week of each month, generating a
  detailed dispatch
• does the job, although SDDP takes a looong time
  to execute in this mode
• still, at least the constant CPU and hard disk
  activity of multiple PCs over a period of several
  weeks kept the office warm

19
AC transmission modelling in SDDP

• SDDP provides a range of options for modelling AC
  transmission
• Simplest and quickest option no AC constraints
  at all, and losses are modelled by pro-rating
  demand forecast inputs, rather than within the
  model. (Effectively this is what was done for
  Auckland 400 kV GIT analysis)
• Can add AC thermal constraints to the model,
  optionally varying by season (Transpower
  currently reworking this aspect of the model)
• Can add AC group constraints for a more realistic
  model of AC limitations
• Can add modelling of AC line losses within SDDP
  (this slows down execution considerably, as
  dispatch is optimised accordingly).
• We are also doing some development work on a
  regional transmission model, with 17 regions
  (Auckland, HLY/WKM/WRK, Taranaki, etc),
  incorporating current AC constraints, inter- (but
  not intra-) regional AC losses, and potential
  future augmentations

20
Next steps

• Field trip to Brazil?
• Continue to improve model, in conjunction with
  Transpower bug fixes, augmentations, spelling
  corrections in datafiles, etc, etc
• Continue updating SDDPs inputs demand
  forecasts, hydrological inflow records, lists of
  generating plants, etc, etc
• Continue to develop utilities for running SDDP,
  producing inputs and post-processing outputs
• Use SDDP for 2007 SOO (will doubtless be a
  learning experience)
• Develop regional model
• Another in-development project using SDDP to
  build up a lookup table of runs, so that the
  net costs in a year with given levels of load,
  generation, and HVDC transfer capacity can be
  estimated by interpolation from the lookup table
  rather than by running SDDP
• Parallel SDDP supposed to be coming out soon
  (?)