Instituto de Pesquisas Tecnol

1
University of Sao Paulo Department of Electrical
and Computer Engineering Intelligent Techniques
Laboratory
Energy Cost Optimization in Water
Distribution Systems Using Markov Decision
Processes Paulo T. Fracasso, Frank S. Barnes and
Anna H. R. Costa
2
Agenda

• Anatomy of Water Distribution Systems
• Problem relevancy
• Markov Decision Process
• Modeling a Water Distribution System as an MDP
• Monroe Water Distribution System
• Experiment results
• Conclusions

3
Water distribution system

• It is a complex system composed by pipes, pumps
  and other hydraulic components which provide
  water supply to consumers.

Focus of my work
4
Problem relevancy

• About 3 of US energy consumption (56 billion
  kWh) are used for drinking water (Goldstein and
  Smith, 2002).

2 billion/year
Source Electric Power Research Institute,1994.
5
Markov Decision Process - MDP

• MDP is a model for sequential decision making in
  fully observable environments when outcomes are
  uncertain.
• Advantages of MDP compared to other techniques
• Real world operates in uncertain and dynamic
  domains
• Planning generates control policies to
  sequential decisions
• Optimal solution guarantees to achieve a higher
  future payoff
• Disadvantages of MDP
• Discrete domains (state and action)
• Course of dimensionality

6
Markov Decision Process - MDP

• MDP is defined as a tuple
  where
• S is a discrete set of states (can be factored in
  Nv features)
• A is a discrete set of actions
• T is a transition function where
• R is a reward function where

7
Markov Decision Process - MDP

• Solving an MDP consists in finding a policy ,
  which is defined as a mapping from states to
  actions, s.t.
• Bellamns equation allows to break a dynamic
  optimization problem into simpler sub-problems
• The optimal value of the utility is
• The optimal policy are the actions obtained from
  

8
Water Distribution System modeled as an MDP

• Topology of a typical water distribution system
• States (everything that is important to control)
• Time range
• discrete
• Tank level range
• discrete

9
Water Distribution System modeled as an MDP

• Actions (what you can manipulate)
• Triggered directly
• Associated with a VFD range
• discrete
• Transition function (how the system evolves)
• Calculated by EPANET
• Reward function (how much an action cost)
• Consumption
• Demand

10
Water Distribution System modeled as an MDP
Demand
Final result
MarkovDecisionProcesses
Electrical power
Control policy
Energy priceschema
Constraints
11
Understand MDP results

• Control policy
• Maps state variables into a set of actions
• States variables everything that is important
  to control (tank level and time)
• Set of actions what you can manipulate (pumps)
• Indicates controllability (avoid black region)
• Correlated to demand curve

Tank level
Time
12
Understand MDP results

• Controller
• Uses control policy map to produce actions
• Actions are based just on tank level and time
• Easy to implement and fast to run in PLC (lookup
  table)

Tank level
Pump trigger
Time
13
Monroe Water Distribution System

• Characteristics
• 11 pumps
• 1 storage tank
• 4 pressure monitoring
• 40k people served
• 182 miles of pipes
• Diameters varyingfrom 2 to 42 inches

14
Monroe Water Distribution System

• Demand curve (during summer season)
• Average 6 700 GPM
• Minimum 4 188 GPM
• Maximum 8 389 GPM
• Pressure restrictions (in PSI)
• J-6 65 P 70 ? J-131 45 P 55
• J-36 50 P 60 ? J-388a 40 P 90

15
Monroe Water Distribution System

• Pumps (E2, E3, E4, E5, E6, E7, W8, W9, W10, W11
  and W12)
• Energy price schema
• On-peak (0900 2059) 0.04014/kWh
• Off-peak (2100 0859) 0.03714/kWh
• Demand (monthly) 13.75/kW

16
MDP apply to Monroe WDS

• Mathematical model
• Set of states where
  and
• Set of actions
• Transition function
• Reward function
• Data flux diagram

EPANET DLL
.INP FILE
MATLAB
17
MDP results in Monroe WDS

• Expected electrical power

E5 and E7 consume 144.3kW
W11, E2 and E6 consume 320.4kW
18
MDP results in Monroe WDS

• Number of activated pumps (27 possibilities)

onE2,E6
onE5,E7
onW12,E3,E4,E5
onE2,E3,E4,E5
19
MDP results in Monroe WDS

• SCADA records
• obtained from historical data (July 6th, 2010)
• 75 of WTP consumption is considered to be used
  in pump
• One day is extrapolated to one billing cycle (30
  days)
• Both approaches started in the same level (19.3
  ft)

Energy expenses SCADA records MDP Difference
Off-peak energy /month 3 210.57 2 608.32 -23.1
On-peak energy /month 3 750.78 3 768.51 0.5
Demand /month 3 836.25 3 603.67 -6.5
Total /month 10 797.60 9 980.50 -8.2
20
Conclusions

• MDP avoids restrictions (level, pressure, and
  pumps) and reduces expenses with energy
• To reduce energy consumption is different to
  reduce expenses with energy (demand is the
  biggest villain)
• Summer season imposes small quantity of feasible
  actions
• Verify if it is possible to reduce the number of
  pump combination
• MDP policy is easy to implement in a
  non-intelligent device (PLC)

21
Contact
Thank you for your attention
PAULO THIAGO FRACASSO paulo.fracasso_at_usp.br Av.
Prof. Luciano Gualberto, trav.3, n.158, sala
C2-50 CEP 05508-970 - São Paulo, SP - Brazil
Phone [phone]